How This Site is Built

2019-08-15

0e1.org Homepage

This is the first entry on 0e1.org. I put together this blog / website so I have some place to record what I learn and create on a day-to-day basis. I would like to spend more time engaging with the world in a way I find meaningful. Right now I am unemployed, and it’s becoming pretty apparent that I need some artificial contrivances to keep me on task, and I think this personal learning journal will serve that function pretty well. And I might as well make it anonymous and public, since I have this otherwise unused domain name sitting around.

This site consists of a collection (sparse, at the moment) of markdown files in a source tree. GNU make is used in conjunction with a couple awk heavy bash scripts in order to turn the markdown files into a website and generate the main index page at 0e1.org/index.html. Pandoc is used to do the markdown -> html heavy lifting.

I do not have any previous experience with make and very little with awk, so this solution turned out to be very unsophisticated. There are 3 main files used in the generation of the website: makefile (of course), makeindex.sh, which creates the main index based on YAML meta-data blocks at the beginning of each markdown file in its search path, and blog.sh, which simply creates a markdown file within the blog directory with the naming scheme ‘YYYY-MM-DD.md’. If this file already exists, blog.sh simply opens it for editing in vim.

At the time of writing this, the directory structure looks like this:

├── src
│   ├── blog
│   │   ├── 2019-08-15.md
│   │   └── folderinfo.yaml
│   ├── buildres
│   │   ├── footer.md
│   │   ├── header.md
│   │   └── indexheader.md
│   ├── index.md
│   └── res
│       └── style.css
├── makefile
├── makeindex.sh
├── blog.sh
└── deploy
    ├── blog
    │   └── 2019-08-15.html
    ├── index.html
    └── res
        └── style.css

If the deploy directory were empty, one could rebuild the site by running make. Running make will do 3 things:

  1. If any file within src/res/ has changed, it will be copied into deploy/res/

  2. Any new or modified *.md files within the src/blog directory are converted into .html files using pandoc and placed in the deploy/blog directory. The makefile is designed such that adding more directories in the future alongside blog, such as stories or other categories, is trivially easy. The header.md and footer.md files are automatically included with the creation of each file.

  3. makeindex.sh creates an index of the markdown posts within a list of directories. The script extracts title, date, and abstract metadata from a YAML metadata block at the beginning of each post. The script uses folderinfo.yaml at the root of each directory to get the proper title for the directory. Currently, the only directory it searches is blog, but this can be easily changed in the future.

Since the posts are named according to the date of their creation, it is easy to correctly order the index. makeindex.sh uses the file names to order the index for each subdirectory it indexes. Since the intended use of this blog is as a journal, I will not be writing posts over a long period of time or revisiting past posts often, so the inadequacies of a non-descriptive name are not an issue. It is a primitive solution, but it is easy and straight-forward.

There are a number of very obvious shortcomings to the site at the moment, including that only the main /index.html is generated. I will be making small improvements as I run into issues, but overall am happy with how the site generation process works at the moment. Despite its simplicity, this will serve my purposes for the time being.

I am including the source for each of 3 main files as they are today. I will not be updating these in the future, but may host the living files on github or gitlab or a similar service at some point.

blog.sh

#!/usr/bin/env bash

if [ ! -e src/blog/$(date -I).md ]; then
  printf "%s\n%s\n%s\n%s\n%s\n" \
    "---"\
    "title: $(date -I)" \
    "date: $(date -I)" \
    "abstract: |" "---" \
    > src/blog/$(date -I).md
fi
vim src/blog/$(date -I).md

makeindex.sh

#!/usr/bin/env bash

if [ $# -gt 1 ]; then
  if [ -d $1 ]; then
    cd $1
    for dirname in "${@:2}"; do
      if [ -d "$dirname" ]; then
        if [ -e "$dirname/folderinfo.yaml" ]; then
          echo "## $(awk -F: '/title:/ {print $2;exit;}' $dirname/folderinfo.yaml)"
        else
          echo "## $dirname"
        fi
        for filename in $(find "$dirname" -name "*.md" | sort -r); do
          title=$(awk -F: '/title:/ {print $2;exit;}' "$filename")
          date=$(awk '/date:/ {print $2;exit;}' "$filename")
          abstract=$(awk \
            '/abstract: \|/,/---/{if(/---|abstract: \|/) next; print}' \
            "$filename")
          htmlfn=$(echo $filename | sed 's/.md$/.html/')
          printf "### [%s](%s)\n" "$title" "$htmlfn"
          printf "Created: %s\n\n" "$date"
          printf "Summary: %s\n\n" "$abstract" 
        done
      fi
    done
  else
    echo "$1 is not a directory"
  fi
else 
  echo "Correct usage: ./makeindex.sh <src dir> [<subdirectories with .md files> ...]" 
fi

makefile

srcdir := src
subdirs := blog

resdir := $(srcdir)/res
reslist = $(shell find $(resdir) -type f)
outreslist = $(reslist:$(srcdir)/%=$(outdir)/%)
stylesheet := /res/style.css

buildresdir := $(srcdir)/buildres
buildress = $(wildcard $(buildresdir)/*)

panoptions := -s -w html --css=$(stylesheet)

outdir := deploy

srcs = $(foreach dir,$(subdirs),$(wildcard $(srcdir)/$(dir)/*.md))
objs = $(srcs:$(srcdir)/%.md=$(outdir)/%.html)

$(outdir)/%.html : $(srcdir)/%.md $(buildress)
    mkdir -p $(@D)
    pandoc $(panoptions) -o $@ $(buildresdir)/header.md $< $(buildresdir)/footer.md

all : resources html index

resources : $(outreslist)

$(outreslist) : $(reslist)
    mkdir -p $(@D)
    cp $< $@

html : $(objs)

index : $(outdir)/index.html
  
$(outdir)/index.html : $(srcs) $(buildress)
    @rm -f $(buildresdir)/blogindex.md
    ./makeindex.sh $(srcdir) $(subdirs) >> $(buildresdir)/blogindex.md
    pandoc $(panoptions) --toc --toc-depth=2 -o $(outdir)/index.html \
    $(buildresdir)/indexheader.md $(buildresdir)/blogindex.md $(buildresdir)/footer.md
    @rm -f $(buildresdir)/blogindex.md

clean : 
    rm -rfv $(outdir)/*
    rm -fv $(buildresdir)/blogindex.md