Website refresh - now with lots more Lua!

Ever since I made the initial version of this website about an year ago, I've always wanted to tweak a few things about it...

In particular, the way that articles appeared on my blog page as just a single link, without any image, felt really unappealing to me. However, the particular way in which I had implemented the blog listing page meant there was no easy way to do that, so I ended up just leaving as a plain, boring list.

Yet, if you hate lists, and love cards with images—rejoice! The new blog list design is a whole set of cards now!

Left—old article listing. Right—new article list: snazzier, cooler, almost masonry-like!

In addition, I have finally released the source for this website up on Codeberg! If you are curious how I made any part of it, feel free to jump in and poke around—the whole thing can be built and run locally, and it's likely that parts of it can be reused for your own Tup-Pandoc-based website! (Which is not how you should be making a website, but hey, it's your website, and I can't stop you 🙃)

Troubles with templates

My journey of revamping my website started with having to... replace the templating engine.
You see, a major limitation of how I made this website was is that I made use of Pandoc's templates. And those are... woefully underpowered. (And as it turned out while experimenting, quite underpeformant too!)

All that Pandoc templates allow you to do is: substitute variables, iterate over arrays, and check for a variable's existence. After processing the article, you get a complete HTML file. Tada.

However, to make a listing of articles, I need to somehow extract those articles' titles and publication dates. What I did last time was go over the source markdown files and use Awk to collect parts of them into a single YAML file. The problem with this is that I couldn't access any part of the finished article, and so I both didn't have easy access to the article's image, and I ended up parsing parts of the articles multiple times, messing up the layout whenever the <!-- SNIP --> came too early.

In order to be able to reuse the metadata of processing the article, it seemed like I would have to split my processing into two steps: converting the article to some intermediary format (while applying all the Lua filters that would extract the correct cover image and such), and converting that to the final article page. Then, I could reuse the intermediary step results to produce the listing page.

Before and after of the article processing pipeline

Here, I experimented with a few options:

  1. First, I tried using Bash with here-docs to write my templates, but after witnessing what the OpenGraph code would have to be written as, I decided squarely against that solution.

    (non-working) OpenGraph code in Bash 
    cat <<HTML
<link rel="icon" href="/favicon.png" />
<meta property="og:title" content="${ogtitle:-$title}" />
<meta property="og:site_name" content="$sitetitle" />
<meta property="og:description" content="${ogdescription:-$summary}" />
<meta property="og:url" content="$host$path" />
<meta property="og:image" content="${ogimage:-$host${firstimage:-/contact_avatar.jpg}}" />
HTML
if [ -n $ogprofile ]; then
    cat <<HTML
<meta property="og:type" content="profile" />
HTML
    # TODO: iterate over keys and values somehow
    cat <<HTML
<meta property="$key" content="$value" />
HTML
elif [ -n $ogwebsite ]; then
    cat <<HTML
<meta property="og:type" content="website" />
HTML
else
    cat <<HTML
<meta property="og:type" content="article" />
${date:+<meta property="article:published_time" content="$date"/>}
${mdate:+<meta property="article:modified_time" content="$mdate"/>}
HTML
    # TODO: deal with og:author
fi

  2. Afterwards, I revisited the idea of using XSLT for templates (which I already use to make the Atom feed look like the rest of the website), which went much better... up until the point I needed a CLI capable of transforming the XML article plus XSLT 3.0 stylesheet, and only found one: Saxon. But it's written in Java! And I'm not adding that to my website's build system 😅

    ..Okay, fine, there's actually a really cool looking new XSLT/XPath tool, called xee by Martijn Faassen, written in Rust. However it still doesn't have XSLT support! And I do want my website this week! 🥲

  3. At some poinet, I also checked out jqt, but it too looked a bit too complicated for what I wanted it to do. If it was closer to pure jq perhaps..

  4. So... at that point, I went full-circle back to Pandoc. Since I had to use it anyway to process the Markdown files, I started looking of ways I could use its inbuilt Lua scripting to make some kind of templates. And.. after a bit of tinkering, I ended up with a PHP-like template system in Lua, which I ended up sticking with.

    (Note: I'm not the only one to have done something like that. There's Danila Poyarkov's lua-template which inspired part of the idea of just rolling something with Lua, though I used a different code generation approach than them.)

    Here's an example of it in use:

    <!-- <?= lua_code ?> executes lua_code and substitutes the resulting value in its place -->
<!-- <? lua_code ?> places lua_code directly into the compiled template, allowing for conditionals, variables, and loops -->
<title><?= doc.meta.title ?> — <?= doc.meta.sitetitle ?></title>
<? if doc.meta.highlighting_used then ?><link rel="stylesheet" href="/highlight.css" /><? end ?>
    (working OpenGraph code with Lua templates 
    <meta property="og:title" content="<?= esc_attribute(doc.meta.ogtitle or doc.meta.title) ?>" />
<meta property="og:site_name" content="<?= esc_attribute(doc.meta.sitetitle) ?>" />
<meta property="og:description" content="<?= esc_attribute(doc.meta.ogdescription or doc.meta.summary) ?>" />
<meta property="og:url" content="<?= doc.meta.host ?><?= esc_attribute(doc.meta.path) ?>" />
<meta property="og:image" content="<?= esc_attribute(doc.meta.ogimage or doc.meta.firstimage or doc.meta.host .. '/contact_avatar.jpg') ?>" />
<? if doc.meta.ogprofile then ?>
<meta property="og:type" content="profile" />
<? for key, value in pairs(doc.meta.ogprofile or {}) do ?><meta property="profile:<?= esc_attribute(key) ?>" content="<?= esc_attribute(value) ?>" /><? end ?>
<? elseif doc.meta.ogwebsite then ?>
<meta property="og:type" content="website" />
<? else ?>
<meta property="og:type" content="article" />
<? if doc.meta.date then ?><meta property="article:published_time" content="<?= esc_attribute(doc.meta.date) ?>" /><? end ?>
<? if doc.meta.mdate then ?><meta property="article:modified_time" content="<?= esc_attribute(doc.meta.mdate) ?>" /><? end ?>
<? if doc.meta.authorurl then ?>
<meta property="article:author" content="<?= esc_attribute(doc.meta.authorurl) ?>" />
<? elseif doc.meta.author then ?>
<meta property="article:author" content="<?= esc_attribute(doc.meta.host) ?>/contact" />
<? end ?>
<? end ?>

So, with that, I used the new Lua-powered system for my blog's listing page, which is now able to load all of the subpages, read their computed first image, and use that for the cover image! With a bit of CSS, it looks way better than before! Success! 🎉

For bonus points, I'm actually using the Lua templates directly in the blog main page's Markdown, like so:

Normal *markdown* above the article list.

<? for doc in in_file do ?>
<a href="<?= doc.meta.path ?>"><?= doc.meta.title ?></a>
<? end ?>

Text just continuing after the articles list.

I can envision a lot of ways in which that kind of templating would come in handy...
such as, massively simplifying how I am currently doing slideshows! 😁

Fixing the build system

With that out of the way, I decided I'd use the opportunity to also revamp the build system and restructure some of the project so it's more presentable.

Earlier, I got away with throwing all the build commands into one Tupfile. However, as the website grew (especially with the addition of my programming course), I started having more and more haphazard commands in that file, since every new folder anywhere required duplicating 3-4 rows of the Tupfile.

Here, I started by moving out all the !-macro commands into a separate file, that I can include from the other files.

Then, I made a common file of rules that processes all the PNG, JPEG, SVG, and Markdown files in a directory, looking like this:

include ./rules.tup
: *.png |> !png |> $(PROJECTROOT)build$(WEBDIR)/%B.png
# ...

Finally, I made a bunch of small Tupfile-s that include that template file, like so:

PROJECTROOT = ../../
WEBDIR = /blog
include $(PROJECTROOT)src/template.tup

(Here, include_rules plus TUP_CWD might have alleviated the need for PROJECTROOT; but WEBDIR seems unavoidable with how few facilities tup provides)

However, that ended up not working for a very bizarre set of reasons:

  1. Not every folder has an index.md file that serves as a listing of the other files.
  2. Not every directory with an index.md file has other *.md files in it.
  3. Since Tup rules may not have missing dependencies outside of foreach, so by reason 1, the Tup rule has to start with : foreach index.md | ....
  4. This leaves only order-only inputs (%i) for the list of the other files.
  5. However, the Tup command fails to execute when there are no order-only inputs (i.e. when there's no non-index files)—thus failing by reason 2.

So... in the end, I rewrote the template.tup file in Lua, since Tup allows for Lua code execution (yay, infinite extensibility!), and Lua can correctly check for the existence of an index.md file. Phew!

This change to the build system makes adding new folders much much easier; I have to copy and lightly edit a single, short Tupfile, rather than weave my way through a sprawling mess of the large Tupfile from before. Hopefully this means more complex parts of the website are coming soon!

(...For a curious aside, I'm surprised at how many of the tools I used for this website ended up having Lua support. I promise I picked them just for their general reliability and/or boringness, not for Lua! 😂)

Publishing the source

Ever since getting challenged by Benjamin Hollon on Mastodon to make websites that can be understood with the "View source" function, I've wanted to publish the source of my website somewhere.

Well, it's out now! 🎉

If you are one of those people who want to see the lowly hacks an experienced programmer resorts to just to get CLI tools with bizarre interfaces to work together, you can check it out at:

codeberg.org/bojidar-bg/bojidar-bg.dev

Conclusion

All things considered, I've had a blast reworking parts of my website. I hit a nice flow state while coding the Lua templates, and I've throughly enjoyed using them so far! There are definitelly kinks to work still, but with a fully-fledged programming language at my disposal, those should be a matter of time, rather than a matter of figuring out extra tools to use.

This has been my fourth post of #100DaysToOffload. Expect a few more pieces of the website to come soon; in particular, a list of my bookmarked articles 😊

That's all for now from me. Have you toyed around with a customized static site build pipeline yet? Or, would you even want to?