The road towards a better DevLog
Featuring: a static site generator written in Rust
There's a famous story about Donald Knuth developing TeX (which then evolved into LaTeX) because he didn't like how his books looked when printed. It took years.
I'm no Knuth by far, but I decided to spiritually follow his tracks and take some weeks off the project to make something about how I publish the DevLog, hopefully improving the experience for both me and the readers.
Frustrations about post editing/management
There are quite a few things that annoy me about the current editing process:
I write in Markdown, so I need to manually convert some formatting when I prepare a post for publishing using the Substack in-browser editor
I don't have the level of control I'd like when inserting images and videos
I can't use directly embed mermaid diagrams, which I find very useful. I'm forced to convert them to images, which is suboptimal
I can't keep the "final version" of a post, with its resources, into a repository
Frustrations about content visualization
I find the Substack visualization good for generic text content, but not great for technical posts containing code snippets:
there's no syntax highlighting
code blocks wrap, while it would be better to keep the indentation and let the user scroll
the fanciful HTML formatting of emails sent by Substack makes the code blocks unreadable (even when the code is less than 78 character columns wide, something which was fine even in text only emails as specified by RFC2822 a couple of decades ago)
there are annoying popups
Frustrations about the state of the Internet
I thought that using a platform like Substack would help me reach a wider audience, but it doesn't feel like it is helping. Actually, I sometimes feel like it is damaging me: I keep reading about social media platforms sabotaging each other lowering the reach of posts including links, or links to content hosted on specific platforms.
Additionally, some people are actively avoiding Substack as a matter of principle, after some controversial content was not banned from the platform.
Do I agree with that? Do I implicitly endorse Substack by publishing content there?
Why do I even need to spend time considering these things?
Isn't developing a videogame and documenting the process stressful enough?
I don't care at all about the competition between tech giants, nor I want to be dragged into debates on freedom of speech and censorship.
I stay humble: maybe I'll give an opinion on "tabs vs spaces", on a good day.
Frustrations about other static site generators
I used Hugo for the Binary Charm website, and it wasn't a pleasant experience.
I quickly checked Zola and I felt like I would have hated that too.
I'm not saying they are bad, but I felt like, in terms of design, I wanted something different.
It was about controlling the processing logic and being able to define higher level data types on the source side. Basically, I wanted to do more processing inside the generator itself, and less on the "web" side of things. Template systems like Jinja allow to put a bit of logic into them, but they are limited, the syntax is awful, and debugging is basically trial and error.
So, around February 2023, I started working on a static website generator written in Rust, with the long term goal to handle the generation of my personal website, featuring a portfolio section done the way I wanted. I worked on it on and off, prioritizing other things (like Particular Reality, of course!).
The first website I published that used the generator was actually the game homepage, when I announced the project last year. A very simple "monopage" use case.
After some other months, at the beginning of 2024, I finally managed to add a first version of the portfolio management features, and I reached my initial goal of generating a new version of of my personal website, organized the way I wanted.
The generator was still pretty simple, with a few hacks and hardcoded things here and there, but did its job and I was quite happy about the result.
And, of course, it was something that gave me the joy of working the way I wanted, which should never be underestimated, especially when one is working on something personal with no strict time or budgeting constraints (that often force sacrifices and trade-offs on the technical side of things).
Extending my generator to handle the DevLog
The new stage I reached with my generator, paired with the unsatisfaction I discussed in the first paragraphs, pushed me to do another round of development, this time aiming to handle the DevLog entries, so that I can self-host them and fix all the issues I have with the current state of things.
As an added bonus, I will be able to port to the new system also some articles I wrote a long time ago, finally killing the Wordpress instance I am still forced to keep around at the time of writing.
Yes, Wordpress. Guess what? A dozen years ago I started using Wordpress for my website because it looked like the "wise" decision, the "not reinvent the wheel" approach, but even that time, I quickly realized that I hated it and that I did not really need it.
In my generator, every item is defined by a YAML file and a "side folder" with the same name.
The YAML file contains a "front-matter" with metadata about the item, and an optional body (a Markdown text).
The "side folder", if needed, contains one or more files making up the item. For example, an image resource side folder will contain a single file, while a video resource side folder will contain both the video file and an image, to be used as "cover" of the video when it's not playing.
The ability to define in the generator itself high level structured data (like a portfolio entry) is what makes it so convenient to use for me. I can define custom processing for each kind of item: for example, an image resource will produce multiple output images, at different resolutions, to automatically have a responsive image in the output. For video content, I'm currently embedding the Youtube version but I could automatically transcode the source video to a more compact, web-friendly format, and embed that.
To handle "articles", I revised the handling of the core resource type of the generator, the "page" one, which actually produces a web page in the website and can embed other "media" resources.
I had to add custom processing so that the Markdown text of an article can
call "generator side" logic, via shortcodes
handle syntax highlighting for code blocks
handle Mermaid diagrams definitions
This is a video showing the generation in action at the current stage of development:
What's missing?
I still have to work on the DevLog "index" containing summaries for all the articles, and highlighting the latest entry. I must also add tags, something which I already did for my portfolio entries.
Then, I need to add the generation of sitemap and RSS feed. Yes, the RSS feed: it's time for their resurrection.
In visual terms, I'm not crazy about the colors used by the syntax highlighting and the Mermaid diagrams. I would also like to add support for a dark theme to the website, and add a dark/light switcher.
It shouldn't be that much work, and the hard parts should be done, but I decided to stop for now, and go back to the game for some weeks. Then, I'll take another "break" and come back to work on the generator, completing these last things.
What then?
I might change my mind, but the idea is port the old articles, a couple per week, to the new format, until the self-hosted version catches up. I might keep using Substack to publish a short summary of the articles, linking to the full length version, so I can still benefit from the newsletter and social functionalities offered by the platform. Or I might stop using it completely. We'll see how things go!
Meanwhile, see you in a week for some game development - much better than this web stuff.