jasmin's little garden

Small collection of fish functions for working with the Hetzner Cloud

2026-08-15T10:49:09Z

Both in private and at work, I’m using the Hetzner Cloud CLI hcloud extensively. The latter shouldn’t be surprising, given the fact that I’m working for them.

hcloud supports having multiple, so called contexts. It makes it easier when working with multiple projects, since you can store multiple API tokens and project configurations locally. However, with each new context I’ve added, I felt the need to find a comfy solution for some workflows. Nothing an hour or two writing only shell scripts wouldn’t fix. :>

Because I’m using fish, the snippets below are also all written for that. That being said, it should be pretty easy to adjust them to your favorite shell, feel free to copy the ideas here.

`hcloud-token` – Print the current token

This is just a small helper and allows me to get the current token whenever I need it. For example, I can call tools that require the current Hetzner Cloud token via a HCLOUD_TOKEN by prepending them with HCLOUD_TOKEN=(hcloud-token).

function hcloud-token --description 'Prints the current token'
    hcloud config get token --allow-sensitive
end

`hctx` – interactive context chooser

hctx combines hcloud with fzf and allows me to set the current context interactively. When I do not select any context, it’ll be unset.

function hctx
    # `string trim` is necessary, because `hcloud` appends whitespace to each entry for the columnar layout,
    # even if there's only one column.
    set available (hcloud context list -o 'noheader' -o 'columns=name' | string trim)
    set chosen (printf "%s\n" $available | fzf)
    or begin
        echo 'no context chosen, unsetting hcloud context...'
        hcloud context unset
        return
    end

    hcloud context use $chosen
end

While it’s a minor improvement, it would only be half as useful without the next function.

`fish_hcloud_prompt` – prints the current context in the prompt

What I love about fish, is that fish_prompt is just a regular fish function. It’s one of the design choices that I really admire, since it makes extending the prompt quite easy. My fish_prompt calls fish_hcloud_prompt, which itself prints information about the current context. I won’t go into detail of the fish_prompt configuration, you can find my configuration on Sourcehut.

The fish_hcloud_prompt itself will print the name of the current context and the name of the project it’s residing in. It’s using an undocumented API to query information about the project. If you click around in the UI with the DevTools of your browser being open, you can see quite a number of them, including the /v1/_tokens/current endpoint.

There’s closed issue #362 in the hcloud-cli requesting this feature, but given the lack of a public endpoint, I do get why it’s not implemented yet.

function fish_hcloud_prompt
    # Skip the hcloud prompt if we don't have it installed.
    command -q hcloud; or return

    # Skip it if there's no active context.
    set -l context_name "$(hcloud context active)"
    test -n $context_name; or return

    # Prefix the prompt with the name of the context
    set -l hcloud_prompt ' (hcloud: '
    set -a hcloud_prompt $context_name'/'

    # Add the name of the project
    set -l endpoint (hcloud config get endpoint)
    set -l path /_tokens/current
    set -a hcloud_prompt (curl -s $endpoint$path --header "Authorization: Bearer $(hcloud-token)" | jq -r '.token.project.name')

    set -a hcloud_prompt ')'
    echo -n -s (set_color red)$hcloud_prompt(set_color normal)
end

That is by far the most recent and most useful addition to my prompt. It displays something like (hcloud: masto/Mastodon), displaying the name of the context and the actual name of the project I’m working on. This avoids me caring too much about the actual context names and displays them right in my shell.

It might make sense to cache the output instead of invoking curl on every invocation of fish_hcloud_prompt, otherwise you might run pretty quick into rate limits. I’d leave the implementation of that as an exercise for the reader. :p

Fixing a FRITZ!Box 6660 Cable to get updates again

2026-08-09T08:55:47Z

One week ago, I realised that the router at home wasn’t recieving updates. It’s a FRITZ!Box 6660 Cable and I have to admit, I was very confused about the fact that the device doesn’t receive any updates. I knew that this was more common back in the days when it wasn’t possible to use arbitrary routers with your ISP. That practice was known as the famous Routerzwang and it was forbidden a couple of years ago. I bought this router second hand for a few bucks and when I bought it back then, I didn’t think about it at all. I assumed I could use it and it would work fine. And it actually did, we we’re able to use it and replace the even shittier router that we got as part of the contract.

At first, I thought that the device might have reached it’s end of life. A quick look at the product page however made me realise it’s well supported. But given that the device couldn’t find any updates and, additionally, I wasn’t able to provide my own update files via the web interface, I fell into a small rabbit hole.

After all: The device I’m using is well-supported and therefore it should be possible to run a newer version of the firmware provided by the manufacturer, right? It’s not like I want to install a custom Linux system on it, no, I want to install a different firmware.

Turns out: It’s possible.

The way to unlock the box, however, is a bit more complicated than I anticipated. In an ideal world, it would be the press of a button, together with some “I know what I’m doing” checkbox. I’m glad that I’m not the first one who’s trying to do this and hence, there is information out there on how to get it resolved.

What I did

After a bit of digging, I found the ffritz repository on Bitbucket, with the README-6591.md being the most important file for me.

First, I tried to get more information about the firmware version is running. To do that, I had to download the “extended support information”, which is a huge .txt file and do a search for firmware_version. In my case, this returned:

firmware_version	avm
DMC	RTL=n,SL1

Those are good news! Because the firmware version is avm, I know this isn’t some box by some ISP, it’s a “regular” box. The only difference is that RTL=n (RTL = retail) is set to no and I only have to override that setting. Apparently, this can be done via ftp, which I think is some wild way of configuring the system. 😄

To do so, I then set the IP of my notebook to the 192.168.178.2, set the gateway to the 192.168.178.1 and the subnet mask to 255.255.255.0 respectively. I connected it via the “LAN 1” port on the box, cut the power, waited a couple of seconds and then turned it back on. On my notebook, I opened ftp and executed the following:

$ ftp # opens the interactive FTP utility
ftp> open 192.168.178.1
Connected to 192.168.178.1 (192.168.178.1).
220 ADAM2 FTP Server ready
Name (192.168.178.1:jasmin): adam2
331 Password required for adam2
Password:
230 User adam2 successfully logged in
Remote system type is AVM.
# Now, we set the DMC variable to mark this box as a proper retail box
ftp> quote SETENV DMC RTL=Y
200 SETENV command successful
# And this reboots it. That can take a while, I disconnected from the FTP client by pressing Ctrl+D.
ftp> quote REBOOT

What’s important to know is that the 192.168.178.0/24 subnet has to be chosen, irrespectively of your actual router settings. In our case, the device is usually reachable under the 192.168.188.1 (a legacy from the past which wasn’t changed to the factory defaults so far), yet I was only able to reach it via the 192.168.178.1.

After the reboot, it took a moment longer than I anticipated for the device to come up again, but after a minute or two, it was back again. And when logged in, I could finally navigate to the update screen and: it found a new update! My quest was done, we have a happy ending!

Two years since the last update, however, jeez. I probably don’t want to know how many unpatched devices are out there. 🙈

References

Writing (regularly) is hard

2026-08-04T21:08:20Z

There’s nothing much to say, except that writing, whether it’s in a professional context or on thís blog, is just hard. My brain isn’t used to it, words do not, unfortunately, just come out of my brain and onto the screen. I have lots of ideas of things to write about, yet, I don’t find the time to get it actually done. It surely doesn’t help either that I have to work eight hours a day, time solely dedicated to work and not this blog.

It doesn’t matter if it’s documentation, a novel, or just a random README.md. To those who have mastered this profession: You have my respect and I admire you.

Writing is, I think, a very underestimated skill. I’ve seen too many meeting notes that we’re incomplete or just a very very loose list of bullet points. No one understood them three weeks later, half of the discussed points were missing. Not to speak of the documentation that is halfway dead even before it was started. To get your thoughts sorted and to put them in a concise manner, that’s just chef’s kiss.

It’s a skill that needs practice. A lot of it. It’s a real bummer nonetheless. I want to write about things, yet I know it takes a lot of effort, because it’s hard, and so I don’t practice. welp. Guess, I am literally this meme:

Games I played recently

2026-07-27T20:37:26Z

Originally, I planned to write posts for each of them, but honestly, that feels more like a chore. Hence, this is just a quick summary of the games that I’ve played the past couple of months and what I think about them.

Split Fiction

I wrote about Split Fiction in May 2026 and I’d still recommend it.

Assassin’s Creed Shadows

I got this game as a Christmas gift in 2025. Since then, I’ve been occassionally playing it whenever I felt like it. Because it’s 2026 and open world games are filled with a ton of things to do, it’s not finished yet. And I already spent 90 hours in it.

But I also have to say that I try to enjoy such games and grasp the open world. This means, that depending on my mood, I might me just wandering around there, slaughtering random enemies, riding the horse and discovering the world. I don’t have to rush it. If I remember correctly, I started my journey into the Assassin’s Creed (AC) series with Unity, the part that played in France, if I remember correctly. I got really into it with AC: Origins (the part that played in Egypt). Ubisoft started to include more RPG elements with that part, going away a bit from the stealth game that it once was. Because it was their first attempt, it wasn’t really balanced. As a player, it was too easy after a while. Equipped with burning arrows, poisoning attacks and a burning sword, it was possible to kill everyone in a highly manned castle.

I also played the games that came afterwards. Assassin’s Creed Odyssey, playing in Greek (and which I really love because you can literally fuck around and be a whore lol), Assassin’s Creed Valhalla (less gay, but still enjoyable to a certain extent) and now Assassin’s Creed: Shadows. I skipped Mirage, I was occupied with a different game at the time and it didn’t seem worth it.

Anyway, back to AC: Shadows. At first, I was worried about the open world. Sure, most of the time, game publishers can create a beautiful world, but that’s not worth anything if it feels dead. The worst example for me is Tom Clancys: Ghost Recon Breakpoint. A massive open world, but apart from some enemies, it’s dead. You don’t see many civilians, birds or other critters. Just drones and enemies. (I got sidetracked again, didn’t I?)

Shadows however, doesn’t feel dead. It feels alive. There’s a certain kind of randomness to it and the game design really helped here. In previous games, the structure was hierarchical. You started with the easiest enemies and worked yourself up the ranks unless you can defeat the end boss. Shadows took a different approach: Instead of having one hierarchical tree of enemies, you have smaller clusters of groups to defeat. And a lot of them, which are all discovered over time. And there are no hints about them early on, you just have to play the game! I really enjoyed that game design. It made the game feel less like a chore and more like an adventure.

I also enjoy the character design and character switching. If you don’t know anything about the game: You can switch between Naoe (shinobi) and Yasuke (samurai) and depending on the situation one might be better suited for a task than the other. As I mentioned it earlier, you we’re quite overpowered in the game with all the utensils available for a fight and the stealth tactics. Shadows really forces you to play stealthy or offensive. Naoe isn’t able to bow-hunt and Yasuke isn’t able to sneak or assassinate enemies. In the early hours of the game I died multiple times because I was caught and wasn’t able to defeat!

There’s only thing I really hate about the game: Ubisoft should finally get rid of the Abstergo layer. They try to push it into there and each time it’s more disconnected. In Shadows, there’s a “Anubis” menu where you can also buy stuff with Helix credits (wtf) and it’s one of the worst user experiences I can imagine out there. Even if I wanted to use it, I wouldn’t bear with that for one reason: It is fucking slow. It’s like running in 5 frames per second on the PS5. The in game mechanics? Smooth as hell. Rendering a menu with 10 entries: slow as fuck. I really wonder how they achieved that.

Need for Speed: Unbound

A racing game for the moments my mind is humming “d-do do do, Max Verstappen”. It’s a Need for Speed. Full of dad jokes, unrealistic driving mechanics and fast cars. I wouldn’t spend more than 20€ on it, but since I got it “for free” (as part of the PlayStation Plus abonnement), I can recommend it nonetheless. It’s really good for a quick gaming session in the afternoon, you really don’t have to think when playing it.

Star Wars: Outlaws

Another one of the games that I played only thanks to PlayStation Plus. It’s an open world game in the Star Wars universe. The best part of the game is Nix, you pet companion, which you can pet all the time. In comparison, the main character, Kay, has a rather blunt character. Maybe I didn’t get the character design, but her dialogues felt… disconnected? I don’t know, it’s hard to describe. It felt like the people designing the character and story and the voice actors we’re never in the same room. That was, unfortunately, a common theme until the end.

The end of the story didn’t make sense as well in my opinion. Do you know this moment when you watch a movie, which had a rather slow build up over 90 minutes and suddenly, in the last 20 minutes, everything gets resolved at once and there’s a happy ending?! That’s Star Wars: Outlaws in a nutshell.

Life is Strange: Double Exposure

gay gay homosexual gay. 11/10 much lesbianism, I approve.

No seriously, I can only recommend it. It’s dealing with a lot of negative mental health stuff, but luckily the game allows to enable content warnings on a case by case basis in the menu. If I remember correctly, you have to set it right at startup, so you don’t get exposed without being warned.

It also has cute lesbians. And older trans lesbians with leather jackets. Come on, play it. :>

But be warned: You’ll cry. I’d love to tell you more about it, but given the fact that I finished the game after 12 hours, I’m afraid that everything would be a spoiler. And the game doesn’t deserve that I spoil it to you.

moo!

2026-07-01T21:24:06Z

I don’t know why, but for some reason, YouTube has an ASCII cow.

And I love that. I stole it. It’s mine now. >:3

moo! 🐄

Building an image-based Debian 13 disk image using mkosi

2026-07-02T11:32:31Z

mkosi, a project by the systemd folks, describes itself as a “fancy wrapper around dnf --installroot, apt, pacman and zypper that generates customized disk images with a number of bells and whistles”.

What it basically does is: It bootstraps a system using the package manager of your choice, configures the partitions via systemd-repart and at various steps in the process, you can hook into that. You can add additional files that should be part of the partitions, you can configure users, you can probably configure that system exactly to your needs. If you’ve used Packer in the past, it’s similar to that. Except that it doesn’t take existing images, it builds entirely new ones.

However, like all toolings out there, it has opinions. Disk images are UEFI by default, the dependency on several systemd dependencies is, given the origin of the project, inherited and – and that’s probably the most unfortunate part – it doesn’t have that much documentation or blog posts explaining the concepts. On top of that, the configuration file mkosi.conf changed a bit in the past couple of years and so older blog posts might not be actually adaptable for the current version, which is v25 as of writing this. (heck, I didn’t realise that I updated the mkosi documentation in the Arch Linux Wiki already a year ago, how long have I been postponing this blogpost?)

This week, at the company-internal hackathon, me and my team lead Julian met and together, we tried to deploy a minimalistic immutable Debian VM onto the Hetzner Cloud. It was an adventure and the crappy hotel wifi didn’t make it easier for us. But, after a while, we got it working. And so, this blogpost was born.

(Disclaimer: I am employed by Hetzner Cloud. Opinions in this blogpost are my own and my employer has and had no influence about the contents in this post.)

The goal: Debian, but image-based

What we tried to achieve was the deployment of a minimal Debian system. In our case, minimal means: What is the absolute minimum of configuration required to deploy a working disk image that can be deployed as-is to the Hetzner Cloud? Because that would’ve been too easy, we also wanted it to be image-based (also commonly referred to as “immutable”).

That means: /usr is read-only and there’s no package manager to install additional toolings.

With this goal in mind, we started experimenting. The biggest hurdle was the fact that many of the less expensive server types still use a BIOS. For mkosi, UEFI is required. From our experiences however, the CPX models all seem to use UEFI by default. And so, we chose a CPX22 server for our experiments. For the rest of this post, I assume that you have mkosi installed and a plain mkosi invocation without any configuration or parameters works as intended on your system. (NixOS enjoyers, I’m particular looking at you. 😘)

The beginnings: Preparing the cache

One of the things that really help with speeding up builds is the existence of a mkosi.cache directory. If it exists, package files downloaded as part of the process are cached in there. We also want to make sure that our created files do not clutter the project folder. If a mkosi.output folder exists, mkosi writes its files into that. So, without further ado:

mkdir -p mkosi.{cache,output}
echo "/mkosi.{cache,output}/" >> .gitignore

Step 1: Setting up our `mkosi.conf`

We’ll be starting with a minimal mkosi.conf. I set the Timezone, Keymap and Locale explicitly instead of sticking to the defaults. Personally, I’m a fan of documenting things explicitly. That reduces the ambiguity when I haven’t looked at the configuration for a longer time.

[Output]
Format=disk

[Distribution]
Distribution=debian
Release=trixie

[Include]
Include=mkosi-vm

[Content]
Bootable=yes
Timezone=Europe/Berlin
Keymap=us
Locale=C.UTF-8
Autologin=true

One of the most important lines is the [Include] block. Without that, a bunch of packages aren’t installed. This includes the kernel, systemd and a lot of other packages. If it isn’t included, mkosi would be failing with:

$ mkosi
// ... omitted for brevity
‣ An EFI bootable image with systemd-boot was requested but a systemd-boot binary was not found at /usr/lib/systemd/boot/efi

If we run that image via mkosi vm, we have indeed a custom Debian image. Kudos to the mkosi maintainers for making it so easy. But hey, let’s make it more complex now. :>

Sidenote: Debugging the configuration

Trying to understand what’s included in the current mkosi configuration and what isn’t is not that easy. Gladly, there’s the mkosi cat-config command which also prints not only the current mkosi.conf, but all the other upstream configuration files from the mkosi project as well.

Step 2: Configuring the partitions

If we have a look at the image that we’ve built, we see two partitions:

$ mkosi
$ sudo systemd-dissect mkosi.output/image.raw
 File Name: image.raw
      Size: 1G
 Sec. Size: 512
     Arch.: x86-64
Image UUID: ffceab95-d3d2-4248-b198-ee60d799ab12
Image Name: image

OS Release: PRETTY_NAME=Debian GNU/Linux 13 (trixie)
            NAME=Debian GNU/Linux
            VERSION_ID=13
            VERSION=13 (trixie)
            VERSION_CODENAME=trixie
            DEBIAN_VERSION_FULL=13.5
            ID=debian
            HOME_URL=https://www.debian.org/
            SUPPORT_URL=https://www.debian.org/support
            BUG_REPORT_URL=https://bugs.debian.org/

    Use As: ✓ bootable system for UEFI
            ✓ bootable system for container
            ✗ portable service
            ✗ initrd
            ✗ sysext for system
            ✗ sysext for portable service
            ✗ sysext for initrd
            ✗ confext for system
            ✗ confext for portable service
            ✗ confext for initrd

RW DESIGNATOR PARTITION UUID                       PARTITION LABEL FSTYPE ARCHITECTURE VERITY GROWFS PARTNO
rw root       b590b49b-09d4-4390-abf4-40ceb47748d5 root-x86-64     ext4   x86-64       no     yes         2
rw esp        b0641635-8d07-487f-a43c-f62598cab948 esp             vfat   -            -      no          1

Our root partition is rw (read-write). That means that arbitrary processes can modify the rootfs and any other file in it. To change the partitions, we have to put partition definitions into the mkosi.repart folder.

Beware: An empty mkosi.repart folder does not mean that the defaults are used, it means that you don’t want to have any partitions at all. We figured that out the hard way.

ESP partition

Just like in the current setup, we need two partitions: One for the esp/boot partition and the other one with the other system components of /usr. The mkosi.repart/00-esp.conf is defined as follows:

[Partition]
Type=esp
Format=vfat
CopyFiles=/efi:/
CopyFiles=/boot:/
SizeMinBytes=1G
SizeMaxBytes=1G

That creates a boot partition with a size of 1Gi, definitely large enough for anything we’re aiming to store in it. Depending on your requirements, you can reduce the size as well, but I personally discourage that. Increasing the size later on isn’t worth the effort to save like 500MB, unless you absolutely have to.

erofs for `/usr`

That being said, we also add a /usr partition, defined in a mkosi.repart/12-usr.conf:

[Partition]
Type=usr
Format=erofs
CopyFiles=/usr:/
Minimize=yes

The EROFS filesystem is read-only by design, therefore we don’t have to mount it as read-only later on. It also means that it’s impossible to write stuff into, even if we (or an attacker) wanted to.

Let’s build and start our system again with mkosi vm -f. The system is starting until…

[    **] Job dev-gpt\x2dauto\x2droot.device/start running (15s / 1min 30s)
// ...
[ TIME ] Timed out waiting for device dev-gpt\x2dauto\x2droot.device - /dev/gpt-auto-root.
[DEPEND] Dependency failed for initrd-root-device.target - Initrd Root Device.
[DEPEND] Dependency failed for sysroot.mount - Root Partition.
[DEPEND] Dependency failed for initrd-root-fs.target - Initrd Root File System.
[DEPEND] Dependency failed for systemd-pcrfs@sysroot.service - TPM PCR File System Measurement of /sysroot.

(btw, since this is starting QEMU under the hood: You can terminate the currently running VM by pressing Ctrl-A+X)

`root=tmpfs` to the rescue?

Womp womp. Why is that? I think (although I’m not 100% sure), that we need a / root partition and we don’t have one. There are several ways we can provide one. One of them would be the root=tmpfs kernel command line, which would then be interpreted by the systemd-fstab-generator. So, let’s just do that.

[Output]
Format=disk

[Distribution]
Distribution=debian
Release=trixie

[Include]
Include=mkosi-vm

[Content]
Bootable=yes
Timezone=Europe/Berlin
Keymap=us
Locale=C.UTF-8
KernelCommandLine=
+   root=tmpfs
Autologin=true

Building and starting the VM will now stop at:

         Starting initrd-switch-root.service - Switch Root...
[FAILED] Failed to start initrd-switch-root.service - Switch Root.

And if we look into the status by executing systemctl status initrd-switch-root.service, we get the following information:

Failed to switch root: Specified switch root path '/sysroot' does not seem to be an OS tree. os-release file is missing.

sighs. I have to admit, this is confusing at hell. Because, yeah, it’s right: ls -l /sysroot | wc -l returns 0. The folder is empty. However, /usr/lib/os-release actually contains the files we need. I am actually wondering where the /sysroot is coming from. (Update 2026-07-02: It is the default folder of systemctl switch-root.) Since the systemd-fstab-generator mentions the mount.usr=dissect boot flag as well, we added this. Did it help? No. This time, sysusr-usr.mount failed with:

mount: /sysusr/usr: fsconfig() failed: dissect: Can't lookup blockdev.
       dmesg(1) may have more information after failed mount system call.

We first assumed, that systemd-dissect was missing in the initrd. But, as it turns out, it wasn’t that. /dev/vda gets mounted, dmesg doesn’t seem to report any errors as well. So, what’s going on? Turns out, mount.usr=dissect is not supported by systemd v257 (which is the current version in Debian 13). When looking at the documentation, the current version displayed is systemd v260. We we’re looking at the wrong version of the documentation. And so, mount tried to find a device named dissect instead.

How can we circumvent it? Well, since mount.usr takes in the arbitrary arguments one can pass to mount, we can add a partition identifier to let it discover our /usr partition in a different way. For example, by passing a UUID of the partition. When looking at the docs of mkosi, it even says:

Otherwise, if the value of this setting contains the literals root=PARTUUID or mount.usr=PARTUUID, these are replaced with the partition UUID of the root or usr partition respectively.

So, without further ado, let’s do that:

# mkosi.conf

KernelCommandLine=
    root=tmpfs
+   mount.usr=PARTUUID

And now it works! Running mkosi vm will now also trigger the systemd-firstboot.service, which allows us to enter a default root password. For now, let’s just keep it empty and press Enter twice.

Moving stuff from `/etc` to `/usr/share/factory/etc`

The system runs, but fails with:

login: PAM failure, aborting: Critical error - immediate abort

Why? There’s a bunch of files that we’re installed in /etc. But we don’t copy any of these files over, we just take the /usr partition as is. /etc is empty. That means we’re probably missing some crucial configuration files that are required by the PAM stack. And, indeed, when looking at the mkosi.postinst.chroot script in the ParticleOS repository, it mentions fixes for the PAM stack.

But that’s not the only important script we should copy from that repository. The mkosi.finalize script is equally important! It copies all of /etc to /usr/share/factory/etc. You might wonder, how it all will be copied back into /etc? That’s done by systemd-tmpfiles, the etc.conf has quite a lot of symlinks. So, let’s copy those over as well. After it’s done, there should be three new files in the same folder as the mkosi.conf:

mkosi.extra/usr/lib/tmpfiles.d/etc.conf
mkosi.finalize
mkosi.postinst.chroot

Unfortunately, this didn’t fix the PAM errors for me. After hours of experimenting and comparing it with an existing Debian 13 VM, I stumbled across PR #84 in the ParticleOS repo, which mentioned a Debian-specific fixes. That’s how I found the etc-debian.conf. Adding it next to the existing tmpfiles.d definition and rebuilding the VM fixed the issue! 🎉

(Note to the mkosi/particleos maintainers: As someone who isn’t working with mkosi everyday, it’s not that easy to figure out where to search for the configuration snippets. Sometimes it’s part of mkosi itself, sometimes the fixes for solutions can be found in the particleos repository. It’s a lot of trial and error. 😕)

Of course, it’s very basic. Depending on what you’re trying to achieve, I think it makes sense to install cloud-init and qemu-guest-agent into the VM as well. Remember: You can’t install things via apt, if something is missing like fdisk, debugging becomes a lot harder.

Uploading the image to Hetzner

Julian’s hcloud-upload-image tool has a ~~WIP~~ merged(!) implementation of a hcloud-upload-image write-to-disk command. So, let’s just use that:

$ export HCLOUD_TOKEN=(hcloud config get token --allow-sensitive)
$ # Build a compressed disk image to reduce the amount of bytes that have to be transfered.
$ mkosi build --compress-output=yes
$ hcloud-upload-image write-to-disk \
    --server mkosi-exp \
    --image-path mkosi.output/image.raw.zst \
    --compression zstd

Alternatively, if you don’t want to use the tool, you can also do something along the lines of:

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null 'zstd -cd | dd status=progress of=/dev/sda bs=4M conv=sparse' < mkosi.output/image.raw.zst

Recap

Let’s recap what we’ve build so far. We have a Debian 13 disk image, which contains the necessary (and some extra quality-of-life) components to run the system. apt and other package managers are not part of the system. The whole system is bootstrapped out of a read-only /usr partition. Due to / being a tmpfs, all state is lost on reboot.

I can imagine this scenario being already useful for things like ephemeral CI runners or DNS servers. And for servers that run longer than a workday, the concept of erasing your darlings has it’s benefits as well.

If however, it’s a goal to manage systems that have state and store said state locally, I’d argue to look into repart.d and the repart.d files by the ParticleOS repository. With those, you can easily bootstrap any number of partitions (swap, state, home, etc.) declaratively. I, for my part, will also look into the Unified Kernel Images (UKI) options more.

And, to make it easier for you to start working on it: I’ve prepared a Git repository containing all the files in place. You can find the nachtjasmin/mkosi-debian-experiment on Codeberg.

Happy hacking!

Using Apache2/httpd instead of Caddy

2026-06-01T21:17:25Z

In the beginnings, this blog was hosted on Uberspace. Because of that, it was served by the httpd server, also known as Apache2. After migrating it to my own server, it was served by Caddy instead.

The Caddyfile I used to serve this homepage was rather simple:

https://jasminchen.dev {
    root * /usr/share/caddy
    file_server
}

However, it came with a caveat, which I only realised recently. Caddy doesn’t support the .htaccess files by httpd. This wouldn’t be an issue for most things, but I was using that for redirects of old URIs. And since I’m convinced that cool URIs don’t change, I wanted to fix that.

The maintenance of my redirects is done via an aliases entry in the frontmatter and a Nunjucks template htaccess.njk, which iterates over all pages and writes Redirect 301 entries to a .htaccess file. I haven’t found a suitable “Caddy-native” alternative which didn’t feel like a hack. I also don’t want to maintain a custom configuration snippet and reload that every time I deploy my homepage. And since it’s just a static homepage, I don’t have any special requirements at all. I just need a web server that serves a bunch of HTML and CSS.

Given that I use Fedora as my server OS, httpd was installed pretty quickly via dnf install httpd mod_md. mod_md is the module that provides ACME support for Apache out of the box, so I don’t need certbot/acme.sh/… as well. The configuration file that I placed in /etc/httpd/conf.d/homepage.conf is a bit longer than the Caddyfile above, but I still think it’s pretty concise:

MDomain jasminchen.dev
MDCertificateAgreement accepted
MDContactEmail acme@example.com

# Redirect all http to https, automatically sets the HSTS header.
MDRequireHttps permanent

<Directory /var/www/html>
    Options Indexes
    # Enable .htaccess files
    AllowOverride All
    Require all granted
</Directory>

<VirtualHost *:443>
    ServerName jasminchen.dev
    DocumentRoot /var/www/html/

    SSLEngine on
    # Enable HTTP2
    Protocols h2 http/1.1
</VirtualHost>

And with that, all my redirects for old URLs are working again, yippie! :3

Converting a systemd-homed user to a regular one

2026-06-01T21:17:25Z

When I set up our homeserver, I tried out the newest kid on the block for user management on systemd-based distros: systemd-homed. I no longer remember why I chose it, I guess it was just the regular “oooh, shiny, want to play with it”.

Yet, the fact that I wasn’t able to just rsync anything onto the remote server easily without activating the session first by logging in via a separate SSH session annoys me. Therefore I decided to ditch the homed approach and replace my user with a regular one.

To do so, I performed the following steps:

Stop the current session: homectl deactivate jasmin.
Create a backup of the homedir by executing sudo rsync -aHAUXv /home/jasmin.homedir/ /home/jasmin.homedir.bak/.
Remove the user via homectl remove jasmin.
Create the new user using sudo useradd -G wheel -m -s /usr/bin/fish jasmin.

This automatically adds me to the wheel group for the sudoers, sets my shell to fish and creates a new home directory at /home/jasmin.
Restore the backup with sudo rsync -aHAUXv --chown jasmin:jasmin /home/jasmin.homedir.bak/ /home/jasmin/. Since the old files were owned by nobody:nobody, the --chown jasmin:jasmin ensures that the new user actually owns their files.

And voilá, now I converted my homed user into a regular one.

In retrospect, I should’ve specified the UID of the new user to be the old one. I converted the old files and folders that belonged to me afterwards by running:

# Starts a transient systemd unit to find all directories
# in /data that belonged to the UID 60424 (my previous one) and
# changes the owner of them to the new user and group.
#
# fd is: https://github.com/sharkdp/fd
systemd-run -- fd . /data -o 60424 -tdirectory -x chown jasmin:jasmin

/data is the root directory of the ZFS pool. Of course, you can run it for other folders as well, I also ran it for /etc and /var, just to be sure.

I blame the hyperscalers for the IPv4-first mindset

2026-06-01T21:17:25Z

For reasons, I’m experimenting a bit with Kubernetes again. And, since I planned at one point in 2025 to stop using IPv4 (or better said: dual stack), I wanted to set up Kubernetes in an IPv6-only mode.

Kubernetes does support it. It’s supported out of the box. And yet, if you’re not using kubeadm but instead want to take an easier approach using any of the “small” variants (k3s, k0s, etc.), it seems like no one thought about IPv6. And I would be fine with that if it’s just an issue in early versions of the toolings.

But no, it’s not. I setup a new Hetzner VM without an IPv4 address. Just to figure out what’s possible.

First attempt: k0s

Given the curl | sh workflow, I followed their setup guide and did a

curl --proto '=https' --tlsv1.2 -sSf https://get.k0s.sh | sudo sh

Guess what’s not available via IPv6? Right, get.k0s.sh.

Second attempt: k3sup

k3sup is just like k3s, but connects via SSH to the remote server. Different approach than with k0s, but hey, I’m experimenting. What I got instead?

; k3sup install --ip '2a01:4f9:c013:b1c9::1' --ssh-key ~/.ssh/id_ed25519
Running: k3sup install
2026/05/20 23:35:03 2a01:4f9:c013:b1c9::1
Public IP: 2a01:4f9:c013:b1c9::1
Error: unable to connect to 2a01:4f9:c013:b1c9::1:22 over ssh: dial tcp: address 2a01:4f9:c013:b1c9::1:22: too many colons in address

Fucking christ. But hey, we can wrap the IP address in brackets, that will fix it, right?

; k3sup install --ip '[2a01:4f9:c013:b1c9::1]' --ssh-key ~/.ssh/id_ed25519
Error: invalid argument "[2a01:4f9:c013:b1c9::1]" for "--ip" flag: failed to parse IP: "[2a01:4f9:c013:b1c9::1]"

😭

come on, pls. The issue for IPv6 support is 3,5 years old. The patch for that is a +9 -6. It’s Go. Instead of concatenating strings, just use net.JoinHostPort and voila, you have IPv6 support.

The elephant in the room: GitHub

GitHub is still IPv4-only. And I doubt it’ll ever change. But hey, at least they’re working hard to become irrelevant, one downtime at a time. I won’t rant about it anymore, at least not today.

Why I blame hyperscalers

All of the big hyperscalers are based in the US. And because they are, IPv4 addresses are still cheap for them. The rest of the world struggles with the scarcity of the 4 billion addresses, but for 'murica, it’s still the default. They don’t have to deploy “solutions” like CGNAT. They can just hand each resident their own stable IPv4 address. And so, they default to their IPv4 only networks. Classical “works for me”, but more like “It works in my country”.

And because it’s the default for them, the rest of the world has to adapt to that mindset. The rest of the world (including me), have to deal with NAT64, DNS64, 464XLAT and all that other stuff. Just because AWS, Azure and Google are just like: You get an IPv4 and you get an IPv4. Want security? here, have a 100.0.13.12/24 and also deploy our NAT gateway. NAT! You want a NAT! You could enable IPv6, but we make it scary. There are now letters in your IP? 👻 And those IP addresses, they are globally routable! Globally! Are you sure you want to discuss this with your compliance team? Better stick to NAT and a 10.0.0.0/8, right? Firewalls are for losers.

To close the circle: Kubernetes defaulting to IPv4 in most of their documentation surely isn’t helping here. I’ll write a guide for a truly native IPv6 cluster in the future. For now I just had to vent about the situation.

Game review: Split Fiction

2026-06-01T21:17:25Z

Last weekend, I finished Split Fiction with my brother. It’s another one of EA’s coop games that have to be played with a friend. I think, A Way Out, where you have to escape from a prison, is another game of them.

Aaaaaanyway, back to Split Fiction. The story is relatively simple: You’re playing a writer who wants to publish a new book. The evil capitalist publisher has different ideas and after a short moment, you’re finding yourself in different virtual realities which all were created by either yours or your partners brain. You’re in this together now. That’s the story, more or less. I won’t go into detail here, you can find that on the internet.

It might sound boring, but it’s absolutely fun. You’re constantly switching between sci-fi and fantasy, you’re fighting evil machines and a cute little kitty. The game has so many wonderful and fun moments, it’s hard to get frustrated. Even in some tougher situations, like boss fights, there are numerous checkpoints that assist the two of you to enjoy the game. It’s challenging enough to not be boring, but also forgiving enough to not be frustrating. I love games like these.

I also want to appreciate the sheer amount of different environments you’re playing in this game. They are absolutely incredible, crafted with a lot of detail and small easter eggs here and there. It’s colorful, vibrant and dark at the same time. It’s like a cut citrus dipped into sugar: contrasting tastes, yet they fit together perfectly.

And given it’s co-op and not against each other, me and my brother had a great time playing it.

There’s probably only one warning I have to give you: The final chapter is longer than we expected. If you’re tired, go to bed and continue it on the next day. Don’t repeat our mistake of hustling through it cause you think you’re close to the end, only to find yourself in a different room.

Apart from that: 4,5 out of 5 🐾 Absolutely worth every penny.

About journaling and ADHD

2026-06-01T21:17:25Z

Since a couple of ~~days~~ weeks now, I’ve been writing down a daily note every day. This started roughly about the same time as I started using Obsidian. It’s not much, it’s just a simple template that looks like this:

---
tags:
  - note
  - journal
  - review/daily
created: "<% tp.file.creation_date("YYYY-MM-DD[T]HH:mm") %>"
aliases:
- "<% tp.date.now("dddd MMMM D, YYYY") %>"
parents:
- "[[<% tp.date.now("YYYY") %>]]"
- "[[<% tp.date.now("YYYY-MM") %>]]"
- "[[<% moment(tp.file.title, "YYYY-MM-DD").locale("de-DE").format("YYYY-[W]ww") %>]]"
---

<%\*
await tp.file.move(`Journal/${tp.file.title}`);
let titleDate = moment(tp.file.title);

// # Sunday April 5, 2026
tR += '# ' + titleDate.format('dddd, DD MMMM YYYY');
%>

---

- <% tp.file.cursor() %>

In case you’re wondering, that’s the syntax for the Templater plugin, which I adopted recently to play with it. All it does is to generate some metadata and start the template with a h1 of the current day. That’s it. There are a lot of very, I’d say, overengineered templates out there, where they also track their mood, their calories and what not. I’m not interested in any of that. For me all that matters is that I wrote down what I did at what day. In bullet points, of course.

How it all started

First, I only used that to be able to properly answer the question What have you done yesterday? in the daily at work. In the past, I always tried to remember what I did and that worked most of the times, but not always. Sometimes, I was just too tired and forgot about things. Writing things down whenver I start working or have worked on a task helped me a lot to answer this question properly. So I continued.

Each one of us humans probably has another reason for journaling. I, for most of my life, haven’t written a detailed journal. Sure, I used the app “Daylio” a lot in the past but ultimately stopped using that, because it felt like a chore to me. Maybe my current system in Obsidian will turn into the same, I don’t know. Right now, I’m enjoying it a lot.

And, following the idea of the periodic notes plugin, I started writing a review of my week every Sunday. That’s also the reason why I formatted the date in the de-DE locale in the template: My week starts at Monday, not at Sunday just like they do it on the other side of the ocean. Anyway. So I started writing a weekly review. It’s a basic Markdown file, named after the year and the week, like 2026-W11.md or so. I review my week by having a look at each individual day again and summarise what I did. Of course, I keep the boring parts out, I just write down what I enjoyed. Whether it was watching a movie in the cinema, a nice compliment I got, stuff like that. I also write down what went wrong and sometimes also things that I want to do next week.

The awakening

What I realised after writing the weekly reviews and later, the first monthly review that I did for March, is: My life is full of things to be excited about! Especially in the review I did for March I remembered all the good things that happen. And I also realised: heck, I was doing a lot, no wonder I feel so stressed all the time! The lack of proper time perception, one of the things that this ADHD thingy gave to me, makes me forget certain things and overfocusing on others. I forgot that I was in the cinema in March until I reread my notes again. In my mind, it felt like it was ages ago, when in fact, it was only a mere two weeks in the past. Admittely, I was surprised about the fact of how I easy I can forget things. I mean, it’s one of the most common symptoms of ADHD, it’s just that I hadn’t struggled with it in the past so much. The reason for that, unfortunately, was not that I’m some kind of super brain, no, I was living with constant anxiety. Gonna tell you that, it sucks. It sucks a lot. And it took some years of therapy to get rid of all those fears, of all this inner anxiety, that I might disappoint someone. And now I’ve got rid of the fears, I just can admit that I forget things. Is this great? nope, not at all. But it’s part of who I am and I’ve come to peace with that.

If you have ADHD or you think you might have it, you might see yourself in that paragraph. And everyone of us works different. But, maybe you aren’t writing a journal right now and maybe you think “yeah, this is a new hyperfocus I should jump into!”. And in that case, I can only recommend doing so. And remember, it doesn’t have to be perfect, it doesn’t even have to be complete. It just needs to work for you, only for you.

The questions I’m asking myself every end of the week now are:

What is my highlight of this week?
What did I enjoy? (books, memes, a silly cat, whatever)
What did I learn?

I’m regularly surprising myself each week about what I did in the past seven days. Maybe, the same thing will happen for you, dear reader.

cheers.

Impressions of the the KubeCon 2026 Europe

2026-06-01T21:17:25Z

Last week, I attended my first KubeCon, or as it’s officially named, the KubeCon + CloudNativeCon Europe 2026 in Amsterdam. In fact, this was my first conference in the tech space that’s not affiliated to the Chaos Computer Club in any way.

I wanted to visit Amsterdam for a while now, because their bicycle infrastructure is known to be ahead of everyone else. I wanted to experience this on my own. Originally, my plan was to ride a bicycle on my own while being there, but due to the KubeCon being tightly packed and full of interesting discussions with my colleagues, I didn’t had the chance to do it. Maybe another day. ^^

My impressions of the conference

Anyway, back to the KubeCon. To summarise it: It was awesome. However, that’s not really related to the event itself, but rather the occassional chit-chat with colleagues and strangers alike. It’s somehow refreshing to see a lot of humans, all trying to have a nice time. Of course, with the KubeCon being a really expensive and corporate event, there were a lot of sponsors there, all of them trying to sell you their stuff. And, because it’s 2026, the sheer amount of them advertised something related to “AI”. I wonder if they realised how silly their stuff sounded like. Seriously, why does someone want to interact with their infrastructure using a chatbot? I don’t get it. But I guess, I’m not the target audience, so I ignored all talks and booths which mentioned “AI”.

The remaining talks I attended felt somewhat flat, none of those sparked the feeling of experimenting with the tech itself. They were all pretty boring.

Also, the food: Apparently, they used pre-packaged sandwiches last year. This year, we got better options: warm meals, with a lot of variation to choose from. There’s just one complaint: The portions were rather small. I guess the benefit of that is that visitors do not get tired because they ate too much. 😬

Discussing things, whether work-related or not, with my colleagues and everyone I met there, was the best thing of the conference. I learned a lot about the company-internal lore, which helped me tremendously understanding why things are the way they are. I also learned things about the cluster-api, which I really want to try out soon.

Impressions of Amsterdam

The city is so. damn. beautiful!. Most of the times when I was outside, the sun was shining. There are bikes everywhere. And I can understand why. The bicycle infrastructure is really something else. At no point I felt unsafe, either as a pedestrian and I can imagine that cyclists feel the same. The paths are wide, with lots of room for anyone to travel safe. The narrow alleys, the wide streets, the amount of pedestrians — it won’t be my last time attending this place. Next time I’ll be here, I wanna ride a bicycle and drink a fresh cup of coffee on one of the boats.

With that being said, here’s a photo I took while being there:

Project Hail Mary

2026-06-01T21:17:25Z

Last Sunday, I watched Project Hail Mary in the local cinema. I’ve read the book before, which I enjoyed as well! At first, I was a bit sceptical about the movie. I really hoped that they didn’t turn it into a Marvel Cinematic Universe (MCU)-like movie, full of self-ironic comments, action scenes with a lot of cuts, you know, just another big Hollywood movie. The book was surprisingly calm and it could transport the fears that Ryland Grace, the protagonist, must have felt. The sheer emptiness of space.

I’m glad to see that they succeeded with the movie. I re-experienced the same feelings I had when reading the book. The fascination, the sad and the cheering moments, they’re all in the movie. The script is really, really close to the book.

The only thing that was a bit frustrating, is that they squeezed the last 15 minutes or so. There’s some story that could’ve been told, but was likely cut to not bore the audience. I really hope that there will be an extended cut in the future, even just half an hour could do a lot. But that’s the only thing I have to complain about. Besides that: 5/5 stars, I can only recommend it!

now with human.json!

2026-06-01T21:17:25Z

Originally found over at the blog of Seth Larson, the human.json thingy is a nice concept I adore. You just add a lil human.json to your webpage, <link> to it and 🎉, you are hooman now!

And now I have one too! Right now, without any vouchers. Have to think about what it means to vouch for someone. Until then, I keep it empty.

Building a pollen overview in HomeAssistant

2026-06-01T21:17:25Z

In this flat, we have an old Android tablet serving as the dashboard. Given it’s spring again, nature is getting horny again and as a result, there are more pollen outside. For someone with an allergy (hi, it’s me), this is pretty annoying.

Therefore, I wanted to have a list of pollen and their current prevalence in the air, so that I can my assess my need for antiallergens.

To do this, I ended up with the following Jinja2 template inside a Markdown card:

{% set pollen = expand(integration_entities('dwd_pollenflug')) | sort(attribute='state', reverse=True) | list %}

{%- for state in pollen if float(state.state) > 0 %}
- {{ state.name }}: {{ state_attr(state.entity_id, 'state_today_desc') }}
{%- endfor %}

This is using the DWD Pollenflug integration, which I installed via HACS previously. Given that I prefer text much over images and colors to indicate something, I chose the Markdown card. The integration also provides some visual examples, if you’re more into that.

The list is sorted, which, after renaming the entities, ended up in the following result:

First impressions of Obsidian

2026-06-01T21:28:32Z

Two days ago, I noticed that I mounted the wrong folder for my Outline database deployment. I originally set it up because I wanted something to document this flat, similar to what Luke Hsiao described in his blog. It was a pretty minor mistake. Instead of mounting /var/lib/postgresql I mounted /var/lib/postgres, resulting in a loss of the whole database.

Synctrain + Obsidian works pretty nice, even on an iPhone

I haven’t setup backups either, as this was a thing I had on my bucket list for this weekend. Honestly, this was and still is pretty frustrating. I wrote a lot of documentation and now it’s lost. As a result, I was experimenting with other tools again, notably Obsidian. Although it’s proprietary, it has a massive user base. And, together with Synctrain, an iOS app for Syncthing, it allows me to sync notes from my iPhone to the desktop as well. The mobile editing experience of Outline was subpar, so that actually was a pretty nice upgrade.

Importing existing notes

After setting up the sync and making sure it’s properly backed up this time on the remote host, I started to copy some notes I had locally on my iPhone in Apple Notes over to Obsidian. There’s an importer for that as well, but it only works on macOS and I have such no device at hand, so manual copy & paste was the way to go. After that was done, I had a quick glance at my ~/Documents folder and copied existing plain text notes into the Obsidian folder as well. Not because it was necessary, but rather out of excitement I think. I have this new note organisation tool, it only makes sense if I copy all my existing notes into it.

On Obsidian plugins

There’s a pretty huge ecosystem around Obsidian and a lot of plugins people have written for it. So far, I haven’t used any of them. I’m “limited” to the built-in core plugins right now and even disabled a good amount of them, like the Canvas plugin. I don’t have a need to use that right now and since it somehow clutters the view, I disabled it.

Things I like

The plugin system is probably one of the core strengths of Obsidian. I like what they’re doing with e.g. the “Daily note” plugin, which allows me to write a small note every day. Nicole recommended to write an engineering notebook, which is a thing I haven’t done so far. Dealing with a lot of simultaneously things at work really made me consider writing one, however. I think I’ll stick to a digital form for my first experiments, might switch to a paper-based one later. I also like the editor experience and the ability to customize all the shortcuts. I use the keyboard pretty extensively and therefore it’s always nice to have everything at your hands.

Things I dislike

Notes are missing some metadata by default, like the creation date. I wish that those would be tracked automatically, e.g. from the file metadata. So far, I haven’t found a way to display that. On the one hand, that’s good, because documentation and notes are things that evolve all the time. On the other hand, it’s metadata that I’d like to see sometimes to actually know when I wrote something down. For now, I backfilled the date manually. I have to figure out whether that’s something that’ll block me in the future from using Obsidian.

Next up

So far, that’s it. This blog post is also written in Obsidian and I’m wondering right now, whether I want – and to which extent – hook this up to my 11ty-based setup. Writing blog posts in Obsidian sounds like a thing I want to consider. Maybe I’ll begin to use Obsidian as some kind of CMS for my homepage, but I’m not so sure about that yet. Maybe I’ll also ditch it completely and replace it with a DokuWiki-like solution.

Experimenting a bit with coding fonts

2026-06-01T21:17:25Z

Recently, I played a bit more with the fonts I use for my programming tasks. There’s not a strong reason to do so, I just like to change the appeareance of my code from time to time. Also, using patterns change and that also causes new fonts to be explored. For example, I used Iosevka especially in a time where I was reading logs a lot of the day and the way it’s condensed made it really good for this task!

When having a look at the history of the fonts, I used, there are actually a lot. Like, I used Fira Code for a long time. Then, after a while, I switched to Microsoft’s new font, Cascadia Code immediately when it came out. For a while, I even used the much “beloved” Courier New, just as an experiment.

The font I used for the last year was Lilex, which is based on IBM Plex Mono. I guess that’s the beauty of open source licenses, you can just take something you like and adapt it even more to your own taste. And it’s a good font! I really liked it!

Today, however, I had the inner desire to switch to a more cozy font. A bit more rounded, a bit softer, yeah, I wanted the vibe of the font to be like a sunny day in spring. Imagine drinking a hot cup of coffee (or tea) in a small café in April. The sun is shining, the sky is blue and from time to time, there’s a breeze of wind going through your hair. That’s what I want my font to be right now.

And that’s why I’m choosing Maple Mono for the next adventure. I installed it today on my work device and now, when I’m writing this blog post, I’m also using it on my private device as well. And I really love the effort that went into this. It doesn’t feel too rounded (which is a problem I had with Comic Mono). It’s also not too wide, which I worried about initial. It just feels right. I really appreciate the effort that went into this.

Now, let’s see how long I’ll stick to this font. I just changed it to be the default font for this blog as well and I really like how it turned out!

Turns out, resident IPv6 addresses appear to be stable

2026-06-01T21:17:25Z

Recently, I migrated the <queer.group> Mastodon server to a new server. Primary reason for that was a lot of cruft that was ín the old Debian installation. Another reason to switch was to migrate to a distro that has most, if not all of the dependencies in its package repositories. Long story short, the distro I chose is Arch Linux, which even has a mastodon package in the Arch User Repository, which I was able to use as reference.

Because I just wanted to experiment a bit before exposing the server to the public, I was setting up a firewall upfront. Luckily, Hetzner makes this quite easy and because the server was IPv6-only at the time of setup, I just copied my public IPv6 address into the allowed incoming IPs and disallowed everything else.

I expected this to break after a couple of hours. It’s quite common here in Germany for IPv4 addresses to be rotated daily, so I expected that my IPv6 address would be changing as well. Well, maybe not quite as often, but I at least expected the last 64 bits to be changed on reboots and such. Turns out, they don’t. I was quite surprised by this! After mentioning said observation in the fediverse, I got a reply that provides more information about the stability of the IPv6 address, mentioning three changes within 10 years. That, of course, only applies if you’re sticking with the same ISP for the whole duration and do not replace your device.

I have to admit, I’m quite pleased by this. I have to conduct some more experiments, but if it remains stable, it means I might just point hardcode individual IPv6 addresses in, e.g. my Prometheus configuration instead of sticking to a solution like Tailscale.

IPv6-only servers have another net benefit. In my experience, it’s sufficient enough to keep most of the malicious scrapers out.

I am so sick of the slop.

2026-06-01T21:17:25Z

I am so tired of this bubble. It’s been, ugh, seven years or so. Yes, the “AI” bubble. I am so tired of the endless justifications, the endless repetitions of “in six months, we will see [insert unrealistic expectation here]”.

And I am one of the lucky ones. I do not have to work with it. No manager is forcing this bullshit down my throat. At least, they don’t threaten me with it.

But I swear to god, the two occurrences of “Yesterday, I chatted a bit with Gemini and this is what it came up with, what do you think?” were more than enough. In both cases the output was mediocre at best. Even though I try to be polite, sometimes it’s hard.

Two examples where it got worse

And even if I don’t interact with one of the slop machines directly, I do notice the degradation in the software I use (or better: have to use) every day. The Bitwarden client extension randomly answers with “wrong master password”, because it’s trying to decode the wrong vault. They have a multi-user functionality, apparently no one is testing it. And it worked in the past.

Speaking of developer toolings: GitHub got massively worse. Just yesterday, the scrolling position was randomly reset when I was looking at a bigger patch on my phone. It used to be a good website, from a technical perspetive. But that used to be.

Consequence: Lack of trust

I could open a bug report, but honestly, I don’t want to anymore. Either it’s one of the issues that’s closed by fucking stalebot again or the half-baked solution doesn’t fix it either. I lost my trust in such processes and only do I have some trust into the maintainers to actually have a look at the problem. That’s what this bubble did to me. I hate talking to a chatbot, I also don’t want that some slop machine just looks at my issue and interprets it in its own ways.

You must be kidding me, right?

Slop at work is one thing. But slop by credible artists, that’s a whole different thing. I just wanted to watch your concert, Jean-Michel. But you said something along the lines of “We shouldn’t be afraid of technology” and blabber something about how great it is for artists to express themselves (Minute 55:50 ff. of this concert on ARTE). Go fuck yourself. Seriously, I hope you realise about the damage you’ve done with that.

Further notes

If you understand German, there’s a good video by Ultralativ about the feeling of betrayal if you receive slop and how the sender more or less told you that they do not respect you at all as a person.

Did I mention that I’m tired? Cause I am. But writing this all down helped me to let the frustration go away. At this point, I just hope that the bubble bursts rather sooner than later.

Using OAuth as a Git credential helper

2026-06-01T21:17:25Z

When I was hacking on Forgejo in the hospital recently, I was blocked by their firewall which blocked outgoing access via SSH. Sure, I could’ve started my VPN and try to circumvent their silly policies, but that’s not the first thing that came to my mind. The first thing, in fact, was to use the https URLs for pushing to Git.

Usually when using HTTP endpoints, you’re either using your username/password combination, or like in my case with two-factor authentication being enabled, a personal access token. So, I went to the “Applications” settings on Codeberg and… found old access tokens. Never used, but they’ve been lurking there for two years now.

And I hate that. I hate that there are theoretically tokens that I just forgot about. The idea of short-lived credentials, as in “tokens that expire after a couple of hours” is, at least in my opinion, better suited for such things.

Luckily, with the help of the hickford/git-credential-oauth project on GitHub, pushing to a remote repository now just opens the browser. Most of the time I’m already authenticated there, so the operation almost instantly succeeds. I really like this approach! And although I think that SSH is still beneficial in most cases, I’m thinking about dropping it entirely for most repositories and stick to HTTPS + OAuth instead. After all, how often do you rotate your SSH keys? If you’re like me, maybe once every five years or so. 🙈

One little adjustment in the Git configuration later, and I’m good to go. Thanks a lot for this tooling, M Hickford ❤️

[credential]
	helper = cache --timeout 43200	# 12 hours
	helper = oauth

(Also, while thinking about it, I should separate the SSH keys I use for signing and authentication.)

IPv6 is still a pain in the ass

2026-06-01T21:17:25Z

When I was setting up a new server in an IPv6-only fashion, I was like:

Yeah, I know that there are certain things that won’t work out of the box cough GitHub cough, but overall, it should be fine, right? Right?

And oh boy, I was not prepared for that journey.

Let’s start with the environment. It’s 2025, and I am playing around with Fedora 43 (aarch64) on a Hetzner VM. So far, so basic. The operating system is not an outlier, the cloud provider is known and ARM architectures are so common nowadays that it’s only a mere sidefact.

In fact, when setting up the system, it was much more likely that I found aarch64 binaries on IPv4-only hosts than only x86 binaries on dualstack/IPv6-only hosts. And that, excuse me, fucking sucks.

IPv6, checks notes, gets 30 years old in two months. Thirty years. And yet, it’s treated as some outlier, because some ~~dickheads~~ nice folks at big cloud providers are like: oh nu, please no hexadecimal numbers 🥺 only AI! buy more AI, so we can fire you 🥺🥺

Himmelherrgottnochmal, it’s not that hard. Come on, use that “agent” and let it explain IPv6 to you. Vibe engineer your network infrastructure if you want to, but please, implement IPv6 now.

(jasmin, come on, calm down.)

So, let me tell you a story. To manage the installation of several tools in use, I am using mise at the moment. And it has a nice feature: If cosign or slsa-verifier are installed, it can check the downloaded binaries against that (if they use either one of them). Which works pretty nice, unless, you’ve guessed it, you don’t have an IPv4 address.

That’s right, sigstore.dev, the thing that should help with security, the thing that claims “Making sure your software is what it claims to be.”, the thing that is supported by no one less than GitHub, Hewlett Packard, Cisco, Google and Red Hat, yea, that thing does not support IPv6.

$ dig AAAA sigstore.dev
; <<>> DiG 9.18.41 <<>> AAAA sigstore.dev
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4442
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;sigstore.dev.			IN	AAAA

;; AUTHORITY SECTION:
sigstore.dev.		300	IN	SOA	ns-cloud-a1.googledomains.com. cloud-dns-hostmaster.google.com. 1 21600 3600 259200 300

;; Query time: 69 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Sun Nov 02 00:12:45 CET 2025
;; MSG SIZE  rcvd: 134

This is a joke, right? This must be. I mean, come on. If you are launching a new product (and it says Copyright 2023 in the footer) and you’re not incorporating IPv6 right from the beginning, are you even trying? I mean, what is this? Are citizens of the so called United States of America all gifted a whole IPv4 subnet on their birth or why are they like this?

I am not angry. I am just frustrated. I don’t understand it. I wish I could, but… I can’t. Yes, IPv6 is different from IPv4. There’s a lot of stuff that’s different. There’s a lot of stuff that’s not possible with IPv4, like SLAAC and that might be scary first. But please, for the love of your business partners and users alike, please implement/rollout IPv6 now, if you haven’t done it yet.

Thank you.

Update 2025-11-02: Reply from the sigstore.dev team

Complaining in the internet about stuff is one thing, but it doesn’t help. Which is why I reached out to the sigstore folks via their mailing list to understand what’s going on there. The reply left me even more speechless.

Our main website host (used for our homepage and docs) unfortunately does not support IPv6 unless we use their DNS provider, which we’ve chosen not to do due to a lack of functionality. As far as our various endpoints used as part of the signing or verification process - it’s on our roadmap to support IPv6, but we don’t have a definitive date set yet. Bob Callaway (via Google Groups)

According to this random hosting checker website I found, the page is hosted on AWS. It’s wild that they are trying to make a business out of IPv6. capitalism is wild.

This homepage is now IPv6-only

2026-06-01T21:17:25Z

If you can read this in your feed reader, congratulations.

Why?

I migrated this homepage to another server. And because I don’t think that it’s worth paying for IPv4, I decided to just get rid of it. After all, it’s 2025, literally anything should(!) support IPv6 now.

And if it doesn’t, it’s broken.

Calibrating a PS5 controller on Linux

2026-06-01T21:17:25Z

Recently, I bought a used PS5 to play some video games. Because it was used and not new, the controller had this pretty common problem called “stick drift”. While there are possibilities to fix this in the long term (thanks to all the awesome ifixit folks who write guides for this!), I wanted to know whether there’s a short-term solution.

And to my surprise, there is! Someone built a DualShock Calibration GUI, which uses WebUSB to recalibrate the controller. It was worth a try. So, after searching for a proper USB cable, I plugged it into the computer, opened the page in a Chromium browser (because Firefox doesn’t support the WebUSB APIs) and:

Failed to open the device.

Adding missing permissions via udev

sighs I love it when things would just work. But eh, because my keyboard, the ZSA Moonlander, needed some adjustments as well, so that I can use their online flashing tool, I already knew where to look! The software I had to configure is udev. When your system also uses systemd (and almost all of them do that for a good reason), this likely applies to you as well.

udev itself is configured using rules. Those rules can do a lot of stuff, but in this case, all I needed to do was to change the permissions. By default, devices are only accessible to the root user and I certainly don’t wanna run my browser as root, hell no.

There’s a lot one can do with udev and to be honest, I haven’t looked into all the things that are doable with that. All I wanted was to recalibrate my controller. Similar to the udev rule for my Moonlander, I tried to create a new one for my controller in /etc/udev/rules.d/60-playstation.rules. This was the one I ended up with:

# Vendor 054c is Sony. This rule is for the PlayStation controllers.
KERNEL=="hidraw*", ATTRS{idVendor}=="054c", MODE="0660", TAG+="uaccess"

Update 2025-10-29: After I read more abot udev, I stumbled upon the following information in the Arch Wiki:

The OWNER, GROUP, and MODE udev values can be used to provide access, though one encounters the issue of how to make a device usable to all users without an overly permissive mode. Ubuntu’s approach is to create a plugdev group that devices are added to, but this practice is not only discouraged by the systemd developers, [2] but considered a bug when shipped in udev rules on Arch (FS#35602).

The above udev rule was therefore adjusted to reflect this change.

Sidenote: How to figure out the USB vendor

I figured out the vendor ID via lsusb. lsusb outputs a list like this:

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
...

The first part of the ID is the vendor. In case of the USB hub above, it’s 1d6b.

Results

After saving the rule above, I rebooted the computer. If I remember this right, this must be done in order for the rules to take effect. The above rule adds write permissions to ~~the plugdev group~~ currently logged in users. ~~My user was already part of it, as I’ve created the plugdev group earlier.~~

Anyway, after the reboot, I tried my luck again and: it worked! I was able to recalibrate my controller! With the recalibration in place, I was able to continue playing Humanity (can fully recommend it!!) without getting annoyed by the drift. Nonetheless, I hope that Sony releases a new version of the PS5 controller with a replaceable battery soon, which would allow me to buy a new one and mod it with tunnel magnetoresistance joysticks, like the Guilkit joysticks in the iFixit store. But that’s another thing to look at in the future.

Update 2025-10-23

Because the stick drift is a mechanical problem, the above solution is only a partial fix. Sometimes, after rebooting the controller, the drift appears again. I find that rather annoying, even though I expected it. welp. Guess, my plan to replace the joysticks has to happen soon.

System administration contradicts perfectionism

2026-06-01T21:17:25Z

Or perfectionism contradicts system administration. I am not sure. However, after my last experiments with system administration (sidenote: I’m experimenting with Alpine right now), I think I had to make the realisation that no matter how hard you’re trying, the “perfect” system does not exist. As with everything in life, it’s all a matter of compromises.

However, it’s up to you to decide on the compromises that are made. Is it important that the whole state of the system is persisted in some kind of Git repository, fully reproducible? Then NixOS might be for you. However, if you’re not a fan of Nix itself, then maybe use something like rpm-ostree. Other compromises have to be made for that.

I aim for low maintenance. And for this, I have to make the compromise that I might not have the whole state described in some repository. Ansible, after all, also has it’s limitations. But that’s okay. I came to my mind with that. The rant from earlier this year still applies to a certain extent and I’m still convinced that there’s an easier (more manageable) solution than Ansible out there.

The philosophical(?) perspective

Maybe the reason, why we (as in system administratiors and software engineers) strive so much for perfectionism, is because our whole world is just virtual. We do not resolder our mainboards every two years, we don’t reinvent bridges or tables or everything else out there every year or so. That stuff just works. If it doesn’t, it can be easily replaced/fixed. And even for big things (thinking of a skyscraper), you do the engineering once and then you’re done. No one expects you to rebuild the whole thing from wood or bamboo, because it’s suddenly cheaper than concrete. In our world, it’s however (more or less) required to do so.

Maybe the tech industry is also very very bad at teaching. I mean, we have how many tools for managing servers? I could easily name five. Likely even more. Have I used any of them? Nah. I heard of them and know that they’re the “old” way to manage servers (thinking specificially about Puppet and Chef here) and that all most of the folks out there use Ansible now. If you have the need to get everything as a managed cloud service, you might even manage everything inside OpenTofu files. It’s just a very, very expensive system then.

Another example: Recently I read the PhD study of Joe Armstrong (2003), one of the co-inventors of the Erlang programming language. It’s titled “Making reliable distributed systems in the presence of software errors” and reading it was such a delightful experience. Erlang, if you aren’t familiar with the programming language, was designed for scalability in… 1989. And if you look at the paper, closely at page 27, the expectations of stability were “defined” as:

Typically having less than two hours of down-time in 40 years

I have to say, I have enormous respect for Mr Armstrong and other folks from that era. The sheer computation limitations led to some extremely interesting ideas and solutions. They didn’t have to invent Kubernetes to run a mobile network. Computation power was expensive then and so, different compromises had to be made.

Not saying that everything was better then, but the tech industry really seemed to have taken more care back then, when computers we’re not that powerful and cheap as they are nowadays. What I’m trying to say? I don’t know either. I guess, it might be a good time to learn some old stuff again. To learn from and more about the early beginnings of the tech industry.

Choosing an jujutsu revision interactively using fish and fzf

2026-06-01T21:17:25Z

During optimizations of my fish configuration, I wanted to make it easier for me to choose revisions in jujutsu. They are the core component of my workflow, but choosing them, even with the already provided dynamic completions was sometimes rather cumbersome.

The existing workflow required me to open the log and to remember the change ID I was going to work with. This “worked”, but it was annoying me and especially late in the day, it simply was too error prone.

As a consequence, I checked the fzf section in the jujutsu wiki and at the end of the wiki, there’s a link to a little gist that allows for interactive choosing of revisions. It was not quite sufficient for me at the time of writing, for example, it only allowed to be executed in the root of the repository.

After some smaller adjustments, this is what I came up with:

function fzf_jj_choose --description "Interactive choosing of a jj revision using fzf"
    # Source for the original idea: https://gist.github.com/tdaron/f5d0985687d8aed06714c8901dfb5fcb
    set -l template '
        change_id.shortest()
        ++ "\t" ++ description.first_line()
        ++ " " ++ bookmarks.join(" ")
        ++ "\n"
    '

    # Check whether we're inside a jj repo
    jj root --quiet &>/dev/null; or return

    set -l name (jj log --no-graph -T $template --color always | fzf --ansi | cut -f1)
    commandline -it "$name"
end

This function is placed in .config/fish/functions/fzf_jj_choose.fish and needs a new keybinding, which is added to the fish config. For me, it’s in the .config/fish/conf.d/jj.fish.

# Bind Ctrl+J to interactively select a revision using fzf.
bind ctrl-j fzf_jj_choose

And with that binding in place, I can just press Ctrl+J at any given time and select a jujutsu revision ID with a breeze. \o/

Again, big thanks to Théo for the original idea!

Smaller feed adjustments have been made

2026-06-01T21:17:25Z

If there were a lot of older posts appearing in your feed reader today, sorry!

It’s fixed now, the post identifiers are now stable, thanks to the helpful ideas of Bob, so it won’t happen again (famous last words lol).

ZFS dataset decryption using the TPM with the help of systemd

2026-06-01T21:17:25Z

On my/our homeserver, the pretty awesome ZFS filesystem is used for the data. Because this is the first experiment with ZFS, it’s not the whole pool that is encrypted, just individual datasets. This is especially important for information like the documents, that are are stored in paperless-ngx.

Because those were my first experiments, the ZFS encryption was made via passphrases instead of specific files on the disk. This, however, had one big caveat: On each reboot, those keys had to be entered again using zfs load-key. It is possible to use raw keys that are stored on the disk instead of using passphrases.

But, there’s one problem with the current setup. The disk where those keys would be stored on, is unencrypted at the moment. Yes, yes, consider your threat model yadda yadda, I know. It wouldn’t be much of a problem to just leave it there, but it doesn’t feel right. I considered using a YubiKey or setup some webserver for entering those keys, like Filippo Valsorda did it with his NAS frood (really can recommend the article, I like his approach a lot!).

The next idea was the TPM and I remembered that I used this already to setup automatic decryption on all my other devices with systemd-cryptenroll. But… that’s just for LUKS volumes, weh. Yet, systemd can leverage the TPM, so I continued searching and found the page about systemd credentials and therefore found out about the systemd-creds tool!

That’s exactly what I need! Credentials aka the dataset passphrases, encrypted and decrypted via the TPM, making access to the disk worthless without running it inside the computer that it’s desired to run in. Awesome!

systemd to the rescue!

ZFS mounts can be generated via the zfs-mount-generator, which needs a bit of setup upfront. After that, we can make use of the zfs-load-key@.service, which itself is automatically loaded for all ZFS datasets that are encrypted aka zfs get encryption <dataset> doesn’t return off.

The templating of units becomes really useful here to share a common behaviour: For example, the dataset tank/documents would automatically depend on zfs-load-key@tank-documents.service. But since I planned to use the same approach for decryption of all my datasets, I can just override the default behaviour for all invocations of zfs-load-key@.service.

Long story short, after a lot of tinkering, I came up with the following systemd dropin:

# /etc/systemd/system/zfs-load-key@.service.d/import-key-via-systemd-creds.conf
#
# This overrides the default loading mechanism for ZFS dataset keys and allows them
# to be placed on disk.
[Unit]

# Required to be set, otherwise the encrypted keys on disk are not available on boot yet.
RequiresMountsFor=/etc/credstore.encrypted/

[Service]
# Loads a credential with the same name as the dataset mount.
LoadCredentialEncrypted=%i
# Remove the default implementation…
ExecStart=
# … and replace it with systemd-creds.
ExecStart=/bin/sh -euc 'systemd-creds cat %i | zfs load-key $(systemd-escape -u %i)'

The above snippet loads the credential from the credential store, e.g. /etc/credstore.encrypted/tank-documents, outputs it (since it’s a passphrase) via systemd-creds and passes this to zfs load-key. The systemd-escape in there is required to transform the escaped value tank-documents into a proper path tank/documents, which is the format required by the zfs CLI.

Example usage

In order to use it, the key has to be persisted in the credentials store. In bash, this would be achieveable like this:

# Note: all of the commands below assume that you're root.
export DATASET="tank/documents"

# First, create the dataset. `zfs` is going to ask for the passphrase.
zfs create -o encryption=on -o keyformat=passphrase $DATASET

# Next, we store the passphrase in the local credential store, encrypted only(!) via the TPM.
systemd-ask-password "Encryption phrase for $DATASET" | systemd-creds encrypt --with-key=tpm2 - "/etc/credstore.encrypted/$(systemd-escape -p $DATASET)"

The important line is the invocation of the systemd-creds command, which will read the passphrase from stdin and store it in the /etc/credstore.encrypted folder.

Important note

The files can only(!) be decrypted on the same host. A backup of the key is useless, you have to store the passphrase somewhere else, if you want to recover the data in the future on a different host.

Summary

And that’s it! With that I was able to just use the existing passphrases and store them in the TPM. On reboots, the datasets are now automatically decrypted and I no longer need to enter the passphrases manually.

And yes, the above ideas can be adjusted to use encrypted raw keys as well. Instead of systemd-creds cat it’ll likely be something along the lines of just using zfs load-key together with the $CREDENTIALS_DIRECTORY environment variable.

Sieve filter to auto-sort mails from Git forges

2026-06-01T21:17:25Z

Because I’m interacting with a lot of open source projects online (both at work and in private), I also get a lot of mails being sent by GitHub. And because I’m a member of the Codeberg e.V. as well, where a lot of discussion is happening via issues on private projects, my mailbox became cluttered over time.

Filtering out mails from Dependabot & Renovate already helped with the noise, yet I was overwhelmed by it a lot of the times. Luckily, my mail provider, Migadu (which I can only recommend!), supports Sieve scripting.

I won’t go into detail here how to use them and what’s supported, but with the following Sieve script, I’m able to automatically sort all my incoming mails into proper folders.

require "variables";
require "fileinto";

if header :matches "List-ID" "*/* <*.github.com>"
{
    fileinto "Code/${1} | ${2}";
} elsif header :matches ["X-Forgejo-Repository-Path", "X-Gitea-Repository-Path", "X-GitLab-Project-Path"] "*/*"
{
    fileinto "Code/${1} | ${2}";
}

The folder pattern is {org} | {project}. An earlier version combined them into both, but this was rather tedious because of all the unnecessary folders in between. The default separator is / in URLs and it’s also in mailboxes, which means that for each organisation a folder was created, with a project folder below that.

That was pretty annoying to me, so I switched that to the vertical bar.

From time to time I’m also using this script to clean up empty IMAP folders, which does the job astonishingly good.

And yea, that’s it! Now I have all my Git-related mails cleanly organised in subfolders below Code/ :3

hamsterradeln

2026-06-01T21:17:25Z

hamsterradeln, Verb.

Bedeutung

Ausdruck der Monotonie des Alltags und das Auslassen des Frusts darüber.

Herkunft

Wortneuschöpfung aus dem Substanstiv Hamsterrad und dem Verb radeln.

Beispiel

Morgen wieder hamsterradeln, gar keine Lust.
Da sie morgen hamsterradeln wird, ist sie heute kraftlos.

System administration of Linux servers still sucks

2026-06-01T21:17:25Z

When playing with the homelab (and my other servers that I’m administering), I realize again and again that system administration done right ™ still sucks. What I mean by doing right? Properly documented.

It’s fairly easy to get a system running and do whatever I want to do with that. Download the ISO/qcow2, boot it, login, and do anything. However, over time, the system begins to become dirty. Packages, only needed for a little experiment, configuration files that are no longer needed, users and heck, even state in /var/lib is going to clutter the system.

And as the server gets older (installation-wise), that’s inevitable.

Ansible

But: There are solutions to work around those. Ansible, the most common known tool (I think) for configuring servers allows for a relatively easy configuration using so called playbooks. A playbook here, a playbook there, and suddenly, the whole server is set up.

Eeeeexcept, it still doesn’t avoid that you just login and do stuff outside of Ansible. And so the problem remains. Even at work, where we’re having a fully-fledged architecture full of roles, playbooks, even Ansible AWX hosted by another team, this was inevitable. Of course, you can always blame the persons maintaining the roles and the playbooks of not using them more. But I think that’s a bit too easy. Ansible, although it tries to reduce the friction a lot, is still friction. Maybe not so much for installing arbitrary packages, but as soon as it involves a bit of scripting, it’s becoming cumbersome. You have to understand the Jinja2 templating language, you have to make sure that the playbook is doing what it should do and all that. Not to mention that Ansible is a software that you need to update and oh boy, they’re releasing quite a lot of major releases.

Does the content of the playbook change? Not at all.

Yet, now you’re no longer maintaining just a server, but the setup infrastructure as well. This might be a valid compromise if you’re maintaining a fleet of servers and you have to make sure that everything is running as intended or the company loses money, no doubt. But for me, who’s just maintaining a handful of servers which are completely independent from each other? It’s not worth it.

Also, it’s getting fun if you have to delete stuff again. Just deleting the stuff from the playbook won’t do it, you have to actively invert the action. Usually it’s “just” rewriting legacy stuff to state: absent, but there might be more involved.

NixOS

Let’s have a look at the second contender in the room: NixOS. Based on their homepage, Nix

[…] is a tool that takes a unique approach to package management and system configuration.

And that certainly is true. NixOS is built on the Nix language. And although you can do a lot with it, it’s also another thing to learn. Not that I’m disliking the fact that I have to learn a new thing, nope, not at all. But the last time, I looked at it, it didn’t allow me to learn the way I intended to. It might be a fact of the changing times in the Nix community, but when I searched for Nix tutorials back then, I found more problems that Nix users are having than actual solutions.

The videos on the homepage are great and they make it look soo easy to understand what’s going on. On the other hand is the actual maintenance work. And based on my observation from the folks in the Fediverse that I follow, maintaining NixOS-based systems is not that easy and requires a lot of processing power. And the fact that there’s a major release every six months also drove me away from even trying it. I use(d) Nix for development environments, but even there, it more and more became a maintenance burden. Not to mention that telling “just install Nix” to my coworkers is more an insult than actually helpful.

I have respect for everyone who’s using Nix to manage their systems. It’s just not for me and that’s okay.

Kubernetes

One might argue that Kubernetes is not a configuration management tool, but to be honest, it kinda is. You “just” need a system to run it on. And that, well, must be configured. (Not to mention the whole complexity of control planes, etcd backups, whatever.)

It’s not a lack of knowledge for me, I use Kubernetes at work the whole time. Do I want to have it on my own systems? Not really. It’s good, definitely has it’s usecases, but is much more maintenance than I want to have in my homelab.

OSTree

OSTree is a rather “old” technology out there. And somehow, it also never took off. Why? I don’t know. Maybe there’s too much setup needed upfront, maybe it’s too complicated. Or maybe, people are really angry at Red Hat for some reason, I don’t know.

Yet, it’s one of the things where I appreciate the ideas behind it. It certainly is useful I think and the fact that it’s used for the immutable Fedora distributions also kinda undermines that it’s useful in production.

But: The last time I used it (Fedora 39 & Fedora 40) (see also the blog post back then it was really really slow. Not sure whether that was due to the LUKS encryption on there or because my old ThinkPad was seeing it’s age. (the heck, it was from 2017, it’s not that old). It was noticeable. I’m writing this words on a ThinkPad running Fedora Workstation (the regular, old-fashioned way of running Fedora) and the dnf updates are never slow. They are usually done pretty quickly. And also done in one transaction at startup, with a Windows-like “Updating your system, please wait” splashscreen. It’s not perfect, but it’s good enough.

Because of the reasons above, I’ve ditched OSTree. Until I came back to it recently, because of bootable containers.

Bootable containers

If you haven’t been living under a rock, you have heard of the terms “container” before. Although it’s just a number of tarballs together with some metadata combined, it changed the way of sysadmins, whether they like it or not.

And now, Red Hat, likely due to it’s work on it’s Kubernetes variant called OpenShift, wants to bring this to operating systems as well. It’s not new magic, it’s basically OSTree, but with containers as the infrastructure to distribute the images. I do why they’re trying to build it. Container registries, much like S3 buckets, are the things that are so ubiquitous in today’s Cloud Native ecosystem that they’re already present. Managing a new format, like for OSTree can be seen as a mainteance burden not strictly required.

In a Kubernetes environment, where it’s not that uncommon, to have a so-called “machine controller” in your cluster, delivering whole operating systems via containers makes sense.

Just as a heads-up: a machine controller in a Kubernetes cluster is responsible for the whole lifecycle of a VM. It’s talking to your cloud providers API, spawns a new VM, registers it as a node and also ensures it’s deleted via the Kubernetes primitives.

If it’s setup in your cluster, it is, in fact, pretty pretty nice to work with. All machines managed by that are basically the same, except for minor things like the IP addresses, the hostname and that. The rest of it? Complementely managed by Kubernetes. Which also means that you don’t need SSH on those machines and can isolate them pretty good from the outer network!

If you’re trans and you need an argument to look into it, it’s core component bootc is whispers written in ✨Rust✨.

My experience: Containerfiles aren’t still my favorite, not even with the introduction of heredoc-like strings in RUN commands. They’re usable, sure. But for managing a whole operating system, it is a bit cumbersome to me. Also: Container registries are pretty common and you can get several ones for free. But: Why do I have to rely on an external dependency to update my systems now? Sure, I can rebuild the container nightly via GitHub Actions and with proper caching it should be pretty fast, but: Is all of that really neccessary? That’s a lot of infrastructure for the two or three servers I’m having.

And that’s also the reason why I ditched the idea of using them.

What do I actually want?

Actually, I’m not sure about that either. You know how Windows has a thing like “Reset, but keep applications and user data”? I want that as well. I want to get rid of all the files that might have cluttered my /var in the meantime, I just wanna keep the stuff that’s relevant.

I want a system where I have a history of what I’ve done in the past. Which files I created, which services I modified, all that. And it’s all tidied up somewhere in a Git repository, where I can then annotate it and where it’s getting synced back from. Yes, I want bidirectional sync. etckeeper does one half of the job, it just won’t work for my Fedora systems because of lack of support for dnf5. :c

I want a Time Machine-like experiences for my servers. I want BEGIN and COMMIT. I just wanna hop onto a server, start my start-recording script, do whatever the fuck I can imagine and then, when I’m done, I stop the recording and I receive a wonderful summary of all the things I’ve done.

And the output can be an Ansible playbook, a Nix module, whatever. When I invoke useradd, I want to configure when, at the end of my session, an actual entry to /etc/shadow is written or maybe something like systemd-sysusers file. I want that the computer figures out how I can document the things I’ve done.

And even if all I get is a plain directory full of the changes I’ve done, that’d be enough for me. I just want to avoid that state of either really fucking up your system by accident, using all the CPU cycles to rebuild it (whether it’s a container or a Nix flake doesn’t matter for this argument) — I just want to play with my computer and know what I’ve done on them. I don’t need to track every little patch update that is done. I only want to keep the info that I installed vim and if, if, I really want to pin the version, every package manager on this planet provides me some way to do this.

All of the above is possible and probably pretty easy to implement on filesystems that support snapshots. It’s just wild to me that it’s apparently 2025 and closest thing we’re having are auto-snapshots (via btrfs, e.g. in openSUSE). We could make our computers could assist us much better in that regard.

and yet, we degrade them to produce slop. because number go up. and that sucks.

"Clean code" is not an excuse to avoid writing comments

2026-06-01T21:17:25Z

Good code does not need documentation.

This quote, appearing in several forms over the last couple of weeks in my professional and personal life have been driving me nuts. Usually it is not phrased exactly like that, but more as a question that goes in the following direction:

Why do I have to write documentatiom, it’s clear based on the code alone?

My opinion: This statement never holds true.

You might say that domain-driven design, properly named variables and a good software architecture already is enough documentation. And that there is no need to comment the following piece of Go code:

func queryInvoiceByID(ctx context.Context, id string) (*Invoice, error) {
  // implementation omitted for now
}

If you don’t know Go: The function above takes two parameters and returns a pointer to an Invoice struct, together with an error. However, the programming language is not important for my point here. You can replace that function with any other in your mind.

So, what does the code do? Right now, we cannot know, as I haven’t provided any implementation. The next question probably is: What do you expect it to do?

Test-Driven Development to the rescue!

If you’ve been programming for a while, you probably heard of Test-Driven Development, commonly abbreviated as TDD. The idea is simple: Write down the requirements as tests and only after then, write only the necessary amount of code required for the tests to pass. In an ideal world, tests translate to business requirements.

And in a lot of tutorials out there, the authors also argument that this documents the code. If we have a test called TestQueryInvoiceByIDReturnsInvoiceIfInDatabase, we know that this method returns the invoice if it’s in the database. However, a question to you, dear reader? Does this actually provide additional value for the understanding of the function? To me, it does not. Not at all. It introduces a new external component, called “database”. But apart from that, that’s it.

So, there are a lot of questions that are still unanswered. I haven’t provided any additional context to the system where this function is coming from. As readers, we need that to understand this gear in the other set of gears! Invoices are everywhere, so the code snippet above could also be taken from almost any application that has to manage invoices somehow. It can be the interface of a cloud provider, from the website of your local bike shop or even from a system like paperless-ngx.

Improvement No. 1: The bare minimum

// queryInvoiceByID queries the Invoice by the id.
func queryInvoiceByID(ctx context.Context, id string) (*Invoice, error) {
}

If another developer would read this, they would respond with a “no shit, sherlock” reaction. Does this comment provides a benefit? Not really. But: There’s a lot of possible improvement here!

The meaning of comments in code

Comments in code are not limited to the same boundaries as the code itself. Comments can use metaphors, links to external resources, maybe even little ASCII diagrams. Good comments are art, to a certain extent. They can express ideas, thoughts and even little musings for the next reader!

When writing comments, I try to include as much information as possible. In the past, the following things ended up as comments in my code:

Behaviour in error cases, esp. if it’s deriving from the norm
Ticket numbers that introduced it
External limitations that influence the implementation
Earlier implementation ideas that did not work out (and the reason why!)

One could argue that this partially belongs in the commit message. I don’t disagree! A lot of those things can also end up in the detailed description of your commit, but it doesn’t exclusively have to be there. If in doubt, I always stick to the code itself as the place to write down details for behaviours. That way it’s easily there for anyone to read and not hidden somewhere in the Git history.

No doubt, this is a lot. If you’re not used writing documentation for your code, this can feel very overwhelming. Admittedly, I struggled with this for a huge time of my career as well. In the past, I was arguing in favor of refactorings over writing documentation. I always assumed that this should be enough.

But this was from a time where I was working mostly alone on my projects. Working more and more in teams (aka in companies, instead of my free time), this started to shift. I caught myself communicating my intents as comments more often. I started to add explanations to bash scripts that invoked external commands. In the end, it’s easier to just read a comment right next to it than to decompose what rsync -aivtS is doing under the hood.

And to be honest, we, as developers, should have clear communication as our primary goal. “Clean” code, how good the intentions of the programmer were, will never reach that.

Improvement No. 2: The detailed comment

So, how could a “good” method be annotated?

// queryInvoiceByID queries the invoice and its associated metadata by its id from the database.
// Callers must ensure that the transaction is opened and attached to the context.
//
// If it cannot be found, an ErrNotFound is returned.
// For other errors, the underlying error is wrapped before.
//
// Note: Although invoices are having an UUIDv4 ID nowadays, a string is required here to ensure
// backwards-compatibility with the old invoice system.
func queryInvoiceByID(ctx context.Context, id string) (*Invoice, error) {
  // Although passing complex parameters (like transactions) via the context might be considered an
  // anti-pattern in Go, it is used here to extract common behaviour.
  //
  // This should be removed in the future as part of the database overhaul (PROJ-1337).

  // ...rest of the implementation
}

Notice something? I still haven’t provided even just one line of implementation, yet readers already know what to expect and what to take care of when calling this function. Additional bonus feature: Because it’s a comment, future programmers don’t have to read the code at all when queryInvoiceByID is called somewhere. They can rely on their editors assistance, which can display the comment when hovering the function! I know I am repeating myself, but: Providing only code and tests – which all have to be parsed! – cannot provide this.

Does this take practices? You bet it does. But it’ll make your job and that of others a lot easier, so: let’s do it together! Word by word, phrase by phrase, let’s write comments! ✍️

Automatically enable auto-merge for all merge requests on GitLab

2026-06-01T21:17:25Z

GitLab 17.4, released in September 2024, introduced a very welcome change to its auto-merging feature.

GitLab now supports Auto-merge for all checks in merge requests. Auto-merge enables any user who is eligible to merge to set a merge request to Auto-merge, even before all the required checks have passed. As the merge request continues through its lifecycle, the merge request automagically merges after the last failing check passes.

When I read this first, my eyes were immediately wide open. Up to that point, our merge request workflow at work was:

Person 1 opens the merge request (MR)
Person 2 approves it
Person 1 merges it

The time between step 2 and 3 could be a couple of seconds, but we observed several days as well. Pretty cumbersome if you ask me. And what’s the developers’ solution to a manual and repetitive task? You guessed it, automations!

First problem: The Merge Request REST API is not ready for auto-merge

The REST API documentation as a dedicated /merge endpoint, with a merge_when_pipeline_succeeds parameter. My first idea was to use this API, but in my tests I always received a 405 Method Not Allowed error. According to the documentation, this means that the MR cannot be merged. Technically, this is correct, yet I hoped that it’d automatically trigger the auto-merge functionality.

It did not. I actually do not blame GitLab for this. In retrospect, implicit behaviour like the auto-merge feature would be a change in behaviour and therefore might’ve introduced unexpected side-effects for existing API users.

Yet, I had to learn it the hard way that there’s no auto-merge functionality within the REST API.

Second try: The GraphQL API

The REST API, however, is not the only API. GitLab also has a GraphQL API. If you’re not familiar with GraphQL: It’s another type of API specification, just in cool, because Facebook invented it, I guess. In the end, we’re still executing HTTP requests.

After tinkering for a while with the GraphQL explorer (which is admittely one of the cooler things that GraphQL allows), I ended up with the following result:

# This can be tested here: https://gitlab.com/-/graphql-explorer
mutation acceptMR($projectPath: ID!, $iid: String!, $sha: String!) {
	mergeRequestAccept(
		input: {
			projectPath: $projectPath
			iid: $iid
			sha: $sha
			# This is the important thing you'll have to adjust to your project-specific settings!
			strategy: ADD_TO_MERGE_TRAIN_WHEN_CHECKS_PASS
			shouldRemoveSourceBranch: true
		}
	) {
		errors
	}
}

The value for the strategy is important here! Since we use Merge Trains at work, I set it to the ADD_TO_MERGE_TRAIN_WHEN_CHECKS_PASS strategy. As of GitLab 17.9, the following strategies are available:

MERGE_TRAIN: Use the merge_train merge strategy.
ADD_TO_MERGE_TRAIN_WHEN_PIPELINE_SUCCEEDS: Use the add_to_merge_train_when_pipeline_succeeds merge strategy.
ADD_TO_MERGE_TRAIN_WHEN_CHECKS_PASS: Use the add_to_merge_train_when_checks_pass merge strategy.
MERGE_WHEN_CHECKS_PASS: Use the merge_when_checks_pass merge strategy.
MERGE_WHEN_PIPELINE_SUCCEEDS: Use the merge_when_pipeline_succeeds merge strategy.

Use the strategy that fits most to your project. Place this file in .gitlab/auto-merge.graphql.

Using the GraphQL mutation in a pipeline

To use this mutation query in a CI pipeline now, we have to add a step to the .gitlab-ci.yml. The following snippet for your .gitlab-ci.yml will do:

enable_auto_merge:
  stage: deploy
  needs: []
  rules:
    # Copied from: https://docs.gitlab.com/ee/ci/yaml/workflow.html#skip-pipelines-for-draft-merge-requests
    # Disable this job for drafts. Not necessary, but it reduces the amount of jobs.
    - if:
        $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TITLE =~
        /^(\[Draft\]|\(Draft\)|Draft:)/
      when: never

    # Enable it for all other MRs.
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

    # And ignore it if we're not in a MR scope. Explicitly mentioned here for clarity.
    - when: never
  image: alpine:3.21
  before_script:
    - apk add --no-cache curl jq
  script:
    # Until https://gitlab.com/groups/gitlab-org/-/epics/3559 is not implemented, we cannot use the
    # job token to do this. Therefore, we have to use a project token for that.
    #
    # Because the "merge_when_pipeline_succeeds" parameter via the REST API fails,
    # we use the GraphQL API and its mergeRequestAccept mutation.
    #
    # The `AUTOMERGE_BOT_TOKEN` is defined in the CI/CD variables of this project.
    - >
      jq -n --rawfile query .gitlab/auto-merge.graphql '{
        projectPath: env.CI_PROJECT_PATH,
        iid: env.CI_MERGE_REQUEST_IID,
        sha: env.CI_MERGE_REQUEST_SOURCE_BRANCH_SHA
      } | {variables: ., query: $query}' | curl --header "PRIVATE-TOKEN: $AUTOMERGE_BOT_TOKEN"
      --json @- $CI_API_GRAPHQL_URL

As you can see, it’s requiring a dedicated token for this. Any token with the api scope will do. I decided to use a project access token for this and persisted it within the AUTOMERGE_BOT_TOKEN CI/CD variable.

Has it helped us?

A lot. The time to merge is effectively zero. Once the merge request is approved, it’ll get merged automatically. Over the last couple of weeks, this increased the velocity a lot and allowed for a much more asynchronous work.

Low-effort self-hosting using Podman

2026-06-01T21:17:25Z

Because every one in the homelab community is hosting their services differently, I thought I write about my way of hosting containers.

Server setup

My primary server is a Lenovo ThinkCentre thin client (not sure about the exact model), with an Intel Core i5-6500 and 8GB of RAM. This might not sound much first, but this machine has proven itself to be quite capable for self-hosting.

Right now I’m hosting (in alphabetical order):

AirConnect
iSponsorBlockTV
Handbrake (used on-demand for pre-transcoding videos)
Home Assistant
Jellyfin
nginx
paperless-ngx
Zigbee2MQTT
…and several other smaller webservices

As the operating system, I am using Fedora Server.

Why Fedora, and not a Debian-based distribution?

I chose Fedora for my server because personally I was searching for a distribution that fulfills the following requirements:

It should work out of the box.
It has a community and sufficient documentation for common tasks.
It provides up-to-date software without sacrificing stability or comfort.

Fedora provides all of this. I was considering Arch Linux first, but even with the archinstall helper utility that’s assisting with the setup of a new Arch setup, I wasn’t motivated enough to use it. You know, sometimes I just want things to work.

And there’s an additional benefit of using Fedora: By using an operating system that has overlap with the Red Hat folks, podman is available out of the box in an up-to-date version. Which means less maintenance work, as no additional package repository is needed.

Why Podman instead of Docker?

There are a couple of reasons for it.

It’s in the upstream repositories. dnf install podman is sufficient to run containers. No additonal package repository required.
It’s daemonless. Docker has it’s socket running on /var/lib/docker.sock running all the time.
Rootless containers. Podman, in comparison to Docker, has a pretty good support for running unprileged containers. I hope that it changed already in Docker, but the last time I checked it, running docker as a regular user effectively meant sudo permissions on the machine.
It is respecting my network policies, especially the firewall. When creating a container using docker with published ports, it works by creating additional iptables rules, effectively circumventing the firewall.

Which means, if I’d run docker run -p 8080:8080 my-fancy-service:latest, it would be available to the whole web, even if I’d configure the firewall to block this port. To workaround this, you’d have to bind it to localhost only (docker run -p 127.0.0.1:8080:8080), which is – as far as I can see – almost exclusively never done in any READMEs or tutorials out there.

podman, on the other hand, is less surprising and does not do such shenanigans.
Tight integration with systemd.

Quadlets aka containers managed by systemd

The tight integration of systemd, especially with the availability of a unit generator out of the box is the most undervalued feature of Podman in comparison to Docker, I think. Docker is known for requiring docker compose for basically everything that exceeds the default usage of containers that are executed just one time using docker run. The systemd unit generation of Podman can be seen as the answer to that. It is translating .container specification files to a podman run command. By following the architecture of systemd, we automatically gain a fully functioning systemd unit as a gift.

Personally, I just love this. I write my jellyfin.container and automatically can manage the Jellyfin instance using systemctl. I can use systemctl stop jellyfin to stop it, I can use journalctl --unit jellyfin to see its logs, it no longer feels like it is isolated from the rest of the system.

How does that `.container` file look like in practice?

I’ll take Jellyfin as an example here, because it’s one of those applications that are relatively complex in their setup. My Jellyfin container specification, located in /etc/containers/systemd/jellyfin.container can be seen below.

For this blog post, I added additional comments to make it easier to understand.

# The [Container] section is read by Podman, therefore all of the following
# parts are interpreted by Podman, not by systemd.
[Container]

# Using `:latest` here allows me to update Jellyfin by pulling a newer image
# and restarting the container. The whole procedure is explained in a later section.
Image=ghcr.io/jellyfin/jellyfin:latest

# Required for hardware acceleration. I'm passing the built-in GPU to Jellyfin here.
# https://jellyfin.org/docs/general/administration/hardware-acceleration/intel
AddDevice=/dev/dri/renderD128:/dev/dri/renderD128
GroupAdd=105

# Networks are explained later in this blog post.
Network=services.network
Network=external.network
HostName=jellyfin

# The CONFIGURATION_DIRECTORY environment variable is set by systemd automatically,
# when using the `ConfigurationDirectory` directive in the [Service] section.
#
# This effectively creates a folder /etc/jellyfin automatically, which is then mounted
# into the container. The `:Z` at the end is required for SELinux, enabled by default
# on Fedora.
Volume=${CONFIGURATION_DIRECTORY}:/config:Z
# Same as above, just with /var/log/jellyfin.
Volume=${LOGS_DIRECTORY}:/config/logs/:Z
Volume=${LOGS_DIRECTORY}:/config/log/:Z

# And also /var/cache/jellyfin. By using the systemd-integration, we can clean
# the cache later by running `systemctl clean jellyfin`.
Volume=${CACHE_DIRECTORY}/cache:/cache:Z

# I first used RuntimeDirectory, but the ramdisk was too small to hold the
# on-the-fly transcodes. Therefore I moved it to the cache later.
Volume=${CACHE_DIRECTORY}/transcodes:/config/transcodes/:Z

# The NFS mount to my media library.
Volume=/media:/media

# [Unit] and [Service] directives are passed as is to the generated systemd unit.
# There's nothing spectacular in here.
[Unit]
Description=Jellyfin media server
RequiresMountsFor=/media/music
RequiresMountsFor=/media/video
RequiresMountsFor=/media/books

[Service]
CacheDirectory=jellyfin
LogsDirectory=jellyfin
ConfigurationDirectory=jellyfin

# Create subfolders required for binding the folders
ExecStartPre=mkdir -p ${CACHE_DIRECTORY}/transcodes
ExecStartPre=mkdir -p ${CACHE_DIRECTORY}/cache

# Inform systemd of additional exit status. Copied as-is from the Jellyfin
# documentation.
SuccessExitStatus=0 143

Explaining the networks

For my setup, I use two networks. The primary one, services, is a bridge network. It does not have any special configuration values and is located in /etc/containers/systemd/services.network. Almost all containers do land in here.

[Network]
NetworkName=services
Driver=bridge

The second network, external is only used in exceptions, mainly by Jellyfin and Home Assistant. It’s a MACVLAN network, which assigns those containers individual MAC adresses and makes them appear as individual devices within the network. Jellyfin benefits from this for DHCP, Home Assistant for the auto discovery feature.

[Network]
NetworkName=external
# macvlan networks have to be bound to an interface. The default interface on my
# host is named eno1, which is passed as an additional argument here.
PodmanArgs=--interface-name=eno1
Driver=macvlan

# Since 2024-10-26, the subnet is predefined. By doing so, we do not need DHCP
# leases and therefore can avoid the weird router bugs.
#
# This shouldn't be necessary usually, only with very very weird routers.
Subnet=192.168.188.0/24
Gateway=192.168.188.1
IPRange=192.168.188.220-192.168.188.250

One thing that can be noticed from the above Jellyfin configuration: none of the ports are exposed. In my setup, they don’t have to. What’s set, is the hostname (HostName=), hardcoded to jellyfin.

External access using nginx as the reverse proxy

To reach my services, I am using the linuxserver.io/swag container. It is a nginx reverse proxy, together with automatic ACME support out of the box. I use nginx, because I am familiar with it, but you’re not bound to this decision as well. If you want, you can use any reverse proxy out of the box.

My nginx container is the only container that publishes ports for HTTP and HTTPS communication.

Network=services.network
PublishPort=443:443/tcp
PublishPort=80:80/tcp

Because nginx is also connected to the services network, it can resolve all the other containers by it’s hostname using the internal DNS of Podman.

To keep this blog post focused, I won’t explain further details of my reverse proxying configuration here.

Applying defaults to all containers

The Jellyfin configuration above is lacking several features, including but not limited to:

automatic updates
automatic start on boot

I am implementing those by using drop-ins. Drop-ins are smaller configuration files that apply to all of my containers. I love this feature a lot, because it allows me to enforce a common set of rules for my containers. Those drop-in files are located in /etc/containers/systemd/container.d. For example, I have a 01-registry-auto-update.conf with the contents of:

[Container]
# Enables automatic updates for all containers. This is done via the
# podman-autoupdate systemd timer.
AutoUpdate=registry

This ensures that all of my containers automatically update once a day (as defined in podman-auto-update.timer). The automatic update mechanism is pretty simple in comparison to tools like Watchtower, but it served me for a year now without any problems. I can only recommend it.

02-enable-automatic-startup.conf ensures that all my containers start when booting the machine.

[Install]
WantedBy=default.target

Another file, 03-restart-on-failure.conf, defined below, automatically restarts my containers. Another useful thing that you definitely don’t wanna miss out.

[Service]
# Automatically restarts the container if it exists with a non-zero exit code.
Restart=on-failure

Starting the container

After the container is configured, I reload the systemd daemon (systemctl daemon-reload) and start the service (systemctl start jellyfin). They’re enabled automatically due to the [Install] directive.

That’s it! Jellyfin is now running and I can watch my videos. :3

How does the generated unit file look like?

The systemd generator in /usr/lib/systemd/system-generators/podman-system-generator generates systemd units that are placed in /run/systemd/generator/. The important part is this:

# /run/systemd/generator/jellyfin.service
# Automatically generated by /usr/lib/systemd/system-generators/podman-system-generator
#
[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
KillMode=mixed
ExecStartPre=mkdir -p ${CACHE_DIRECTORY}/transcodes
ExecStartPre=mkdir -p ${CACHE_DIRECTORY}/cache
ExecStart=/usr/bin/podman run --name systemd-%N --cidfile=%t/%N.cid --replace --rm --cgroups=split --hostname jellyfin --group-add 105 --network services --network external --sdnotify=conmon -d --device /dev/dri/renderD128:/dev/dri/renderD128 -v ${CONFIGURATION_DIRECTORY}:/config:Z -v ${LOGS_DIRECTORY}:/config/logs/:Z -v ${LOGS_DIRECTORY}:/config/log/:Z -v ${CACHE_DIRECTORY}/cache:/cache:Z -v ${CACHE_DIRECTORY}/transcodes:/config/transcodes/:Z -v /media:/media --label io.containers.autoupdate=registry ghcr.io/jellyfin/jellyfin:latest
ExecStop=/usr/bin/podman rm -v -f -i --cidfile=%t/%N.cid
ExecStopPost=-/usr/bin/podman rm -v -f -i --cidfile=%t/%N.cid
Delegate=yes
Type=notify
NotifyAccess=all
SyslogIdentifier=%N

As you can see, in the end it’s just a podman run invocation. If there’s something wrong with it, it’s pretty easy to debug it, either by changing the command or by adding additional commands ahead of the podman execution, for example to set a value for sysctl.

That allows you to run a common setup.sh or something else without resorting to Makefiles or custom bash scripts that run docker compose in the end!

Summary

Without additional system dependencies (apart from podman), I now have a container that is:

automatically booting on startup
updating itself every day
not exposed to the public internet, but behind my reverse proxy
managed via systemd, therefore automatically picked up by any monitoring solutions that monitor it
declaratively managed
following a consistent pattern of configuration paths just like non-containerised services

And, most importantly: Once they run, I don’t have to touch them anymore (unless they break because of an update).

Sorting series in Jellyfin by the DVD order

2026-06-01T21:17:25Z

Today I learned that it’s quite common for TV series to be aired in a non-chronological order. This can make the media management a bit more annoying, since automatic tagging systems might take the airing date as the ordering.

This also happened to me with Cowboy Bebop, a series where the “first” episode was actually the 13th. When opening the TMDB, I saw like there’s not just one way to order the episode, but a mere six of them!

Anyway, I wanted the correct order in Jellyfin instance. Fortunately, I found a solution in the forum for this exact problem:

Edit the metadata of the show (not just the season).
Scroll down to the “Display order” and change it. In the case of Cowboy Bebop, I changed it to “DVD”.

In my case, I also had to refetch all the metadata and replace it in order to take effect, but it might work without that step in your case. After that, the series is in the correct order with no manual change on the metadata being necessary.

Cleaning up my home directory on a schedule with systemd-tmpfiles

2026-06-01T21:17:25Z

I don’t know about you, but I am using both ~/Desktop and ~/Downloads as some kind of temporary workdir. Almost everything I work on ends up in there, from memes to Linux ISOs to archives made via wget. And over time, my ~/Downloads ended up to be a total mess which I randomly forgot about.

Furthermore I realised that the XDG_CACHE_DIR (commonly located in ~/.cache) tended to fill up with random stuff over time. I am not sure whether it’s the responsibility of the user of the application to clean up files in there, but heck, there were a lot of them. And when I glimpsed over it using ncdu, I realised that most of it hasn’t been touched for a long time.

Therefore I needed a plan to clean up those folder. I first thought about writing a simple bash script that checks the age of my files and folders and is executed on boot. But then I remembered systemd-tmpfiles and it’s support for user-specific configuration entries in ~/.config/user-tmpfiles.d.

And that’s how the configuration below was born:

#Type Path                                     Mode User Group Age         Argument

# Delete all downloads older than one month
d %h/Downloads/ - - - 30d

# Delete cache files older than a month
d %h/.cache/ - - - 30d

Although presented in a single block here, I split them up across two files, downloads.conf and local-cache.conf respectively. systemd-tmpfiles provides a lot of useful parameters, but I am using the d type here.

d: Create a directory. The mode and ownership will be adjusted if specified. Contents of this directory are subject to time-based cleanup if the age argument is specified.

With the configuration line above this translates to:

create ~/Downloads, ignore existing mode, user and group and delete everything older than 30 days

And that’s exactly what I need. Now I just need to enable the respective timer (systemctl enable --now --user systemd-tmpfiles-clean.timer) and I’m done. By default, the timer runs five minutes after boot and every day after that. So, even if I’d forgot to turn off my devices, the files would be cleaned up properly.

I've just released older projects to the public domain

2026-06-01T21:17:25Z

Because copyright licenses are a weird concept to me (and because licenses all suck to a certain degree, at least for me), I decided to relicense some of my older repositories to the public domain.

Therefore, the following repositories are now part of the public domain:

As part of the license change, I also moved some of my repositories to Sourcehut in order to remove my dependency on GitHub.

For the future, I plan to move all repositories to the public domain before I “retire” them. It’s not much, but if it helps at least one other person on this planet, it’s a win for public knowledge sharing. :3

If you wanna do the same, I’d encourage you to place the Unlicense license within your repository.

Addendum: after publishing the post I realised that the first day in the new year is also referenced as Public Domain Day 🙈

The retrospect of a social media break

2026-06-01T21:17:25Z

Aaaand another blog post, close to the end of the year. That should go into my go-to explanation guide “how adhd can be perceived in adults” – if I had one. Do not expect any kind of structure, this post is going to be a stream of consciousness.

If you’ve followed me on the fediverse, you are aware of the fact that I took a break. I might write about the consequences of that as well and the thoughts I had in my absence of social media. But not this year anymore, because, yeah, of course. It’s almost 2025, I’m just waiting for ny partner in order to build up a small fire to roast some Stockbrot and marshmallows.

Anyway, back to topic. One of the realisations I had in the last couple of weeks is that social media always served as some kind of knowledge sharing to me. Whenever I stumbled across something or was playing around with some technology, I shared that on the fediverse (or previously on Twitter). And now, when I took a break I realised pretty quickly: I miss that knowledge posting. Not just mine, but also from others. I missed and still miss the ability to learn from others. Taking a break from social media has it’s advantages, but for me, this was by far the biggest disadvantage.

And the more I thought about it, the more I realised that I’m not sure whether I want to use social media in the way I did before. This is not going to be a post that says “build your own website” or “own your content” (jeez, I hate the word content so much), it is merely a reminder that all the knowledge you’ve shared on any of those platforms is going to be lost one day. And I don’t wanna lose it, I want to keep it.

There are, of course, enough posts that aren’t meant for longevity (jokes and such, I think) but the things you’ve learned about and even shared with others? I’d love to read all of them later. And even though Mastodon as a micro-blogging platform is better than the other corporate-branded platforms in terms of the open web, it’s still far from perfect.

One example? Lennart Poettering, one of the core maintainers of systemd, started to share some information ahead of the releases itself in forms of little toots. This can be seen e.g. in this thread, where run0 was explained. Lennart also has a blog, where this just ended up as just another link to the actual thread.

I understand it to a certain extent. Writing several smaller has threads on a social media platform has a lower friction than to write a fully featured blog post. On the other hand, I miss the ability to link to the actual blog posts of people and their website. Social media profiles all look the same, websites do have some variation. (Though I also remember the “old” times, where everyone was just using the default Twenty Something themes of WordPress.)

And the same goes for my friends. I missed all their knowledge when I took a break and the joy to learn something new. I really wish I could read their ideas and knowledge in a different form. And I also regret not storing my own knowledge in a different place as well. There are tools like Notion, Obsidian or a plain MediaWiki installation somewhere and sometimes I wonder why we keep it private. Why don’t we share our own wikis with the world? Wasn’t this the whole intention of the web that Tim Berners-Lee imagined it? Why don’t we do it?

(oh, almost time for the fire, now I have to rush a bit…)

Especially for me, who’s living in a tech-savvy bubble, I don’t think it’s technical problem. Did we got so bad at concentrating? As for me, with ADHD, I might answer that with maybe. It is (at least for me), pretty hard to write a longer post like this. And the reason why it’s working this time is because of the fact that it’s purely text-based. (Working with images in a static site generator is painful, change my mind.) I’m just in a flow and reduced the friction for me to write longer texts.

But it’s not the texts I want to keep nor do I expect that everyone is capable of maintaining their own wiki.

I just have a simple wish for the future and to you, dear reader: Please share your knowledge with the world in a format that allows for longevity. Install a wiki, start a blog, heck, even start a forum for your community. But please, don’t have it get lost on Discord, Matrix or the fediverse. You don’t have to hide.

(And I don’t have to as well, therefore this is one of my goals for 2025 as well.)

Hello, new ThinkPad!

2026-06-01T21:17:25Z

After carrying my “old” ThinkPad T460 for a while (seven years!) now, it was time for a replacement. And, well, I have to admit that it took me a while to actually buy a newer device for me, as the old ThinkPad isn’t broken. It’s just, let’s say, not in its best conditions. The two batteries have a combined lifetime of like 20 minutes and are so close of becoming a very fancy spicy pillow. That was the first reason to buy a newer device.

The second reason was the lack of Thunderbolt, which made it impossible to integrate the device with my work setup. This always resulted in me sitting in the bed next to the charger. Bad for the back, even worse for long-term posture.

One day I took the initiative and lurked a bit on ebay, searching for a good deal for a newer ThinkPad. I peeked at the Framework devices, but they’re still to expensive for me. I didn’t want to spend more than 400 EUR in total and originally aimed at 200-300 EUR. Not much, but plenty enough for a newer device, even refurbished.

Sadly, none of the deals online looked particularly good to me. They we’re reasonable, but never reached my “this is a no-brainer, I have to buy this now” line. I mean, what do I want with a newer device that only has a display resolution of 1366x768 pixels? Even if everything else is in perfect conditions, this isn’t a good deal. At least not for me.

It took me some days until I found a deal that was too good to be true. A ThinkPad T14 Gen 2 (11th Gen Intel Core i5), which is the first generation of the Thunderbolt 4 devices! All other devices had “only” Thunderbolt 3, that device really stood out. To my surprise, it even was within my desired price range with a cost of ~360 EUR, which is, holy fuck, pretty pretty cheap! So, there has to be a quirk somewhere, right? And from the description alone, my “this is a scam” radar hit pretty quickly. The description was almost empty (it was more or less just a “ThinkPad to sell, charger included.”) and the seller was pretty inactive as well. But hey, there are some protection mechanisms by eBay itself and the account was a stunning 20 years old (holy fuck, eBay is a pretty old webpage), so I just decided to give it a go and hope for the best.

And that gamble paid off. I’m now the owner of a device in an almost perfect condition. Even the battery reports a health of 100%! The luck really really paid off and hopefully, that little guy is going to serve me another seven years. 😊

jujutsu made me realise how bad the UX of Git actually is

2026-06-01T21:17:25Z

Nowadays, Git is the common tooling that every software engineer has to know about. And due to the rise of DevOps approaches, not only developers, but operators (or “infrastructure engineers”/“DevOps engineer”/some other fancy term here) as well.

And if you’re a Gen Z kid like me, Git is likely the only version-control system (VCS) that you’ve ever learned. A GitHub account is more or less ubiquitious nowadays, which is unfortunate, I think. Because now they’re used interchangeably and they aren’t. Git has limitations, but GitHub has even more.

Don’t get me wrong, Git is not bad per se. The decentralised approach, the ability to work offline and its abilities to deal with merge conflicts are pretty powerful. And yet, the default user experience of the git command could be better.

How much, you might ask? Well, that’s the thing I realised when I tried the new kid on the block, jj. I never tried one of the other VCS systems out there, whether it’s Mercurial, Subversion, Fossil or the VCS of Microsoft that I forgot the name of.

But what is bad about the user experience of Git?

Git has a really, really high learning curve. You have to learn about commits, branches, what the HEAD pointer is, the difference between “untracked” and “staged”, etc. That’s quite a lot for newcomers to deal with.

To continue, you have different ideas what a commit is per team/employer/ project. One team might stick to Conventional Commits, which I already ranted about in 2022, another one might enforce squashing on merge/pull requests and so on.

And you have to learn all of this in order to commit the smallest changes to a project.

Yeah, of course, it’s a pretty powerful tool! It takes time to practice it!

That’s certainly true, yes. But: I stopped counting how many workshops/tutorials/wiki pages I wrote for others to explain “basic” stuff of Git itself, just so that I, as a reviewer, have an easier job to do. No matter how much practice someone has in Git, there’s always a thing that could be improved from my point as a reviewer.

The problem I faced with most of the commits I’ve seen over the last decade is: an unclear scope of a commit. Changes that are actually separate from the description itself (a typo here, some code formatting there) are so common that if I’d count the “days since it last happened”, the counter would never go above 30. Those “leaks” of changes are happening at least(!) once a month. And that is pretty annoying, especially as a reviewer.

So, what do in this case?

To be honest, I ignore most of such “change leaks” all the time, as long as it’s limited to formatting. It annoys me, sure, but I accept it. I know that the Git user experience is bad and therefore treat those diffs as a symptom of the tooling rather than a lack of knowledge.

The problem of Git in a nutshell

Because the Git user experience is so frustrating to most of its users, it encourages you to do less commits. It is possible to just use git commit -am "wip" or do some combination of git commit --fixup together with interactive rebases. And this is working pretty nicely, as long as your changes are limited in their scope. Except: that’s almost never the case.

I cannot speak for the majority of developers out there, but I never work on just feature, it’s always at least three or even more. And those features/changes are not mutually exclusive to each other. Sometimes I have fixed something in one branch (even if it’s just a npm run format-all), which then has to be repeated on the next branch again.

Not to mention the fact that at least I don’t know always what I’m working on next. I just open the repository locally and start doing something. Not every change has a proper bug ticket or a clear scope, sometimes it’s just refactorings all over the place. Even when I’m working on a dedicated bug: No codebase is perfect, there’s always something that I see and work on.

And yes, I can interrupt my current edits and create a small intermediate commit using git add -p, but even then: It’s on the same branch as the feature I’m working on, so I have to get it out of my branch history. As soon as you’re doing this multiple times a day or a week, you’ll get annoyed by it.

An alternative approach: stacked diffs

I won’t go into detail here, but the idea of stacked diffs does not rely on branches at all. Instead, each commit is an independent and theoretically isolated change. Instead of having one branch feature-1, which contains commits A, B and C, you’re working on something that’s commonly called a changeset.

This changeset has a unique identifier and evolves over time, but it always ends up in one commit. Tools working with changesets, for example Gerrit, allow reviewers to see how a change evolved over time.

It looks similar on a first glance, but it fundamentally changes the way one as a developer and reviewer alike is working with repositories. It encourages developers to keep their individual commits clean and focused. And what’s more important: you aren’t no longer working with individual branches, you can always commit straight to main.

That’s just my idea of how I approach them, there’s a more detailed explanation for stacked diffs by Jackson Gabbard, which I can only recommend for reading.

My experiences with jujutsu

But let’s head back to the introduction of this post and the little tool mentioned in the title, jujutsu, or short, jj. I found out about it back in May 2024, according to this commit from my dotfiles.

jujutsu is a relatively new version-control system (VCS), which describes itself as “both simple and powerful”. And even though I was very skeptical when I read it first, I tried it out.

Adoption is easy

Since jj is using Git under the hood, I was able to try it out immediately on any code repository that I have checked out locally. This is a massive benefit, as it requires no changes to existing infrastructure. I don’t have to convince my team to use it, I don’t have to introduce new tooling to add server-side support. Especially in a business context where everything needs some kind of approval or discussion beforehand, this is a massive benefit.

The lack of a staging area is actually nice

jj reduces the “areas” from Git to just two: either the file is untracked or it’s tracked and committed. There’s no manually managed staging area. The current state is committed as soon as you execute jj. That’s always the case, unless the --ignore-working-copy is given. As a consequence, executing git add is no longer required as it’s done for you.

Branches are no longer used for communication

Because branches are only used for pushing changes to review, they don’t have to be named ahead of the change itself. At first I still created the branches manually in order to follow the naming guide within the team. But it turned out to be a burden that’s almost always unnecessary, because: jj has commands to simplify pushing branches with the name being autogenerated!

It’s the change ID (which doesn’t change even when committing stuff) and therefore, together with my user-configured prefix, all my branches are named seemingly “random”, even though the name carries an actual meaning. My branches are now looking like: joster/change-adjslkangk1da.

The usual reactions at work were:

what the heck even is this branch name?

did you just keysmash?

And of course, if you’re used to branches like JIRA-123/add-new-fancy-feature or refactoring-of-user-service, those branches do seem random. In retrospect however, I do not miss manual branch names. For the case I have to use them anyway, I can do it with a simple jj bookmark set refactoring-of-user-service -r adjs.

git-lfs is still missing

I think this is one of the features that I’m still missing from time to time. I’ve used git-lfs in the past and on repositories that use it, I cannot use jj as I’d love to. The open issue regarding git-lfs support provides a lot of interesting insights about the feature itself and although it’s still a bummer for me, I can live with the current situation as is. Especially because I know that once the feature is going to land, it’ll likely be much more polished than the native behaviour anyway.

The ability to work on multiple things at the same time is astonishing

It is absolutely incredible how, yeah, almost perfect the user experience of jj is. When working on any project now, my workflow is:

open the repository
start working in the codebase

As soon as I think I’m done, I check the diff (jj diff) and if the scope is already clear, I just jj desc-ribe and push it (jj git push -c @ (@ is the current revision)). If I have to split the changes up, I use jj split (the naming is so clear, it’s wonderful), which opens an editor where I can select what’s going into each of the patch. This might be repeated multiple times for as often as required and that’s it. Rebasing, merging, etc. — that’s all done automatically. And if I’m not happy with the current commit log, I can rebase the changes and easily place my commits whereever I want them to be.

Dealing with conflicts got so much easier

Previously, when working with Git, merge conflicts were always a pain. They always had to be resolved immediately. With jj however, the conflicting commits are just marked as “conflicted” and do not need to be resolved now, they can be resolved at any time later. Dealing with conflicts is becoming just another commit, which then gets squashed into the conflicting one.

jj has a pretty low learning curve!

To be quite honest, I had my doubts when switching to jujutsu. Learning new tools when you already have reliable ones working, is something that needs to be carefully considered. Especially the new syntaxes for revision sets and templates were bugging me off at first. But in the end, they were actually pretty easy to understand and so, so incredibly powerful that I don’t know how I’d work without them today.

Also missing: pre-commit hooks

pre-commit hooks are still fascinating to me. Not because the idea behind is magic, but because how many devs seem to use it. Meanwhile I always found every millisecond blocking me from committing changes rather annoying and prefer to do the code formatting and such manually.

Therefore, the lack of pre-commit hooks are not a blocker for me, but I wanted to mention them here anyway for completion.

“bare” jj isn’t quite as useful, but extremely powerful after configuration

jj has a powerful syntax for filtering revisions, called “revsets”. It allowed me to configure jj to my likings. For example, jj open shows all of my unfinished changes and with jj remain ready I can rebase all ready (read: all open, but without the unfinished ones) revisions automatically on top of the current main branch. Yes, all of them. This is like git rebase, but on steroids!

Recap: I don’t miss `git`

It took me maybe a week or two to understand jj. But it is so powerful that I can use it for ~95% of all of my work in repositories. It even is so good that I stopped installing git on new machines altogether. That’s huge.

Martin and all the developers have my deepest respect for creating such a wonderful piece of software.

And maybe, you want to try it out as well now. In this case, head to the project page and start exploring it.

jasmin wrapped

2026-06-01T21:17:25Z

The holiday season is near, therefore it’s time for me to revisit what I’ve done (fuck, now Linkin Park is playing in my head) in the last year. Instead of answering questions like I did last year, I’ll use a free-form approach this year.

tl;dr: 2024 was fucking stressful.

Things I wanted to do, but haven’t done

I wanted to exercise more. Maybe it was my fault in the first hand that I haven’t set clear goals for that. Anyway, it didn’t work out as expected. I should try to do it more. It’s just… hard. Moving to a new city (more on that later) surely haven’t made it easier.

Next year it’s gonna work out, I’m sure. 🫣

Things that happened…

…at work

Originally hired as a software engineer, I’m now a “cloud platform engineer” and therefore working together with the team on all things Kubernetes, Terraform and so on. The whole cloud business bingo. And although we had some pretty rough times in the past couple of months (wanna join? we’re hiring!), I’m very grateful to have such a wonderful team. It’s unfortunate that some of them had to leave for personal reasons. In case you’re reading this, Z. and M.: I really enjoyed the time with you. You both taught me sooo much and I can only hope that you’re doing well right now.

There are ideas in the room that could make the work even more exciting. Hopefully I’ll be able to mention them in the recap for next year. 👀

Anyway, Kubernetes is actually an interesting piece of software and I do understand it’s appealings to the industry now. And yet, I hope that something else might step into the middle ground between “doing stuff by hand” and “running a Kubernetes cluster”, especially for a diverse set of hosts. (I have some ideas on that, might write a blog post about them one day.)

…mentally

Another thing I realised more often this year is the stability of my mental health. I’m still stunned how much more enjoyable the life can be when you’re not constantly depressed. (say whaaaat?) Therapy did a lot and I’m unbelievably grateful that I had such a good therapist.

Nonetheless, I almost burned out. I can only hope that it won’t repeat next year.

This year I chose to use more time for myself and only for myself. It was and is one of the better decisions I made this year.

…in my friends circle

It just expanded. I made new friends in 2024. And I think that I might have resolved the open conflicts with my ex. Not sure about that yet, but that’s a thing I’m definitely going to clear out this year, sooo: 7 days to go, I’d say. 🫢

…on my projects

The fediverse instance got a new administrator and moderator. But I also had to take a break from the fediverse, because the moderation work wasn’t good for my wellbeing. I’m still trying to pinpoint the exact cause, but so far it’s been likely that it’s a combination of the constant load and, more importantly, the cultural “need” to respond to anything within a timely manner. Especially the latter got incredibly hard, because most of the times things are seen as black or white with no nuance in between. And yea, this also applies to some actions I did in the past.

I also loosened the ties with my hackerspace this year. Due to the move into a new city, it wouldn’t have been responsible to continue the work in the awareness team there. Less responsibilities, more me time. It was a good decision. Yet, I miss a lot of the entities there. 🥺

All other projects of 2024 we’re unfortunately postponed for… idk for how long. 😕

Plans for 2025

Hopefully(!) launch a new product at work.
Get a new bicycle.
Tighten the existing friendships.

That’s it for this year, fellas. See you next year 👋

Just migrated the queer.group S3 to Hetzner Object Storage

2026-06-01T21:17:25Z

I planned a lot of things for today. But as usual, if you have plans, your head just decides to say: nope, not today.

A little benefit of this is the fact that: Because I am tired, I also had no spoons for overthinking technical problems. As a result, I thought that it is a pretty good idea today to migrate the S3 storage for the Mastodon instance I’m administering to a different provider, in this case: Hetzner Object Storage, which is just a fancy name for S3 on Ceph.

Why the switch?

The current provider, Backblaze had several problems. First and foremost, it’s an US-based company. While I usually don’t have problems with this, and also leveraged their hosting location in the EU, it always feels a bit better to use a provider in the EU.

I am aware of Scaleway (French-based) and their offerings, but to be honest: Their user interface is kinda overwhelming. Yes, they offer a lot of products and features, but to me, they offer too much. I’m sure that they’re great and all, but one big benefit Hetzner always provided to me is: simplicity. A carefully designed user interface, just enough options to be capable of doing everything I need and transparent billing.

Plus, one additional benefit that definitely shouldn’t be underestimated: Since it is fully integrated into the Hetzner billing and authentication infrastructure, I have:

one account less to worry about,
one UI less to understand (even if everything is managed via OpenTofu),
and one thing less to document.

Since Hetzner already allows you to invite others to your projects, my fellow co-admins, gim and Nina now see the S3 buckets as well without any action needed from my side.

Aren’t you risking vendor lock-in there?

Yes, actually, I do. The whole instance including the media files is now hosted on Hetzner infrastructure. The backups are excluded, but I might transfer them as well. This increases the risk for shutdowns a lot, if someone reports my nipples or something.

I trust Hetzner in so far that they provide me some time to migrate off their infrastructure, in case that happens.

At this point it actually is pure irony that I work for an Austrian cloud provider and don’t use any of their products, because the UI sucks. But hey, not my fault. '^^

The migration process

In order to provide a smooth transition from one provider to another without too much impact, I decided to do the migration in the following order:

Change the proxy to ask the new S3 backend first, fallback to the current
Adjust Mastodon configuration to point to the new bucket
Copy all files from the current to the new bucket

Adjust nginx to use two S3 backends for proxying

Since the media files are already proxied behind nginx, it was pretty easy to add the Hetzner S3 as one additional named location.

location @s3hetzner {
  limit_except GET {
    deny all;
  }

  resolver 8.8.8.8;
  proxy_set_header Host BUCKET_NAME.fsn1.your-objectstorage.com;
  proxy_set_header Connection '';
  proxy_set_header Authorization '';
  proxy_hide_header Set-Cookie;
  proxy_hide_header 'Access-Control-Allow-Origin';
  proxy_hide_header 'Access-Control-Allow-Methods';
  proxy_hide_header 'Access-Control-Allow-Headers';
  proxy_hide_header x-amz-id-2;
  proxy_hide_header x-amz-request-id;
  proxy_hide_header x-amz-meta-server-side-encryption;
  proxy_hide_header x-amz-server-side-encryption;
  proxy_hide_header x-amz-bucket-region;
  proxy_hide_header x-amzn-requestid;

  // Note: The following two headers might not longer be needed after the beta phase.
  proxy_hide_header x-debug-bucket;
  proxy_hide_header x-debug-backend;
  proxy_ignore_headers Set-Cookie;
  proxy_pass $s3_backend$uri; // Note for the reader: Set this to your own value.
  proxy_intercept_errors on;  // Note: change this to off as soon as the migration is done.

  // Enable proxying via TLS
  proxy_ssl_protocols TLSv1.3;
  proxy_ssl_name BUCKET_NAME.fsn1.your-objectstorage.com;
  proxy_ssl_server_name on;

  proxy_cache CACHE;
  proxy_cache_valid 200 48h;
  proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
  proxy_cache_lock on;

  expires 1y;
  add_header Cache-Control public;
  add_header 'Access-Control-Allow-Origin' '*';
  add_header X-Cache-Status $upstream_cache_status;
  add_header X-Content-Type-Options nosniff;
  add_header Content-Security-Policy "default-src 'none'; form-action 'none'";

  // Fallback to the existing backend, if there's no match.
  error_page 403 = @s3;
  error_page 404 = @s3;
}

I also had to change the default location at the top to the new location.

location / {
  try_files $uri @s3hetzner;
}

A little lesson I learned along the way

The following snippet did not work, since it looks like that always the last location is chosen. That’s a problem, since new uploads are going to be made to the new bucket in the next step.

location / {
  try_files $uri @s3 @s3hetzner;
}

Telling Mastodon to use the new S3

That step is pretty simple. Adjust the S3 settings inside the .env.production file to point to the new location and restart all Mastodon processes. For the record, I use the following values:

S3_ENABLED=true
S3_BUCKET=BUCKET_NAME
S3_ENDPOINT=https://fsn1.your-objectstorage.com
S3_ALIAS_HOST=pdn.queer.group

After I restarted Mastodon, I quickly uploaded a random image with the visibility set to private and posted it. It worked as intended, so I went on to the next step.

Mirroring the old bucket to the new onini

At this point, the functionality of Mastodon itself was no longer impacted. Existing media files were working and new ones could be uploaded as well. What remained was the migration of the remaining data.

For this I used the Minio client, known as mc on the command line. Since Lix is installed on this machine, I did it via nix-shell. This spawns a new shell with the minio-client package installed in it.

$ nix-shell -p minio-client

After that, I setup two aliases and mirrored the “folders” using mc mirror.

# Replace `old-bucket-name` and `new-bucket-name` with the actual names of your buckets.
#   -a preserves any attributes (shouldn't be needed, but better safe than sorry)
#   --retry retries the upload, in case an error occurs for whatever reason.
$ mc mirror -a --retry backblaze/old-bucket-name/accounts/ hetzner/new-bucket-name/accounts/
$ mc mirror -a --retry backblaze/old-bucket-name/backups/ hetzner/new-bucket-name/backups/
$ mc mirror -a --retry backblaze/old-bucket-name/custom_emojis/ hetzner/new-bucket-name/custom_emojis/
$ mc mirror -a --retry backblaze/old-bucket-name/images/ hetzner/new-bucket-name/images/
$ mc mirror -a --retry backblaze/old-bucket-name/media_attachments/ hetzner/new-bucket-name/media_attachments/
$ mc mirror -a --retry backblaze/old-bucket-name/site_uploads/ hetzner/new-bucket-name/site_uploads/

Those are all the smaller folders, which – depending on your instance – might only consume a couple of gigabytes. The biggest folder, the cache is likely substantially larger than the rest. In my case, it’s a whopping 250GB! Therefore I leveraged systemd-inhibit in order to prevent my notebook from sleeping.

And yes, I am using my notebook for the migration, even though it’s neither the fastest nor the most reliable solution. But I really don’t care for this one-time job to follow “industry best practices”, I just want the job to be done eventually one day.

Now I had to wait for hours and hours, until the migration was done. Eventually I wrapped it inside of systemd-run and committed a crime with this level of nesting, but hey, it works. :p

$ systemd-run -u "sync-mastodon-assets" \
  --user \
  systemd-inhibit \
  nix-shell -p minio-client \
  --run 'mc mirror backblaze/old-bucket-name/cache/ hetzner/new-bucket-name/cache/ --retry -a'

Eleventy is nice, actually

2026-06-01T21:17:25Z

It is time for a rewrite again. Why writing blog posts when you can just refactor your homepage again and again and again…

Well, this time I chose Eleventy (11ty) for it. The primary reason behind this migration was: stable releases. The previous version of this homepage was using Hugo. And although Hugo is nice, if you’re already in the Go ecosystem, the lack of a stable version really put me off this time. You know, sometimes I just want to do something on my homepage. And when I can’t, because something changed again and I have to fix my templates (again), I’ll just get distracted.

And that’s why I ditched Hugo. Now, you could ask me: why not Zola or some other static site generator that’s been statically compiled? Why 11ty, which itself is written in JavaScript?

To be honest: because it feels correct. It is the language for the web. And it’s not even fundamentally slower than Hugo. Hugo took like 1 second, 11ty takes 1.1 or so. Never measured it, because: it doesn’t matter for me.

Benefits of the migration

With the rewrite, I also took the opportunity and ditched Tailwind CSS. I wanted to refresh my CSS skills, since a lot of good stuff was added to it. I’m a big fan of the :is() selector!

Eleventy also has the ability to compute bundles of CSS/JavaScript per page. It’s plain, but it works pretty good. You can see it on the individual blog post pages, which get progressively enhanced with anchor links if JS is available.

Oh, and another big improvement: I no longer minify the outputs. I think that in the time of HTTP/2 and compression, it doesn’t add that much of a benefit. The CSS is just 2KB big, it’s not even shipped with one of those “reset stylesheets”. Go ahead, look at this smol stylesheet! It is so frickin’ simple, I love it.

With the removal of the minification, you can just open the source code of this page and learn from it! For me, that’s a much better feature than, idk, 10 milliseconds of reduced transfer time.

Still missing

The dark theme. It’s on my todo list.
The project list page: I’ve ditched it completely for now, gonna add it back later on.
Proper lang tags for German content.

Plans

One thing I really want to add to this page is some touch of personality. Right now, it feels pretty bare. I already have ideas for some eastereggs, let’s see when I actually implement them 🙈.

Anyway, that’s it for now. If you have feedback of any kind, just mention me on the fediverse or via mail.

Over and out
Jasmin

GPN22: Lektionen aus 1,5 Jahren Fediverse-Moderation und -Administration

2026-06-01T21:17:25Z

Ehrlich gesagt, kann ich immer noch nicht wirklich glauben, dass ich dieses Jahr auf der GPN einen Talk gehalten hab. Aber nun, äh, hab ich.

Jetzt steht meine Meinung im Internet. Bissel gruselig ist das ja schon. Aber nun. ^^

Der Talk ist auf media.ccc.de abrufbar. Und die Folien hab ich separat für diesen Post nochmal hochgeladen. Also, einmal die Folien als PDF, ebenfalls unter CC-BY 4.0 lizensiert.

Analysing the source code of Truth Social

2026-06-01T21:17:25Z

Truth Social is America’s “Big-Tent” social media platform that encourages an open, free, and honest global conversation without discriminating against political ideology.

After Evan Boehs got the freeze peach connosieurs of Truth Social to release their source code again, I was immediately hooked. Not because this sparks joy, but because I’m a silly curious pup. 🐶

This is more or less just a stream of thoughts, probably a bit more structured than by thoughts.

Which version are we running on?

Since I’m involved with the codebase of Mastodon from time to time (cough), I knew immediately where to look. lib/mastodon/version.rb. According to this file, truth.social is still based upon Mastodon 3.4.1. This isn’t new, but still, I’m fascinated that they haven’t incorporated at least the other changes from the stable/version-3.4 or even the stable/version-3.5 branches.

Did they adapt the source code and yes, to what extent?

I’ve continued by cloning the Mastodon source code with 3.4.1 as the tag. From there, I started to compare the Truth Social source code against the Mastodon one using my beloved delta. A simple

delta ./truthsocial/source ./mastodon-3.4.1

is doing the trick for the following report.

Integration of the Prometheus exporter

Apparently they have added the prometheus_exporter library to the dependencies. Unfortunately, it doesn’t seem to be available under https://truthsocial.com/metrics, that would’ve been too easy. According to the ./app/lib/prometheus/application_exporter.rb, they’re using it for the following metrics:

number of created statuses
number of retruths
number of replies
number of favourites
number of reports
number of blocks
number of login attempts
number of registrations
number of uploaded media files
number of accounts following account
number of accounts unfollowing accounts
number of posted links
number of approved users
number of ad impressions
number of chats
number of chat messages

Besides the custom metrics mentioned above, it is also integrated into Sidekiq, as a middleware for each request and probably some other places. So, they’re using Prometheus metrics for observability. Since New Relic is also mentioned in the code, they’re using their services for monitoring and stuff.

RSS feed mocking

Mastodon has this neat feature, where you can follow the public posts of any profile using a RSS reader. Apparently truth.social does not like this, at least they’ve added the ./app/controllers/api/mock/feeds_controller.rb whose sole work is to create empty RSS feeds.

Feeds

But they also have a ./app/controllers/api/v1/feeds_controller.rb which seems to be a different feature. And DEFAULT_FEEDS_SIZE = 18 with a value of 18 is not a dogwhistle. no no. it’s all fine. /sarcasm

Following ./db/seeds/feeds/feeds.sql, this feature is similar to the timelines of Mastodon. We have the following feeds:

Following: A chronological timeline of Truths from accounts you follow.
For You: Truths we think you’ll be interested in
Groups: A chronological timeline of Truths from the Groups you’ve joined.

iOS-/Android-specific handling

The app/controllers/api/base_controller.rb got several notable changes. Since I don’t know whether some of them we’re cherry-picked commits (and I’m too lazy to find out right now), I’ll focus on the stuff that’s definitely a change of the source code by them.

Apparently, Truth Social contains code for SMS verification of users. And some iOS/Android specific stuff for device registration.

Pleroma? Inside my Mastodon codebase?

They added code to handle Pleroma-specific API requests.

app/controllers/api/pleroma/
- accounts_controller.rb
- user_settings_controller.rb
app/controllers/api/v1/pleroma/
- chats
  - events_controller.rb
  - messages_controller.rb
  - reactions_controller.rb
  - search_controller.rb
  - silences_controller.rb
- chats_controller.rb
app/controllers/api/v2/pleroma/
- chats
  - events_controller.rb

(surprise surprise, the nazis are supporting each other. If I should name one software that yells “hello, please block me” as a Fediverse moderator, it’d be Pleroma.)

The compatibility with Pleroma is also mentioned in the app/controllers/api/v1/accounts/credentials_controller.rb. This is likely for the support of Soapbox as a frontend, also mentioned with the OAuth secret inside db/seeds.rb.

Yes, they also added advertisements, including their app/controllers/api/v4/truth/ads_controller.rb. In order to built on top of this feature, it’s also mentioned across several places within the codebase, e.g. in the app/controllers/api/v1/accounts/credentials_controller.rb.

You can also report an ad if you don’t like it. They are using the following advertisement networks:

On iOS they’re using the SkAdNetwork API from Apple, which is accessing a new endpoint in .well_known (./app/controllers/well_known/skadnetwork_controller.rb)
revcontent, referenced in ./app/workers/track_revcontent_ad_impressions_worker.rb
Rumble, which is referenced in ./app/workers/track_rumble_ad_impressions_worker.rb.

There’s also an API key(?) for one of those in ./app/views/layouts/application.html.haml in line 46.

Truth Social has a lot of TV integrations. Yes, including an EPG. They learned from the Nazi Party and their propaganda networks, that’s for sure.

Affected files/folders:

./app/controllers/api/v1/admin/tv/
./app/controllers/api/v1/tv/
./app/models/tv_account.rb
./app/models/tv_carousel.rb
./app/models/tv_channel.rb
./app/models/tv_channel_account.rb
./app/models/tv_device_session.rb
./app/models/tv_program.rb
./app/models/tv_program_status.rb
./app/models/tv_program_temporary.rb
./app/models/tv_reminder.rb
./app/models/tv_status.rb
./app/serializers/rest/tv_program_serializer.rb
./app/serializers/rest/v2/tv_carousel_serializer.rb
./app/serializers/rest/v2/tv_channel_guide_serializer.rb
./app/serializers/rest/v2/tv_program_serializer.rb
./app/services/p_tv/
./app/workers/scheduler/tv_create_program_records_scheduler.rb
./app/workers/scheduler/tv_refetch_channels_list_scheduler.rb
./app/workers/tv_accounts_create_worker.rb
./app/workers/tv_accounts_login_worker.rb
./app/workers/tv_create_tv_program_status_worker.rb
./app/workers/tv_program_reminder_notification_worker.rb
./public/tv/
./spec/controllers/api/v1/admin/tv/
./spec/controllers/api/v1/tv/
./spec/fabricators/tv_account_fabricator.rb
./spec/fabricators/tv_channel_account_fabricator.rb
./spec/fabricators/tv_channel_fabricator.rb
./spec/fabricators/tv_device_session_fabricator.rb
./spec/workers/tv_accounts_create_worker_spec.rb
./spec/workers/tv_accounts_login_worker_spec.rb
./spec/workers/tv_create_tv_program_status_worker_spec.rb

According to the ./app/serializers/rest/tv_program_serializer.rb, some of the content is hosted on https://vstream.truthsocial.com.

They’re really focusing on videos.

In the ./app/models/media_attachment.rb, they increased the maximum size for videos from 40 to 450(!) MB. Furthermore, they’re also allowing application/octet-stream as a content type for videos. This might end up in a vulnerability, but I think they’re preventing this by performing a magic byte detection in ./lib/paperclip/media_type_spoof_detector_extensions.rb

rumble.com

Speaking of videos, apparently you can also upload directly to rumble.com (is this just another YouTube clone?). The logic for that is in: ./app/services/concerns/upload_video_concern.rb

Newsmax

Newsmax, a TV broadcasting company that describes itself as “real news for real people” (wtf) is also tightly integrated into Truth Social. You can fetch the electronic program guide (EPG) directly from Truth Social, according to the ./app/services/p_tv/programme_guides/newsmax_service.rb.

One America News Network (OANN)

One America News Network, also known as One America News, is a far-right, pro-Trump cable news and political opinion commentary channel founded by Robert Herring Sr. and owned by Herring Networks, Inc., that launched on July 4, 2013. The network is headquartered in San Diego, California, and operates news bureaus in Washington, D.C., and New York City. – Wikipedia

That’s also integrated into the EPG, see ./app/services/p_tv/programme_guides/oan_service.rb.

Real America’s Voice (RAV)

EPG also integrated into Truth Social, defined in ./app/services/p_tv/programme_guides/rav_service.rb.

Real America’s Voice is a right-wing to far-right streaming, cable and satellite television channel founded in 2020 and owned by Robert J. Sigg. The network and online presences have promoted right-wing and far-right conspiracy theories, including COVID-19 misinformation, 2020 election conspiracies, and QAnon. – Wikipedia

Weather Nation (WN)

Weather Nation, whose operator is also responsible for RAV, is also integrated in ./app/services/p_tv/programme_guides/wn_service.rb.

API endpoints for follower information?

They also added controllers to return the follow-relationships or account information as JSON. Sadly, for us antifascists, it’s locked down to administrators with write(?) permissions. The controllers can be found in:

./app/controllers/api/v1/admin/accounts/follows_controller.rb
./app/controllers/api/v1/admin/accounts/statuses_controller.rb

Mass-import of accounts

Did they actually add CSV imports to import users? Apparently so. I can only assume about the reasoning behind it, but yeah, line 71 of the ./app/controllers/api/v1/admin/accounts_controller.rb actually suggests that:

if !@account.user.not_ready_for_approval? && !@account.user.ready_by_csv_import?

And when you continue, you actually realise that they built their mass-import tool to create new users. And also, to help the staff, they also added the possibility to process multiple accounts in advance (./app/controllers/api/v1/admin/bulk_account_actions_controller.rb), which only seems to be used to trigger the account validation using SMS again.

Chats

“Plan the next attack on the White House, exclusively on Truth Social!” could be a potential advertisement for this feature. Added in ./app/controllers/api/v1/admin/chat_messages_controller.rb, they’ve build a chat feature into their social network.

Their chat feature has everything a chat needs:

reactions
a functioning search
read receipts

In the background they’re using a RabbitMQ queue system (./app/lib/events/event_bus.rb) to support that, the chats itself are stored in the database. Chat messages are limited to 500 characters.

Groups

They have groups, just like Facebook. It actually seems to support a lot of stuff, including but not limited to:

invites
promoting users
kicking out users
group-specific timelines
group-only posts

Moderation

Blocked links

Truth Social has a pattern-matching based ability to block links in ./app/controllers/api/v1/admin/links_controller.rb.

Moderation actions

Besides the already existing moderation tools of Mastodon, Truth Social has the following moderation actions:

add/remove the “sensitivity” of a post (probably the same as the media sensitivity in Mastodon?)
The ability to change the privacy level of a post from public to private and vice-versa.

Those got added in ./app/controllers/api/v1/admin/statuses_controller.rb. I mean, it’s not federating and such, still: fascinating that they added moderation features that are (partially) not even in Mastodon itself.

Rules

According to ./config/locales/en.yml, Truth Social actually has rules. Content or accounts that does not comply with with the following rules, seems to get deleted:

Illegal activity and behavior: Content that depicts illegal or criminal acts, threats of violence.
Intellectual property infringement: Impersonating another account or business, infringing on intellectual property rights.
Sensitive content: Depictions of violence, gore, nudity.
Underage content: Sexually explicit content involving underage children.
Prostitution: Solicitation or advertising for illegal sexual activity or sex for hire.
Privacy violations: Violate or post content that violates a person’s privacy rights.
Illegal sales: Sale of or promotion of illegal drugs, counterfeit services and goods, or illegal products and services.
Doxxing: Sharing or threatening to share the private information of an individual without their consent or breach of privacy rights of others.
Spam: Fraudulent or malicious content or links, inauthentic engagement, repetitive replies, ReTruths, or direct messages.
Troll: This user is a troll and/or I dislike their content.

“ModerationAI”

Apparently, they’re also using so-called “AI” named “ModerationAI” for moderation:

We use artificial intelligence (AI) to assist our hardworking moderators, and some Truths are flagged for deletion or marked “sensitive” by AI. While the AI we use is very good, it is not error-proof. Assisted by technology, our moderators use their best judgment to ensure compliance with our Terms of Service. Please give our team time to review your Truth to determine whether it violates our Terms of Service. After a thorough review, we will reinstate the Truth or uphold its removal.

See also: ./lib/tasks/generate_moderator_ai_admin.rake.

Federation

Truth Social has removed most of the ActivityPub (AP)-related code. The inbox/outbox controllers got removed, actions are passed the skip_activitypub: true parameter if possible, et cetera. All references to Mastodon are either replaced with “Truth Social” or just “Truth” or just removed. This includes:

links to the documentation
the instance behind a profile (so donald@truthsocial.com just becomes donald)
renaming the “Toot” button to “Truth” (lol, lmao even)
removing features like:
- polls,
- content warnings,
- post privacy buttons,
- bookmarks
The whole app/services/activitypub just got cleaned of all the federation features, it’s only used for fetching remote content now.

please don’t block us from scraping your content 🥺

For outgoing requests, they’re using a proxy by default, as seen in ./app/lib/request.rb. Also, they are hiding that they’re Truth Social by omitting the instance name from the default user agent. Usually, Mastodon requests content with a user agent that contains the following substring (Mastodon/1.2.3, https://example.com) with 1.2.3 being the version and https://example.com the public URL of the instance.

They removed that in ./lib/mastodon/version.rb, which also means that website administrators who want to block Truth Social cannot do that with just the user agent.

Other stuff

They renamed the “toot” button to “truth”.
They renamed the “reblog” button to “retruth”.
Instead of stars there’s a fire emoji for likes/stars.
The character limit is 1000 characters.

OpenAPI documentation (for authenticated users)

They integrated Swagger/OpenAPI into their application. If you have an account on Truth Social, you can open the API documentation of Truth Social, this got added via the ./app/controllers/apidocs_controller.rb.

Deployment via GitLab and Dokku

In the ./app.json, Dokku is mentioned. Looking further, they’re also using the GitLab CI and a Dokku instance hosted at dokku.tmediatech.io, which is mentioned in the ./bin/ci-pre-deploy.

Potential SQL vulnerabilities

They also commited a (potentially outdated) brakeman report under ./brakeman-output.json, which was last executed on July 17th 2023 in /Users/markmorales/Code/social-v1_groups. Said scan mentions a lot of potential SQL injections.

Removal of the code of conduct

Not surprising, but they removed the code of conduct. Spreading hate is far easier without one, I get that.

Summary

It’s pretty obvious that they’re building their own echo chamber. Full of far-right content, lots and lots of video content and even though they took Mastodon as their codebase, they’re not interested in federation. They probably just want to benefit from all the apps/clients that are built for that.

Eure Prüderie kotzt mich an.

2026-06-01T21:17:25Z

Inhaltswarnung: Sex, vulgäre Sprache

Hallo liebe Hetzner-Menschis,

ich hab in den letzten zwei Tagen ein bisschen mehr mit euch geschrieben. Und eigentlich hab ich auch immer sehr viel von euch gehalten.

In den letzten fünf Jahren hatte ich nie ein technisches Problem mit eurer Infrastruktur. Im Gegenteil: sie lief und lief und lief. So gesehen, müsste ich die glücklichste Kund*in sein. Tatsächlich hatte ich mich sogar bei euch beworben, aber das ist dann leider nichts geworden.

Aber in den letzten beiden Tagen habt ihr mich doch enttäuscht. Technisch ist alles weiterhin bestens, daran liegt’s nicht. Eher so das menschliche, you know, eure allgemeinen Geschäftsbedingungen und so. Vielleicht bin ich auch selber schuld, dass ich da im Voraus nicht nachgefragt hab, wie eure AGB auszulegen sind.

Es ist richtig, dass pornografische Inhalte von Sexarbeitenden, auch wenn sie volljährig sind und alle Inhalte mit Einverständnis für Produktion und Reproduktion erstellt wurden, gegen unsere AGB verstoßen und daher auf unseren Servern nicht erwünscht sind. Die Entscheidung, bei dem Verbot pornografischer Inhalte über die Wertungen des strafrechtlich erlaubten hinaus auch legale Formen der Pornografie nicht zuzulassen ist eine Grundsatzentscheidung der Unternehmensleitung. Deshalb sind unsere AGB insofern auch weitergehend als die gesetzlichen Regelungen. Zitat aus der E-Mail der Hetzner-Rechtsabteilung

Anyway. Eure Prüderie kotzt mich an. Ich versteh ja, dass ihr bei Beschwerden von außen reagieren müsst. Und sobald wir von der Darstellung Minderjähriger sprechen, sind die Regeln da auf lokaler und internationaler Ebene sehr streng. Versteh ich. Aber Pornografie ist nun eben ein sehr weites Spektrum. Erotik auch. Kann so richtig auch niemand definieren. Aber warum ist eure Geschäftsführung da so prüde? Warum habt ihr mit Pornografie so ein Problem? Ich ahne es. Es ist fürs Image, ne? Damit neben super-mega-startup.de nicht gleich noch tittenbericht24.com liegt. jaja, böse titten.

Wisst ihr nur, warum das schade ist? Weil für mich als queere Person die Nacktheit und damit auch im weiteren Sinne die Pornografie ein Akt der Selbstbestimmung ist. Es ist mein Körper, es sind meine Bilder, alleine die Produktion dieser Bilder ist schon eine Auflehnung gegen die gesellschaftliche Mehrheit. Gewissermaßen ein Akt der Rebellion. Das System fickt uns, also ficken wir zurück.

Lasst mich doch meine Selbstbestimmung feiern. Ob nun im Bikini oder oberkörperfrei, solange keine gesetzlichen Grenzen überschritten werden, seh ich keinen Handlungsbedarf. Aber naja. In einem Internet, in dem jedes bisschen Nacktheit direkt zur Sperrung führt, reiht ihr euch leider nur mit ein. Bin ich überrascht? Nicht wirklich. Aber enttäuscht.

Ich hab nicht immer Lust, nur auf den Tag zu warten, wo ich auf der Abschlussliste stehe und dann von irgendwelchen Meldesystemen vertrieben werde. Ich würd mich gern auch einfach mal so entfalten dürfen. Brustwachstum statt Wirtschaftswachstum feiern.

Werd ich auch. Aber dann wohl nicht mehr auf eurer Infrastruktur.

Schade Schokolade.

Goodbye Arch, hello Fedora

2026-06-01T21:17:25Z

How do you know that you’re getting older? Easy: installing updates is just another chore and does not cause excitement anymore. And that’s the reason why I replaced Arch on both of my computers. My current device, a ThinkPad T460 (with the lovely hostname twinkpad) was the last device which was running Arch.

Why I switched away

Even though I used Arch for years now, I decided to switch. The primary reason is: I’m not using my TP that often. Maybe once or twice a week. And with each usage, I ran the obligatory paru && poweroff before I put it away. And that turned out to be a larger chore over the last couple of weeks. Reviewing the PKGBUILD files, waiting for the packages to compile (even tho I tried to use the *-bin packages wherever possible), I got annoyed by almost everything.

And so I decided to replace the operating system. Since my dotfiles are already managed via Git, I did not have too many files to backup. Just the documents, some downloads and the repositories. Most of the stuff was backed up anyway, but eh, better be safe. rsync was a massive help here.

Hello there, Fedora Kinoite :3

For my next system, I chose Fedora Kinoite. There are multiple reasons for that:

I’m already familiar with Fedora. Fedora Workstation is installed on my work laptop, Fedora Server on my homeserver.
It’s immutable and still easy to understand.
KDE. ❤️
✨ shiny new stuff ✨
hopefully less work with updates.

I could’ve chosen Fedora Workstation, but I really wanted to explore the “immutable” part of it. “Immutable” might not be the correct term, indeed, “image based” might be better. I still have my mutable filesystems. But the system and all it’s packages are no longer mutable by default. This is different from my existing devices which all use a “classic” mutable operating system. I really wanted to try out NixOS, but to be honest: learning Nix to manage my system is out of my scope. Maybe if I have enough time, I might look again into it.

But the biggest chore, the software updates, are a breeze with that setup. I can use Discover, the software manager of KDE to update the whole system with one click and it’s automatically applied on the next reboot. And even if something breaks, I can still access the previous state and therefore (at least in theory) always end up with a working system.

Using toolbx

The recommended way on immutable systems like Fedora Kinoite is to use toolbox. For those who are unfamiliar with that, I’ll quote a part of their project description:

Toolbx environments have seamless access to the user’s home directory, the Wayland and X11 sockets, networking (including Avahi), removable devices (like USB sticks), systemd journal, SSH agent, D-Bus, ulimits, /dev and the udev database, etc…

And so I was exploring with that as well, only to realize pretty quickly that I want a custom container image for that. For example, I use fish as my primary shell instead of bash, kitty as my terminal emulator (and therefore kitty-terminfo should be installed) and so on. That’s how my custom container image was born. I also configured toolbox to automatically use this image for new containers.

I’m not sure whether I want to setup a toolbox environment per project (maybe in combination with direnv) or if I want to use one container for everything. Only time can tell, I guess. 😄

The future

I’m definitely curious about the long-term simplicity of this setup. If everything goes well, this setup might last for years with little to none maintenance work. If not, well, then I’ll replace the OS again. easy (and annoying) as that. ^^

Jasmin wrapped

2026-06-01T21:17:25Z

Inspiriert von Jascha, dachte ich mir, jo, Jahresrückblick dieses Jahr wäre mal nice. Ich hab für sowas in der Vergangenheit halt einfach die sozialen Medien (sprich damals Twitter, heute das Fediverse) genutzt, aber doch realisiert, dass dort einiges einfach untergeht. Und daher, mach ich das diesmal hier.

Zusammenfassend war dieses Jahr wirklich nur von Turbulenzen geprägt. Ich bin das zu einem gewissen Grad durchaus gewohnt und komme relativ gut damit klar, nur dieses Jahr ist es wirklich sehr viel gewesen. Es sind sehr viele gute Dinge passiert und ich für meinen Teil bin insbesondere mit der Entwicklung meiner Persönlichkeit sehr zufrieden. Die Therapie, die dann in diesem Jahr nach zwei Jahren ihr Ende nahm und mein Rückblick auf einige meiner getätigten Postings im Vergangenheit haben mir gezeigt, dass ich da sehr gut Fortschritte gemacht habe.

Haare länger oder kürzer?

Kürzer. Undercut for the underdog. Oder so. Es tut auf jeden Fall gut, mehr Androgynität im Stil zu haben.

Haare gefärbt?

Ich glaub, drei mal. Am besten gefallen hat mir das floureszierende Neongrün, welches ich auch auf der GPN getragen habe. Leuchtende Haare sind einfach bombe, das wird 2024 definitiv wiederholt.

Mehr Kohle oder weniger?

Neuer Job, neues Glück. Ich habe dem öffentlichen Dienst den Rücken zugewandt und einen Job gefunden, der genügend Geld einbringt, um in der Theorie drei Menschen zu versorgen.

Mehr ausgegeben oder weniger?

Kann ich nicht sagen, ich führe keine Buchhaltung. Aber immerhin ist der Stromverbrauch dieses Jahr gesunken. Auf der Liste des Erwachsenseins ist das wohl eines der Dinge, über das ich mich echt freue.

Vermutlich aber durchaus mehr Geld ausgegeben, 2023 wurden nicht selten die Haare gefärbt. Von den drei Piercings im Ohr mal ganz abgesehen. Ups.

Mehr bewegt oder weniger?

Leider deutlich weniger. Ich hatte in 2022 wenigstens noch regelmäßig einen Spaziergang drin, das ist 2023 deutlich weniger geworden. Homeoffice in Vollzeit trägt sein übriges dazu bei, mein iPhone berichtet mir so im Schnitt an die 500 Schritte täglich.

Das ist viel zu wenig, daher steht für 2024 deutlich mehr Bewegung auf dem Plan. Werde mich wohl mal wieder häufiger aufs Rad schwingen.

Der hirnrissigste Plan?

Meine Verlobte dazu ermutigen, den bisherigen Job und damit das gesamte Haushaltseinkommen an den Nagel zu hängen. Rückblickednd war es eine sehr gute Entscheidung, nur eben auch mit sehr viel Risiko verbunden.

Die gefährlichste Unternehmung?

2023 war komplett ohne gefährliche Unternehmungen und wenn es nach mir geht, darf das genau so bleiben.

Die teuerste Anschaffung?

Ich bin in der Regel sehr zurückhaltend was Anschaffungen angeht. Vermutlich ändert sich auch das nächstes Jahr, da plane ich mal, den Kleiderschrank auszumisten und zu erweitern.

Das beeindruckendste Buch?

Ich bin es noch am Lesen, da ich dieses Jahr leider nicht viel gelesen hab. Aber Identitätskrise von Alice Hasters dürfte mein Favorit des Jahres werden.

Der ergreifendste Film?

Auch da führe ich keine Statistik, weiß aber, dass ich vermutlich bei Elemental mehrfach pausieren und heulen musste. Aus Gründen.

Die meiste Zeit verbracht mit …?

Meinen Verlobten bzw. meiner Verlobten und meiner Ex-Verlobten.

Die schönste Zeit verbracht mit …?

Meinen Verlobten bzw. meiner Verlobten und meiner Ex-Verlobten.

Vorherrschendes Gefühl 2023?

Uff. Schwere Frage. Ich denke: Zuversicht. Insgesamt ist es doch die Zuversicht, die nach der Depression zurückkehrte.

2023 zum ersten Mal getan?

Eine Beziehung beenden müssen. Leider.

2023 nach langer Zeit wieder getan?

Mich unter Menschen begeben. Das waren in 2023 die Easterhegg, die GPN und die MRMCD und zum Jahresabschluss die Weihnachtsfeier auf der Arbeit.

Das hat jedoch auch sehr viele Nerven gekostet. Insbesondere musste ich lernen, wann und wie:

die Überreizung stattfindet,
ich entsprechend meine Bedürfnisse kommunizieren kann,
und all dies auch rechtzeitig zu erkennen.

Die Weihnachtsfeier mit den knapp 400 Menschen und der lauten Musik war somit auch ein wenig der Stresstest (quite literally), wie gut ich damit klarkomme. Und im Gegensatz zu den Chaos-Events davor auch ohne Begleitung in Form meiner (Ex)-Partner*innen. Ich habe ehrlich gesagt nicht damit gerechnet, insgesamt so gut zurecht zu kommen, auch wenn ich an der Früherkennung für Überreizung noch arbeiten muss.

Aber für den Fall hab ich inzwischen auch immer ein paar Ohrstöpsel dabei, das sollte beim nächsten Mal helfen. :3

Drei Dinge, auf die ich gut hätte verzichten können?

Das abrupte Ende der Beziehungen.
Die COVID-Infektionen, die ich gerne vermieden hätte, aber nun, iat halt wirklich ungünstig gelaufen.
Der Stress, den die Jobsuche so mit sich bringt. Ich bin für die unehrliche Selbstvermarktung nicht geschaffen.

Das schönste Geschenk, das mir jemand gemacht hat?

Es gibt nicht das schönste Geschenk für mich. Das größte Geschenk, und ich weiß wie kitschig das klingt, sind meine Partner*in und Freund*innen. Und ganz egal ob es die Behausung oder eine Umarmung ist, ich bin dankbar für all die Liebe und gemeinsame Zeit, die ich mit euch verbringen darf. 💜

Überraschendstes Ereignis?

Die Wahl des Awareness-Teams. Ich hab ehrlich nicht damit gerechnet, dass es einerseits diese Zustimmung bekommen würde und zeitgleich auch mir als Person so viel Vertrauen entgegengebracht wird.

Und ohne zu sehr ins Detail zu gehen: es war anscheinend auch bitter nötig.

2023 war mit einem Wort… ?

Kräftezehrend.

Pläne für 2024?

Heiraten. Aus Liebe und steuerlichen Gründen.
Mehr, beziehungsweise überhaupt mal wieder Rad fahren bzw. sportlich betätigen.
Den noch ausstehenden Konflikt mit meiner Ex-Partnerin abschließen.

Besonders dankbar für…

Wie eben schon erwähnt, bin ich allgemein sehr dankbar für alle meine Freund*innen. Dennoch möchte ich einen besonderen Dank aussprechen, an (Sortierung alphabetisch):

derf: Wir haben dieses Jahr einiges an Zeit verbringen dürfen, sei es im Chaosdorf oder beim gemeinsamen Waffeln futtern. Du bist eine super liebe Freundin und ich bin sehr dankbar, dass du mich das ein oder andere Mal dann doch liebevoll zurückgepfiffen hast, wenn ich mal wieder über die Stränge gesprungen bin.
Lari: Unsere langjährige Beziehung wär dieses Jahr fast zersprungen. Ich bin dir unglaublich dankbar für das entgegengebrachte Vertrauen, für die Unterstützung und all die gemeinsamen Momente. Und ich bin sehr zuversichtlich, dass die kommenden Jahre weniger stressig werden, als es 2023 noch war.
maximemelian: Du warst für mich da, als ich es am Dringendsten brauchte. Danke. Ich werde dir das nie vergessen. Und natürlich bist du abseits davon auch eine absolut wunderbare Person und ich habe jeden einzelnen Moment mit dir genossen.
Tessa: Auch wenn unsere Beziehung in 2023 begonnen hat und wieder endete: Danke für alles. Jeder einzelne Moment mit dir war wunderschön. Ich wünschte rückblickend, dass ich dir einen Teil der Achterbahn hätte ersparen können, aber ich hab leider erst durch die Beziehung mit dir gelernt, dass Fernbeziehungen nichts für mich sind. Tut mir leid. Ich freu mich schon auf all die Momente, die wir in 2024 dann verbringen können.
Vivien: Leider gab es zwischen uns im vergangenen Jahr sehr viele Konflikte, von denen ich mir wünschte, dass sie nicht existiert hätten. Aber wir haben auch sehr viele schöne Momente gehabt, an die ich sehr gerne zurückdenke. Und für die möchte ich mich bei dir bedanken. 2024 wird bestimmt auch wieder welche davon mit sich bringen, die Zuversicht meinerseits besteht zumindest.

The promise of the fediverse hasn't been fulfilled yet

2026-06-01T21:17:25Z

After thinking about starting yet another project (as if I don’t have enough ideas already) and what could be improved, I fell into a rabbit hole of ideas. This blog post is the result of that.

Which promise, Jasmin?

What do I mean with “the promise of the fediverse”? Basically it’s the idea that because of ActivityPub and interoperability, we can now each use a different software to interact with others. If you are on Mastodon, you can follow your friends on Pixelfed, know what they’re reading on Bookwyrm and so on. You get the idea.

The thing is: social media on the fediverse is dominated by Mastodon. Which itself isn’t that bad, in theory, it’s just a simple software which could be replaced with something better. And we have ActivityPub, so all those interactions from Misskey, GoToSocial, are still visible to Mastodon users and vice-versa. Unfortunately, this leads to one thing: literally everything has to be compatible with Mastodon. Not because it’d be necessary, but Mastodon itself is seen as the fediverse and therefore it’s expected that I can interact with everything from my Mastodon account. (On a technical note, everything ends up being a Note in ActivityPub.)

And to be honest: this isn’t social. Social media, as we collectively name the different platforms, became somewhat “antisocial”. Mastodon itself is just Twitter, decentralised. The idea of Twitter itself is “create an account, connect with others (friends, corporations) and tweet sometimes”. It was fun back then. Limited in expression, due to the 140 character limit. Later that got expanded to 280 chars and that was maybe the best thing that could’ve happened, at least to me. I found a lot of friends, I could connect with others without promoting myself all the time. I was just a username, a profile, and a short bio. I still love that idea and I do get why Mastodon copied it.

The fediverse just lacks one thing: corporations. For me and others, this is a selling point. No ads, no shady business scams. (Well, they do exist, but as an admin, I can block them) But Mastodon itself is built on the idea that corporations might use it one day. And therefore even the fun things like the “toot” button got lost. The color scheme is no longer blue, but blurple. I kid you not, that’s the “official” name inside the source code.

Everything is blurple or “neutral” today. Discord and Mastodon are blue-purple-ish, Twitter is just black and white (if fascism would be a color, Twitter would use it) and the rest of the internet has the same corporate color scheme. Sometimes the changes are good, like for the Wikipedia redesign in 2022, but most of the time, I just feel disconnected. The thing where the internet nowadays fails: connecting with people and friends. Nowadays there’s just @handle@platform.tld as a way to use social media. This doesn’t feel right. I am one body, but I have a lot of personas (well, I’m plural, but that doesn’t matter here). For some I am Jasmin, their coworker, for others, I’m Jasmin the good friend and for others I’m the kinky trans femme from the neighbourhood. On Mastodon, and I use it as an example, because I use it, I’m all of that at the same time. I cannot choose how to interact with others and what others should see of me. In terms of plurality, I cannot even properly let the other headmates front. That sucks.

I am 25 years old. I never used Myspace and for some reason, it still exists after 20 years. No one I know uses it. I used Facebook in the past, as it was the lowest common denominator to exchange homework back then. I started using Twitter in the beginning of 2013. There’s even a forum post (that I won’t link to, cause it contains my deadname) from January 6th, 2013 where I asked why Twitter was so popular. I didn’t get the idea back then. We all know what happened to Facebook: it became the social media platform for boomers, right-wing extremists and so on. Renaming the platform to “Meta” and imagining the “Metaverse” as some kind of VRChat without furries didn’t help avoiding the collapse.

But I have to admit, Facebook did one thing good: You could actually connect with friends. You could chat with them, plan events, exchange stuff in groups. To my knowledge, no other social media platform has tried to fill that gap. Facebook is not cool, so you don’t copy it. Of course, because of ads and stuff you could do a lot of other stuff as well, like playing little flash games and competing with your friends. (Nowadays, Discord fills in that gap.)

Post privacy options and Google+

Then there was Google+. I used it and do you know what? It was the first place in the web where I felt home. Facebook always felt like a competition (who has the most friends, who changed their relationship status and when) and on Twitter I felt like a stranger. I was just there, shy and not knowing where I belong. Google+ only had some big corporate accounts and “tech influencers” as I like to call it, but the idea of circles never came back. The idea is simple: instead of having just one profile, you could decide how others perceive you. You could post to the “friends” circle that you want to hangout, immediately followed up by a post to the “wider friends” circle that you like the weather today. Or something like that.

Today, at least on Mastodon, you only have four options for the visibility of your posts:

Public: everyone can read it.
Unlisted: everyone can read it, it just doesn’t appear in the federated or local timeline.
Followers-only: only people who follow you or those who have been mentioned can read it.
Direct: only mentioned people can read it.

That doesn’t reflect any of the personas I mentioned earlier. For example, if I wanna post nudes on the fediverse, I can safely do that. But I cannot exclude that stuff from minors, even if I want to. Either I create a second NSFW-only account or I just don’t post it and hope it’ll be handled by them properly. But I know how I felt as a minor on the web and I loved it when others interacted with me, because I felt valued. Seen as a person and not treated like a child. I tend to forget that, and this blog post from hellbeast, where pup described pups own experience, reminded me of my own.

I think we all know toots/posts/images etc. that we’re not comfortable to share with one group or another for whatever reason, nudity and sexual imagery is just one of them.

Other things that suck right now

The good old calendar scheduling problem

I miss the Microsoft Exchange functionality in Mastodon. I am not joking. Have you tried to meet with friends in 2023? Either you all have their phone numbers or you decide on a chat platform, which usually is Discord or Matrix in my circles, to collectively decide on a date. And everyone has to put it in their own calendar. The .ics format is as old as me, come on, at least a little calendar subscription should be doable.

I can plan events in Discord, I could do that in Facebook back then. I cannot do it with a fediverse software right now. Even if I’d just use it for events like the 37c3 in Hamburg, a little meet up of friends, whatever, I miss that. In the microblogging world, things like that are out of scope, for a good reason. It just has nothing to do with blogging. I can write “I’m at the grass at GPN” and that’s one way to connect, but if wanna meet up for a monthly potluck or something like that, I have to coordinate that somewhere else.

Moderation and federation

Disclaimer: I’m administrator and also the sole moderator of queer.group. I can only speak from my experience as the admin, I haven’t experienced that topic purely from an users perspective.

If you are not aware with federation, let’s provide a quick glance at what’s possible with Mastodon right now:

Silencing an instance. This has the effect that posts from there won’t be visible to you, unless you follow the respective accounts.
Suspending an instance. Communication is impossible now, they won’t see your posts (with AUTHORIZED_FETCH turned on) and you won’t see theirs.

I could write a blog post about why the federation model is good and bad at the same time, but eh, not today. I’ll try to focus on the actual effect of these options for now. And turns out, both options suck.

Silencing sucks, because half of the timeline suddenly is not being fetched/rendered (in case you open a thread for example) and you don’t even have the “Load more” button like e.g. on Twitter. You know the stuff is there, you just need to open it on their web page. In the best case, you can read the thread and be happy, in the worst case you also see the other problematic instances and might have a bad day then.

Suspending is better, because at least it doesn’t hide things from you or suggests something to exist, nope, it’s completely erased from the instance. At least it feels like that. The problem occurs with bigger instances, where I as the moderator have to choose between cutting an enormous amount of connections or to protect minorities from racist/ableist/… attacks. Personally, I tend to choose the latter, as it’s possible for the so-called good accounts to switch their instance.

Account portability and the ephemeral nature of posts

At least in theory. Practically, switching an instance is not that easy. There are migration tools inside Mastodon and in theory, you won’t lose the relationships to the accounts. But all your posts, pics, etc. they won’t get moved. Personally, I don’t mind. “We” (as mods) tell others that their posts are ephemeral, they shouldn’t treat the Mastodon instance like some kind of archive. It was never made for that.

But: We all use social media platforms in a different way. Maybe you appreciate exactly that part of posting, maybe you use the fediverse as a diary. Sure, I can say “start a blog”, but honestly: I cannot expect others to start a blog when I complain that the web got so hard to understand at the same time. Back then, when I tried to setup a WordPress instance (10 years ago or something like that) it was just:

extract this zip folder
open FileZilla
copy it to your host

Theoretically, WordPress still can be installed this way, you just need to be quick enough with the setup. “Host your own fediverse instance” - oh boy, I wouldn’t recommend that. Sure, doing fun with computers can be fun, but you need to learn a lot of concepts. Install this, install that, ensure a firewall for this, create backups, and so on. It’s too complicated. You naturally need to rely on someone else to do this stuff for you. That’s just how the society works. Everyone in this little society has their abilities and skills and we help and support each other. I do computer stuff, you can do woodworking and together we help the disabled person who can’t do any of those things. That’s how I imagine a healthy society. (Side note: healthy in the sense of collaboration, not in some eugenic way)

Back to the topic of account portability: There’s no way of moving the account to a different instance right now. It’s not even possible to switch the domain, for example from exampl.com to example.com without setting up a new instance. And it doesn’t has to be this way.

Instances don’t technically exist: https://www.w3.org/TR/activitypub/#server-to-server-interactions – ThisIsMissEm

We do not have to focus around instances. Mastodon decided to do so and everyone else followed. Likely it’s not even possible to revert some of the decisions that were made back then.

So, what do you want, Jasmin?

Maoam. Jokes aside, I want a social media platform (everything’s a platform nowadays, but that’s a rant for another day) which is built around people, not accounts. I want to:

Connect with my friends, knowing what they’re up to.
Want to post my silly thoughts into the fediverse
Want to connect in smaller groups (where everything I post is only visible to that group)
Chat with people occasionally without the need to switch to Discord, Matrix or something else. For things like “hey @ <10 people>, wanna meet at 9pm in ” and stuff like that.
Schedule events and see what’s going on in my region
Use different personas for different things, even if connected to the same account, basically plurality-driven development.
Essentially, I want to get away from instances as part of your online identity. I’m Jasmin, not @jasmin@queer.group. The instance should matter for moderation stuff, but it shouldn’t be part of the identity.

So, you just want Facebook, but federated?

Probably, yes.

Is this even possible?

I haven’t looked enough into the ActivityPub spec into what’s possible and what’s not possible. Maybe there are limitations, but if there are any, they probably can also be removed. We, as creatures on this planet, are more than a collection of Note objects. The software should be built around us.

What about the safety of minorities?

I’d actually love this idea to be “allow-list by default”. Right now, if you open up an instance, it just federates with everyone. Nazis, TERFs, all kind of douchebags. Tools, like thebad.space exist to keep the worst actors out and yet: if you don’t actively moderate your instance, you’ll end up with a bunch of threatening people in your mentions, it’s just a matter of time.

If the approach would be inverted and therefore limited by default, this problem could be avoided, maybe coming with a bunch of a lot of other problems. As I said, this whole blog post is just a collection of thoughts.

But: what if we take the concept of mutual trust and apply it to the fediverse? What if I could add you as a friend and with a little “I trust you” checkmark, I just “subscribe” to your list of allowed accounts/instances? I don’t need to be friends with any of them, but when I used Twitter, I found this information (“X and 37 others of people you follow, also follow Y”) very valuable. We also used BlockTogether there, to effectively subscribe to the list of blocked accounts by our friends. Even tho Twitter could’ve been a shithole, we built our community there. We took care of each other as good as possible.

The fediverse isn’t there yet.

Update (2023-12-29)

After posting this blog post to the fediverse, I got a lot of feedback. And maybe, the little project called Letterbook is going to be the Fediverse solution I desperately seek. Thanks to Jennifer for the exchange!

I cannot promise anything, but I plan to contribute to the project as well. :3

Mastodon on Debian 12: libssl.so.1.1: cannot open shared object file

2026-06-01T21:17:25Z

Recently, I’ve migrated our Mastodon instance from Debian 11 to Debian 12. Although it wasn’t necessary, because Debian 11 still gets a lot of security updates, I wanted to do it. New software always good.

And everything worked like a charm. I just followed the Debian upgrade guide, restarted the server and got happy that Mastodon was still working like a charm.

Well, libretranslate, which we’ve installed via pip, was no longer working due to PEP-668, but that was mentioned in the release notes, so I was prepared.

The Debian provided python3 interpreter packages (python3.11 and pypy3) are now marked as being externally-managed, following PEP-668. The version of python3-pip provided in Debian follows this, and will refuse to manually install packages on Debian’s python interpreters, unless the –break-system-packages option is specified.

So, everything went smoothly? Nope.

Meet dynamic linking, my beloved.

On Saturday, I got alerted by the monitoring system. The weekly scheduled task to clean up the Mastodon media usage was failing with the following message:

libssl.so.1.1: cannot open shared object file

The funny thing: the file existed. So, I guess, something something kernel update or so. Tried several solutions, including bundle install --redownload. Which presented me with even funnier error messages.

Bundler::GemNotFound: Could not find rake-13.0.6.gem for installation

# ... (omitted for brevity)

An error occurred while installing rake (13.0.6), and Bundler cannot continue.

In Gemfile:
devise-two-factor was resolved to 4.0.2, which depends on
    devise was resolved to 4.8.1, which depends on
        responders was resolved to 3.0.1, which depends on
            railties was resolved to 6.1.7.4, which depends on
                rake

At this point I was just like: “yeah, love you too. deletes your vendor folder” And that actually was the solution:

$ sudo su - mastodon    # switch to Mastodon user
$ cd live               # go inside the installation folder
$ rm -rf vendor         # delete the installed packages
$ bundle install        # install all packages again

So, yeah, if you are experiencing the above errors as well, just reinstall your dependencies.

Translating things with a comma? Too hard for this language model.

2026-06-01T21:17:25Z

Sometimes I don’t understand computers. Take, for example, the following sentence.

sorry for the short outage, got distracted during the update :blobcat_dead:

How hard it could be to translate it? Obviously very hard. The translation is just the emoji.

You know what’s even funnier? If you remove the comma, everything works. You could even place the comma at a totally different location and the translation still works. Yes, I’ve tested it.

But sure, “AI” is gonna save the world. /i

(not blaming the developers here, they’re doing their best. but it’s still fascinating tho.)

Meine erste GPN

2026-06-01T21:17:25Z

Letztes Wochenende hab ich meine erste Gulaschprogrammiernacht (GPN) besucht. Bereits im Vorfeld zeigte sich, dass das indirekte Motto der Gulash pride night eher zutreffen könnte und für mich war es das auch.

Ein Wochenende, in dem es für mich darum ging, nette Menschen kennenzulernen. Und das war so verdammt befreiend. Bis auf einen Zwischenfall gab es keine nervigen Typen, kein Geglotze, afaik keinen fefe und keine misogynen Kackbratzen. Oder sie haben sich einfach sehr unauffällig verhalten.

Es war sehr schön und sehr bunt und sehr queer. Und ich hab meine ersten Furries kuscheln dürfen 🥺

Danke. Danke für diese Erlebnisse. Für all die netten Gespräche. Für das Kuscheln. Für den gegenseitigen Respekt. Danke, dass es ein Event gibt, in dem ich einfach auch ich selbst sein kann. Ganz egal, ob das gerade bedeutet, dass ich meine Ruhe brauche oder halb nackt (innerhalb des Rahmens) über das Gelände hüpfe.

Kurzer COVID-Einwurf zum Schluss

Ich hätte mir aber dennoch gewünscht, dass deutlich mehr Menschen Maske tragen. Die Anzahl an Maskenträger*innen war leider echt unterirdisch. Ich kann ja verstehen, dass das zum Essen unpraktisch ist. Und auch in der Küche, wo es warm ist, kann das eventuell nervig sein. Aber dennoch, Leute, bisschen Respekt und so wäre super.

Oder wenigstens irgendwie Schutzkonzepte in Form von einer PCR-Test-Station oder halt, irgendwas. Hoffen wir einfach, dass meine Tests weiterhin negativ bleiben.

How I fixed my ThinkPad's display

2026-06-01T21:17:25Z

Sometimes it’s the little things that matter. For those who don’t follow me on Mastodon, the display on my older ThinkPad T460 broke. It didn’t even fell or something like that, it was just broken within a minute.

I mean, I could’ve still used it. But if one quarter of your screen is flickering, that’s annoying. And I hate annoying things. A lot. So I wanted to fix it. Luckily, there’s an iFixit guide for the LCD replacement.

Can’t be that hard to follow a guide, eh?

The first disassembly

(insert a dis ass can fit so many embly joke here)

Sorry, had to do it. When I first followed the guide, I had the hope that it was just a problem with the contacts. That I just need to clean the contacts and everything will be working after that. Turns out, I was wrong. I’ve disassambled the whole part, only to find out that it didn’t improve anything. It didn’t got worse, so at least I haven’t broken something. One of the fears that you have when you don’t this every day 😅

Anyway, I asked the Matrix room of the local chaos community. If someone uses ThinkPads, then definitely hackers. And catgirls, puppygirls, foxgirls, you name it. I got forwarded to the Xelent store. The display that I’ve linked is actually better than the default. If I were to buy a new display, at least I can get an upgrade as well, eh?

That’s what I did. The shop is based in Germany, so a plus for customer right protections. If it wouldn’t have fit properly, I could send it back. But I’ve checked the number of pins and even peeked under the base cover of my partners T470 to be extra sure.

It took quite a while, around three weeks. But one day it arrived, hooray!

The actual replacement

Actually, having a new display in front of me was quite scary. It’s a lot of money that could be destroyed with a simple mistake. I was really really careful with the disassembly and all the screws. I really wanted this to succeed.

So I followed the above guide again, step by step. With dramatic music in the background and no lights whatsoever, I tried to fix it. Of course, I’m exaggerating. Neither the eye of Sauron nor some dark music was following me. I am used to play some 80s rock in the background, and that’s what I did this time as well. (At least I think I did, I actually don’t know it for sure.)

Around half a hour later, I got it working! I really did that! I really fixed the display! One of the core components, one of the harder parts! I did it!

I’m really really proud of myself. And now I’ve got an even better display than it was before. Checked it with the built-in maintainability tool, all the pixels are working.

And with that, I am now ready for the GPN.

TIL: What's a search domain?

2026-06-01T21:17:25Z

I never thought about the search domain setting in my network settings. It was set to fritz.box (cause I use one of their routers) and I never knew why. It was just… there.

Because of a toot by Katze, I got a bit hooked what that thing called actually is.

What’s the search domain?

Basically, if you have a search domain set, like my router did it, you can resolve arbitrary, uh, subdomains. It’s useful for company networks and especially internal DNS systems.

For example: If you set the search domain to acme.org, people can resolve meow.acme.org by simply entering meow in their browser. The domain would be resolved like this:

ooooh, you don’t have a top level domain on this request. Imma ask acme.org to resolve this.
acme.org responded with 10.0.13.12.
I can send my request to 10.0.13.12, yaay! :3

Can I use that for my own network?

Theoretically yes. Practically, it’s complicated. The reason for that is DNS rebinding, a method of DNS spoofing. The technical details are explained in the linked Wikipedia article.

However, as a safety precaution, certain routers, like my beloved (ahem) Fritz!Box do have a filter for such DNS requests. Even if I’d set the A record for home.jasminchen.dev to 192.168.178.100, the router would block such DNS requests.

With DNS over HTTPS (DoH) or DNS over TLS (DoT), I could easily circumvent such filters. But that’s something to explore on another day.

Determine the Go architecture in a justfile

2026-06-01T21:17:25Z

These days, just is my favorite tool to invoke build scripts, run deploy commands etc. It’s just a binary, the justfile syntax is clean, there’s nothing to hate about it.

Unfortunately, when using a lot of Go-based programs, there’s a slight mismatch between the architecture reported by the arch() function and the architecture used by the Go compiler. As of today, there are about 20 reserved architectures reserved by the Go team. Not all of them are used (anymore).

But if I want to download a Go-based tool, like dbmate for running the database migrations, I either need to hardcode the architecture to amd64 or map from x86_64 to amd64. I would’ve used the hardcoded amd64 value in the past, cause ARM architectures were not as common back then. But, with the increased popularity of ARM for servers and clients, this is not longer suitable.

Therefore, I wrote this handy, just-native ugly if/else-block, which covers the most common architectures:

# Maps arch() to one of the known values inside the syslist.go
# https://github.com/golang/go/blob/3e9876cd3a5a83be9bb0f5cbc600aadf9b599558/src/go/build/syslist.go#L56
go_arch := if arch() =="aarch64" {
    "arm64"
} else if arch() == "arm" {
    "arm"
} else if arch() == "wasm32" {
    "wasm"
} else if arch() == "x86" {
    "386"
} else if arch() == "x86_64" {
    "amd64"
} else { error("unknown architecture {{ os() }}") }

It can be even extended to cover almost all supported architectures:

# Maps arch() to one of the known values inside the syslist.go
# https://github.com/golang/go/blob/3e9876cd3a5a83be9bb0f5cbc600aadf9b599558/src/go/build/syslist.go#L56
go_arch_complete := if arch() =="aarch64" {
    "arm64"
} else if arch() == "arm" {
    "arm"
} else if arch() == "mips" {
    "mips64"
} else if arch() == "powerpc" {
    "ppc"
} else if arch() == "powerpc64" {
    "ppc64"
} else if arch() == "s390x" {
    "s390x"
} else if arch() == "sparc" {
    "sparc64"
} else if arch() == "wasm32" {
    "wasm"
} else if arch() == "x86" {
    "386"
} else if arch() == "x86_64" {
    "amd64"
} else { error("unknown architecture {{ os() }}") }

Now I can use the variable to determine the correct download URL for dbmate, just like this:

install-tools:
    curl -fsSL -o ./bin/dbmate https://github.com/amacneil/dbmate/releases/download/{{ TOOLS_DBMATE_VERSION }}/dbmate-{{ os() }}-{{ go_arch }}
    chmod +x ./bin/dbmate

Enabling accent character selection on KDE Plasma

2026-06-01T21:17:25Z

After re-installing my Arch setup from scratch and installing my favorite desktop environment, KDE Plasma, again, I realized that I was missing one specific feature: holding down the “regular” keys to get a selection menu for accented/alternative characters.

An example? One of my metamours has the à inside their name. Unfortunately, I’m lazy and out of this laziness I first chose the regular à. Although I think that it’s not a big deal, I still find it disrespectful.

Luckily, I found Nate Graham’s blog post about said feature really quick. I’ve opened vim, edited /etc/environment and added the QT_IM_MODULE=plasmaim line.

Voila, now it’s enabled and I can write the name of my metamour as intended! 😄

Note (2023-08-14): It looks like the above environment variable breaks with the so-called “dead keys”, e.g. ^. Therefore, I just set the QT_IM_MODULE to the empty string, in order to have my dead keys on Wayland again.

Removing orphans from the Mastodon cache

2026-06-01T21:17:25Z

As part of the ongoing migration of the queer.group media files from local storage to S3, I’ve realised that I should do a cleanup of the orphaned files before copying them to the S3 bucket. Fortunately, Mastodon offers an easy solution with its `tootctl’ command.

tootctl media remove-orphans

And very soon, files got deleted.

# removed for brevity
Found and removed orphan: cache/accounts/headers/109/384/607/004/860/065/original/039a300fdeba0dc0.jpg
Found and removed orphan: cache/accounts/headers/109/427/244/149/514/100/original/b464439464ab98ff.jpeg
Found and removed orphan: cache/accounts/headers/109/440/021/041/888/306/original/04dda84036e651b6.png
Found and removed orphan: cache/accounts/headers/109/440/021/041/888/306/original/2777348e3afd634d.png
Found and removed orphan: cache/accounts/headers/109/440/021/041/888/306/original/2aa59f9af035ba23.png

During the removal process, it crashed after about 200,000 scanned files. This was probably due to a temporary database error. But it has a --start-after parameter with a very unhelpful explanation, I would say.

The Paperclip attachment key where the loop will start. Use this option if the command was interrupted before.

What the heck is a “Paperclip attachment key”? Where do I find it? Turns out, it’s the filename. I marked it in the following quote.

Found and removed orphan: cache/accounts/headers/109/440/021/041/888/306/original/2aa59f9af035ba23.png

To provide an example command on how to use it, here’s how I did it:

RAILS_ENV=production bin/tootctl media remove-orphans --start-after=cache/accounts/headers/109/440/021/041/888/306/original/2aa59f9af035ba23.png

Proxying S3 with NGINX to Backblaze B2

2026-06-01T21:17:25Z

Sometimes a simple forward slash (/) can drive a lot of frustration. For the migration of our Mastodon instance from local storage to a S3-compatible one, I’ve wanted to do it without any bigger service disruptions. To do that, I’ve followed Leah’s post about Mastodon scaling. She states:

You can also build everything to allow migration in the background with new media directly uploaded and served to/by S3 and old media, that isn’t migrated yet, served as fallback by the old local storage.

Yes! This is exactly what I want! In the full config example that she’s linked to, this line caused the frustration.

proxy_pass https://s3.example.com/mastodon/$uri;

My assumption was that mastodon is the S3 bucket name. Fortunately, Backblaze provides hostnames where the bucket name is already included, like https://bucket.s3.example.com. Therefore I’ve updated the line to:

proxy_pass https://my-backblaze-bucket.s3.eu-central-003.backblazeb2.com/$uri;

Looks similar to the config above, right? Except: it doesn’t work. The reason for that is: $uri starts with a forward slash (/) and therefore the proxied URLs look like this:

https://my-backblaze-bucket.s3.eu-central-003.backblazeb2.com//something.jpg

You see that double slash? Normally this wouldn’t be a problem, as hosting providers can easily get rid of them. Backblaze doesn’t. Therefore, in order to make the proxy_pass setting work as intended, the line needs to look like this:

proxy_pass https://my-backblaze-bucket.s3.eu-central-003.backblazeb2.com$uri;

That’s all. That single character.

Side note: According to a comment made on StackOverflow you shouldn’t use $uri, but $request_uri, because $uri might open the instance up for HTTP header injection vulnerabilities. All in all, the config line should look like this now:

proxy_pass https://my-backblaze-bucket.s3.eu-central-003.backblazeb2.com$request_uri;

Why I think that Conventional Commits are a bad idea

2026-06-01T21:17:25Z

When writing commit messages, we have a lot of flexibility. We can write the largest commit message that the world has seen until then or we can do a git commit -m "wip" and that’s it.

Writing commit messages takes a lot of practice, no doubt. There are guides, patterns and all that stuff around the internet. One example are Conventional Commits. Conventions and standards are good, at least most of the times. However, I’m not sure if this is the case with conventional commits.

If you are unfamiliar with it, here’s the quote from their page:

The Conventional Commits specification is a lightweight convention on top of commit messages. It provides an easy set of rules for creating an explicit commit history; which makes it easier to write automated tools on top of. This convention dovetails with > SemVer, by describing the features, fixes, and breaking changes made in commit messages.

The commit message should be structured as follows:
<type>[optional scope]: <description>

[optional body]

[optional footer(s)]

Sounds easy, right? Now we have a type, a scope and that’s it. And the types are fix, feat, refactor and some else. That’s not a lot of new concepts, nice! I’ve adopted Conventional Commits around 2021 and have been using them since.

Simplicity does not exist.

Although it sounds easy to prefix your commit messages with a type, that part of categorization is actually hard. At least for me.

Example: Updating a major flaw in the docs

There’s a major problem with the docs, they’re out of date and they written behaviour leads to unexpected results. Let’s assume that the commit actually solves this problem. Because it’s going to fix the instructions for new users, it’s probably a fix. Maybe even a fix(docs). Or does it belong to the docs scope? It could even be a chore.

It depends on the personal perception of the issue severity. I wouldn’t judge anyone for picking the wrong scope.

Guidelines, Jasmin, guidelines!

Yes, of course, there should be a guideline for the team. But even then, writing good commit messages is hard. git itself is a monster. No one should deal with git. I understand it very well (at least I think so). I’m working years with this thing. I’m no longer “deleting and re-cloning” repositories, just because I broke something. Nevertheless, it’s still a shitty software. It works, but it’s definitely not user-friendly.

But what about the automation?

Actually I don’t think that auto-generated changelogs are useful. If your changelog is just a categorization of your commits, it does not provide any value to me. I don’t care about the hundreds and thousands commits by @dependabot. Either admit that you don’t want to write a changelog or write one. Yes, it’s work. Yes, it’s annoying. But it needs to be done.

And now?

Actually, I don’t have a proper solution for writing commits. But I know that I won’t be writing Conventional Commits much longer than needed. Instead, I use the good old style: regular titles. Instead of feat: add new menu button I’m going to write Add new menu button. And I’ll put more effort into my changelogs.

hello_world.md

2026-06-01T21:17:25Z

So, let’s try it out again. Maybe I’ll actually write a blog this time. Would be nice. But I cannot promise anything. Maybe this is also the goodbye post, who knows.

but for now: Hi. 👋

jasmin's little garden

Small collection of fish functions for working with the Hetzner Cloud

hcloud-token – Print the current token

hctx – interactive context chooser

fish_hcloud_prompt – prints the current context in the prompt

Fixing a FRITZ!Box 6660 Cable to get updates again

What I did

References

Writing (regularly) is hard

Games I played recently

Split Fiction

Assassin’s Creed Shadows

Need for Speed: Unbound

Star Wars: Outlaws

Life is Strange: Double Exposure

moo!

Building an image-based Debian 13 disk image using mkosi

The goal: Debian, but image-based

The beginnings: Preparing the cache

Step 1: Setting up our mkosi.conf

Sidenote: Debugging the configuration

Step 2: Configuring the partitions

ESP partition

erofs for /usr

root=tmpfs to the rescue?

Moving stuff from /etc to /usr/share/factory/etc

Uploading the image to Hetzner

Recap

Using Apache2/httpd instead of Caddy

Converting a systemd-homed user to a regular one

I blame the hyperscalers for the IPv4-first mindset

First attempt: k0s

Second attempt: k3sup

The elephant in the room: GitHub

Why I blame hyperscalers

Game review: Split Fiction

About journaling and ADHD

How it all started

The awakening

Impressions of the the KubeCon 2026 Europe

My impressions of the conference

Impressions of Amsterdam

Project Hail Mary

now with human.json!

Building a pollen overview in HomeAssistant

First impressions of Obsidian

Synctrain + Obsidian works pretty nice, even on an iPhone

Importing existing notes

On Obsidian plugins

Things I like

Things I dislike

Next up

Experimenting a bit with coding fonts

Turns out, resident IPv6 addresses appear to be stable

I am so sick of the slop.

Two examples where it got worse

Consequence: Lack of trust

You must be kidding me, right?

Further notes

Using OAuth as a Git credential helper

IPv6 is still a pain in the ass

Update 2025-11-02: Reply from the sigstore.dev team

This homepage is now IPv6-only

Why?

Calibrating a PS5 controller on Linux

Adding missing permissions via udev

Sidenote: How to figure out the USB vendor

Results

Update 2025-10-23

System administration contradicts perfectionism

The philosophical(?) perspective

Choosing an jujutsu revision interactively using fish and fzf

Smaller feed adjustments have been made

ZFS dataset decryption using the TPM with the help of systemd

systemd to the rescue!

Example usage

Important note

Summary

Sieve filter to auto-sort mails from Git forges

hamsterradeln

`hcloud-token` – Print the current token

`hctx` – interactive context chooser

`fish_hcloud_prompt` – prints the current context in the prompt

Step 1: Setting up our `mkosi.conf`

erofs for `/usr`

`root=tmpfs` to the rescue?

Moving stuff from `/etc` to `/usr/share/factory/etc`

How does that `.container` file look like in practice?

Recap: I don’t miss `git`