fun memory violations

My first scifi short story

Katharina Fey — Fri, 24 Nov 2023 00:00:00 +0100

You may not know this about me, but I've been writing a SciFi novel for the better part of a decade now. This year I (somewhat) participated in NaNoWriMo to work on it. The full story still isn't nowhere complete.

But when I sent some friends individual scenes for feedback (or just to showcase the world and my writing style) someone suggested I should consider publishing one of the chapters as a stand-alone short story. And that's exactly what this is.

So here you go! I hope you enjoy this random bit of SciFi world building. Maybe I will publish other short stories like this in future, but who knows. Certainly in another 10 years the novel should be done. If you're an editor at a publisher and you like what you read: call me, lol.

The Royal Regatta

A holoscreen blared in a bar somewhere on Terminus 361. It was showing two presenters sitting at a desk, a vaguely masculine and a vaguely feminine looking person, both of them wearing loud, bright suits, having their puffy hair styled backwards, and wearing gold glitter around their eyelashes and hairline. Anyone only casually looking at the cast, or viewing from a distance, would have had trouble keeping the two figures apart.

“That was quite the recap Mel,” the masc presenter commented.

“Absolutely Mev, absolutely. Coming up next, I am personally very excited by the next position. Team Alhama, from the Great Westerland,” she turned over to Mev, “two time regional champion, first time contender in the Pandam regatta.”

“They are certainly here to fight but the question is, whether their pilot, someone who has only been racing needle ships for three years, is up to the task.”

“Oh yes, oh yes”

“As everyone knows, that is not a long time to be behind the proverbial stick of such a powerful engine.”

“So who is this new pilot?”

“Her name is Zhenzha Vae. And I think we have a clip of her.”

The feed switched over to another flat view. It showed a woman, maybe in her early thirties, with deep black hair, braided into several small locks, tied together with a fluffy hairband. She had dark green eyes and her skin was a medium brown, her mouth twisted into a narrow smile, glaring at the camera.

“I know we’re the underdogs in this season but believe me when I say,” she paused for emphasis, “your ostentatious engineers have never seen what a real Westerland engine can do.”

The view cut back, the femme presenter smirking.

“She’s got grit, I’ll give her that. But she will have to convince me with her flying, proper.”

“Certainly Mel, certainly. But let’s go live to the preparation hall, while Team Alhama is at this very moment getting their engine ready for the pre-race checks.”

Zhenzha looked disconnected on camera, like nobody was actually watching her. She interacted with her teammates, flicked through a data tablet of flight parameters and coordinates, or simply leaned against a bulky crate behind her as she focused on her breathing. Finally it was time. She was wearing a pitch black flight suit, she shoved her braids into the back of her shirt and put a slim black flight helmet on. Her visor was tinted pink, yellow, and blue, the colours of the Great Westerland flag.

The needle ship was massive, reaching out from the holoscreen. It was at least two hundred metres long and twenty metres high and wide. It’s form was simple: long straight lines, with low-angles, but discrete edges. It slowly sharpened into a tip towards the bow, and had a wide aft plate, housing three sets of engine effectors: one ion and two microwarp.

The hull was made up of a black metal sheets, some of which had sponsor decals, others technical specifications and maintenance indicators printed on them.

The ship was ejected from the hangar and hovered around the starting area, keeping itself steady with the gravity engines to all sides. It’s hull plating had started extending outwards slightly, revealing rows and rows of measuring equipment boxes and additional thrusters. Inside of it Zhenzha had hooked herself into a control matrix, enveloping her arms and legs, providing reverse kinematics to her movement. A shimmering black cable was plugged into a socket at the base of her neck, drooping down slightly as it arched upward into the ceiling. She smirked. The whole ship started to hum.

“And this is it, the countdown has started, the ~~Black Nebula~~ — wow, what a name by the way—“

“Goosebumps!”

“—has just started its priming sequence. Everyone brace yourselves.”

“There are a lot of people who want to see this team, and in particular this pilot, do well this season.”

“Ten more seconds.”

“We can certainly hope that this will be a successful start to their first season here, at the seventy-seventh, legendary, sexennial Pandam Royal Regatta !”

The ship hummed louder, pulsing with energy, countless capacitors charging, and fields subtly adjusting.

“Five. Four. Three. Two. One.”

The countdown hit zero and almost instantly the ship started growling. The space in front of the Black Nebula started to morph and warp out of shape, swallowing the ship, at first slowly at the front, then the rest disappeared with a brief flash of light, dragging a very faint trail behind itself into the distance, barely visible on the holocast. Almost instantly the gravitational waves could be felt around the launch hangar, heard as a burst bass symphony.

“And they are off, the ~~Black Nebula~~, with Zhenzha Vae at the helm, racing for team Alhama of the Great Westerland. What is their first Pandam regatta opening going to be?”

“It would appear as if team Alhama is going for a reverse gravity turn around Pheneus, running the system counter-clockwise.”

“We have only seen this approach taken twice during the qualifiers, so far. Mev, maybe you could give some insight into why this is considered such a bold strategy this season.”

“Certainly Mel,” a view of the solar system came up, with a small window in the bottom right frame still showing the Black Nebula, severely warp shifted and distorted as it was visible from one of the many thousand orbital cameras littered around the system, “this season the orbital periods of Pheneus and Vhalstella, the Vhey system’s two gas giants — for viewers across the Galactic expanse — are almost directly opposite to each other, with the system core right in the middle.

“Approaching the system clockwise means that the Vhalstella checkpoint can be used as a gravity boost into the lower systems—“

“Free acceleration essentially.”

“—free acceleration, quite right Mel. Taking the counter-clockwise route gives a small time advantage in the beginning, but Vhalstella’s gravity is going to slow them down towards the end of the run. We will see if team Alhama can still keep their advantage.”

“Oh and we are coming up on the first checkpoint…”

Zhenzha had her eyes closed, but there was plenty for her to see. Her arms and legs were swaying in the control struts that were holding her in place. The stars were blurry messes around her, rows of numbers were streaming down her peripheral vision. She felt a soft breeze on her face and her race engineer mumbling statistics in her ears, with several seconds delay (she tried not to think about the exorbitant logistics of providing a feed to her in this moment at all).

The ship around her was humming and occasionally screaming and whining. The warp field emitters were stretched to their legal limits as the Black Nebula hurled itself across the Vhey system with super-relativistic speeds. She could feel the ebb and flow in the gravity waves around her.

“You’re coming up on the first checkpoint. Time looks good.”

Zhenzha tightened the gravity fields, releasing on acceleration and putting the ship into a gentle spin to not lose stability. For a flash of a microsecond she felt something go past her, then it was gone. She widened the fields again and pushed the accelerator. The engine howled and she felt herself be pulled backwards in her control harness.

“Incredible, team Alhama bringing in the first checkpoint time of 34.2217 seconds, that is, Mel, I believe—“

“Even after some compensation, this is the fastest time of this qualifier, so far.”

“Incredible.”

Zhenzha kept pushing.

“Kapo belt is coming up in 350 mk. And watch your port wake tails, you’re dangerously close to the red line.”

She nodded, mostly for herself, then reigned in some of her field projectors, collapsing her gravity wake back to just under two million kilometres, making the ship run slightly more bumpy. Her view showed a wall of object indicators coming towards her. She kept pushing, breaking through the 100 xc limit. One of the team Minds had plotted a path through the belt hours ago, but by now some of the rocks would have drifted. She had to make adjustments to their route.

The first rocks started hurling by her, too fast to register, to briefly to appreciate how close an encounter it had already been. The twelve million kilometre thick Kapo belt was wrapping itself around the core planets.

“Mev! Mev, it appears that Zhenzha Vae is not slowing down.”

“It would appear so, Mel, it would appear so. Team Alhama seems to have chosen a through-route for the Kapo belt. But a belt transfer requires significantly lower speeds and is usually deemed a bad strategy.”

Another three seconds had gone by and she had reached the edge of the belt. The needle ship had flattened its gravity fields from several million kilometres down to just a few ten thousand metres. It was as if the engine itself was rebelling against her choice of manoeuvre, but she didn’t care.

“Check projector 77-b, it looks like it’s spitting out bad metrics.”

She checked the projector and indeed it seemed to be slightly off. But not enough to do anything about it, and anyway, she was too busy.

She was deep within the belt at this point, flicking the ship back and forth to narrowly evade a collision, scraping by another rock close enough to physically touch it. The Black Nebula was in a mild spin, it’s gravity wake flaring wildly behind the ship. With every twist and turn Zhenzha made to adjust her trajectory another few rocks ten thousand metres off her bow were torn to pieces by the gravity forces spiking at the edges of the warp bubble.

“Half way through.”

Zhenzha was in a trance, unable to even register new messages that were coming in, her time, her position, it didn’t matter. She was surrounded by a sea of proximity alerts, surfing through the asteroid belt and weaving and in and out of available pockets of safety.

Finally, she reached the other side and accelerated again, maybe even more aggressively than before.

“Mel, I am at a loss of words.”

“Me too Mev, me too.”

“Dear viewers, we seem to be witnessing the making of a new record time here, at the seventy-seventh legendary sexennial Royal Regatta. Zhenzha Vae of team Alhama of Great Westerland, has just crossed the Kapo belt in just 15.838 seconds. We will have these numbers verified, and I’m sure the referees are already checking the flight footage and statistics.”

“I suppose this shows us that even a belt transit is a valid strategy if you don’t plan on slowing down for it.”

Mev chuckled. “It would appear so, it would appear so.”

The next stretch was easy. Zhenzha hit the accelerator and curved around the mid system to the Dhereliq (the old sandy mining colony) checkpoint, almost forgetting to collapse her wake (ripping a checkpoint gate apart was grounds for disqualification), using the gravity assist to loop around the desert planet once and hitting another checkpoint gate around its seventh moon, then plunging deeper into the system and crossing the inner belt. It was significantly less dense and it became evident that it had been extensively mined for almost a thousand years.

“Another extravagant transit of the Sen belt. Team Alhama has certainly come here to provide a show.”

“They are succeeding. We have times on sections three and four at 11.8945, and 3.8141 seconds.”

“Wow. Really. 3.8141. Wow.”

“I think in all my time presenting I have not seen a sub-four for the Dhereliq checkpoint pair. Absolutely incredible.”

“Incredible.”

Zhenzha had to slow down for the inner system checkpoints. The Vhey star’s gravity was too strong and wake field requirements were much stricter.

Fucking civilian traffic.

She hit the gate around Aquel’qu, then a few seconds later another gate around Pandam’s moon.

“Kapo coming up in 300 mk.”

Zhenzha steadied her breathing and pushed the Black Nebula forward. She was floating in a sea of darkness, surrounded by data, flashes of information, and proximity alerts coming up in front of her. Her mind was razor sharp, absorbed in total flow state as she dodged through the Kapo belt once again, pushing harder this time. Twice she almost collided with a rock, only to make a last microsecond adjustment to avoid disaster. A kilometre to her right an asteroid was ripped apart by a gravity wave spike.

She was through it again, making her final approach to Vhalstella.

“You know the drill. Tight field, use the third spiral fractal. Stay strong. Good luck.”

The Black Nebula was approaching Terminus 222, located on Vhalstella’s 22nd moon. Under normal circumstances she would have to make a gravity turn around the gas giant, shedding an incredible amount of speed, then curving back to hit the inner gate in low orbit around the giant itself, before accelerating for the final section. But these were not ordinary circumstances, and the Black Nebula was no ordinary ship.

Zhenzha banked hard towards the moon she had just shot past. It felt like she took another breath, then fully collapsed her outer gravity fields, while letting the projectors facing the moon cycle through a complex set of spiral fractals. A screaming filled the air around her, then a series of loud bangs cracked through the ship hull. The engine itself felt as if it was sputtering.

The needle ship curved itself around the moon at an unnatural rate, as if the laws of physics had gone completely out the window, as if the little moon possessed the gravitational pull of a small star. The ship was being shaken through violently, creaking and moaning under the load. The next five seconds felt like an eternity to her. Zhenzha could feel her pulse, throbbing in her neck, her breathing went stiff, her muscles started aching.

Finally she had made it. She reset the field arrangement back to a default layout, collapsed them briefly for the checkpoint gate, then opened up all engines, widened her warp field as much as was legal and pushed.

“I do not believe my eyes, dear viewers, what we are seeing here...”

“Team Alhama managed to run the Vhalstella gate pair in just 7.5184 seconds! Please correct me if I’m wrong—“

“That is certainly a new record time as well.”

“Incredible. Approaching the gate in reverse—“

“Something that team Alhama seems hellbent on demonstrating this season, their ability to flip the rules upside down and get something incredible from it.”

“Does this mean Vhalstella’s gravity is not going to slow down them down?”

“Oh indeed, on the contrary. The way that team Alhama is going around the giant now is even going to boost their speed.”

“Incredible.”

Zhenzha left out a breath of relief when she was on final approach. She took off a few limiters and pushed the engine to its absolute limit. Her vision went blurry; even the inertia stabilisation she was under could not save her from the acceleration forces now. The engine had started growling angrily, as if complaining about the load it was under, in tonal intervals of five, from the lowest to the highest possible audible note. She broke the 300 xc barrier. The Black Nebula curved around the system, along the outer edge of the Kapo belt, crossing the entire length of the Vhey system in just a few seconds. Zhenzha started getting dizzy when the gas giant Pheneus was coming towards her view, faster and faster, filling it out. For a microsecond she collapsed her fields to not rip the final gate to pieces, then she cut the accelerator and rode her gravity wave around Pheneus twice, three of its moons that happened to line up in a nice formation for her to circle, then braking hard and coming to a stop near her Terminus 361 team hangar.

They had set a new qualifying record. Back in the hangar she was greeted by camera crews and reporters, the steaming hot needle ship sitting on its four hydraulic legs behind her, and in front of her a raging deck full of engineers, technicians, backup pilots, managers, cooks, janitors, and anyone else who helped run team Alhama that day. She pulled off her helmet, revealing her face and hair drenched in sweat. She was smiling, and waving at people who waved at her. She answered some questions about her flying, about the paths she took, about the strategy that went into the run. She thanked her teams engineers, technicians, and strategy Minds. She made sure to stress that this was not the end of the regatta, that the main field was a very different competition, but that they were honoured and excited to have made such a splash at their first qualifying and that she was excited to see where else they could go.

She removed herself from the crowd by disappearing into a hatch somewhere towards the back of the hangar. Back in her room she stripped off her flight suit, took a quick shower, then collapsed into bed and slept.

Edit 3.3.2024

Remove opening dialog on the holoscreen broadcast. Reworded some sentences to be more direct.

Cleaned up formatting, and switched from underline to italics, and italics to bold.

I am no longer using flakes

Katharina Fey — Sun, 18 Jun 2023 00:00:00 +0200

Before I really get started on what bothered me about flakes, I would like to very clearly point out that this is my opinion, and also that I am a big dumb-ass.

I use my computer for lots of different creative and professional (sometimes both) endeavour, and while I like Nix in a lot of ways (I have even really come to love the syntax of the language), having "managing my computer" as a hobby is not one of them.

I deliberated whether I should write and publish this blog post, but decided that I still wanted to share my experience and frustrations. Maybe the way that I have come to use Nix on my computers is not in line with "how you're supposed to do it", but maybe that's exactly the reason why I should write something about this topic.

Anyway...

What are flakes

Back in 2019 RFC 0049 was proposed, which introduced the Nix community to the new concept of "flakes", a way to manage Nix build inputs and outputs in a pure and reproducible manner. This made a lot of people very angry and after months of reasonable online discourse, the RFC was withdrawn and implementation work began in the new Nix CLIs behind an "experimental" feature flag. This was done in order to work on the design of flakes, and being able to explore problems that came up more organically. I can see however how people might feel that the design was rejected, and was then snuck in the backdoor by the RFC authors. In the meantime a number of smaller RFCs have come up by community members as a way to voice feature requests for flakes.

Responses to flakes are still varied to this day. Some are really into the features they enable your builds to have, and I have to admit that being able to have a single flake.nix file in a project repository, which pins a version of nixpkgs and produces a build of the given application is very nice.

Disdain for even the idea of flakes can still be found in some corners of the Internet however, and I think everyone involved in the development of flakes is rightfully on-edge, given the amount of hostility and abuse that they have been getting for checks notes implementing a new feature in a package manager.

I am aware of this history and I don't want this blog post to become a weapon wielded by jackasses to harass a bunch of people online. Don't be a dick and go look at some ducks instead.

What I did before

When I got started using NixOS I began by maintaining a nixpkgs subtree. That's what all my friends were doing, and it seemed like a neat idea at the time. I was occasionally working on upstream nixpkgs, so having all the code right there seemed like a sensible idea. In hindsight, it really wasn't, and it turned into a frequent source of frustration (see the disclaimer above about how I don't want "managing my computer" to be a hobby of mine).

I stuck with this system for close to 4 years anyway (see the disclaimer about being a dumb-ass), because the cost of migrating always seemed to high. I knew I wanted to try out flakes, but the barrier to entry also seemed too high. The fact that the flakes RFC was withdrawn and development continued behind an experimental feature flag signalled, at least to me, a state of flux and uncertainty about the design of flakes. The fact that the only documentation available was one short page on the (generally excellent, please contribute more to it!) NixOS wiki, and a blog post series by tweag didn't help this.

The two big problems with the nixpkgs-subtree approach are: nixpkgs is huge and is going to make your computer unbearably slow, and unmanageable merge conflicts in case I ever updated a computer and didn't immediately push the update to the repository remote.

And while it would seem like something that is easy to avoid, I ran into this problem again and again and again, to the point where by the end my servers and desktops were running completely disjointed configuration repositories, that only occasionally were updated by sending patches around. Fucking yikes. Yes, this is my fault, but the system I had designed myself into encouraged these sorts of human errors to accumulate.

Finally, late last year (2022) I started a job where I came into much closer contact with flakes, and so, I pulled off the several-year-old band-aid and dove right into it.

Migrating to flakes

This was the state that I inherited and moved over to flakes. It took me a few weeks to get things building again on my NixOS desktop and Pop_OS (with home-manager) laptop. Right off the bat I didn't like a few things. I was using the NIX_PATH as a path lookup for modules in my configuration, so I didn't have to rely on relative paths between modules (I would frequently expand <configuration/foo> and similar lookups). This isn't allowed with flakes, unless you opt into impure evaluation mode, which is what I did. In hindsight, the way that my flakes configuration ended up looking was much more linear than the kludge of horror I had built before, and so this limitation was less of an issue, although it still bothered me in some places.

I also really wasn't, and still am not a fan of the build syntax:

Before: nix-build '<nixpkgs/nixos>' system -I nixos-config="$HOST"
After: nix build .#nixosConfiguration."$HOST".config.system.build.toplevel

This is the commit that ported my desktop over to flakes, if you wanna have a look. In my build harness I removed a bunch of features like being able to quickly build vm and iso targets. This might still be possible with flakes, but I didn't find an obvious way to do things in the moment, and so I opted for removing the features in the meantime.

The NixOS documentation (and SEO!) situation is bad enough, and restricting yourself to results that talk about a subset of the language and ecosystem made it unbearable in some cases to understand what I was supposed to do. I ended up reading several people's configurations to copy things from them.

I ended up with this custom harness function to create a NixOS system because I hate code duplication (and a similar one for the homeManagerConfiguration function):

nixosSystem = root: nixpkgs.lib.nixosSystem (root // {
    inherit system;
    modules = root.modules ++ [
      ({ ... }: {
        nix.nixPath = [
          "nixpkgs=${nixpkgs.outPath}"
          "home-manager=${home-manager.outPath}"
        ];
        nixpkgs.overlays = defaultOverlays;
      })

      ## Include the home-manager NixOS module
      home-manager.nixosModules.home-manager
    ];
  });

It was important to me to have the nixpkgs and home-manager keys set correctly in my NIX_PATH for the "legacy" CLIs to be able to pick them up and use. Nonetheless, this caused countless issues that I still don't fully understand. But more on that in a little bit.

Another rift I saw in the community of flakes users was whether or not to use flake-utils. The two philosophies seem to be:

"use flake-utils because it makes your life easier"
"don't use flake-utils because it makes flakes seem too complex"

Which one is it? Are they both true? I ended up not including another dependency for my configuration, instead writing the wrapper functions myself (as I said, I really quite like writing Nix code!).

Especially because all my computers are x86_64-linux, and flake-utils is often used to work around some issues with flakes regarding cross compilation. Depending on what systems you use with Nix, flake-utils may give you a better bang for your buck because you need to write less obscure loop code that includes your configuration root for multiple system targets.

I imagine this will confuse potential adopters of flakes similarly to how it confused me, and it certainly didn't leave me feeling rosy about this technology I had just built my sand castle atop of. If you're interested in the custom wrapper code I wrote: here are my main utilities and user utilities!

Using flakes

And so, that was my life for about 6 months. I would occasionally run nix flake update, which was easy to complete, almost never caused any merge conflicts due to my dumbassery (and even the one or two times it did, was easy enough to fix by just deleting the lockfile), and life was good.

I also started using flakes for some one-off projects, where I was collaborating with other people (for example jackctl, which you should check out if you make music on Linux), where using flakes made it easy to depend on multiple sources of nix code, and build a single package with a particular postFixup phase or whatever. I will still use flakes for these use-cases, because I think they're the ideal situation to map a simple set of inputs to a simple set of outputs.

Importantly, this is not my development environment, but rather something that would be invoked in CI.

But, I slowly noticed that the development environments littered around my system stopped getting current rustc packages. The way I build those is with shell.nix files, which include <nixpkgs> from my path, which are built by the lorri service. The idea of having different environments, that all pull from the same local package set is just to reduce the space requirements on my computer. I have a big SSD, but might as well not pull in 50 slightly different versions of rustc, cargo, etc.

At first I didn't really care, but with a lot of Rust projects recently dropping down their supported Rust compiler period, I started running into compilation issues. I even briefly installed rustup on my laptop because I was blocked from working on a project and it was the easiest way to get out of my pickle. shudders.

I knew something had to change. I couldn't go back to the way things were before, and the way flakes made me re-evaluate my configuration was good. At the same time, my computer started accumulating strange behaviours I didn't really understand. And maybe it was a matter of not using the new CLIs exclusively, or wanting to rely on lorri to asynchronously build environments for me and injecting them with direnv (something that works very well with emacs too).

The last straw was a build failure as a result of an incompatibility between home-manager and nixpkgs, that had allegedly been fixed, but updating both my nixpkgs and home-manager inputs over the span of 2 weeks didn't allow me to update my computer. Maybe I got extremely unlucky with the unstable channel progressing, but in the end it didn't really matter. I wanted to make a change.

What now

I had previously evaluated different strategies for managing my nixpkgs and home-manager dependencies. I briefly thought about subtrees, where I squash committed everything but decided that I didn't really get any benefit out of that arrangement. I tried a submodule, which was somehow even worse, and then I tried [niv] and flakes. In the end, flakes briefly won out because I wanted to try them. So moving to niv was the obvious choice.

niv was born out of some of the ideas of the initial flakes RFC, taking basically only its input management, encoded in a json configuration, which gets parsed by some magic nix code (which I have to admit, I do not fully understand, nor have I had to look at it because something was broken or unclear as to how it worked).

My build harness script looks like this now (the relevant bit):

NIXPKGS_ALLOW_UNFREE=1 \
    home-manager build -f "$ROOT" \
      -I "nixpkgs=$DIR" \
      -I "klib=$DIR/lib" \
      -I "nixpkgs-overlays=$DIR/overlays" \
      -I "home-manager=$($DIR/lib/find-component.sh home-manager)" \
      "$@"

and the way I import nixpkgs looks like this:

{ overlays ? [], ... } @ args:
let
  sources = import nix/sources.nix;
in
import sources.nixpkgs (args // {
  overlays = (import lib/overlays.nix) ++ overlays;
})

I had to write a small utility for setting the appropriate NIX_PATH for home-manager:

# De-reference current directory and grab the component parameter
# This will break if none was provided so... don't do that :)
DIR=$(realpath "$(dirname "$0")/..")
COMPONENT=$1

# Run nix eval and just return the result
nix eval --expr "(import $DIR/nix/sources.nix).${COMPONENT}.outPath" --impure | tr -d '"'

The nice thing about grabbing the component paths from the niv sources is that the NIX_PATH environment variable gets populated with absolute /nix/store paths. This means that changing something in my configuration won't apply to things on my system until I next build and switch to a new generation:

 ❤  (tempest) ~/sys> echo $NIX_PATH
nixpkgs=/nix/store/7znqbsc1vqyjhcmdvfjrfp73hhvazpwq-nixfiles 
home-manager=/nix/store/0haa6si3qmp8r80irj4a692qvxp51rfh-zb3bip08c8w6jxbhbmb4i852lp4a3d95-home-manager-src
nixpkgs-overlays=/nix/store/q015fg370v3qjwrz840pl5f38km4j5sh-overlays

In conclusion

In the end... is this better? I really don't know.

Flakes felt like magic in a lot of places, even though their functionality should be able to be encoded into a simple input -> output relationship. While using flakes I didn't feel like I understood what my computer was doing. And while my current solution technically allowed me to reduce the amount of code needed for my system to build, my solution is bespoke, and I can understand that someone who hasn't thought about their Nix setup as long and hard as I have, might feel overwhelmed by it. This is especially a problem if everyone comes up with slightly different variations on how they would like to initialise their nixpkgs builds. And maybe I should just have globally installed rustc and cargo and called it a day.

Shit... maybe managing my computer is my hobby.

I can certainly see how flakes can work for many, many people, and I am probably in the minority of users who will bump into problems.

At the same time, I'm not a fan of the way flakes have been developed.

There seem to be a sentiment that flakes are "done", even though they've technically never left the experimental stage. I do not condone the abuse directed towards the creators of this technology! But at the same time, I can see why some people might feel annoyed at the idea that an RFC was de facto rejected (withdrawing an RFC is not generally done because its initial design was perfect), and then having its feature set snuck into the Nix codebase without going through the same RFC process again. At this point, enough users depend on the experimental flakes feature as it exists, it could be argued that any breaking change would be too disruptive, and so the current state must be stabilised as is.

I have also seen and felt the pressure of coming up with perfectly measured criticism against flakes (one of the reasons, why I was unsure about publishing this blog post), because of the way conversations about them often divolve into abusive tirades. No it's not OK for people to be behaving the way they have been. But it's also not OK for the flake developers to dismiss people's criticism, concearns, problems on the basis of some people being dicks.

Since 2019 the documentation situation for flakes has only improved marginally. There are a few more resources out there now, but none are "official" or built by the community, and also none that go beyond the basics of what a flake is and does (yes nix shell is cool, but nix-shell existed before too). There is a kind of chicken & egg problem where documentation must refer to things that are stable, and flakes need documentation to get more people into the headspace to try to use them.

Ultimately, I'm open to the idea of using flakes again. Updating my horrible hacks from before flakes to flakes was hard, changing from flakes to niv was two afternoons. So maybe I will try them again at some point. But for the time being, I decided it's more trouble to me than it's worth.

Thank you for reading this rambly post until the end. Maybe you learned something, or were able to contextualise some of your own experiences or problems. Please don't be a dick. In the end, it doesn't really matter. Computers don't matter, and you shouldn't get too upset about them.

Cheers.

Why I deleted my Twitter, and why maybe you should too

Katharina Fey — Tue, 18 Oct 2022 00:00:00 +0200

I deleted my Twitter account this month. It was a decision that I had struggled with for months (if not years), and one that I want to talk about. This article is a collection of thoughts gathered through conversations with friends on this topic. I was advised to turn these thoughts into a blog post.

If you are struggling with your relationship to social media and having a hard time cutting ties with some platform (in particular Twitter), maybe this blog post will be helpful to you.

Background

I got addicted to Twitter in the run-up to the 2016 US election. I was going through a bit of a moderate phase back then (sandwiched between the punk days of my teens and the anarchy of today) and I became part of "liberal twitter", consumed liberal memes, and enjoyed dunking on conservatives. Of course I didn't think Trump would win.

After the election results this behaviour intensified. I sought refuge from the world that had shattered my understanding of society by sliding deeper into the filter bubble that had sheltered me from reality in the first place. And so I revelled in making fun of Trump and his supporters, celebrating the "covfefe" and body shaming, and intensively following the discourse™. Of course it didn't make me any less miserable.

The band-aid for the pain that I felt, the realisation that the world didn't work the way I had swindled myself into believing, soon didn't satisfy my need for comfort and so I became more dependent on instant gratification through social media. I was unable to truly comprehend the extent to which I was addicted to the thing that was hurting me.

Doom scrolling is a well known phenomenon in our terminally online society and we talk about it as if it was unavoidable. As if it was an individualist problem. We know that it is bad for our mental health, yet rarely do we talk about it on a systemic level. Why do we all put up with these circumstances and why do we all keep making excuses for the systems that make us miserable? This, to me, is the core of how many people engage with modern social media platforms.

I had taken longer breaks from Twitter before, deleting the app off my phone, bargaining with myself about how much I wanted to share. But I always fell back into its orbit. I feel it's appropriate to use the language of addiction around this subject. And it wasn't until I truly admitted to myself that I had a problem that I was able to let go and examine the frameworks through which I had engaged with online society, and the impact that these systems had had on me and my life.

In the following sections of this article I want to outline some of the mechanisms that have kept me hooked for so long and have made it extremely difficult to escape its consequences. Lastly I also want to talk about a possible way out and some alternatives as to how to engage with the online.

The fear of missing out

Social media weaponises "FOMO", the fear of missing out. Twitter is particularly good at this, advertising itself as the platform "to know what's happening". Twitter users know it first is the actual current tag-line to entice you into signing up.

I believe social media has had the strangle-hold over our collective psyche at least in part because "we" have designed a society where being outside your own home is miserable, exhausting, and toxic. To have a good time outside you need to spend money, or you need to engage in some kind of social event. Merely the act of being outside is punished through car-centric design and architecture. Not to mention the noise that personal motor vehicles cause and their effect on the mental well being of city dwellers. I have been fortunate in a way, living in the suburbs of Berlin (for the time being), meaning that I only have to walk for about 5 minutes before I end up in the Brandenburg forest.

This has actually been a life-saver. It has meant that I was able to escape stressful situations over the last few years (of which there have been too many) by going for a walk, disconnecting through nature, and re-grounding myself in a very basic way. It seems silly to even talk about this but having the ability to go for nice a walk has been linked to a significant improvement of mental health¹.

The unfortunate reality is, that many people do not have access to this basic human need, and so, social media can often seem like an enticing alternative. This is an especially important angle considering the way addictive design is used in social media apps to give you a false sense of gratification.

Over the last decade we have all become more and more aware of global events to the detriment of our collective mental health. Permanently wired into "what's happening" creates the illusion of control, where none is possible. I'm not arguing for ignorance here either. I don't think that shutting yourself off to what is happening in the world is a reasonable alternative. But we must examine the ways that our current modus operandi is damaging to us. We must especially become aware of how the fear of missing out will drive us to engage for longer and beyond our personal limits to handle the stress that knowing the horrors of the global capitalist system causes within us.

Becoming a brand

When I created my Twitter account back in 2012 it was for me to shit-post about video games, and to keep in touch with various friends around my life. Around 2017 a slow shift started to appear in the way that I used my account. It was gradual, and imperceptible to me at first. Over time I was changing from being a person to being a brand. This will probably not be everyone's experience. I know people who are 100% the same in person as they are on Twitter (even with large-ish follower counts) and I assume this is not a problem that everyone is going to have.

Still, it is something that I have struggled with, and that I know people much "larger" than me are also struggling with. My audience on Twitter was still relatively small, yet I stopped feeling confident in being able to share myself as-is on the site. In one part this was because it made me uncomfortable to share intimate and private parts of my life with thousands of people online that I didn't know. In other parts it was because my political and societal outlook shifted and radicalised. My views became less socially acceptable, and so, I started filtering myself online. This was small at first, but over time it built up a wall between me and my Twitter brand.

On one hand I felt this weird obligation to use my platform, to leverage the following that I had for the work that I was doing and that was important to me. On the other hand I felt repulsed by the idea of having to pretend to be someone I wasn't to be accepted.

I want to quickly explore the shape and form of online communities (through my experience obviously -- I have done some research into this but sociologically things are more complex than this) because I think it will be helpful to explain how this shift from "person" to "brand" happened.

Broadly speaking, internet communities fall onto a spectrum between specialist and generalistic.

Specialist communities are those built around a specific common interest. This could be part of your identity, it might be a hobby, or it might be a shared profession. It's important to know that this is only a part of you and while it's easy to find people who are also into this thing, often any other overlap is coincidental or not guaranteed to result in connections that you will want to maintain. In my experience these communities end up feeling shallow because the social overlap is limited to a particular subject. It is certainly possible to build otherwise nuanced relationships with people in these spaces, but it's not as easy.

(Again, this model has its limitations, especially for communities by marginalised folks, where a shared life experience provides a common backdrop to society that influences every other aspect of someone's life.)

Generalist communities are those that are polymorphic, meaning that a (more or less) diverse crowd of people come together (maybe under a very loosely defined banner) to talk to each other. While it will be possible to meet people who share a wide variety of interests to you, it can also feel like searching for a needle in a hay stack because there are a lot of people, and finding potential friends in a huge diverse crowd of people who you may not have many things in common with can be difficult.

Most social media sites will reflect any number of community variations on this spectrum and Twitter is no exception in that. There are subtle differences in how these categories work depending on how you engage with the platform. In my time on Twitter as an individual I felt like I was building multiple sets of specialist communities. Once I became a brand however this shifted and the focus relied more on building a generalist audience. No longer could I assume that my followers were sympathetic to me, and so came the consequences of presenting my authentic self in front of an ever-increasing crowd of judges.

The "algorithm" & the timeline

A few years ago social media platforms started pushing for non-chronological, behaviour curated timelines. At first this caused an uproar, when users stopped seeing their friends' posts in their feed and instead started seeing random sponsored content; whether someone had paid for it or merely engaged with it. At this point you will also see content recommendations from groups and hashtags.

But this controversy died down over time, as could maybe have been expected. Ironically running a poll on Twitter these days whether people use the algorithmic or chronological timelines yields in a heavy filter bias, which "shows" that most people are using the chronological timeline. Of course, most of your followers won't actually see your post in the end.

Non-chronological timeline design is at the centre of modern corporate social media sites. The goal is no longer to facilitate meaningful exchange between users and instead the focus has shifted to keeping users "engaged" for longer. The customers are ad-buyers and the commodity is your time. This is another open secret that we all conveniently ignore. You may think that you are above this, that you can't be manipulated into spending more and more of your precious time on this planet in front of your phone or laptop, refreshing, scrolling, refreshing, scrolling, but considering that the designers of these tools fall prey to their own creations ², you are not.

Importantly "algorithms" are also arbiters of content, and in the ideal case, equalisers. Someone with very few followers can have a post "blow up" if the algorithm blesses it, and someone with many followers can have their work buried if the algorithm deems it unsuitable. This might not be a problem for your shit posts. But when you try to use a platform to advertise your work, especially when you are financially dependent on it, this can be a scary prospect.

Twitter is by far not the worst platform for this, nor is it the one with the worst effect. Sites like YouTube, where artists spend hundreds of hours creating videos, are much more affected by this. But the mechanisms are fundamentally the same.

Social capital vs. Perceived social capital

Algorithmic timelines on Twitter feed into the dilemma between our perceived social capital and our actual social capital. Social media has largely become mandatory for branding, for advertising your work in some way. Especially if you are self employed or an artist of any kind, your livelihood may depend on your ability to promote your work and getting whatever you are doing in front of the eye balls of other people. Having a large following on Twitter will definitely help with this, but it's by no means a guarantee. Other platforms suffer from this too (again: see problems that smaller YouTube creators face³ that those who have "made it" no longer do).

I think this is the aspect that I struggled with the longest. I didn't judge my Twitter account by the actual social capital that I gained from it, I judged it by the perceived social capital that I thought to gain from sticking around. The reality of the situation was that any time I would genuinely talk about my work, advertise something that I had made or was proud of, my Twitter audience was useless. Almost as an insult the most popular of my posts were shit posts. Having recently talked about this with some friends, both with smaller and significantly larger audiences, this seems to be a somewhat common experience.

It's extremely difficult to distinguish these two values and to really understand what actual benefit you are gaining from sticking to a known habit. A lot of it is going to be projection. And fantasies will make it hard to quit. Don't think about the one time your work was valued, think about the hundreds of times that your work felt ignored.

Whatever current drama is happening

Twitter is stuck in a perpetual cycle of despair. One could argue that this is a more general trend in the 24-hour news-cycle-society that we live in, but Twitter manages to put its unique spin on it.

More importantly, no matter what is currently going on, you will find a lot of people threatening to leave Twitter if X happens or if Y doesn't happen. But they never do. At this point I don't think anything is going to make them actually leave Twitter. No design changes to the platform did it, the Nazis didn't, Donald Trump didn't do it, this acquisition by a certain apartheid profiteer won't do it either, and whatever the next thing is that's going on in the future won't do it either.

I truly believe that Twitter brings out the worst in people, whether it's the capitalist grifters who need an audience, or self righteous trolls who perpetually feed the discourse. It gives you the illusion of control by letting you curate an experience bubble, while barely even giving you that.

In the end, Twitter has become too large an investment in the hearts and minds of its users to give up. And to be honest, I've had enough of it to want to make excuses anymore.

In conclusion

I was in London at the beginning of October, watching the closing performance of The Prince at the Southwark Playhouse. After the play I walked around Elephant & Castle with a friend and we talked about social media and the performances we all play for each other (in line with the play, which you should definitely watch on or offline if you have the chance).

It was in that moment that I went fuck it.

We only get so many years on this planet and the world is a truly horrific and heart breaking place. And we are going to need all of our collective strength to change things. If we even still can. One thing I am certain of however: social media culture is not going to help us.

Corporate social media has been designed to keep us engaged to the detriment of our mental health. Being informed of something does not necessarily increase your agency over the situation. Instead it may simply be giving you the illusion of control by exploiting your emotional reactions and well-being.

Delete your Twitter account. You can do it. I believe in you :)

/ Afterword

One last thing I want to talk about is alternatives. I am still online, I still talk to friends. I even, in a sense, still have a public persona. I use the fediverse⁴ ⁵ to advertise my work, and have done so for years. In fact, every time I posted something on Twitter, I also posted in on fedi. The differences in feedback were astounding. It wasn't that every post got a lot of attention, but the ones I truly cared about did. Not only were they widely shared, they created conversation... honest to god feedback, discussion, and interest. Something practically unknown to me on Twitter.

Now... as a certain TV writer posted on mastodon.social back in 2017: "I wonder if being on here during my twitter breaks is like when I tried to quit smoking using cigars", and he might be right (love your work Dan).

I think the crux of the issue is how you approach your use of technology, and what kind of interactions that technology fosters. I don't want to turn this article around in the afterword and insinuate that any problematic relationship with social media is your own fault. It isn't. But, I think you have more power over your habits that you give yourself credit for.

I have three accounts on the fediverse: a public one, where I post realistically the same stuff I posted on brand-Twitter before it made me miserable, a semi-public one, where I share selfies, random things I'm working on, and a sort of micro-blog, and a private one where I talk to just close friends. The expectation to have multiple accounts with varying degrees of publicness is ingrained into the fediverse in a way that I've never really seen it on Twitter.

And yet, I wonder how much time I spend on the elephant website and whether it is useful to my life. In the end everything is a balancing act. And while I can still see a lot of good reasons to stick around on the fediverse (imperfect as it may be⁶), I can no longer say the same for Twitter.

Giving up on Twitter doesn't mean giving up on online society. Especially if you live in an isolating place (like the suburbs, or more generally North America), it can be incredibly helpful to find a wider community online.

But please don't use a switch in platform as an excuse to not examine the underlying relationships you have with the online.

https://www.nature.com/articles/s41598-022-15004-0 ↩
Watch "The Social Dilemma (2020)". It's not a great documentary; in my opinion it falls short in its analysis of what social media and the manipulation of public opinion actually means, outside of a very narrow "someone help me balance this, my democracy is dying". But there are some insightful interviews in it, by people who have worked on various social media sites, talking about their own experiences with social media addiction. ↩
https://www.youtube.com/watch?v=0HiNVQkamA4 ↩
https://fediverse.info ↩
https://wordsmith.social/elilla/a-futuristic-mastodon-introduction-for-2021 ↩
https://sporks.space/2021/02/02/mastodon-really-is-crumbling-and-it-will-only-get-worse/ ↩

Embrace being proven wrong

Katharina Fey — Wed, 13 Apr 2022 00:00:00 +0200

I recently posted an article on my Twitter feed about the shortcomings of Flatpak. It resonated with me and my opinions on a subject that, it turned out, I didn't know enough about. I had conversations with people who were far bigger experts in this area than me, both some in favour of Flatpak, and some sceptical, and came to new conclusions.

Don't get me wrong: I'm still not 100% convinced. I still have criticism of Flatpak, as I think anyone can have about anything.

But I opted to delete my tweet because I didn't feel comfortable with the hyperbole of the article. Worse even: in some corners of the internet the article had garnered a reputation for "speaking truth to power", feeding into a weird Red Hat conspiracy theory that I felt even more uncomfortable with.

Discourse neurology

This style of rhetoric, the "taking apart an argument so I don't have to think about it too hard", has become very common in modern discourse. This is driven by the "fight or flight" response of the amygdala in our brains¹. A feeling of physical anxiety floods us when we are on the defensive, for whatever reason. And all logic goes out the window.

Many of the arguments you see online (and many offline too, don't get me wrong!) center around this emotional response and it is one that our culture fosters as well.

Admitting to being wrong is seen as a weakness and connected to shame, a loss of status, and humiliation. And so, we are never wrong. We attack our intellectual oponents in order to relieve ourselves of the responsibility of having to engage with an argument that makes us uncomfortable. This is how filter bubbles get created too².

Hope for the future

But that's not how it needs to be. And that's what this blog post is about.

Because let's face it: we're all wrong about most things. There are too many things to be known, and too little time to know them all. And because society favours whitty comebacks at surface level discourse, this doesn't change the fact that we are filtering ourselves from properly engaging with arguments that we might find uncomfortable.

There is an antidote however: radical vulnerability and humility.

Being wrong is an opportunity to learn something new and to broaden your perspective on some topic you didn't understand previously. Approach arguments that make you uncomfortable with an open curiocity and you will find yourself agreeing with more things than you previously thought.

And this isn't about just changing your opinions either. By engaging more openly with things you disagree with you can actually increase your resolution on ideas that you hold dear. I'd also like to point out that not all ideas are equally valid. Fuck you if you think this justifies debating nazis...

Ultimately I want to approach life with a curiocity that doesn't exclude me from new and exciting things that may be happening in the communities I'm in, or from ignoring uncomfortable truths about how the system we are forced to live in works.

And maybe, if enough of us do this, we can make this world just a little bit better too.

My attempt at synthtober

Katharina Fey — Tue, 05 Oct 2021 00:00:00 +0200

It's October! This year was weird, maybe even more so than 2020. The year felt very short, and very long at the same time to me; more so than I usually feel around this time of year.

As the dark season comes around it's good to remind yourself to focus on the things you really care about. Make sure you get plenty of light, maybe change those 2700K bulbs for something 4000K+. Colour temperature can make a real difference whet it comes to SAD...

Anyway, it's October, the month where everybody draws or hacks or drinks themselves to happiness. A friend mentioned last weekend that she was going to try to do synth-tober: creating a small song or jam or whatever every day. I haven't published a song on my bandcamp in almost (or over?) a year so it's high time I get out of my rut.

This blog post will be updated repeatedly throughout the month. I doubt that I will manage to make a song every day, but I want to manage at least two a week.

Some uploads will be very unpolished songs, others might just be me fucking around with interesting sound design. We'll see :)

Happy October!

Entry 1: Uneasy

Technically this started as a jam but then I saved the patterns and went to work so that technically makes it a song, right? I have been playing around with my Octatrack a lot recently so I wanted to get back to basics. I grabbed my analogue drum machine and the Circuit and used it for drums and synths. The result is...eh, listen for yourself!

Entry 2: forest

I haven't picked up my guitar in months. In fact, a thick layer of dust had collected on it. Now... my guitar is old and kinda broken. It only has 5 strings (like all the pros recommend), and it's internal wiring seems a bit shoddy with the top pick-up periodically no longer picking-up.

What I put together here is an effects chain in bitwig, which I fiddled with a bit while and after playing some random chords and riffs. The layers of effects almost hide the fact that I am extremely out of practice at playing the guitar ;) Hope you enjoy!

Devebruary (Week 2)

Katharina Fey — Mon, 22 Feb 2021 00:00:00 +0100

At the beginning of the month a friend a I decided to spend four weeks seriously hacking on our games, and blogging about it. We called this event "Devebruary": this is entry for week 2. You can find week 1 here.

Current state of the project

This post will undoubtebly be much shorter than last week's. I didn't get to work on the game as much as I would have liked to. I did however spend a bunch of time working on basics that I need to continue.

I wrote a map file loader for the server (and client too I guess), which wasn't too hard. The next problem I had was that I didn't just want to write one of these files by hand (which is certainly possible), but already felt like I needed to have a way to edit maps in-game. So I started working on a map editor. You can launch this state by passing --editor to the client as an argument.

As part of this I also needed to deal with the fact that the ggez crate doesn't expose any sort of UI element system. It bundles the lyon crate with some utilities to draw meshes, which I decided to use to write UI elements.

I started with a simple button which can be clicked.

Next up I tackled the input and event system in ggez. There is a central trait called EventHandler, which provides a set of functions that should be called during update, render, and other event times. The problem is that there's no mechanism to provide the main event loop with more than one of these objects, meaning that a lot of user-space code ends up just passing these objects around and calling the correct function on something.

So instead of doing that I wrote an EventLayer abstraction which takes a set of event handlers (which can also be created during runtime!) and proxies events to them. This obviously adds some run-time overhead, but the effect on code quality has so far been very positive.

So this week then...

I hope to finish my input refactoring today, and hook up the UI initialisation code to it. Then I can start building UI building blocks that automatically react to events as they happen.

The same input trait that buttons implement should also be implemented by all other game elements: nodes need to be selectable after all...

I'm wondering if there should be a management type in between the input events and the game entities, or if its sufficient to have them each be managed on their own.

Also hopefully I'll have more screenshots next week

Devebruary (Week 1)

Katharina Fey — Sun, 14 Feb 2021 00:00:00 +0100

Last week a friend and I decided to spend four weeks more seriously hacking on our games and blogging about it. We called this event "Devebruary", even though February had already started. This is my first post.

The current state

I've been working on a game called "RST Node" (read "Reset Node") for the last few years, albeit only conceptually. It is a 2D (top down) RTS, set on a computer network. A match consists of multiple players starting with a single node on this network, attempting to capture as many of the surrounding computers as they can. As part of this they need to build compute clusters to gain resources, and attack the enemy with various exploits.

I started programming on this game last year in January, but only worked on it for about a month before running out of time and energy (did something else happen in 2020? Can't remember). The game is written in Rust and AGPL-3.0, seeing as it uses a component from my research project (Ratman), which is also licensed under the AGPL-3.0.

What I've done this week

The code up to this week was entirely backend focussed. I had layed out the basic structure of units, abilities, and matches. Nothing was being drawn yet. To render a client I'm using the [ggez] Rust crate, which is a Rust clone of the [love] framework. I'm not a huge fan, and if the game was any more graphically ambitious I'd want to find something else. But seeing as I'm only really drawing simple 2D graphics and doing some camera (i.e. viewport) manipulation, this will do.

The following screenshot demonstrates the current client state. Sexy, I know!

The asset loading probably took the most effort in this demo. I'm creating art assets in inkscape as SVGs, loading them at game start, and converting them into sprite textures with librsvg. This way the game can scale across many resolutions while not requiring a very large assets directory.

On the server side I looked into using QUIC for client-server communication, but the only crates I could find were either synchronous, or used tokio. The rest of the game is built on async-std, and I didn't feel like either adding a second runtime or porting the game. So instead for now I'm using the async-std UdpSocket abstraction, and if I run into packet delivery problems later, I'll either switch to TcpSocket, or write an async-std wrapper with the [quinn-proto] crate.

What I'm planning for next week

So far the rendering is largely disconnected from the backend map types. Ultimately the renderer needs to be able to render a mesh of nodes that are distributed through a 2D space, with links between them. Each link also needs to have a length based on the distance between two nodes, that determines how long it takes packets to traverse it.

I already defined a structure of how maps work (as sets of nodes and links), although currently without x/y coordinates. In the next week I want to integrate map loading into the asset loader (and also make the asset loader available to the server which ultimately needs to load the map configurations)

Maybe I'll also make enough progress to have a first packet traverse a link. But we'll see!

Irdest: history of a fork

Katharina Fey — Thu, 04 Feb 2021 00:00:00 +0100

TLDR: we're forking qaul.net to Irdest! This includes myself, and four other developers who have been on-and-off working on qaul in their free time over the last year. We have a new domain, have largely re-branded the existing code-base and website, and are submitting Google Summer of Code projects to find people who would like to contribute code while getting paid.

The background

If you're a regular follower of my work, you may know that I've been the lead developer of a project called qaul.net, a decentralised mesh routing application and app development toolkit. The project itself is not mine and was started around 2011; I joined it in 2016 as part of my Google Summer of Code. But I quickly took over a lot of the core planning and development aspects, and even worked on it as my full-time research job from 2019 to 2020.

In the summer of 2020, the Open Tech Fund, which was funding development on the project, was defunded, and development stopped for a few months. During this time, I was also recovering from some health issues, and needed to take a break from the project.

Where things went wrong

Software projects are social entities of the people who work on them, not merely technical problems to be solved. And while I have enjoyed my time working on qaul.net, I also feel like I can no longer overlook the problems in how the project is organised. I don't want to talk about the specifics, and I won't mention anyone by name. A lot of my issues with the project revolve around how one of the original maintainers sees the project itself, and his role compared to others who work with him.

Ultimately, social relationships, whether personal or professional, need to be rooted in mutual respect and understanding. Neither of those two are the case with regards to the original maintainer in question.

Over the last year and a half, I have repeatedly brought up issues with him, about his conduct, the way that he approached my work, and our interactions, and have tried to engage with him in conversation about our working relationship, and how we might improve it. In all of this time, I don't think that I have had a single productive conversation with him, as equals.

It became apparent to me, that working with this person would continue to negatively affect my mental health, and so I (with the help of some amazing friends!) made preparations to fork the project.

Why now?

Initially we were planning to keep this as a very soft fork. The "upstream" project wasn't really doing anything. Nobody except me was putting in code at that time, and it was a happy neutral ground where we were prepared to take the project in a different direction, but didn't need to force the issue.

Yesterday, the original maintainer announced that qaul.net had received new funding. He e-mailed me to inform me that he had undone a bunch of work I had put in to improve our website and also removed my access from most of our infrastructure.

It became apparent that a low-key fork was no longer possible. So instead, we went ahead with some of the final re-branding we had been putting off, and are going ahead with the fork.

Why write this?

I want to work on this project, whether I get paid for it or not. I'm in the fortunate position of being able to put in time on the project while doing other jobs, or in between clients.

My main concern is not having to interact with the original maintainer in question on a regular basis, and not working in a structure which is authoritatively structured around him.

I write this blog post as a warning to anyone who is interested in contributing to qaul.net, and might have to interact with this person as well.

It would make me very happy to see more people engage with qaul and getting paid for their work. I simply fear that people might be walking into an abusive situation without being aware of the history. And thus...this post.

Ultimately, I'm mostly relieved. The last attempt to communicate I made was back in November, and since then I had heard next to nothing until this announcement. It is clear to me that my opinions and feelings in this matter are not of importance, and so, I feel justified to move on.

Thank you for reading. If you want to learn more about Irdest: https://irde.st, and if you want to support my work directly, I have a GitHub sponsors.

Edits

2021-04-04 - we changed the project name from qaul to Irdest. A previous version of this page can be found in the web archive.

That's the wrong abstraction layer

Katharina Fey — Wed, 11 Nov 2020 00:00:00 +0100

I'm writing this post mostly to my future self, not any specific project or piece of code I've seen other people write. That's not to say that I don't think this is something that probably applies to many projects. Sometimes it's easy to lose sight of what we're doing, and it's good to be reminded.

So to start at the beginning: I've been working on supergit, a Rust library to parse git repositories. It's built on top of libgit2 (and the git2 rust bindings), and aims to create a more Rustic interface and type fascade for git repositories. It also aims to solve issues such as: rename detection, path-history, and subtree management. I'm writing this library for octopus, which will eventually host my monorepo.

In supergit the main workflow is around iterating things, seeing as git is an acyclical graph, and iterators are a decent way to view this datastructure. But git graphs can get pretty big. I wanted the iterator to be configurable in a way that allows someone to write a tool that searches a whole repository history, while also making it possible to step through a history 20 commits at a time (to implement history pagination on a website, for example).

Looking at the current API, this is how you would implement the latter, for a main branch:

use supergit::Repository;

fn main() {
    let path = ... // get your repository path somehow
    let repo = Repository::open(path).unwrap();

    let main = repo.get_branch("main").unwrap();
    let iter = main.get(20);

    iter.for_each(|c| {
        println!("{}: {}", c.commit().id_str(), c.commit().summary());
    });
}

That's easy enough, right? But wait, why am I calling .commit() on c. Isn't it already a commit? Well...sort of. In supergit, this type is a BranchCommit, because this is where things get complicated.

Sort of like a tree, but not really

In git, rarely is a branch just a history of single commits. Maybe this is how some people think about their history, but it certainly has never been the case for any of the repositories that I work on. Basically the second you have more than one contributor, it's very common for a history to have merge-commits in it.

So how do we deal with that in an iterator? The design I chose was to wrap a Commit object in another type, which can convey this state. BranchCommit is an enum and has three variants: Commit (maybe I should rename that to Simple or something?), Merge, and Octopus (if you don't know what an octopus merge is, don't worry about it. Most people don't and they're very rare and weird).

What Merge and Octopus contain are new Branch handles (the type returned by get_branch()), meaning that for every split it's now up to the user to decide whether they want to continue first-parent (i.e. only ever follow the main branch line, ignoring the history of merged branches), or if they want to enumerate the histories as well. Most importantly: for every branch merge, you get to re-decide what your iterator strategy should be: infinate, limited by number, or limited up to a certain commit-hash.

So far so good I thought, this is an okay enough interface for me to work with. But this is where some problems appeared.

File histories (and git internals)

(a slight de-tour through git - feel free to skip)

The main reason why I'm writing this more Rustic wrapper around libgit2 is to make it easier to determine what the history of a file has been. This is pretty simple to find out via the git CLI (git log -- <your file here>), but not something that libgit2 exposes, because that's not how git stores data.

To git, all data is stored in a key-value store indexed by a SHA1 (soon to be SHA256 I think?) hash reference. That applies to files, full file trees, and commits as well. Say we have a file acab.txt, we commit it and it gets the ID da39a3ee5e6b4b0d3255bfef95601890afd80709 (the file ID, not the commit ID!), but then we open it and write ACAB in it, and commit that again. Now the file ID is 99f069b8a0cbe4c9485a14fe50775d0f71deb4e7. Both these files are saved in the git object store, because after all you might want to go back to the older version.

But here's the thing: from the actual commits we can get two things: the file tree at the time of commit, and the commit parents. To figure out what actually changed in the commit, you have to diff it against it's parents, which is exactly what git show does if you give it a reference to a commit.

What this means is that if you want to have a library that grabs the history of a path, well you'll have to go through all commits, and check the tree for changes at that specific path. Furthermore, that won't actually let you know if a file has simply been renamed, only that it has changed. Further logic is required to figure out if the file is the same, but just has a different name.

And all of this is something that supergit implements, behind a nice Rustic API (I hope...).

Bloated abstractions

So I wrote a function that would, for a branch iterator, step along it and check the history of a path, by diffing each commit with it's parents, and tracking a path via the delta information in the diff. But this is where I ran into problems. Because my iterator design always chose the first-parent to step through. Other branches were ignored, and because the function accepted an iterator and stepped it internally, there was no way for my file_history() function to figure out the exact behaviour the user wanted.

My first instinct was to implement branching in the BranchIter itself; allowing it to branch off, essentially pushing commits it would have to get back to onto a stack, and resuming from a previous position. That turned out to be a really bad idea.

It took me about an hour of banging my head against this abstraction before I realised that it wasn't meant to be. Sometimes systems are self-contained, and adding more functionality takes a considerable amount of effort, and begs the question, if it's really the right choice to make. Why add more functionality to an abstraction that works fine on it's own?

Instead, embrace composition, and add another layer on top, that can use the previous. You end up with a much more managable design, and data can flow from one layer to the next. Make sure that your interfaces are flexible enough to be re-used, but don't think that just because a component could technically be responsible for some work, that it really has to implement this work.

And that's it basically. Thanks for reading my ramblings about git and one of my side-projects. I hope I managed to make you think about the way you build systems a bit, and maybe next time you are in a situation similar to this one, don't be like me :)

"The good place" vs. the ethics of society

Katharina Fey — Sun, 20 Sep 2020 00:00:00 +0200

A few months ago I was bored and I decided to watch "The good place". It's a show that had been introduced to me before, and I even watched about half of the first season, before I kinda forgot about it. It had left me feeling mostly irritated, and uninterested, and so I moved on with my life. Up to the point where I felt really bored, and I started watching it again.

I don't really wanna talk about the show from an art criticism perspective. It's quite fun to watch at times, the premise is quirky and all the characters have something to set them apart that makes them recognisable for someone who's bad at differentiating people. But it's a comedy at it's core, and most of the "humour" left me feeling kinda cold. It didn't so much have jokes as much as just vague references at jokes.

Really, the show wasn't special, funny, or even bad enough for me to really care about it too much. There was however something in the moral text, and subtext of the show that bothered me, that I've kept thinking about. And that's what this post is going to be about.

Good vs Evil

The main premise of the show is centred around the idea of "good people" vs "bad people" (the good place vs the bad place). It mirrors heaven and hell, without putting a precise theological term on it, because this concept has existed in various faiths throughout the ages.

The story follows a woman who gets sent to the good place even though she's a horrible person. Most of the first season is dedicated to this mystery. At first she thinks this is a mistake, until it becomes apparent, that bad people being put into a fake "good place" is part of a weird psychological punishment system in the bad place. They are in fact in the bad place. When they find out about this, their memories get wiped, and it starts from the beginning, with slight alterations. But the group figures out that they aren't in their personal paradise again and again, and so their memories get wiped again, and again.

The show wants to demonstrate that people can get better, seeing as a group of "bad people" were sent to a fake "good place", and improved as people. The permanence of "good people" and "bad people" is called into question. Some stuff happens, and the group of four people, and one daemon who started taking a liking to them end up on the run.

Throughout the plot it becomes apparent that the system is broken in more subtle ways too: nobody gets to go to the good place anymore. Nobody is good enough; too high are the standards of what counts as a "good person". Furthermore, when they manage to get into the good place, it becomes clear that eternal bliss with no ups and downs, and no end in sight is just a different type of hell.

The show concludes by restructuring the system, making the "bad place" not into a torturous nightmare, but a place where your actions and emotions are being tested, and called into question. The idea being that there is no such thing as a "bad person", and that everybody could go to the "good place", if they accepted that they have flaws, and worked on them.

They also mildly restructure the "good place" to have "an end", which is death. Isn't that nice, everybody gets to live their perfect life in heaven, then they die.

Good people & bad people?

So that was the plot. As I said, I'm not gonna criticise the show for it's scene-to-scene writing, or even the overarching plot. It mostly tries (and manages) to be wholesome. Although it has issues throughout, that are rooted in a very flawed understanding of philosophy and morality.

The moral compass of the show is a character called Chidi, a professor of moral philosophy who died and was sent to the "bad place". He was deemed a bad person because of his indecisiveness. It is shown that he tried to be a good person, but got too caught up in the details of what that meant, which caused great pain to the people around him (and which got him killed).

Throughout the show he quotes Kant a lot, with some other racist white men from history sprinkled in there. His understanding of philosophy isn't very deep, or nuanced. Either he was supposed to be bad at his job, at which point the show didn't really take the time to develop this enough to be poignant, or it just demonstrates that the show was written by people with basically no knowledge in this field.

I argue that the way that "the good place" portrays philosophy and moral choices in philosophical frameworks is very representative of how our society works, and how people think about "good vs bad".

But let's back up a bit. For most of the show (if you watched it/ will) the thoughts it is trying the hardest to communicate are "there's no bad people", "hell is a bad concept", etc. This becomes pretty obvious. However, the larger system of afterlife remains pretty much entirely un-examined. Why is there an afterlife, and why do we need one, these are questions the show never asks, or attempts to answer. Any critism against the system is phrased in a coy way, that will lead to reform of it, not abolishment, i.e. changing what the "good place" and "bad place" means, not their existence.

Moral individualism

I said the show is representative of how people think about morality, and this doesn't just start and end at "what is a good person". It also applies to how the show deals with individualism.

What is individualism you may ask? I'm glad you did (not really, now this post has to be longer...). Individualism is one of the axiomatic philosophies that the western world is built on. It's the idea that each individual is responsible for their own destiny, and identity.

Used in a (mostly) harmless way it's used to sell things to people that can be "customised" to fit your "own personal style" (without really giving you any autonomy), whereas on a higher and more sinister level it is used to justify the horrors of society. As an out of context Margaret Thatcher would say "there's no such thing as society, only people". After all, society is just men, women and those damn enbies, that all make their own free choices, and if society is bad, then that's just a representation of how people are bad.

This is an over-simplification of course, but it digs at the core of what individualism means to us. It's a way to absolve society of guilt, up to refusing the existence of it all together. Individualism touches many, if not all aspects of society, and it would take too long to really examine them all here. Instead, I want to focus on what this means for "the good place".

There is no society in "the good place"

I don't know if the word "society" is ever used in the script, but it is certainly not a subject of conversation in any of the episodes. None of the characters will acknowledge that there is a human society or what it looks like. The focus is on individuals. After all, the fact that the world is bad is the result of just a few bad people, that need to become better.

This is where the view that "there is no bad people" the show tries to hammer into you falls flat. Because it's a lie.

Human society is structured in a way that a few select people at the top have a lot of wealth and power, while the rest of us live in varying degrees of poverty by comparison. I grew up in Germany so I'm gonna say I don't live in luxury and peace by comparison to others, but we all suffer under the ruling class. This is a reality the show refuses to acknowledge and it makes it's arguments about moral philosophy feel almost dystopian.

Maybe this is controversial, but there are bad people. If it is your job to harass homeless people, you are a bad person. If it is your job to enforce the "war on drugs" that overwhelmingly affects black people, you are a bad person. If you are a billionaire, you are a bad person. You are in a position where you could change society for the better. You could give all your wealth away, and actually help people. But you don't. And no, I don't mean the fake philanthropy that rich people indulge in because those are usually just schemes to pay less taxes and massage on their public image. No billionaire ever gives away so much money that they stop being a billionaire.

The ending of the good place is framed as a beautiful thing where everybody gets to live a life in heaven in the end, if they manage to work on themselves to become better people. And sure, there are "bad people" like sexists and racists, and they'll just get stuck in these tests forever that they won't escape until they become better people. It doesn't matter how much suffering you've caused others, you get to go to the good place if you manage to accept that you were bad.

Why an afterlife?

So I mentioned that in the show, the existence of an afterlife is never explained, rationalised or called into question. It exists in a vacuum, the same way that people in it live in a vacuum.

The ending of "the good place" is framed in a way that is meant to make you feel happy and hopeful, but all it makes me feel is wonder why we needed to wait until the afterlife for people to deserve happiness.

The world is an awful place because of people, sure, but it's the system that makes people into monsters. Not only will it corrupt people going in with good intentions, it will turn people with bad intentions into powerful rulers.

"The good place" fails to or refuses to understand that society exists, and portrays a moral system in which all actions are unconnected from the bigger picture. If you were a nice person to people in person, and generally tried to be g o o d then it doesn't matter if your employees need to pee into bottles, or if your company is burning the rainforest to ash.

Hell, the ruthless business lady in the "medium place" was sent there because she saved someone in her last moment. But the "bad things" she did??? SHE WAS RUDE TO PEOPLE. Don't worry the exploitation through the capitalist machine, that's all fine.

The shape of art & paradise

To wrap up this article I want to at least mention why I'm writing about this. Because I said earlier that I didn't find the show special, funny, or intentionally bad enough to really engage with it. And now here I am, writing upwards of 2000 words about it :)

The media we consume as people shapes us, and influences us in quite profound ways. The way we tell stories is symptomatic of how society perceives itself, and how people see themselves in society. Media that doesn't acknowledge the existence of society then and the suffering it brings will inevitably white-wash reality, and push this influence on anyone consuming it.

At this point I would have liked to mention a better show or movie, or even book, but none really come to mind. I guess it's hard to point to any text and demand it delivers a coherent world philosophy, while also being a story with characters and plot.

As a society we need to grow the fuck up. The stories we tell each other of heroes and villains, the balance between good and evil hanging in the balance, all while these actors exist outside of anything that could be called a power hierarchy, needs to end. Only when we grow up from this world view can we realise that paradise is within us, and that collectively we can create it here on earth.

Not gonna lie though, trains that go through space are pretty cool.

On gender, transition, and re-transition

Katharina Fey — Thu, 18 Jun 2020 00:00:00 +0200

It's pride month. Which actually has nothing to do with this post, but might have inspired me to write something. Anyway, if you're someone who doesn't like thinking about gender, and only follow me for my s p i c y / r u s t / t a k e s, maybe skip this one. Also, if you're a TERF and use any of these words to harm anyone, go choke on a brick you fucking piece of human garbage.

Cool, that's the disclaimers out the way.

The part that's all about me

I guess this is a kind of coming out: I'm trans! I've been "on the internet" for a while now, and have way too many people following me than is reasonable for my boring life. And I guess a lot of you might not know this. I've never made a point of it, and I know from personal experience that people read my gender very differently.

Some timeline stuff I guess. I'm not gonna do the "I've always been trans" thing, because I know that's not true. That being said that I was quite miserable before my transition for a few different reasons I won't get into right now.

I started socially transitioning around 2011, then started HRT in 2013, and then...well, just kinda lived my life. I'm not gonna sit here and pretend that transition wasn't the correct thing for me to do, and I am a much happier and well-rounded person now because of it.

Why am I writing this? Because after all I've been pretty happy just being stealth (meaning not being public about my trans-ness) for a while now. And I'm not the kind of person who wants to share my personal life all that much. It's weird being on the internet when everything you say is gonna be seen by thousands of people.

Well...the reason is that my gender identity is changing. Has changed. I guess I've always been a little butch, but in recent times (meaning the last ~6 months or so) I've been feeling explicitly more masculine. I've wanted to go by he/him pronouns, wear different clothes, express myself differently in public, grow a beard (something I've always failed at lol). It wasn't just that my idea of what being a woman meant changed, I think fundamentally the way I related to my gender changed.

And this is where things become really complicated.

The part that's about society

The way that our society at large handles transgender discourse is toxic. From the very beginning of my coming out, there's been an inscrutible focus on "why" people are trans. There must be medical reasons. Look at this brain scan of this one transgender lady. Her brain looks like a cis ladies brain. This will once and for all prove that trans people are trapped in the wrong body! ...

I understand why this framing has come about and stuck, because it was a great way for somewhat liberal people to convince more conservative people that "no actually trans people are like...real, and not just making it up". This line of reasoning is called "trans medicalism" and it's rooted in the idea that trans people are scientifically the gender that they say they are.

This approach has several problems. It is extremely oversimplifying, and makes assumptions about the nature of gender that many people would not agree with. Worse, even the people who don't really believe in it, who only use it as a weapon against the TERFs, to defend their own identity, end up upholding the European gender model binary, one that would rather many non-binary and even transgender identities didn't exist in the first place. It is a model that will never truly accept you for being trans, only sort of tolerate you, because maybe you're close enough to the status quo to fit in.

And a lot of trans people start internalising trans medicalism as a survival mechanism. This is where this discourse becomes harmful. Because not only does it prevent some trans people from actually expressing their non-binary gender identities, now you have insecure people who are threatened in their identity by the idea that there's no medical truth to being trans, and bully people who don't conform to this.

Even worse, they will sometimes align themselves with TERFs to defend the "true trans people", before "the younger generation ruined everything". And this is where this all comes back to me.

Gender isn't fixed

During the last few months I've been trying to find experiences by people similar to me, and it made me very very scared. I didn't really know what to call myself. Because I'm by no means a cis man, but looking for people who "de-transitioned", I found a lot of people who were hurting, who felt they had been pressured into transition, and who were being rallied around by TERFs who thought these poor souls proved their bullshit points of views.

And I saw a lot of trans people yelling at any trans person even considering "de-transitioning", as some kind of traitor. I guess I understand why. You don't wanna be giving the TERFs more ammunition. You don't want to undermine your own identity. Maybe you don't really believe in it, but your self esteem is built on trans-medicalism. How do you deal with people who de-transition?

I'm still not really sure what I would call myself, because I think de-transitioning is the wrong term for what I would want to do. And really, I think I'd like to think about it as just another section of the life-long transition of my gender. To live means to change, and my gender will change until I die. There's nothing I can do to stop it, and I think trying to control it will inevitably fail.

I think it's also important to point out again that I regret nothing. I'm glad I've been living as a woman for close to 10 years. I don't know how I want to express my gender identity, or on what scale neccessarily. Maybe I'll use different pronouns with friends, maybe only from time to time, maybe I'll change nothing in the end because this is all "just some phase".

But I think it brings me to the core point I want to make here:

Stop pretending as if transitioning into a gender is the end-all-be-all of your gender identity!

So what if something is a phase? In my opinion it doesn't make it any less valid. Transition is a journey, not a means to an end. And transitioning to femme, and back to masc (MtFtM), or vice versa doesn't make someone less trans. How can people believe that gender is a spectrum, while not accepting that people will move around on this spectrum?

The worst thing is: this is something I would have expected to explain to my mum, but I didn't expect this to be such a controvertial thing in the trans community itself.

Why write this?

I was thinking about writing this article for at least a few weeks now. And undoubtebly it'll be many days between writing it as a first draft, and the finished thing on my website. I revealed a lot of personal things in this post, things that I wouldn't otherwise want to share.

I think ultimately I want to be a voice of support for anyone who's feeling similarly to me: "older" trans people (I'm not even 30 lol), who have been doing this "new gender" thing for so long that it became normal, who might feel themselves wanting to either express themselves in much more feminine or masculine ways than before, or at different intensities, or more androgynously.

I think it's important that we remind ourselves that transition isn't a means to an end, that gender is ever changing, and normalising the idea of re-transition. And this doesn't just apply to cis people! Trans people carry the trauma of society with them and can be just as toxic in this matter as the TERFs.

Life is too complex for anything to remain the same forever. We all need to become better at embracing this.

Rust, or: how to run a community

Katharina Fey — Wed, 08 Apr 2020 00:00:00 +0200

This will very much be an off the cuff post about community building with insights that I've seen from various communities I was a part of in the last 10 years. None of this is to be taken as facts, and is entirely personal opinion. I do hope however that this post might make one or the other think about how we run communities. Really… I'm just bored and want to write something.

The communities I've been a part of involve Rust, Fedora, Moonscript (a lua coffee script type), libgdx (a Java game framework), Nix(OS), and the Homeworld 2 modding community that's to blame for me learning to code and unleashing my terrible puns on the world.

There seem to be two avenues of community organising: centralisation and distribution. One favours building a community around a shared name, domain, communication channel, and workflow. People work on the same things, in the same place, under the same name. This can work really well for smaller projects, and generally means less friction when people from different backgrounds have to talk to each other, because everybody is kinda in the same place. This is how most communities get started, and how Rust was working until about one year ago.

There is a fundamental problem with this model though: at some point your community will become too big, and people will start to fracture out into their little bubbles. There's no fixed point at which this happens, but it seems to happen sooner or later for most projects.

I think as community builders it is our job to embrace these fractures, letting sub teams spin off into their own little ventures, while using the shared name to advertise these ventures. I think it's okay to not have a project represent as a closed front to outsiders, but very much embracing the fact that projects are run by many individuals that chose to collaborate on something larger than themselves.

The reason why I'm such a big fan of e-mail collaboration via mailing lists (both for conversations and sending patches) is that it encourages this separation. A project that forces people who cross-collaborate to jump from tool to tool is just as centralised as a project that only has a single communication channel. But there's definitely examples of projects that have grown into little bubbles that still work on a shared "product", without having to all do it in the same place: the Linux kernel.

Whether you agree or disagree with my take on e-mails, I think we should all be aware of the finite size that a community can have. And at what point should we start to embrace community mitosis?

So there's a pandemic

Katharina Fey — Thu, 02 Apr 2020 00:00:00 +0200

(Note: this article was written as is, but never published. I decided to retro-actively publish it because I felt it was important to have it in my log of articles, but I also don't think I can really add much more to this now since the political climate has shifted quite hard.)

I've been meaning to publish this article a lot sooner, but here we are. The last month has been kinda bonkers. Reading this in the future might feel either funny or wistful. Reading this in the now must feel redundant.

Anyway, I've been working on a few things, and also have been thinking about a few blog posts that I've wanted to write. (One has already gone up, so once again I'm a literary genius with getting my ordering right). I'll try to write a bit more, and care less about each article being as polished as previously. I feel like I've said this before, so I'll try not to become too spammy. There's still Twitter…

In this pandemic I've seen a lot of rhetoric mirror the security discourse post 9/11. Additional restrictions on public life are being advocated for, because they will save lives. Checkpoints, new rules and regulations, everybody has an opinion on how things should be running and very frequently run to the state to enforce anything "health officials" say might work.

Ultimately this virus is the perfect threat: you can't see it, you don't even know if you have it, you can endanger people by moving around freely. And there will never again be a time when "but what about a virus" won't bu used to justify restrictions on the lives of people.

And please don't interpret me saying these things as there being some conspiracy to enact authoritariasism, as much as that sounds like a novel you'd read, that's not how the world works. This isn't plotting, it's simple fascist opportunism.

Collaborating with git-send-email

Katharina Fey — Thu, 16 Jan 2020 00:00:00 +0100

There's is a conversation that I keep having with various people, and while some of my thoughts are available in e-mail threads on my public-inbox, I felt like maybe it was time to write a blog post about it as well.

The reason for this is that there is documentation on the internet on how to use git-send-email in theory, but few ever really talk about the resulting workflow beyond a single patch.

I won't pretend that the tools couldn't use some work or that it doesn't take a bit of getting used to, but the reward is well worth it, and something that I feel deserves more attention.

At the end of this I will talk a bit about why I think this mode of collaboration is good, and could potentially be better than existing collaboration models.

The basics

To get into the basics of sending patches by email, I recommend git-send-email.io, which goes into the setup of basics on various platforms. It's one of those things where your setup will vary slightly, depending on your OS and email hoster, and not something that I feel needs too much more explanation.

You can go through that set of slides to send a test patch to the project that's hosted on sourcehut to see if your setup is working properly. This is enough to send short one-offs to projects without having to make an account anywhere (except the e-mail you already have anyway).

Discussion and patches

I think one of the main advantages of git mail collaboration is that the workflow of sending patches and creating meaningful discussion on patches is so interlinked. While you are using different clients to send patches and replying to feedback, the code that you send is still available in your e-mail client. So it's easy to reply to both feedback, while copying parts of a patch for reference.

It's important here to send e-mail as plain text, because otherwise it might cause problems for people to reply to. There's a great website that helps you make sure your e-mail client can and is configured to use plain text: useplaintext.email.

Patchsets and revisions

So having the basics out the way, I think it's important to discuss a more complete workflow. When people send contribtions to projects using pull-requests, often a set of changes will go through several revisions before getting merged. It's also nice to quickly force push to fix a small typo or similar without having to let that typo ever be part of the history of the commits that get merged.

When collaborating with git over e-mail this is still possible via "revisions". When sending a patchset, you can provide a -v parameter with a number. The patches you send will then have a revision number in them, as follows: [PATCH v2]. It's recommended to send newer revisions of your patchset as a reply to the previous one, i.e. [PATCH]: foo being the parent of [PATCH v2]: foo in the same thread.

If you get replies to your patch, you can make changes to your commits, then send out a new revision to the whole set, or just individual patches, if your set of changes contains a lot of code and you want to keep the volume of e-mails down.

The advantage of this is both that people can comment on things as they happen in the history of the code instead of being forced to understand a set of changes all in one go, and that you are automatically encouraged to squash commits with messages like "small fixes" before sending them out to a project's mailing list.

Cover letters

One neat thing that many people also don't know about are cover letters. Sometimes a set of changes is so large and requires some preface to make sense, it's a good idea to write an introduction for someone to read first. This is what GitHub pull-request descriptions were derived from.

To generate a cover letter you need to create your patches in two stages:

git format-patch to generate a series of .patch files that can later be turned into e-mails. This tool takes a --cover-letter paramenter that indicates to it to generate an empty patch called 0000-cover-letter.patch, which contains the diff-stat (git-shortlog) of your proposed changes. You are then free to edit this file in your favourite text editor to write a friendly introduction to your patchset.

Another often overlooked feature here is "timely commentary", are comments in the patch e-mail that won't be part of the patch or the commit message itself. They can be made after the --- marker in a patch mail, but before the actual patch starts. This section is usually used for the diff-stat of that particular patch.

After that you can use git-send-email, almost the same as before, but instead of giving it a series of commits to send (say HEAD~3), you now just say *.patch or wherever you saved the patch files earlier.

You don't have to resend the cover letter every time you send a new revision of your whole patchset. On the other hand, if things have fundamentally changed, it might be a good idea to add one again, just to make sure it's up to date for new people joining the thread for feedback.

An example

I always work well with examples and I think it's good to illustrate how all of this can work, especially for people who might be scared by the concept of collaborating this way.

I'm creating some patches for my libkookie repo and I want to get some feedback from myself, so I decide not to push to master, which I totally could do, but to my public-inbox instead.

There's two commits that I want some feedback on, so I make my commits, and verify that they are indeed what I want them to be:

 ❤ libkookie> git log HEAD~2..HEAD
 commit 3a147c15e998d57d9db877c9cd92d0cf04411cc9 (HEAD -> master)
Author: Katharina Fey <kookie@spacekookie.de>
Date:   Wed Jan 15 21:01:06 2020 +0000

    ws/kitty: setting default shell to tmux

commit d54937fa9414d87971a01dbc0dec5105b97e8f7e
Author: Katharina Fey <kookie@spacekookie.de>
Date:   Wed Jan 15 20:59:40 2020 +0000

    ws: adding gpg submodule by default

Well, perfect. This way I can also verify that the sometimes confusing range syntax in git (HEAD~2..HEAD, meaning all commits HEAD~2, so 2 commits ago, and HEAD, so now) works the way I'm expecting it to.

I think this is quite an impressive set of changes so I decide to reward myself with a good ol' cover letter.

 ❤ libkookie> git format-patch --cover-letter HEAD~2..HEAD
0000-cover-letter.patch
0001-ws-adding-gpg-submodule-by-default.patch
0002-ws-kitty-setting-default-shell-to-tmux.patch

I can go and verify the patches look okay, do a final pass over the typos and then edit the cover letter as well:

From 3a147c15e998d57d9db877c9cd92d0cf04411cc9 Mon Sep 17 00:00:00 2001
From: Katharina Fey <kookie@spacekookie.de>
Date: Wed, 15 Jan 2020 21:06:37 +0000
Subject: [PATCH 0/2] The best patchset in the universe

To whom it may concearn,

I have created the most magnificent patch set in the history of the
universe and I really think you should merge it because otherwise
you'd be a git.

Cheers,
me!


Katharina Fey (2):
  ws: adding gpg submodule by default
  ws/kitty: setting default shell to tmux

 modules/workstation/default.nix      | 1 +
 modules/workstation/kitty/kitty.conf | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

-- 
2.24.1

Perfect, they'll just love that over at spacekookie inc. I quickly exit, save, and close the file and send off the patches:

 ❤ libkookie> git send-email --To "~spacekookie/public-inbox"@lists.sr.ht *.patch 
0000-cover-letter.patch
0001-ws-adding-gpg-submodule-by-default.patch
0002-ws-kitty-setting-default-shell-to-tmux.patch
(mbox) Adding cc: Katharina Fey <kookie@spacekookie.de> from line 'From: Katharina Fey <kookie@spacekookie.de>'
From: Katharina Fey <kookie@spacekookie.de>
To: ~spacekookie/public-inbox@lists.sr.ht
Cc: Katharina Fey <kookie@spacekookie.de>
Subject: [PATCH 0/2] The best patchset in the universe
Date: Wed, 15 Jan 2020 21:10:48 +0000
Message-Id: <20200115211050.31664-1-kookie@spacekookie.de>
X-Mailer: git-send-email 2.24.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

    The Cc list above has been expanded by additional
    addresses found in the patch commit message. By default
    send-email prompts before sending whenever this occurs.
    This behavior is controlled by the sendemail.confirm
    configuration setting.

    For additional information, run 'git send-email --help'.
    To retain the current behavior, but squelch this message,
    run 'git config --global sendemail.confirm auto'.

Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll):

You can get the question about the Cc not to show up by providing --supress-cc all as a parameter, but I find it useful. Basically a Cc is just a ping, and if you're mentioning people by e-mail address in your patchset (for example, if you have Co-Authored-By lines in there) the appropriate people can be pinged for you automatically.

So, I'm happy with things as they are, so I hit "a", for all and send off all three e-mails. (You can find them in the archive here).

I wait, drink some chocolate oat milk, and wait for a reply.

Katharina Fey <kookie@spacekookie.de> (0 mins. ago) (inbox unread)
Subject: Re: [PATCH 2/2] ws/kitty: setting default shell to tmux
To: ~spacekookie/public-inbox@lists.sr.ht
Date: Wed, 15 Jan 2020 21:30:23 +0000

A comment on this commit:

> --- a/modules/workstation/kitty/kitty.conf
> +++ b/modules/workstation/kitty/kitty.conf
> @@ -1,10 +1,11 @@
>  font_size 10
> -font_familt twemoji-color-font
> +font_family twemoji-color-font

This was a typo before but I think we don't really want this feature
anymore, because all the font integration stuff is broken anyway.  I
think it'd be better to remove this line and then add it again when it
becomes relevant again.

~k

What's interesting is how feedback can be layered into the patch itself, to comment on changes that need to be made. This way it's possible to keep track of the relevant lines of code, and also be able to have a threaded conversation.

I guess I have a fair point here, the emoji fonts have been broken on my computer for ages. So while I'm somewhat annoyed by having to change things again, I can also understand why.

What I want to do now is reply with only a second revision on this one commit because I don't know if there's more feedback coming for the rest of the patchset. First, we need to figure out what the Message-Id of the previous reply is, either via you e-mail client, or the public mail archive of the project.

Note: this can sometimes be tricky, but usually you should be able to see the "raw" message in most mail clients to find the Message-Id of the e-mail you care about.

 ❤ libkookie> git send-email \
    --To "~spacekookie/public-inbox"@lists.sr.ht \
    --reply-to "<87r2001k7k.fsf@kookie.space>"
[...]
OK. Log says:
Sendmail: /home/.nix-profile/bin/msmtp -i ~spacekookie/public-inbox@lists.sr.ht kookie@spacekookie.de
From: Katharina Fey <kookie@spacekookie.de>
To: ~spacekookie/public-inbox@lists.sr.ht
Cc: Katharina Fey <kookie@spacekookie.de>
Subject: [PATCH] ws/kitty: setting default shell to tmux
Date: Wed, 15 Jan 2020 21:42:56 +0000
Message-Id: <20200115214256.1770-1-kookie@spacekookie.de>
X-Mailer: git-send-email 2.24.1
In-Reply-To: <87r2001k7k.fsf@kookie.space>
References: <87r2001k7k.fsf@kookie.space>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK

The way that the reply works mean the thread now looks somewhat like this:

[PATCH 0/2] The best patchset in the universe 
 ↳ [PATCH 1/2] ws: adding gpg submodule by default
 ↳ [PATCH 2/2] ws/kitty: setting default shell to tmux
    ↳ Re: [PATCH 2/2] ws/kitty: setting default shell to tmux
      ↳ [PATCH v2] ws/kitty: setting default shell to tmux

I wait a bit longer and I get another e-mail thanking me for my contributions, and saying that the patches have been merged.

Sometimes it can be nice to re-generate a patchset with all the latest versions of patches, even if they've been sent to the list before, just to make it easier to apply them. But that's often also not required.

The conclusion

Hey, you made it all the way to the end of this post, congrats!

I think the way of collaborating I outlined in this post has a lot of advantages over currently popular models (i.e. pull-requests on GitHub or merge-requests on GitLab). People talk about wanting to decentralise development, escaping these walled gardens that companies have built, and they often disagree on how this can best be done.

There's even people who gladly opt into this model because they feel that the added gamification of the platform will get people to work more. Not only do I think that the relationship that people have with maximising a number on a website can be abusive, but also that I've felt better getting patches into projects via a mailing list than any PR has ever made me feel.

I'm not gonna pretend that the tooling for all of this couldn't use some work: git-send-email has 1000 confusing options and also getting the Message-Id to reply to patches with can be hard and annoying.

In fact, I'm working on some tools to make both sending and applying patches easier (as part of the dev-suite project started by my friend Michael. I'll write more about this soon!)

In this model of development there's no need for a central service like GitHub, no need for special software to make pull-requests federate or even for you to host a copy of the project anywhere.

All you need is the code the project provided you, a text editor and a mail address.

Some website design changes

Katharina Fey — Fri, 03 Jan 2020 00:00:00 +0100

Howdy, it's that time of year again where I apparently do design things on this website (the last article is from exactly 2 years ago). Apologies if you've never been to my website and only use the RSS feed. Also: you're cool!

In previous iterations of this series I tended to fundamentally change the way that the website worked, either by changing the way that the html was generated, or by cutting down/ adding categories. This time I'm doing none of that.

Indeed things are getting simpler but mostly on the CSS side of things. I was able to delete about half of my CSS which is a pretty cool. The biggest change in the way the website looks is the article overview and the article pages themselves.

Generally I didn't like the card design style very much anymore so I wanted to change that. But I also wanted to make it simpler to see all my articles at a glance, without any summaries. I feel this makes the blog feel more "web log-y", which I like. It also means it's now consistently bright text on dark background and I think I've gotten the typography down enough to make it all pretty.

Anyway, there's more things I wanted to do but those will come later. I should also point out that my primary code host for this website isn't github anymore. It's now hosted on sourcehut and collaborations are still welcome (if you see a typo or have general comments), albeit not with pull requests anymore. You can submit patches to my public inbox which I have hinted at in a previous article.

In post-congress news I seem to have caught a cold, so: bye.

curl up in bed

Another decade down the drain

Katharina Fey — Wed, 01 Jan 2020 00:00:00 +0100

Now that the last decade is officially over, there's some things I want to write about. I do this less so for others, and more for myself. Around this time of year, people usually make new year's resolutions, which is not quite what this is. Instead, I want to reflect on what the last ten years have meant for me, and some plans I have for the next decade.

I graduated school in 2011, started uni twice, dropped out twice (for consistency), and found computer science and programming as my passion. I moved to Berlin, worked many jobs, at a bunch of different companies and finally, towards the end of this year, made myself independent as a freelancer.

Why am I mentioning any of this? Well, I don't think it's really possible to plan for the future, so I won't try. Instead, I want to set myself a goal for the next ten years. Something that has nothing to do with my career, or even a hobby that I hold right now. But let me back up a bit…

Over 15 years ago (oof) I did an exchange program, which had me living in France for six months. During this time I became pretty fluent in French. Over time, unfortunately I forgot a lot of it again. Then, over the last few years I started learning a few other languages, apart from English and German (I consider myself bilingual). I am conversational in French and Esperanto, and can understand a few words of Russian.

So what's my goal for 2030? Easy: I want to learn ten languages well. This includes becoming fluent in the languages I already know partially (French, Esperanto, Russian), as well as learning a few new languages entirly. Some that I am interested in are Arabic, Catalan, Spanish, Scottish Gaelic and Kurdish. I don't know if it will be this exact set that I will end up learning, but it's a good starting point.

Most importantly, I think that a decade is enough time to undertake this venture. I don't know where I will be in ten years, or what I'll be doing and neither do I think that it's ever really possible for me to guess. But whatever my life looks lke, I hope that I'll be speaking a lot more languages.

Part 1: Against Primitivism

Katharina Fey — Sun, 24 Nov 2019 00:00:00 +0100

This is the first of two blog posts that will be slightly more philosophical than other texts on my blog.

For some of my regular readers this thesis might not be particularly radical, but I still feel like it warrants being said.

What is primitivism?

I think this is the most important question to ask and one that has many answers. Depending on who you ask, and what their political background is, the answer might be "a joke", or even "a slur".

In simple terms, primitivism yearns to return to a simpler time, removing technology from human lives as much as possible. This is meant to address one of the largest sources of anguish and anxiety in our modern society, removing it from the equation. In many places primitivism even frames itself as revolutionary.

The problem with this analysis is that it is inherently linked with privilege. This can take many forms. A mild form would seek to abolish the internet, personal computers and phones, arguing that letting people return to real-life communities will result in more happiness and a more "natural" life.

This fails to acknowledge that these technologies are life saving for many, giving both social outcasts and various disabled people a space to have a community.

But most often it is not those affected who make the case for these measures. Usually it is white, able bodied men that fail to understand how their perception of society is skewed because of their own biases.

An even more extreme form of primitivism would reject more general technological advancements, arguing for things to be "good" because they are "natural". This analysis, even more so than the last, ignores challenges that those who propose these solutions don't have to deal with: what about medicine, what about artificial aids?

Against the internet

It is true, that in the modern world technology has been turned against us. Large companies control the way that people interact with technology, track them, and more. While it is possible to live outside of this system, only a few people actually do. Even a lot of technologists (software developers, hackers, ...) don't fully manage to decouple themselves from the corporately controlled tech bubble. i.e. how many hackers use Google, Twitter, etc.

On some level it is understandable that the narrative of primitivism has emerged. This is not to say that these ideas are in any way new, but in a way they are making a comeback in certain leftist circles.

For someone who doesn't know how to code or has only minor technical literacy, this fight might seem lost. Approaches like the one previously outlined seem welcome. I feel it is important to point out though that the demographic of people coming to this conclusion is already skewed. More vulnerable people that are dependent on technology have a different analytical framework and come to radically different solutions (more on that in a future post)

It is this narrative that inspired these posts, at least in part. I feel that to proclaim to "blow up the internet" (for example) is lazy and counter revolutionary at it's core. It frames all conversation about improving technology and using it in our struggles to liberate ourselves as regressive, and somehow collaborative with an abusive system. Suddenly instead of talking about strategy to our solutions you are thrust to justify your work to people who misunderstand it's basis and see it as part of the thing you are trying to fight.

Misunderstanding technology

So what do I mean by that, and do I have an example? I'm not trying to say that someone has to be a programmer to critique technology. I'm arguing that the same level of engagement people would expect of someone doing art criticism be extended to tech.

There is this notion that computers are fundamentally flawed, not because they are fallible and replicate a human's biases, but because of their foundational inner workings: binary! The sheer fact that computers operate on the basic assumptions of truths and falsehoods means that there is to assume to be universal truths.

Not only are conclusions from this hypothesis often shallow and reductionist, they also misunderstand the performative, interpretational nature of computers. On the wire every signal is analog. It is the translation to binary that gives them meaning. But: this does not mean it is representative of a truth, it is merely a projection of an assumption. The same way that axioms in mathematics are not "truths", but rather assumptions to build discoveries on top of.

The same can be applied to binary data: on the wire all data looks pretty much the same. Again, it is an interpretation that turns something into a text or a picture. There is no truth to data, only relative perspective.

Computers are indeed fallible and as flawed as the humans using them. But this is precisely because there is no underlying truth to computing, only the interpretations of those who make the instructions. This is why I argue that machines are merely an extension to ourselves rather than any "autonomous" system.

I say "autonomous" (in quotes) systems, because it is another term that is deeply misunderstood. But this time it is because the creators of these systems want it to be misunderstood. This is what the next essay will cover.

Rust 2020: the RFC process and distributions

Katharina Fey — Mon, 04 Nov 2019 10:00:00 +0100

I must have missed an e-mail in my inbox, because recently I started seeing people publish Rust 2020 blogposts so I thought, why not. I haven't been incredibly involved in the development of Rust in the last few months, (navigating the delicate balance of being self employed, working on free software and not burning out) but I feel like that might change again. And also, I still have feelings about software.

This post was also largely inspired by my friend XAMPPRocky's post, as well as attending NixCon a few weeks ago, and generally interacting with the NixOS RFC process.

What even is an RFC?

An RFC, or "request for comments" is a mechanism by which a group of people can get feedback from a wider community on proposed changes. The idea is that a written proposal outlines a change's scope, implementation details, rationale and impact on the ecosystem, then people make comments on the proposal. Usually by the time that everybody has stopped shouting at each other, the RFC is ready to be merged, meaning it is accepted and its vision can be implemented. This can either be implementing a feature, or removing unstable flags from it.

Unfortunately I'm not being too flippant here: the procedure of how an RFC goes from "proposed" to "accepted" is very vague and can depend on a lot of factors. Needless to say, this can also be the source of a lot of conflict in a community.

Rust has had an RFC process for a few years now, and most, if not all decisions to the language and ecosystem have gone through it, and the community feedback it entailed. Some go largely overlooked, like this one that I co-authored at the Rust All Hands 2018 in Berlin (it's fine, I understand), others get hundreds of comments. This often results in no meaningful conversations, in large parts because it's hard to have a discussion with 1000 people, and in other part because GitHub is a terrible platform to do anything on (sequel hook).

RFC chaos

I remember this issue first coming up in the module refactoring debates and the three (?) RFCs that were in total created before everybody felt happy enough about it. These were the first large RFCs I witnessed while being kinda part of the community. Many of the people who were involved in them talked about how stressful it had been and I think they might also be the first time that the RFC process, the way that the Rust project implemented it, started showing limitations of scale.

The fact that an RFC is proposed, with no real structure or framework on how to continue afterwards means that either feedback is chaotic and iterations on the design can seem arbitrary, or on the other hand some RFCs remain open for years, in limbo, where nothing really happens on them. Both aren't great outcomes, only add to stress levels of the people who were involved in writing them, and generally just slows down our decision making process.

As XAMPPRocky wrote in her blog post:

When 1.0 launched there was ~30 members of The Rust Programming Language, now in 2019 we have ~200 members. This is nearly 7x the amount of members, yet we've changed very little to be able to adapt to this growth.

While she was talking about how many people get paid for Rust, I feel this is also applicable to the way that we make decisions. Many people wrote about the RFC process for their Rust 2019 posts in rather vague terms, including myself. Well, I'm mentioning it again, because I feel like we should try something concrete.

Shepherds and an RFC committee

The NixOS project has two concepts in their RFC process which I think are valuable and that the Rust project would benefit from: RFC Shepherds and the RFC steering committee.

The RFC steering committee is a group of between 5-6 people, assigned for a year to oversee any new RFC, make sure that shepherds get assigned to it, and also keep tabs on progress that is being made. Are shepherds regularly (in whatever interval they deem appropriate) meeting to discuss the RFC, is feedback being taken into account by the authors, and how is the discussion generally going?

They do not need to actually understand where the discussion is heading, but make sure that a discussion is happening. This would solve the problem of RFCs remaining open for years, without getting any further feedback and un-cluttering the PRs page of open RFCs. RFCs that were forgotten about by their authors or that the community has largely moved on from can be closed/ rejected. It can also give closure to people who have written RFCs that was never rejected, but not accepted either (again, I'm cool, don't worry).

RFC shepherds are then assigned to an RFC (3-5 people) to actually oversee the discussion and consolidate feedback into changes that can be made on the RFC. They are also responsible for regular (again, up to them how regular) meetings discussing the wider implications, as well as small details of an RFC, usually on a video call, taking notes for people who can't attend to read up on afterwards.

An important note here: shepherds don't have to be part of a team that would otherwise oversee the development of a feature (like lang or compiler) and instead can be any community member who feels like nominating themselves or who is nominated by someone else. The idea is that everybody should be involved in overseeing incoming RFCs.

Governance WG and a new process

Generally, I think we all know that the RFC process needs to change. It has a bunch of problems that have lead to people physically and mentally burning out while contributing to Rust. And, as XAMPPRocky mentioned in her post, sustainability is important for Rust to remain a healthy project, 5, 10 or 15 years down the line.

I haven't followed a lot of progress from the governance workinggroup, but reading their charter and some of the proposed RFC stages might address some of the issues in the process. I feel that introducing a new RFC governance body (a set of people who rotate) as well as the concept of RFC shepherds would be beneficial to the Rust project as a whole, and anyone who's involved in any RFC related discussions in the future.

There's some tooling issues to address as well, but I feel those are second to the social ones.

Distributing Rust code

Wow, yea this was supposed to be a post about Rust 2020 and my personal roadmap. While I would love to be involved co-authoring an RFC on changing the RFC process in the ways that I propose in this post, there's some personal projects I want to get going as well.

At the beginning of the year I told myself that my SQL migration crate barrel would see a 1.0 release by the end of the year. This is looking less and less likely, but I want to at least get close to it. And then, next year, there will be a 1.0. There's a bunch of improvements to the crate itself, as well as compatibility with other crates (such as diesel and other migration toolkits), I want to make.

There's clap 3.0 things that are happening although maybe those will all be done by the end of the year. Who knows?

But mostly, I want to address a pain point in application packaging. Over the last year I've been tricked into maintaining a Linux distribution, NixOS. And while I'm not that involved in the development of it, there's some things that often come up with packaging Rust applications that could and should be better.

Mostly this is about applications, written in Rust, that want to distribute artifacts other than their binaries as well. Be that generated man pages, default configuration, or static files for a website. Currently this process is entirely up to the packager of an application and relies heavily on the application in question having good documentation. This is also a problem for all Linux distributions, not just NixOS.

Enter cargo-dist, a tool that can be used by a project to easily declare exportable artifacts and provides a way to tell an external packaging tool (such as nix, or dpkg) where to copy files to make a complete, working application. It ~~steals~~ borrows some concepts from autotools, using a PREFIX and several paths that artifacts can be copied into. This way a Rust application can easily be made into a package by calling cargo dist, which internally does a release build, and exports required artifacts to the appropriate places.

All of this is pretty WIP and local on my laptop right now. But I would love to finish it soon, and see projects in 2020 adopt this as a standard to distribute files for Rust packaging.

Labels are language

Katharina Fey — Fri, 20 Sep 2019 15:38:00 +0200

A phrase that I've heard way too fucking often recently (this edition will contain swearing and might not be suitable for children of ages below NaN) is "I don't care about labels, I want to do politics!"

As one might expect, this sentiment often comes from centrists. But more often than not, it comes from fellow leftists. People who are otherwise somewhat radical in their approach of the world, people who think capitalism's gotta go and (sometimes) that states and borders are bad. And it's a stance that has confused me, and keeps confusing me and which is why I'm now writing a blog post about it because apparently that's what I do.

The problem I have with "I don't care about labels" is that it's analogous to "I don't care about language".

Labels are a linguistic tool to talk about $stuff without having to build up an entire language from first principles in every sentence. Labels are very useful for general conversation about things, like "what is a table?", "what is a train?", "what is art?", etc.

When we look at the definition of labels, there's usually three kinds. There's labels for natural things, with natural definitions, such as the definition of a prime number. These are farely rare. Neither the definition of prime numbers, nor prime numbers themselves are going to change due to cultural context.

Secondly, you have labels that refer to natural things, with cultural definitions. These are things like planets, mountains or rain. Definitions can change and they're also subject to cultural differences. What you and I consider "rain" will most likely depend on where we grew up, if there was frequent rain at all, etc.

The last category are cultural things, with cultural definitions, such as art, sub-categories of it (movies, games, etc), as well as any identity label. Calling myself an anarchist doesn't naturally depend on anarchy as a concept occuring in nature, nor can I define it just by pointing at other properties of natural definitions. Rather, I need to pre-define a whole bunch of cultural context, for you to be able to understand why I am an anarchist and what that means.

And that's the fucking job of labels! We can't have the same 5 conversations over and over again and we can't rely on the trust that people around us are always gonna be on our side. We should have conversations from time to time about what these labels mean to us, especially when it becomes clear that there's miscommunication.

But also, just because we're having a conversation about labels, doesn't mean we need to start bikeshedding their definitions and scope (whether it be anarchy, libertarian socialist, libertarian communists - these are all kind of similar enough to work with). Their context is still there to be used.

That doesn't mean that I am okay with any vaguely leftist label. I have, over the last year or so, become more sceptical of communism, talking about how you want to guillotine people and similar. Being an anarchist means being opposed to state violence, no matter who's state it is. But this isn't a conversation that is easy to have if I don't already know a bunch of labels and can refer back to them. Furthermore, maybe I don't want to have this conversation in certain situations so why would I have to engage with tankies when I don't want to?

Most of the time the people who say "I don't care about labels, I wanna do poltics", never do any politics due to lack of a platform or language to engage with similarly minded people about strategy. That's because political action depends on the people doing it having some understanding of the work they're doing, how it relates to others and themselves. There's a reason why minority groups rely on labels (such as people in the LGBTQ community), and they serve an important role in our discourse.

This is not to say that we should try to make the onboarding easier and use less jargon language when dealing with outsiders. Making people more sympathetic to the radical left is important, albeit not a job everybody might want to do.

Still...I feel labels are important, especially when we deal with internal discourse. For the sake of the conversation, and everybody involved in it.

ociTools in NixOS

Katharina Fey — Mon, 09 Sep 2019 18:00:00 +0200

With the release of NixOS 19.09 any second now, I thought I wanted to blog about something that I've been working on, that recently made it into master, and thus the new stable channel.

What are OCI tools?

Open Container Initiative (or OCI) produced a spec that standardised what format containers should use. It is implemented by a bunch of runners, such as runc (the Docker/ standard Kubernetes backend) and railcar (more to that later) and outlines in exactly what format a containers metadata and filesystem are to be stored, so to achieve the largest possible reusability.

The spec is pretty long and in some places not very great. There's even a blog post from Oracle, talking about how implementing an OCI runner in Rust made them find bugs in the specification.

What are ociTools?

So now the question is, what does that have to do with NixOS/nixpkgs. The answer is simple: I wanted to be able to containerise single applications on my server, without requiring a container daemon (such as docker) or relying on externally built "Docker containers" from a registry.

So, ociTools.buildContainer was recently merged into nixpkgs/master, allowing you to do exactly that. It's usage is farely straight forward

with pkgs; ociTools.buildContainer {
  args = [
    (writeShellScript "run.sh" ''
      ${hello}/bin/hello -g "Hello from OCI container!"
    '').outPath
  ];
}

The args parameter refers to a list of paths and arguments that are handed to a container runner to run as init. In this case it's creating a shell script with some commands in it, then getting the output derivation path. Alternatively, if you only want to run a single application, you can pass it <package>.outPath directly instead.

There's other options available, such as the os, arch and readonly flags (which aren't very interesting and have sane defaults). Additionally to that there's mounts.

Simply specify any bind-mount you wish to setup at container init in a similar way you would describe your filesystem with nix already:

with pkgs; ociTools.buildContainer {
  args = [
    (writeShellScript "run.sh" ''
      ${hello}/bin/hello -g "Hello from OCI container!"
    '').outPath
  ];
  mounts."/data" = {
    source = "/var/lib/mydata";
  };
}

Railcar + ociTools

So that's all nice and good. But what about actually running these containers. Well, as I previously said I didn't want to have a dependency on a management daemon such as docker. Instead, I also added a module for the afromentioned railcar container runner (Oracle please merge my PR, thank you).

It wraps very cleanly around ociTools and generates systemd units to start containers, restarting them if they crash. This way you can express applications purely in nix, give them access to only the things they need, and be sure that their configuration is in line with the rest of your system rebuild.

services.railcar = {
  enable = true;
  containers = {
    "hello" = {
      cmd = ''
        ${pkgs.hello}/bin/hello -g "Hello railcar!"
      '';
    };
  };
};

The metadata interface for mounts, etc is the same for railcar as for ociTools.

Anyway, I hope you enjoy. There is definitely things to improve, especially considering the vastness of the OCI spec. Plus, at the moment ociTools does require a bunch of manual setup work for an application to function, if it, say, runs a webserver. It would be cool if some NixOS modules could be re-used to make this configuration easier. But I'm sure someone else is gonna have fun figuring that out.

Starting a public inbox

Katharina Fey — Sat, 20 Jul 2019 00:00:00 +0200

I've been a lot more active on this blog in the last year than I have previously, and that makes me pretty happy. I'm trying to become less obsessed about publishing only perfect diamonds made of words, but instead also publishing articles that might still have some flaws or that I haven't rewritten twelve times yet ;)

As a result of this I have actually gotten more and more e-mails by people saying that they read my blog, giving me feedback and sometimes submitting patches to my website repository to fix typos or bad formatting. And that's pretty cool.

Recently I started thinking if the format of e-mail might not be well suited for comments as well. Not just to me, but to allow other readers to talk about the stuff I post. I wasn't super sure if this was such a great idea, after all...this is the internet we're talking about. Someone will be an asshole and ruin it for everybody.

Then I discovered Drew DeVaults public-inbox mailing list, which is basically exactly what I thought about creating, hosted on source hut.

It might still be a terrible idea, but it's one I wanna try. I also wanna automatically post new blog posts to the mailing list, as plain text, so people don't have to fuss around with my RSS feed if they don't want to. I will host my public-inbox on source hut for now too, especially considering that I've been trying it out for a lot of smaller personal stuff, it makes sense. And I really quite like it (might write about that in the future too)

So, if you have comments, questions or want to fix typos, feel free to check out my public-inbox

Let's see how this goes then 😀

Bikeshedding disk partitioning

Katharina Fey — Thu, 11 Jul 2019 00:00:00 +0200

I recently got a new Thinkpad. Well...new is a stretch. It's an X230, featuring an i5 and 16GB of RAM.

One of the first things I did with this laptop was to flash coreboot on it. This is something I've always wanted to be able to do, but so far lacked hardware that was supported. And generally, it felt like finally maybe I could have a laptop to tinker around with.

And that's where this post begins...

Encrypted disk

So from the start I knew I wanted to have a fully encrypted disk.

What that means is that your /boot partition (whether it is it's own partition or not), is also encrypted.

Secondly, I don't like (U)EFI... What that means is that I'm installing GRUB in the MBR (with a DOS partition table) instead.

Now: GRUB stage 1 can handle the encryption for us, but there's some limitations

Keyboard layout limited to US
/boot can only be certain partition type
/ and /boot need be contained in an LVM

That last one might not be accurate if you only want to have an ext4 (or similar) rootfs. But because I want to have a zfs root, I need to embed it into an LVM. This is also the reason why /boot needs to be it's own partition. After we've done all this, we will install a linux distribution of choice (which we'll reveal later).

Anyway, let's get started!

Preparing the disk

(Feel free to skip this step)

Something you might want to do is letting your disk look otherwise uninitialised, or "securely erasing" any data that is already on it. But generating random data is a lot of work and /dev/urandom is very slow.

Instead you can create a crypto-disk (luks) on it, then fill it with zero's. But because of the encryption it will seem random.

(/dev/sda is my disk in this example because lolwat is nvme even?)

$ cryptsetup luksFormat /dev/sda1
$ cryptsetup luksOpen /dev/sda sda_crypto
$ dd if=/dev/zero of=/dev/mapper/sda_crypto bs=512 status=progress

This might take a while, but considerably less time than filling the disk with random data. After this is done, you might want to actually wipe the first bunch of bytes.

$ cryptsetup luksClose sda_crypto
$ dd if=/dev/urandom of=/dev/sda bs=1M count=8

Basic partitioning

So what we want to do is setup a single partition on /dev/sda, the same way we did to prepare the disk. Then repeat the previous command to setup a cryptodisk.

What follows is the LVM setup:

$ pvcreate lvm
$ vgcreate vg0 /dev/mapper/lvm
$ lvcreate vg0-boot -l 1G
$ lvcreate vg0-swap -l 16G
$ lvcreate vg0-root -L +100%FREE

I included the swap partition in the LVM instead of as a ZFS subvolume because those can sometimes deadlock and this just makes things easier.

Now we want to create the filesystems. For /boot we can just use mkfs.ext4, but consider that I want to use zfs on /, that will require some more work.

$ zpool create rtank /dev/mapper/vg0-root

Feel free to call your pool whatever! At this point you could also create subvolumes to split /, /home, ... if you wanted.

Mounting & Configuration

So that's all good. How do we initialise this system now? We need to mount the zfs pool first, then /boot and then install our linux secret distribution of choice (spoilers: it's NixOS!)

$ mkdir -p /mnt/boot
$ zpool import rtank
$ mount -t zfs rtank /mnt
$ mount /dev/mapper/vg0-boot /mnt/boot
$ nixos-generate-config --root /mnt

That last line is obviously NixOS specific. You now have a fully encrypted disk setup, without using EFI. Wuuh!

The rest of this post I want to talk about how to make this all work with NixOS and reproducable configuration.

Most of what we need to configure is in the boot option. Let's go through the settings one by one:

boot.loader.grub
- efiSupport = false actually the default but I like being explicit
- copyKernels = true enable this to avoid problems with ZFS becoming unbootable
- device = "/dev/sda" replace this with the device that holds your GRUB
- zfsSupport = true to enable ZFS support 😅
- enableCryptodisk = true to enable stage-1 encryption support
boot.zfs.devNodes = "/dev" to point ZFS at the correct device tree (not 100% if required)
fileSystems."/".encrypted
- enable = true
- label = "lvm" the label of your LVM
- blkDev = "/dev/disk/by-uuid/f1440abd-99e3-46a8-aa36-7824972fee54" the disk that ZFS is installed to. You can find this out by looking at your symlinks in /dev/disk/by-uuid and picking the correct one.
networking.hostId needs to be set to some random 8 bytes

Following is the complete config to make it easier to copy stuff from:

boot.loader.grub = {
    efiSupport = false;
    copyKernels = true;
    device = "/dev/sda";
    zfsSupport = true;
    enableCryptodisk = true;
};
boot.zfs.devNodes = "/dev";

fileSystems."/" = {
    encrypted = {
        enable = true;
        label = "lvm";
        blkDev = "/dev/disk/by-uuid/f1440abd-99e3-46a8-aa36-7824972fee54";
    };
}

networking.hostId = "<random shit>";

And that's it. If you spot any errors in this article (or any for that matter), feel free to e-mail me or send me a PR over on github.

Usable GPG with WKD

Katharina Fey — Tue, 02 Jul 2019 00:00:00 +0200

With the recent SKS keyserver vulnerability, people have been ~~arguing~~ reasonably talking on the GnuPG mailing list about how to proceed with keyservers, public key exchanges and the GPG ecosystem as a whole.

As part of this WKD was mentioned. It stands for "Web Key Directory" and is a standard for making a users public key available via their e-mail provider or server with the domain that corresponds to their e-mail address. There's several clients (such as Enigmail in Thunderbird) that will use this standard to automatically fetch a user's public key, when writing an e-mail to them.

As an example: my e-mails are hosted with mailbox.org, but I use my own website as an e-mail alias. This means that I can make my public key available via my website, and clients using WKS could then get it automatically.

If you don't have your own domain and use a webhoster instead, you might still be able to use this. There's a list of supported hosters that you should check out.

Setting this up

(Note: in newer versions of gpg the tool gpg-wks-client is included, which can handle setting up the folder structure for you automatically).

There's two ways of making your public keys accessable this way: the advanced and the direct way. This post will only talk about the latter, because I find it easier.

You need to create a .well-known/openpgpkey directory on your server. In this directory, place a policy file. This can be zero-length, but is used to check for WKD capability. Next, create a hu folder inside it (~~no idea what this stands for...~~ — as pointed out by an attentive reader, it stands for hashed-userid)

Next, take the prefix of your e-mail address (i.e. in kookie@spacekookie.de, this would be kookie), hash it with SHA-1 and then encode the output with z-base-32. You can use this convenient encoding website.

Edit: Also pointed out by a reader, you can actually use gpg --with-wkd -l <email> to display your hashed User ID instead of using an external resource for this.

Export the binary version of your pubkey (so without -a) and place it in the hu folder, under the name that you just computed.

The resulting folder structure should look something like this:

$ tree .well-known/ 
.well-known/
└── openpgpkey
    ├── hu
    │   └── nzn5f4t6k15893omwk19pgzfztowwkhs
    └── policy

You need to make sure that this folder is accessable through your webserver (this either involves including it in a static site or configuring nginx correctly). But fundamentally, that's it!

You can test if it works by setting a new GNUPGHOME and running this:

$ env GNUPGHOME=$(mktemp -d) gpg --locate-keys <your-email-here>

And that's it! Clients like Enigmail, KMail or GpgOL for Outlook will now automatically fetch your public key for any message they send.

Allocations are good, actually

Katharina Fey — Sun, 07 Apr 2019 00:00:00 +0200

Something that you can often hear in the Rust community, especially from people who were previously C or C++ developers, is the adage "allocations are slow".

The other day a friend asked me how to create a consecutive list of numbers. I pointed her at (0..).take(x).collect() which can be made into a Vec<_>, with a number of her choice. It did made me think however about how this could be done much nicer in a allocation-free manner.

It lead me to come up with the following code which creates a [_; _] slice, depending on which integer representation and length you choose.

(0..)
  .zip(0..x)
  .fold([0; x], |mut acc, (step, i)| {
    acc[i] = step;
    acc
  })

So with this in mind, I wanted to run some comparisons. I chose the numbers so that 32768 consecutive numbers would be generated. I compiled the example with both Debug and Release mode. (All of these measurements are done with rustc 1.33.0 (2aa4c46cf 2019-02-28))

Let's start with the non-allocating version.

$ time target/debug/playground
target/debug/playground  1.45s user 0.00s system 99% cpu 1.457 total
$ time target/release/playground
target/release/playground  0.27s user 0.00s system 99% cpu 0.270 total

Cool! So as you can see, the Release profile is over 500% faster. And performance-wise this is quite reasonable.

Let's see how an allocating implementation stacks up to it. The code used here is the following.

let vec: Vec<u32> = (0..)
  .take(1024 * 32)
  .collect();

So how fast is this gonna be?

$ time target/debug/playground
target/debug/playground  0.01s user 0.00s system 93% cpu 0.010 total
$ time target/release/playground
target/release/playground  0.00s user 0.00s system 85% cpu 0.005 total

What? ...it's faster?!

Well, I guess this does to show that it's not as simple as saying "allocations are bad". Avoiding allocations at all cost can slow you down.

Thanks for coming to my TED talk!

. . .

Yes but why?

Okay maybe you're more curious than that and want to understand what's going on here. So come along, let's read some assembly!

Let's focus mostly on the release profile here, because Debug generates a lot of code that makes it harder to understand. So we have two code snippets that we should throw into godbolt to see what rustc does.

// This doesn't allocate
const length: usize = 1024 * 32;
pub fn slice() -> [u32; length] {
    (0..)
        .zip(0..length)
        .fold([0; length], |mut acc, (num, idx)| {
        acc[idx] = num;
        acc
    })
}

// This does
pub fn vec() -> Vec<u32> {
    (0..).take(1024 * 32).collect()
}

Let's have a look at the assembly that the vec() function generates.

.LCPI0_0:
        .long   0
        .long   1
        .long   2
        .long   3

# ... snip ...

example::vec:
        push    rbx
        mov     rbx, rdi
        mov     edi, 131072
        mov     esi, 4
        call    qword ptr [rip + __rust_alloc@GOTPCREL]
        test    rax, rax
        je      .LBB0_4
        movdqa  xmm0, xmmword ptr [rip + .LCPI0_0]
        mov     ecx, 28
        movdqa  xmm8, xmmword ptr [rip + .LCPI0_1]
        movdqa  xmm9, xmmword ptr [rip + .LCPI0_2]
        movdqa  xmm10, xmmword ptr [rip + .LCPI0_3]
        movdqa  xmm4, xmmword ptr [rip + .LCPI0_4]
        movdqa  xmm5, xmmword ptr [rip + .LCPI0_5]
        movdqa  xmm6, xmmword ptr [rip + .LCPI0_6]
        movdqa  xmm7, xmmword ptr [rip + .LCPI0_7]
        movdqa  xmm1, xmmword ptr [rip + .LCPI0_8]
.LBB0_2:
        movdqa  xmm2, xmm0
        paddd   xmm2, xmm8
        # ... snip ...
        ret
.LBB0_4:
        mov     edi, 131072
        mov     esi, 4
        call    qword ptr [rip + _ZN5alloc5alloc18...@GOTPCREL]
        ud2

(full code dump here)

As you can see this uses the "Move Aligned Packed Integer Values" instructions in x86_64. From some x86 docs:

Moves 128, 256 or 512 bits of packed doubleword/quadword integer values from the source operand (the second operand) to the destination operand (the first operand).

Basically the LLVM can figure out that our numbers are predictable and can allocate them in a way that is batchable.

We will already see how the non-alloc code is going to be slower: because the code that assigns numbers is less unterstandable to a compiler (i.e. assigning values to an array sequencially) this will not end up being batched.

That's not to say that alloc code is going to be this fast on every platform (RISC instruction sets lack many vectoring techniques) and this doesn't even take embedded targets into account.

But there you have it.

LLVM is magic...

... and saying "allocations are bad" really isn't telling the whole story.

home-manager, or: how not to yakhave

Katharina Fey — Thu, 21 Mar 2019 00:00:00 +0100

Don't expect the bait-and-switch titles to remain forever. I just thought it was fitting for this one too 😉.

Some background

Ever since I started venturing into computer programming and using more and more tools that used dotfiles, I've been frustrated at the lack of good tools when it came to synchronising these files.

And that's not for lack of options. Either I disagree fundamentally with what I want a sync tool to do or nobody had come at the problem from the same angle as me before. This is not to bash on other projects or solutions. I know many people who are very happy with either keeping their dotfiles in a large git repo, symlinking manually, making ~ a git repo or using various tools that automate the symlink process.

The problem I had was that not all my computers were the same. As in, I didn't neccessarily want the same configs on all of them. Because this is where it becomes complicated.

I had been thinking about writing a tool to do the things I wanted to do before. Thinking back to my post about failure and limiting scope, I never started it. While making a lot of drawing board attempts, I never wrote a single line of code because I could tell that it would lead me down a dark path.

Reading about people on reddit from time to time who started their own "ansible but for dotfiles" projects that never went anywhere, I felt like I was doing the right thing by not even starting.

So nothing happened for a while. Until recently.

Enter: nix

In case you don't know it, nix is a functional programming language and pure package-manager for unix systems. Yes, that includes MacOS. There is a linux distribution built around it called NixOS which utilises nix as a package manager and configuration language heavily.

So what does pure-packaging really even mean? Have you ever had the situation where you upgraded your system and half way through something failed and now you ended up with a broken system because some of the packages had already changed while others had not?

Yea, that's what people would call "impure packaging".

With nix on the other hand this cannot happen, since packages are atomic, meaning that after something is built, it can't be changed again. Doing an update? New package. Changing a small config? New package. It means that not only can failing upgrades be seemlessly be rolled back but also that two different versions of the same library can easily be installed at the same time.

Yes, that's right: no more "DLL hell"!

Well okay so why am I fangirling about nix here? Apart from the fact that I've been dabbling with NixOS quite a bit recently, to the point where I have basically replaced all my Arch installs with NixOS now...

Enter: home-manager

You already saw it in the title of this post, but I wanted to re-introduce it. What exactly is home-manager?

It's nix, but for your userspace! Not only does it not require root permissions, meaning you can install packages just for you locally (well okay, nix can do this as well but...besides the point).

More importantly, home-manager adds modules and utilities to manage userspace configurations. Everything is sourced from ~/.config/nixpkgs/ (you can move that IIRC) which is then used to generate all your configuration files.

Config files are kept in the nix store (which is usually located at /nix) and then symlinked to their destination.

Right. Now I can practically hear you all saying: "but didn't you say you didn't like tools that just symlinked stuff?"

Well..yes, and no. Obviously symlinking a config is much nicer than having to copy them around. What I disliked about tools that symlink configs to places from some other place was that I was still responsible for manage that "other place".

With nix, I never touch the store directly. In fact you don't ever do that! Instead I edit the home.nix configuration (or sub-configs when it gets too complicated) that then take some inputs, define the outputs and nix makes sure that the configs are then where they need to be.

It gives me a single source of truth, but the best thing is that it's not a dumb source of truth: nix is a programming language! What that means is that I can dynamically adjust some config contents according to what system I'm running on, while not having to worry about keeping it all sane.

I'm really happy I didn't write my own "ansible but for dotfiles" and I think I'd recommend nobody do that (I mean...unless that's your kink - I don't judge!). But I'm even more happy of having been introduced to nix and home-manager in particular.

I'd much rather help write some re-usable modules, that other people can also take advantage of, than reinventing the wheel from scratch. Again.

Rust 2019: how we make decisions

Katharina Fey — Mon, 21 Jan 2019 00:00:00 +0100

I'm late to the party, I know. But writing this post took a little longer. In fact, I wrote it three times, not really sure where I wanted it to go and what I wanted to say. In the end, I think most things have already been said. So this will be short.

Problems

There have been a great number of blog posts about the problems that the Rust community is facing. I recommend you read some of them because they're really good. For example boats who published an article early december about "Organisational Dept". Or killercup about how we count contributions (and other things). Or skade, who published a small collection of articles on organisational subjects.

There's many more under the #rust2019 hashtag on twitter.

My point is: you can read about what the issues are elsewhere, from more perspectives. There's no point in me trying to rehash the same stuff again. I'm not that good a writer that I will bring anything to the table that these people haven't already.

In my opinion the issues we have at the moment are because of two things.

Teams that have too much responsibility
Bad tools

(I won't really talk about the tools issue in this post. In summary: the tools we use to communicate are all garbage. And github and discourse comments are terrible for debates. Really, any linear chat platform is terrible - there might be another article at some point)

As I've said, there are plenty of articles that go into what is the problem with our current process. I want to focus on the why, how and how we stop it bit of that.

And first we need to talk about how things are built.

Vision vs Implementation

Fundamentally there are two parts to design:

Vision is what drives a project forward, creating new concepts and thinking about their implications, while Implementation is finding the most elegant, sustainable and performant way of making vision a reality!

Now, these two can very often not be separated as easily as I make it out there. Take the example of the CLI-WG. When we first assembled earlier this year, we started working on a vision: "how should CLI development look". What followed then was an implementation of the vision, as closely as we could manage with the resources and time available. I would argue that during the implementation period of the vision, some aspects of what we thought should be done were influenced by things we learned about how to do them. Like hitting a moving target. To some extent this is how most software development works, unless you are working towards a very well defined spec!

Having the same group of people be in charge of both the overall vision for a system and it's implementation isn't a bad thing, given that the system is small enough!

And this is where teams come in! Splitting up the community into smaller groups who work on the same stuff, so that this kind of collaboration becomes possible again.

In a way, the Rust 2018 working groups were inspired by the same idea. The difference between teams and working groups being, that the latter has a more loosely defined governance structure and allows people to join and collaborate easier and more quickly. There's less "established structure" in a working group. These lower barriers are one of the reasons why I'm a huge fan of working groups, and feel like the Rust project should expand on them in the future. Maybe some teams could even be replaced.

But that's not fundamentally the issue that the project is facing.

Blurring the lines

Problems arise when these lines are blurred too much. This happens with both discussions, as well as to teams of people, who get involved in too many things. Ultimately, we need to face the fact that days are short, people's time is limited and the number of responsibilities a single person can have isn't infinite!

Througout previous blog posts and conversations with others, the need or "desire" to have better communication channels has been made clear. And I feel that we need to work on the way that we communicate, if we are ever to fix the way that we make decisions.

But before I go into detail about what that means to me, specifically, I want to talk quickly about the core team.

The core team is a medium sized group of Rust developers, who oversee large areas of the development of Rust. As the website puts it:

Overall direction and policies of the project, subteam leadership, cross-cutting concerns.

As such, I feel it would be the perfect candidate to step back from implementation, and focus on both vision for the entire language, as well as communication among smaller teams. Unfortunately this hasn't seemed to be the case in the past and is something the core team should work on this year.

Note that I think, that this is a responsibly that should also not be taken lightly. Most of the members of the core team are also active in other teams, sometimes even leading them. I feel that this is one of the reasons why this role has been neglected.

Solutions

So I would sugest something, that I've heard others talk about before, although rarely publicly, which might be considered a bit of a hot take.

The core team should be rotated.

What that means is that the core team in itself still exists, does a certain number of jobs and should be considered quite a time-intensive commitment. But the people involved in it shouldn't stay involved with it forever. Even if it's currently (practically) already the case that the team rotates, making this more explicit and making the roles of the core team a central part of how other teams communicate and operate, I feel would benefit the overall contribution climate of the rust project significantly.

Ultimately, we have a problem in how we communicate. The lines between vision and implementation get blurred too often. Not only in RFC discussions. Implementation specific bikesheds on github often result in new rationale being pushed forward, that have nothing to do with the actual question at hand. And as such, people often talk past each other.

I don't know how wide the communication scope of the core team should be, but I definitely think that moving it's responsibility away from implementation and back towards communication and fostering collaboration between teams and working groups is the approach we will have to take this year to solve our problems of Organisational dept!

Hacking is political

Katharina Fey — Wed, 02 Jan 2019 00:00:00 +0100

I'm just coming back from the Chaos Communication Congress, a four day event just after Christmas. It was my fourth one in total, and now the third in a row (the first being 25C3 as a smol girl).

It's hard to describe the C3 (abreviation for the congress, opposed to the CCC, the club). Some call it a "hacker conference" which is...in some ways accurate, but often doesn't manage to capture what it is. Not to mention relies on the external definition of a "hacker" to describe it. Other's call it a "tech event" or "tech conference" which really isn't accurate either. There are lots of artists and non-tech people represented and I feel these experiences shouldn't be ignored.

The C3 has been in Hamburg for quite a while but was forced to move last year due to the congress centre there being remodeled (read "torn down"). After remodelling the venue would also not be suitable for the event anymore, meaning that a semi-permanent move had to be initiated.

I didn't enjoy last year's Congress.

Not only was it plagued by loads of logistical problems and bad adaptation of the new space, there were political issues around the organisation of the event and how decisions were made. Last year's congress showed off that the CCC (the club) had a problem with apoliticality.

Apolitical Hackers

I've been rather loud about the apolitical or centrist parts of the hacker movement. Conflation between the terms "maker" and "hacker" have further undermined the movement with capitalist and neoliberal ideas.

That isn't to say that everybody in the hacker scene needs to be continously aware of all political implications of their actions at all times. Danger arises from people who don't feel like political action is important at all or who represent centrist, capitalist and neo-liberal viewpoints. This includes refusal to take action against climate change or supporting the police, regulatory bodies and disregarding free software movements for reasons of convenience.

I'm taking about these things in rather vague terms because I want to avoid drawing a definite line that people can't cross. Really, I would argue that there's many ways to be a hacker. I see issue and a danger to the movement, when people attempt to "leave politics out of hacking" entirely.

This includes fighting capitalism and discrimination against minorities. Hackers, by definition, are the political left! Anyone who isn't and still claims to be a hacker has successfully co-opted the word and is attempting to undermine the movement for their own political gain.

It's not always obvious how the existence of apolitical hackers impacts the movement or the technologies that they build. But just like Tech in general has racism and sexism problems, so does the hacker movement. Society does, really. There's no way to remove a community from time and space. "Stuff" happened before we got here, and pretending that it didn't, is dangerous.

I could talk about the impact of apoliticality on technology and communities that are being built for a very long time but I really don't want to today. Instead I want to focus on something else, something more positive.

A very political congress

I very much enjoyed this year's congress!

Maybe in part that was because of the people I was attending it with this year ( 😉 ) but in no small part, it was also because of the general atmosphere around the event. This isn't something only I noticed. I had conversations about this with others, who apparently also noticed this change from last year.

The first thing you saw when entering the venue was a huge Antifa flag, setting the tone of the rest of the event. Apart from that there were a lot more assemblies this year, many were dedicated to squatting, anarchy and fighting capitalism. There were a few queer assemblies, scattered around the hall, making it so that queer and leftist representations weren't all bundled in one spot but were present all through the venue. Even purely technical assemblies were surrounded by antifa flags and anarchist stickers and flyers. This changed the atmosphere quite significantly.

It wasn't perfect. Just like last year, it was plagued by logistical problems, some smaller, some larger. There were people who had their stuff stolen. There were speakers who made problematic jokes on stage. There were still apolitial and centrist people present. In fact, there were people complaining about C3 "having gotten so political", which really is a good thing. We want the centrist and right-leaning "hackers" (read: libertarians) to feel uncomfortable.

But despite all that, the air, the general atmosphere of the event was different.

I welcome this change. And I hope that it sets a new tone for the CCC and the C3 in general. I would very much enjoy going back next year and finding out that the event had become even more overtly anarchist.

Because we shouldn't forget our core motivation as hackers: being gay, and doing crimes!

The only downside from 35C3? I don't really know what to think about birds anymore...

Failure. Or: why Rust is probably the best programming language ever created

Katharina Fey — Sun, 11 Mar 2018 00:00:00 +0100

This post is two stories. One is about accepting and recognising personal failure, reflecting and growing from it; the other is about an incredibly and seemingly endlessly powerful programming language, called Rust.

In the summer of 2014 I started a project which was kind of insane. I knew it was insane, yet I embarked on that journey regardless. I wanted to write a password manager. I chose Ruby as a language because I didn't know many others and was – in more than one way – still a programmer novice.

The details of development aren't too important. About 6-8 months into the project I had written something rather cool and functional. It wasn't very fast, the code base was a bit of a mess and I was having issues with packaging. But, at the core, I really liked what I had made, which had shifted from just being a password manager to being a universal, platform-independant secrets manager, close to a keychain. In my mind applications could write sensitive information into a "vault" which was managed by this project, without having to worry too much about access rights, authentication or anything else.

So far so good; this is how both stories start.

Over the next few years this project would take me over and, ultimately, destroy me. I had gotten it into my mind that the cryptography should have been handled by something more low-level, something more "advanced". I talked to people, I looked at languages and in the end, thinking I had more experience now, chose C++ to re-write the project in. This was the beginning of the end. It took me another six months to get the basics done, getting caught up on nitty gritty details.

I ended up switching to C, back to C++, then back to C again, not being satisfied with the way that one or the other language handled things. And the scope was out of control. I didn't want to make a cute little secrets manager anymore. I wanted to make a database. It had transactions, sharding, multi-user access, backups, countless optimisations, run modes and even it's own SQL-like query language. I went completely overboard and lost all grasp of what it was I wanted to create. After literally years, re-writing the same parts of the code again and again, creating new libraries to handle even smaller tasks that had been completely trivial in Ruby, I stopped.

What this project had turned into wasn't maintainable. It didn't even really make any sense. It had no use-case, besides "being cool" and that wasn't really enough to motivate me anymore. I was also caught up with other work, getting involved with the Google Summer of Code 2016, then slowly fading work on the project into the background. This wasn't a conscious decision though. In my mind, I was just putting it on hold, learning from all of my failures and then trying myself again at another re-write. I didn't know that not trying again would be the act of learning from it.

As hackers, we are often compelled to take on the world. Everything seems plausible, sometimes trivial. We understand technology in a way that most people don't and in that, we gain confidence in our abilities past the point of reality. Hubris. We want to make things, break things, modify things. And we forget our own limitations, time and scope. We end up starting so many things that we never finish. Or we get obsessed with something that doesn't make any sense.

It took me over 2 years to understand that I can't let my impulse to adventure drive the way I work. I love open source and I love working on things that are just free and out in the open. I want to help build an ecosystem of tools and applications that help people, without any cost or baggage of being for a closed down system. But learning, that there were things that I can't do, that maybe the way that I viewed work, problems and how to tackle them was fallable, that took some more time to understand. In the end, everything I did on this project was a collosal waste of time. It's still on my github, more as a reminder to myself of how failure works...

It has nothing to do with not knowing how to solve a problem. It has nothing to do with failing to understand code or a language or a toolkit... It has everything to do with not knowing how to limit a project, and when to stop...

This is the end of story one

It's been nearly a year since I worked on this project (or the 5th iteration of a re-write anyways), in the meantime I've worked on many small things, trying to keep in mind what I want to do, what is plausible and also useful. And in the meantime I've come into contact with a magical programming language: Rust!

I had started programming in Rust before, during a very stressful time in 2016. And I never managed to get into it much. This year was different though. The toolchain had matured and maybe I had also matured as a developer. And maybe I was in a better state of mind to learn new concepts. Whatever it was, I'm glad it happened.

Rust is a systems programming language by Mozilla. It's a compiled and safe language which prevents segfaults and allows for fearless concurrency (as they put it), without sacrificing speed. In fact, it runs almost as fast as regular old C code.

Now though, that's only half the reason why Rust is amazing, and I will show in a moment how the first part of this story is in any way relevant.

After creating a few smaller projects in Rust, I started thinking again about password managers. The landscape looks a little different now than it did in 2014, yet I'm still not 100% satisfied. Everything ends up being an add-on to keepass which, in my opinion, doesn't have a very good database layout or file format. And end-user applications are usually very complicated or badly designed. I know I have high standards when it comes to UI/UX design so please don't consider this slander about these projects. I just didn't want to use them and had been sticking to my Windows application running in a WINE for the last few years.

Now I knew Rust. Because the amazing thing about Rust is only partially the language. The other half is the entire toolkit that comes with it. From a built-in version manager to a kickass package manager, an ecosystem of crates that can be easily included, following the UNIX philosophy of enabling you to do small things, in good ways, yet somehow always fitting together.

I remembered what I had thought about before. Limitation of scope, accepting limitation of time and limits also in my own abilities. And I started writing a project very close to what it had once been in Ruby. And within a week or so, I was close to the feature set of the version I had finished, before beginning my descent into madness. It took me a week of collecting external crates, writing a few hundred lines of code myself, playing around with different crypto backends and there I was, in the process of building something cool again.

Rust makes it incredibly simple to do rapid prototyping. Yes, the language is very strict sometimes. Yet it has this feeling of "throwing shit against the wall" and seeing what sticks. With the added benefit that there are compile-time checks that make sure that there are no serious issues with your program. You can still write bad code, it just seriously limits the damage you can do. And that makes it incredibly fun to write with.

What's the point of all this?

Well, first that I love Rust 😝.

But secondly that sometimes failure looks different than what we might expect. It's not about failing on a technical but either on a social or planning level. And maybe that this is something we should talk about and foster in the hacker community.

Rust is an amazing language for many things but it also has it's limits. There are countless people in the hacker culture who stick to their technologies because they feel familiar, dragging others into their little bubbles because they want to expand their influence, never considering if what they're advocating is sensible or scalable.

This isn't just something we (as a culture) do with tools, it also happens on a social level. And in the end, shouldn't we all strife to learn new things, broaden our horizons and, last but not least, choose the right tool, for the right job?

Sometimes a new technology can enable us to break out of our bubble and achieve something that we previously thought impossible.

Rebuilding my Website (again)

Katharina Fey — Wed, 03 Jan 2018 01:31:00 +0100

It's winter, rebuilding my website is a tradition...right? Happy new year everybody.

This has been a long time coming. I've not really been happy with the way my website looked for a while and have been playing around with new designs for the past few months. I also took that opportunity to throw out a few old articles, fix formatting on others and generally do house-keeping.

The whole thing is still using Pelican to generate pages but now with a completely new theme and new plugins 🎉

This re-design also decreases complexity. The old theme was massively too complicated and I've now taken it down to 3 (or 4?) templates. Working around the old theme and what Pelican expected was an interesting experience. Especially since it felt less like building a website and more like working with a game engine where small changes make lots of magic happen and voilá.

There are a few things I want to write about...soon. Until then, there is an easteregg hidden somewhere. Let's see who finds it first 😉

Dabbling with Moonscript

Katharina Fey — Sat, 06 May 2017 11:55:00 +0200

Recently I've started learning/ using Moonscript. It's a language that compiles to lua and as such can run in the LuaJIT, an alternative lua engine which allows very easy and fast ffi calls into native code. This makes lua code capable of writing very performant applications and games that use native rendering, window creation or general libraries.

But in my opinion lua has always felt a bit cumbersome. I use awesomewm so I had to write it occasionally to customise my UI layout. And this is where Moonscript comes in. It's a lot of syntactic sugar on top of lua as well as some other concepts such as object orientation which lua just plain out doesn't have. And while yes, you can write good code without OO (cough C cough) it is a nice tool to have in your pocket, especially when writing GUI applications or games.

The language

class Thing
  name: "unknown"

class Person extends Thing
  say_name: => print "Hello, I am #{@name}!"

with Person!
  .name = "MoonScript"
  \say_name!

As you can see Moonscript is an indentation based language which (in my opinion) combines syntactic elements from lua and ruby together. In the snippet above (which is from the moonscript website) you can see classes, inheritance as well as the with keyword which allows you to initialise/ work with objects without typing it's variable name over and over again.

If you want to learn more about the language, I can only recommend you have a look at the Moonscript in 15 minutes guide

How to use it

You can just write Moonscript files, add #!/usr/bin/env moon to them and get going. Obviously that's pretty cool for little scripts that you just want to get going. But not so great for larger applications because a) you don't have access to ffi via luaJIT and b) it adds additional startup cost.

So instead for my projects so far (which so far are a game and a desktop app) I use a Makefile to build and run the Moonscript compiler and then execute the init.lua with luajit.

SOURCES := $(wildcard *.moon) $(wildcard **/*.moon)
LUAOUT := $(SOURCES:.moon=.lua)0

.PHONY: all run build

all: run
build: $(LUAOUT)
%.lua: %.moon
        moonc $<
run: build
        luajit init.lua

Wrapping up

So...I'm kinda excited about this. Most of the code I write is either in C or Java (depending on what exactly I'm doing). And those two strongly typed and compiled languages have served me well and will continue to be my go-to solutions for a lot of problems.

But I've long been looking for a dynamicly typed, interpreted/ just-in-time compiled language that I can use for anything from little scripts to medium-sized desktop applications. I used to use python for this but have recently (over the last 6-9 months) fallen out of love and developed a rather passionate dislike of it and it's ecosystem.

My current project will get it's own little article at some point but I don't mind teasing the progress here. I'm writing a new UI for redshift which works with X11 linux backends and is heavily inspired by f.lux on MacOS. It's written in moonscript, with my own forked version of redshift (which I call libredshift). It's on github and licensed under MIT.

Hope I've made you a little curious about Moonscript :)

LibGDX interface containers

Katharina Fey — Tue, 24 Jan 2017 00:14:00 +0100

Let me tell you a factual statement: UI programming is terrible

Let me tell you an even more factual statement: UI programming in LibGDX is even more terrible

I am a big fan of LibGDX. It's a really nifty library/ framework to get started with game development if you're more comfortable inside a code editor than a full blown game engine that is more targeted towards designers and artists. And I put my money where my mouth is: I have a series about LibGDX development for beginners on this blog and work almost exclusively with it when it comes to my own projects.

Yet, there is something that bothers me and there didn't seem to be a great solution to fix it. UI code structure. In this post I want to highlight a utility I have written for LibGDX which is very easily embeddable into your existing projects which will you help structure UI code more efficiently.

The root problem

The reason I dislike UI programming with LibGDX is that it usually results in very long code files or passing dozens of parameters into sub-classes that are needed to update the UI for button presses, etc.

This goes so far that I have written an editor for game assets before just to realise that (once the development was complete) it had become completely unmaintainable and I had to start from scratch with better structure. It is incredibly easy to just throw out a UI design with Scene2D and LibGDX but unfortunately it is equally easy to produce very bad code which will turn into a big spaghetti mess.

Let's look at an example problem that I wanted to solve.

Looking at this structure we have three main components that interact with each other. We have a class that handles UI logic (setting up actors in tables, adding listeners, etc), we have a window state which in the particular case which made me write an alternative was a "Lobby Handle" which coordinated what players were going to enter a match, the map, game mode and if everybody in the multiplayer match was set to "Ready". Lastly we have the actual network signal handlers that listen to TCP/ UDP packets and execute code to write/ read from the window state as well as update UI elements.

Implementing this structure with Scene2D and LibGDX will result in a lot of very ugly code. Because the network signals need to know everything about the UI (how it is structured, etc). And our window state can be written to by two different sources which means that we need to mutex it to avoid race conditions.

Maybe a solution

So, what was I trying to solve? First a bit of limitation of scope. Because a lot of UI problems have been solved over and over again and usually at the cost of runtime performance or with a lot of extra code.

UI code doesn't have to be embedded in a screen
All UI code can access the shared context of the screen
UI elements can update each other
Clean API that can be called on from anywhere (with a reference to the handle) that triggers range of functions.

So with that in mind, this is what I did.

class MyUIHandle extands UIHandle {
    public static enum UI implements UI_BASE {
        PLAYER_LIST;
    }

    { /** Initialiser block for new objects */

        registerHandle(new PlayerList(), UI.PLAYER_LIST);
        // ... more handles
    }

    @Override
    public void initialise(Stage s, Object ... var) { ... }

    public class PlayerList extends UIContainer {

        @Override
        public void initialise(Stage s) { ... }

        // Define more API here ...
    }

}

When we initialise a new UIHandle the initialiser block will create our PlayerLists and register them with the UIHandle. That code is hidden away from you. You can see that we're implementing a different enum type that we overload with values so that we can address submodules via a compile-time checkable value (such as enums). From inside (and outside) this class UIContainer's are available via handle.get(UI.SUB_HANDLE). Obviously keeping your enum labels short will make your function calls snappier :)

The following graphic will sort-of explain the layout in more detail.

What you might also notice is that the UIHandle has an initialise function with variadic parameters while the UIContainer class only takes a stage. That is because window context is stored once in the UIHandle and then accessable from all UIContainer classes. This way we only need to do the inversion of control pattern once instead of for every sub-component.

You can keep the UIContainer classes outside this code-file. Then you might however want to provide a construct that does another inversion of control so that an external UIContainer can access the context provided via initialise!

public class PlayerList extends UIContainer {

    private MyUIHandle parent;
    public PlayerList(MyUIHandle parent) { this.parent = parent; }

    // ...
}

Now let's talk about that public API. In our original example we wanted to have networking code update some UI elements. And we want UI elements to update other UI elements. So first of all, we keep context in each UIContainer about what UI elements are accessable to it. So what we can do in every of our submodules is this:

    parent.get(UI.PLAYER_LIST).updatePlayers(playerList);

It also means that if we get new data from – say – a network socket or AI simulation, we can very easily update data in some random UI element.

    handle.get(UI.PLAYER_LIST).populate(playerList);

So all in all, we have solved the following problems:

We have access to all game state in the UI code without passing too many parameters into lots of sub-classes
UI code can be moved into lots of files for easier understandability
Context isn't duplicated
UI code can update other UI code without needing a direct reference to it.

The individual UIContainer instances are essentially independant of each other via dependency injection.

This library isn't done yet. Most of this is kinda hacked together to fit into my game. But I'm interested in making it more generic and putting it on Github. Especially because I can see myself using it again in the future.

Hope this might be useful to somebody out there. If you have questions, comments, hatemail...

Twitter or E-Mail

Post 33C3, what next?

Katharina Fey — Tue, 03 Jan 2017 12:28:00 +0100

Howdy everybody,

I just came back from the annual hacker conference in Hamburg, Germany known as the "Chaos Communication Congress" (or CCC for short). It was the first time I was there for the entire venue and the first time I was able to go at all since 2008. So yay!

It was a lot of fun and I have a lot of nice memories to hold onto now. I talked to a lot of interesting people, learned new things, got inspired to do new things and continue on old things.

More importantly, I loved the chance to get in touch with some other women in the tech industry (via Haecksen & Queer Feminist Geeks), talk about problems, attempt to come up with solutions and just generally rant about things :)

I also found out that I am in no way, shape or form a dancing person. Although electronic club music is fun!

Some talks I went to

Following is a non-comprehensive list of the talks I went to. I am filling this from memory, so some talks might have been missed or dropped. And maybe I'll just edit them in later without anyone ever knowing.

How Do I Crack Satellite and Cable Pay TV?

A really quite epic lecture about using glitching to extract keys from a very dated security layout. Not that anyone should do this (it's not worth doing it anyways...never anything good on) but it will teach you a lot of stuff about hardware security

Bootstraping a slightly more secure laptop

A talk about the flip-side of TAILS which aims to introduce trusted computing into a world where the machine can't be trusted. HEADS on the other hand uses coreboot and cleverness to create a verifiable machine environment to build an OS on top of. Made me want to get an old thinkpad on ebay to play with 😊

The Nibbletronic

A relatively short talk about the creation of a musical instrument. Learning by doing and failing. Quite interesting for me as a hardware designer (as a hobbyist) but also a musician.

Shut Up and Take My Money!

If you have a bank account with N26...stop having a bank account with N26. Their security is absolutely horrible. And while, yes, all of these security issues have been fixed, it shows a rather lacking attitude towards security from their engineering team. Best demonstration of client-side security gone wrong. And why ReST APIs are fucking aweful!

Untrusting the CPU

This was a great talk given by a close friend of mine about one of his super crazy projects. The idea being to construct an FPGA powered PCI-E device for laptops and/ or desktop computers that intercepts messages to the display, encodes and decodes text into them to provide an interface for encrypted messages without using the CPU. It's really quite interesting and I can't wait to see what he does with it.

Making Technology Inclusive Through Papercraft and Sound

One of my favourite talks was about an engineering toy kit that was aiming to be more inclusive. The problem it attempts to tackle are the incredible low numbers of women in computer science and engineering (significantly lower than in other scientific fields). There are a lot of reasons why women aren't well represented in the fields and they are all cultural. This talk was about trying to change the culture around teaching people about electronics and code to be more inclusive towards groups of people (mostly girls/ women) who would otherwise be missed.

I really enjoyed the talk on a lot of different levels. One was the technical aspect of creating a childrens toy on the cheap that is inclusive and universally programmable through audio encoding. Quite worth a watch.

I don't think that just with projects like this the culture around women in tech will change. But it's a start. What we realistically need is a change in culture throughout all layers of society. I think the problems around women in tech are quite complicated. And unfortunately usually result in a bunch of assholes starting to shout either about how feminism is evil or how diversity isn't important. And biases aren't actually thaaaaat bad, right? 😝

I could rant here forever and it's questionable how many people would actually care 😅 I can recommend this talk. Let's leave it at that :)

The Moon and European Space Exploration, Interplanetary Colonization

Those were just the first two talks from a series of space talks. The first one was from one of the heads of ESA about their plans to colonise the moon for profit! And science of course... It was quite funny and definately worth watching.

The second one I almost liked more, though mostly the first part of it. Liz George manages to explain incredibly well in a very short amount of time what challenges exist when discovering exo-planets. The second part (by somebody else) is a bit more vague about how to actually get there and is less science, more fiction. But hey 😝

Going into 2017

So in short: 33C3 was pretty epic! And I honestly can't wait for next year. It's not clear yet where it will be held but it will be epic non-the-less. And who knows, maybe I have a talk to hold by then 😊

Which brings me to this year. Last year was fucking shitty. Politically...On a personal level it actually went quite well. And I got a lot of shit done. I did Google Summer of Code, I made huge progress on my game project (yes, I will post about that at some point). And especially in the last months of the year, I redesigned and rerouted the Open Plantb0t board. On january 1st, 2017 the revision A2 design went into production.

I hope to get all my parts together soon and build up a second prototype series which (hopefully) works better than the last 😉 I will keep y'all updated on that. pel Until then, I hope you've had a happy new years eve and not an all too terrible year...yet 😉

Winter update

Katharina Fey — Fri, 02 Dec 2016 10:43:00 +0100

Howdy everybody!

As the year is winding down and we're all getting ready for the jump to take us out of what has (in my opinion) been a very shitty year, I looked at my blog and could only shake my head.

I had moved this over from Wordpress to Pelican and basically replicated all of the layouts to the extent that some of Pelicans own functionality had to be abused to make it work. But as I kept publishing things on here I realised that most of the features I had implemented went unused.

And so, for the last few days I have tweaked the layout (and design - as some might notice) to be a bit more traditional again.

I was also considering to change theme but after not finding anything I liked I decided to hack the fuck out of my current one instead. You can check out all of my horrible changes here.

I've also finally done some stuff that I've wanted to do for ages - such as pimping up the front page, adding a proper projects page and go through some of my old tutorial series, fix their formatting (yea right "perfect wordpress import...") and update them to newer API's of libraries. Some articles have just been dropped because I would have had to re-work their formatting and they were no longer relevant. Stuff will slowly be introduced again, with proper formatting 😊

Everything else

In the terms of literally everything except my blog: I'm looking forward to the 33C3. I'll be joining with the c-base assembly. My first congress in almost a decade! Expect maybe an update from that. And maybe there might be some christmas hacking. It's always more fun to do silly RGB LED stuff if it ends up annoying people on the tree!

Also, with the blog now in a bit better shape I will try to keep a closer journal of what I'm doing. But hey...no promises, right? 😉

I shall leave you with this piece of relaxing GIF.

What I have done in GSoC 2016

Katharina Fey — Fri, 19 Aug 2016 18:13:00 +0200

Google Summer of Code is coming to an end. And as the final bugs are getting squashed and more code is being prepared for the big merge, I am sitting here, trying to think of how to represent my work.

I thought I would write up a little blog post, explaining what I've done and what still remains to be done.

The TLDR

My main contributions are all available here (spacekookie/qaul.net on the qaul_crypto branch). I did a lot of small commits. Most of my code can be found in this sub-directory.

In addition to that I ported an existing project (from python) to C to be relevant for future front-end endevours of the client. It's called librobohash. I didn't end up finishing the port because there were more pressing issues in qaul.net and the UI was delayed.

While most of my work has been in hidden backend systems there is a demo you can run. The source compiles and has been tested under Linux (Ubuntu 16.04 and Fedora 24) and is located under the src/client/dbg/ directory. The demo creates two new users (to simulate communication between two nodes), adds the public keys to the keystore and then continues to sign and verify messages. If the demo returns lots of "0" and "OK" it went okay 😊

Feel free to play with the demo; for example, switch out message for fakemessage during verification 😊 The source for the demo can be found under src/libqaul/qcry_wrapper.c

The good (aka what I have done)

The two main components that I've written during GSoC2016 are internally referenced as qcry_arbit and qcry_context. They are two modules that make up the new crypto module in qaul.net.

As I explained in my first blog post on the Freifunk blog the Arbiter provides a static API for the rest of the library (libqaul) to interact with the crypto module.

The context holds the actual magic of holding user keys, signing and verifying messages and (theoretically) encrypting messages as well.

Possible with this API at this time is to create users, to sign messages with a users private key and to verify messages that are sent to you from other users. Originally it was planned to split the arbiter into the actual API and a dispatcher which would allow for concurrent access to the inner functions. However it was established through tests that the design was overkill and was thus scrapped.

A keystore was added in addition to the user store already existing in qaul.net to provide an easy way to store public keys (mapped against fingerprints) that are received from flood events on the network.

In total the crypto submodule adds another ~2.2k lines of code to the project.

The bad (aka what I haven't yet done)

So far completely un-implemented is encryption. Unfortunately working with the crypto library selected for the task turned out to be more challenging than expected. With almost no documentation and a few very niche examples I basically went through the library line-by-line to understand how it worked.

As such, my focus was set on signature exchanges at first because the verifiability of messages and the change to address users by their fingerprints was deemed more important.

My contributions to qaul.net won't end with the end of Summer of Code. The function stubs are already provided and I plan on implementing the encryption features in the coming weeks.

The ugly (aka what I can't do yet)

Signatures (and also encryption) of private messages (so messages that aren't flooded to everybody) is currently impossible. This is due to the way that the communication system in qaul.net works.

I have talked to my mentor and he said that they were currently in the process of re-writing the communication sub-system in libqaul. This means two things:

I need to wait for those changes to be done until I can finish what I set out to do
Some of the code I wrote (hooking into the current communication system) is being made obsolete 😞

In conclusion

What I can say is this: qaul.net has gotten a very big step closer to becoming a more secure network of communication. The crypto submodule is tested and easy to use. What might happen is that parts of the code get merged (the crypto submodule itself) without merging any of the code that hooks into the communication stack.

I had a lot of fun working on this project and I am looking forward to more contributions. I have a few cool ideas that I want to discuss with the rest of the team and I am glad that I participated in the Google Summer of Code.

I was interested in open source before and I contributed to my own projects on github. But the experience I gained this summer will be helpful for me, not just for my own work, but to be less reluctant to join other developer communities.

And I look forward to seeing my code get merged into qaul.net 😊

Read you soon,

~Kate

First steps...baby steps

Katharina Fey — Thu, 02 Jun 2016 19:56:00 +0200

So it's been almost two months, the community bonding period has passed, blog posts were written, talks held and slowly but surely I'm working myself into the qaul.net codebase.

It's always weird joining a larger project and seeing established build setups, code conventions or generally things where your first thought is "I would have done that differently...". But it's really fun.

I'm currently working myself into mbed.tls which is the crypto library which was chosen to power the cryptographic backend for libqaul (which powers qaul.net).

That includes some code that will probably not make it into a later version of my branch: the debugger.

The De-bugger?!

Well...debuger might be a bit of a strong word, it's basically a way to develop core functions of qaul.net without having to start a GUI, going through NetworkManager dialup or oslr bootup.

There I am currently busy writing a wrapper around a new namespace added to libqaul: qcry (short for qaul crypto) and properly integrate all the mbed.tls sources into the library so they can be accessed by libqaul. The idea being that I don't have to leave vim and the terminal to develop on the core cryptographic components such as:

Key generation
Identify generation (with private key fingerprints)
Identity verification
???

Only in the last step of the last bulletin do I actually have to involve the GUI of qaul.net. And until that point I wish to not come in contact with it (if avoidable).

So most of next week will be getting to know mbed-tls as I have never worked with it before. But hey...can't be worse than the gcrypt documentation¹ 😂

Hope to read you soon with more updates (probably rants).

Kate o/

¹I am sure I will eat my words in 4 weeks

I got accepted to GSoC 2016

Katharina Fey — Wed, 27 Apr 2016 18:47:00 +0200

The title should be self explanatory about that one 😊

But let me go back a little bit. A couple of weeks ago I sat in the basement of my local hackerspace talking to a friend about crypto when somebody joined the conversation, asking if I was a student and if I might be interested in Google Summer of Code.

After I looked up the project and familiarised myself with what had to be done, I thought it would be interesting to try to apply. And so I did. I wrote a long-ish proposal of what I wanted to do, how I would do it and when exactly I would acomplish my goals. (You can read my original proposal here)

In the meantime I actually had a sit-down with my mentor (the person joining the conversation in that basement) and made further plans how to implement things.

And so this is it. The next month or so I will have time to get to know the code base of the project (although I partially already have), meet more people from the community and generally get into the rythm of what GSoC is.

I will be posting three blog posts on the official Freifunk Blog, one in a couple of days/ weeks, one in the height of the project and one that will go into the aftermath of the project.

But in the meantime I will be keeping my blog up to date about what I am doing, how things are going, my challenges and things I learn. In the hopes that people might find it useful and lean things from it. Or just to save my insane ramblings in some narcissistic pleasure...to think that I am relevant in the world 😜

Read you soon,

Kate

Open Plantbot – Rev A

Katharina Fey — Wed, 16 Mar 2016 12:08:00 +0100

Spring is coming in Berlin and thus my thoughts – as every year – are with plants and growing them. I live in an appartment with a tiny tiny balcony so I don't have much space but that has never stopped me from wanting to cram as many plants into the space as possible to the point of starting nuclear fusion.

In addition to that I have a few house-plants and very water-sensitive trees in my appartment. My current approach is to go around with a jug of water every couple of days and water them individually – making sure the soil has a certain moisture and doesn't exceed a certain limit – but I've always had the dream of being able to automate away as much as possible. That's where the idea of Plantb0t started. And I want to tell you a little bit about it.

The basic idea is to have a little controller in each plant-pot that measure the moisture of the soil and reports that back to me via an ESP-12 SOM (System on a Module). The ESP has WiFi capabilities and would log to an MQTT server on my home media server. This way (when I'm at home – none of that IoT shit) I can see how my plans are doing.

Current state

So that's what Revision A of Plantb0t is. I also added a second sensor slot which is meant to be populated by a temperature sensor but could theoretically house a second moisture sensor. In the end the probes are only sticks in the ground that have a resistance between them.

Here is a dynamic render of the board (that went into prototype production on the 29th of march, 2016).

As you can see it's powered by an ESP-12 and comes with it's own programmer (The lovely CP2102) and micro-USB header. The USB-Port is currently the only way to power the board.

In the future it is planned to bypass the USB power and only use it for the programmer and otherwise drive everything off an externla powerboard which provides 3.3V for the Plantb0t.

In the bottom you see two constant current sources that can power two analogue sensors that get multiplexed into the ADC of the ESP-12.

GPIO pin headers are included for external gismoz such as a pump to act on the moisture data as well as screwholes to mount the whole thing in a 3D printed case.

In total the board is only 5x5cm big!

Future plans

A few things I want to realise with this project in the next coming weeks:

Primarily the Rev A board needs to be tested to make sure that the programmer works
Figure out a good way to calibrate the sensors. Maybe drive a button via GPIO?
Design a power board that generates 3.3V for the board (but not the programmer!) from a solar panel and a battery to decouple the entire sensor-board from all power-sockets.

For the next revision of the board (Rev B) I want to include more sensor slots. Maybe work on the part spacing a bit and increase footprint sizes. It should be easier to solder and someof the parts are ridiculously small. I mean...I have the whole back to work with?

I also have some crazy ideas for a "Plantb0t+" Version with even MOAR SENSORS (Including a pH-value sensor!). But that's all faaaaar in the future.

Either way...I'm excited for my boards to get here (hopefully in the next 7-8 days) as well as all the parts I need for the prototypes.

I leave you with a screenshot from KiCad where you get to see under the hood of the board. Cheers o/

(The project has a Github repo where I will try to populate the wiki with as much info as possible)

[Update] Jolly Christmas Decoration

Katharina Fey — Fri, 27 Nov 2015 15:30:00 +0100

You might remember I played around with Kicad a few months ago and made this tacky little thing. Just about 2 1/2 weeks ago I went onto DirtyPCB to get them actually made. I wanted to have gone through the production process and get something built before I started doing more complicated projects.

Unfortunately I discovered a little mistake with the design in the layout that ended up at the manufacturer (Rev 3.1). I tried to fix them but Rev 3.2 didn't make it in time, which means my boards will be a bit more complicated to power. However not too complicated as the power-in are just throughholes so I can actually strap anything behind it to power it.

But without further a due, here is the result from DirtyPCB (which I am actually quite impressed with).

Now, I'm new to all of this so I started doing beep-tests on the pads to make sure things were properly connected and all the boards passed them. The production quality is pretty good. Unfortunately I can't start assembling them yet just because none of the parts I ordered for them have arrived yet. The manufacture and shipping of the boards actually beat the shipping of off-the-shelf parts!

Anyways, I'm kinda excited. First time making an electronics project. I might post another update on when the parts arive and post a few gifs of the finished products. If I don't I'll probably tweet about it though.

Now, I have another smaller electronics project in the making where I am, again, waiting for parts to arrive to do some testing. And already designing a modular PCB board. (Limited a bit with the 10x10cm limitations on DirtyPCB I need to design my project in a way that I can take a bunch of smaller panels and stick them together into a large one, which would take hundreds of dollars to make elsewhere).

But realistically for the production quality I saw with these, I'd be happy to give them my money again for future projects. Especially at that price, just unbeatable.

😊

Anyways, enough ramblings. Read you later.

Update...update

Right...so after tinkering with the bauble a bit I found out a few things. The most important one being that I made some mistakes. Some big ones :)

Pin 9 of the shift register was connected to both input A and input B of the XOR gate. Which meant that both inputs were always the same...which also meant that the output was always 0.
The 555-timer clock ran at several hundred kilohertz. I had to change the capacitor down to ~12µF and the resistors to ~4.7 ohms.
The coin-cell battery didn't have enough juice to run it. Two had to be put in parallel. Even then, two batteries would not be able to run for very long.

To make the bauble work I bridget the xor gate completely, so just feeding back the shift register end to the beginning.

In addition to those things some of the LED's sometimes didn't work. I'm not sure if that is due to broken shift registers, traces or LEDs. All in all I do consider it to have tought me quite a lot about electronics, going through the process of producing a PCB and debugging electronics once it arrived and inevidably goes wrong :)

I am currently in the process of redesigning the entire circuite from scratch. And making it easier to solder. I want to make it into a beginner soldering kit that people can both learn how to solder with and also have something to hang off their christmas tree in the jolly season.

Recovering a destroyed LUKs container

Katharina Fey — Thu, 19 Nov 2015 11:41:00 +0100

So...funny thing happened to me the other day. And by funny I mean not funny. Actually I mean quite the oposite of funny. I booted my laptop after shutting it shut down for the first time after several weeks of activity and...nothing.

I stared at my plymouth boot screen while nothing prompted me to type in my passphrase to decrypt my harddrive and the first thought through my mind was:

Fuck...I don't have a backup.

How to debug

Now...not to worry, after some time I was dropped into a recovery console where I could ask very simple questions like what kernel modules were present and what Systemd had been up to. And at first I thought the problem was clear: Module failed to load: vboxdrv and other messages populated my screen – all about VirtualBox kernel modules.

So the problem was clear. I had fucked up something when installing a new kernel or VirtualBox or anything else. So I blacklisted the modules and moved on...just...that it didn't. The problem persisted. Thinking that I had fucked something up when dealing with the GRUB config or the GRUB recovery console I got my trusty Fedora 22 live-USB out and booted off that.

How not to panic

Looking at the partitioning on the disk I realised that my 256GB SSD was only 500MB full (which was rightfully detected as an ext4 formatted volume. The rest of my drive was marked as unpartitioned space. 😱

Now...here is where things get and got interesting. But first let's have a look at my setup.

sda (the actual drive)
├── sda1 (ext4, mounted as /boot, contains my kernel)
└── sda2 (LUKS Encrypted Volume, contains subvolumes)
   ├── vc-root (RootFS)
   ├── vc-home (HomeFS)
   └── vc-swap (guess c:)

So as you can see my boot drive is outside the LUKS container and unencrypted which was why I even got the chance to enter a recovery console. The rest of my system is encrypted. And seeing that only sda1 was being picked up it meant that the partition table on my disk must have had been destroyed to the point that it no longer knew sda2.

Knowing this didn't help very much though and it took me a few hours to fix this.

Restoring the Partition Table

So the main problem was that my partition table was broken. I don't want to start speculating as to why this happened. Maybe my SSD just lost a few blocks, maybe it was bombarded by solar radiation or maybe (just maybe) I was obducted by aliens in the night, refused to give out my master passphrase in my sleep and because of frustration of not being able to get to my data they deleted some junks from my partition table just to spite me.

Either way, a combination of two applications saved my life and hopefully will save yours.

testdisk and cfdisk

At first, make sure you have backups ;) And don't blame me if you fuck it up. Also you need to know EXACTLY what your layout is to restore this. Otherwise BAD THINGS WILL HAPPEN waves hand around warning-ly

Run testdisk on your drive, enter through the screens, let it do a deep search and just say yes to everything it wants to do. This restored the LUKS header for me again at which point my computer at least started seeing the encryption container again. Didn't mean I could log in because keyfiles couldn't be found (they're not in the header apparently).

After that, I ran cfdisk. What this program does (or can do) is rebuild your partition table. After letting testdisk have it's go it found my LUKS header and completely destroyed my ext4 bootpartition. So in my case this is what it looked like.

What you will want to do is hit NEW, select the correct size of your partitions. Depending on how running testdisk went for you it might have found different parititions, all of them or none. Please! For the love of god, make sure you get your sectors right. Becase if you don't it will seriously damage your system and might make it completely unusable. In my caseit was easy, I filled in my boot partition, marked it as bootable and set it's type correctly, also fixed the type error where before sda2 was being picked up as an LVM and not a LUKS container (this is obviously from my running system). And this is what I ended up with.

Make sure you write your changes, exit and reboot. And if you did everything right, you will have a working system again. And that's that. I hope this article will be of use to someone at some point. And remember: make backups!

Cheers o/

Jolly Christmas Decoration

Katharina Fey — Thu, 17 Sep 2015 15:30:00 +0200

Christmas is getting closer (not really but let's just roll with it) and I wanted to learn KiCad a software that let's you create circuits and design PCB for manufacture.

I found a tutorial series online by a guy named Ashley Mills (with quite a legendary beard) who showed off a simple circuit using a 555-timer, a shift register and an XOR gate made from NPN transistors and resistors to display and repeat a pattern on several LED's.

The series focused on getting to know KiCad and all it's features. And while I did that in the first revision of my board, I've diverged from it since. I can however recommend his videos on KiCad to anyone who wants to dive into PCB design, has no clue about the software and could use a little chuckle while also learning some really awesome software (youtube channel link above).

My Christmas Bauble

So this is what I've got.

As you can see it's a round PCB with simple 5mm LED's around the edges. It no longer uses NPN transistors but rather a single SMD XOR gate. Much easier to wire up, cheaper and less prone to errors as well.

In general I've switched the entire design over to primarily use SMD components as they're smaller and more elegant. And it theoretically allowed me to get the footprint of the board down to something that isn't too excruciatingly expensive to produce.

It took me two more revisions to get the board to a state where it's not too complex and actually fit on a single layer (!) with no vias except for the holes for the LED's obviously.

It uses a round cell battery on the back of the board to hide it away and has a hole at the top to actually hang off a christmas tree. Theoretically the battery should lasta few days, so maybe have a few spare ones around in the christmas season.

What now?

I haven't manufactured this yet. I am still thinking about refining the design slightly. I have the entire back to work with and add things. I was thinking about adding a simple bluetooth chip so that patterns could be pushed to the device via an android app. But that's the future. For now it should actually be functional and maybe I'll order some Revision 3 boards just to see that everything worked.

Here is a dynamic render from KiCad as well.

And be sure to checkout my Github repo for the project if you want the KiCad files. Either to play around with them or to manufacture some baubles yourself. If you do, I'd be interested in pictures of the decorations on your christmas trees so I can add them to this article as a slideshow 😊

Chaos Communication Camp 2015

Katharina Fey — Tue, 25 Aug 2015 15:30:00 +0200

Hey everybody, long time no read.

As I returned from vacation on the Chaos Communication Camp 2015 (Not sure if I'll post more about that) and probably starting a new job next week (pssst not sure if I should talk about it 😉 ) the rest of my summer is still ahead of me and I'm booming with ideas and inspiration to do stuff.

I've started more intensively coding on the newdawn branch of Reedb, the C port of the database and planning some features for the old codebase via the backports branch. Because the new codebase will use a different crypto backend (from OpenSSL to gnu_crypt) a migration agent will be neccesary to migrate between 0.11.x to 0.12+ vaults. But as very few people currently use Reedb and most setups are for testing purposes only that isn't a very big priority right now. Depends on how the current version of reedb develops :)

But that's talk for another day. What else has been going on? After the Chaos Communication Camp 2015 I've been playing around a bit with my rad1o badge.

But not much has resulted from that yet. The distribution I'm using (Fedora 22) at this time unfortunately has a broken arm-gcc package which means that a linker for embedded systems isn't working properly. So hacking on that will have to wait a little bit. But I will very likely post more stuff about that in the future.

Until another day, Kate