# codefionn — full content > Full text of every post on https://codefionn.eu, newest first. Companion to [/llms.txt](https://codefionn.eu/llms.txt), following https://llmstxt.org. Posts are written in English and German. # The bad design of LLM APIs > What building an LLM gateway taught me about the state of AI *Published on 2026-10-04 · en · https://codefionn.eu/the-bad-design-of-llm-apis/* I'm currently developing many different AI-enabled applications and specifically created [llmleaf](https://github.com/codefionn/llmleaf), an AI gateway proxy that unifies and enriches all kinds of LLM API providers. During this I had some issues with these interfaces and sometimes with their documentation. This article covers the issues that the AI providers somehow ignore. ## SSE How would you implement a bidirectional communication protocol? Single requests? Custom binary protocol? Maybe WebSockets? The answer chosen by the industry was of course SSE. It's a web standard for streaming events from the server to the client, which is good. Crucially, it doesn't stream the other way around, which is bad. Maybe this was a good decision in the early ChatGPT days, but the post-Opus 4.8 era needs something different. This has two issues. When the client has to send messages back to the server, which in the age of agentic development includes a lot of tool call results, a completely new request has to be made to the server. This is made worse by prompt caching being a requirement for cost reasons and by thinking being hidden or encrypted. When the new request is sent to answer a tool call, the API has to infer the last server response from the context, probably via hashing. Once it's identified, caching works properly and the user hopefully pays less for tokens that were already processed. The implementor has to ensure that the messages sent are detected as the same ones sent previously. There are even special API features that try to work around this, e.g. Anthropic's `cache_control` or OpenAI's `previous_response_id`. All this reconnecting has a latency cost, which would not be paid by proper implementations. If you want to experience this, just use Codex with the OpenAI subscription. It already uses WebSockets.
The agent sends the full history to the LLM API, gets an SSE stream until the next tool call, and sends the tool result as a new request with the full history again
Over SSE, every tool call means a new request with the whole history
The agent keeps one WebSocket open, messages flow both ways over it and only new tool results are sent, while the LLM API keeps the state in memory
Over a WebSocket, the connection stays open and only new input is sent
## MCPs The previous MCP specs were so bad that the 2026-07-28 spec had to fix them. FYI: the previous specs required stateful connections, which made stuff like letting your customers' agents read your documentation over MCP way harder and more resource intensive than it needed to be. The 2026-07-28 spec finally makes MCP stateless. Another issue is that the previous specs made every agent session spawn its own MCP connections, which made 3 or more agents very costly with some larger MCP servers. ## The list endpoint AI providers have endpoints for listing supported models. The issue: they often don't provide necessary information. In the worst case, you will just get model ids, names and release dates.
curl request to the OpenAI models endpoint for gpt-6-astra, returning only id, object, created, owned_by and shutdown_date
OpenAI's model list: no context window, no pricing, no capabilities
But there's a lot of information required for implementing an AI-enabled application: context window length, max output tokens, pricing, supported parameters and capabilities like thinking, input and output modalities (e.g. can receive an image), automatic context window compression (summarization). This is why providers like OpenRouter are an easy solution for applications that support "any" provider, because they provide essential information. ## The future I hope that more providers see how great WebSockets are in agentic workflows and support them more. MCP got fixed with the 2026-07-28 spec, but it will probably take years for the old MCP programs to disappear. List endpoints most likely won't get fixed due to lock-in effects, where you have to optimize your software for a specific provider. Providers don't have many reasons to fix some of these issues, so gateway proxies like [llmleaf](https://github.com/codefionn/llmleaf) or [Portkey AI Gateway](https://github.com/Portkey-AI/gateway) will have to fill the gaps. ## Further resources - [Using server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) - [Speeding up agentic workflows with WebSockets in the Responses API](https://openai.com/index/speeding-up-agentic-workflows-with-websockets/) - [OpenAI WebSocket mode](https://developers.openai.com/api/docs/guides/websocket-mode) - [The 2026-07-28 MCP specification](https://blog.modelcontextprotocol.io/posts/2026-07-28/) --- # Das schlechte Design von LLM-APIs > Was mir die Entwicklung eines LLM-Gateways über den Stand von KI beigebracht hat *Veröffentlicht am 2026-10-04 · de · https://codefionn.eu/das-schlechte-design-von-llm-apis/* Ich entwickle zurzeit viele verschiedene KI-Anwendungen und habe dafür [llmleaf](https://github.com/codefionn/llmleaf) gebaut, einen KI-Gateway-Proxy, der alle möglichen LLM-API-Anbieter vereinheitlicht und mit zusätzlichen Informationen anreichert. Dabei hatte ich einige Probleme mit diesen Schnittstellen und manchmal auch mit ihren Informationen. Dieser Artikel behandelt die Probleme, die die KI-Anbieter irgendwie ignorieren. ## SSE Wie würdest du ein bidirektionales Kommunikationsprotokoll umsetzen? Einzelne Anfragen? Ein selbst erstelltes Binärprotokoll? Vielleicht WebSockets? Die Antwort der Branche war natürlich SSE. Das ist ein Webstandard, um Events vom Server zum Client zu streamen, was gut ist. Entscheidend ist aber, dass es nicht in die andere Richtung streamt, und das ist schlecht. Vielleicht war das in den frühen ChatGPT-Tagen eine gute Entscheidung, aber die Zeit nach Opus 4.8 braucht etwas anderes. Das führt zu zwei Problemen. Wenn der Client Nachrichten an den Server zurückschicken muss, und im Zeitalter agentischer Entwicklung sind das viele Tool-Call-Ergebnisse, muss ein komplett neuer Request an den Server gehen. Verschlimmert wird das dadurch, dass Prompt-Caching aus Kostengründen Pflicht ist und Thinking versteckt oder verschlüsselt wird. Wenn der neue Request einen Tool-Call beantwortet, muss die API die letzte Server-Antwort aus dem Kontext ableiten, vermutlich über Hashing. Sobald sie gefunden ist, funktioniert das Caching richtig und der Nutzer zahlt hoffentlich weniger für Tokens, die schon verarbeitet wurden. Der Entwickler muss dafür sorgen, dass die gesendeten Nachrichten als dieselben erkannt werden, die schon vorher gesendet wurden. Es gibt sogar spezielle API-Funktionen, die versuchen, das zu umgehen, z. B. `cache_control` von Anthropic oder `previous_response_id` von OpenAI. Dieses ständige Neuverbinden kostet Latenz, die eine vernünftige Umsetzung nicht hätte. Wer das selbst erleben will, nutzt einfach Codex mit dem OpenAI-Abonnement. Das verwendet bereits WebSockets.
Der Agent schickt den ganzen Verlauf an die LLM-API, bekommt einen SSE-Stream bis zum nächsten Tool-Call und schickt das Tool-Ergebnis als neuen Request, wieder mit dem ganzen Verlauf
Mit SSE bedeutet jeder Tool-Call einen neuen Request mit dem ganzen Verlauf
Der Agent hält einen WebSocket offen, Nachrichten fließen darüber in beide Richtungen und nur neue Tool-Ergebnisse werden geschickt, während die LLM-API den Zustand im Speicher hält
Mit einem WebSocket bleibt die Verbindung offen und nur neuer Input wird geschickt
## MCPs Die bisherigen MCP-Spezifikationen waren so schlecht, dass die Spezifikation vom 2026-07-28 sie reparieren musste. Zur Info: Die bisherigen Spezifikationen haben zustandsbehaftete Verbindungen verlangt. Das hat Dinge wie den Zugriff der Agenten deiner Kunden auf deine Dokumentation über MCP viel schwieriger und ressourcenhungriger gemacht als nötig. Die Spezifikation vom 2026-07-28 macht MCP endlich zustandslos. Ein weiteres Problem war, dass mit den bisherigen Spezifikationen jede Agenten-Session ihre eigenen MCP-Verbindungen aufbauen musste. Mit einigen größeren MCP-Servern wurden dadurch schon drei oder mehr Agenten sehr teuer. ## Der List-Endpunkt KI-Anbieter haben Endpunkte, die die unterstützten Modelle auflisten. Das Problem: Sie liefern oft nicht die nötigen Informationen. Im schlimmsten Fall bekommst du nur Modell-IDs, Namen und Veröffentlichungsdaten.
curl-Request an den Modell-Endpunkt von OpenAI für gpt-6-astra, der nur id, object, created, owned_by und shutdown_date zurückgibt
Die Modellliste von OpenAI: kein Kontextfenster, keine Preise, keine Fähigkeiten
Für eine KI-Anwendung braucht man aber viel mehr Informationen: Länge des Kontextfensters, maximale Output-Tokens, Preise, unterstützte Parameter und Fähigkeiten wie Thinking, Input- und Output-Modalitäten (z. B. ob das Modell Bilder annehmen kann) und automatische Kontextkomprimierung (Zusammenfassung). Deshalb sind Anbieter wie OpenRouter eine einfache Lösung für Anwendungen, die "jeden" Anbieter unterstützen, denn sie liefern die wesentlichen Informationen. ## Die Zukunft Ich hoffe, dass mehr Anbieter erkennen, wie gut WebSockets für agentische Workflows sind, und sie stärker unterstützen. MCP wurde mit der Spezifikation vom 2026-07-28 repariert, aber es wird wohl Jahre dauern, bis die alten MCP-Programme verschwinden. List-Endpunkte werden wahrscheinlich wegen Lock-in-Effekten nicht repariert, weil man seine Software auf einen bestimmten Anbieter optimieren muss. Die Anbieter haben wenig Gründe, einige dieser Probleme zu beheben. Deshalb müssen Gateway-Proxys wie [llmleaf](https://github.com/codefionn/llmleaf) oder [Portkey AI Gateway](https://github.com/Portkey-AI/gateway) die Lücken füllen. ## Weiterführende Links - [Server-sent events verwenden](https://developer.mozilla.org/de/docs/Web/API/Server-sent_events/Using_server-sent_events) - [Speeding up agentic workflows with WebSockets in the Responses API](https://openai.com/index/speeding-up-agentic-workflows-with-websockets/) (Englisch) - [OpenAI WebSocket mode](https://developers.openai.com/api/docs/guides/websocket-mode) (Englisch) - [Die MCP-Spezifikation vom 2026-07-28](https://blog.modelcontextprotocol.io/posts/2026-07-28/) (Englisch) --- # The age of small personalized software is here > The age of small personalized software is here and it's yours to exploit *Published on 2026-03-06 · en · https://codefionn.eu/the-age-of-small-personalized-software-is-here/* It started for everyone with the introduction of ChatGPT, this new wave of LLMs and other AI models. The generation of text, incrementally even better generation of text with other capabilities allows writing software big and small, allowing developers to move faster and everyday people to write small programs. The past software development cycles have show us that with every introduction of new technologies that make software development easier, more software is written. We had the introduction of textual assembler, Basic, programming languages, easier ways to edit code and open source.
Timeline from textual assembler, BASIC, programming languages, easier editing and open source to AI coding agents, with a curve for software written that rises at each step and jumps at AI coding agents
Each step made code easier to write, and more of it got written
The introduction of AI coding agents made code cheaper—a lot cheaper. This means competing with basic features and feature count is not actually that productive. Large software projects have to survive for a long time and must compete on quality. Fast, well working software that captures the user's intent correctly is a joy to witness. We can take the same amount of time to implement features, but use our newly acquired time to improve quality and longevity, while maybe being a bit faster still. And those items in your backlog? Maybe you can fix them now. Our users can also use AI. Auto translation, more intelligent automations, speech, image recognition and even writing one-off extensions inside of software products. Let them experience what we are experiencing right now. We as software developers have to focus on delivering robust and joyful software that has good guardrails and allows our users to focus on the hard and human stuff. --- # Das Zeitalter kleiner persönlicher Software ist da > Das Zeitalter kleiner personalisierter Software ist da und es steht dir zur Verfügung *Veröffentlicht am 2026-03-06 · de · https://codefionn.eu/das-zeitalter-kleiner-personalisierter-software-ist-da/* Es begann für alle mit der Einführung von ChatGPT, diese neue Welle von LLMs und anderen KI-Modellen. Die Generierung von Text, schrittweise immer bessere Textgenerierung mit weiteren Fähigkeiten ermöglicht es, Software großer und kleiner Art zu schreiben, produktivere Entwickler und den Menschen dort draußen kleine Programme zu schreiben. Die vergangenen Softwareentwicklungszyklen haben uns gezeigt, dass mit jeder Einführung neuer Technologien, die die Softwareentwicklung erleichtern, mehr Software geschrieben wird. Wir hatten die Einführung von textuellem Assembler, Basic, Programmiersprachen, einfachere Möglichkeiten Code zu bearbeiten und Open Source.
Zeitstrahl von textuellem Assembler, BASIC, Programmiersprachen, einfacherem Bearbeiten und Open Source bis zu KI-Coding-Agenten, mit einer Kurve für geschriebene Software, die bei jedem Schritt steigt und bei KI-Coding-Agenten springt
Jeder Schritt machte Code einfacher, und es wurde mehr davon geschrieben
Die Einführung von KI-Coding-Agenten hat Code billiger gemacht – viel billiger. Das bedeutet, dass der Wettbewerb mit grundlegende Funktionen und einer Funktionsanzahl nicht wirklich produktiv ist. Große Softwareprojekte müssen lange überleben und per Qualität konkurrieren. Schnelle, gut funktionierende Software, die die Absicht der Benutzer korrekt erfasst, ist eine Freude zu sehen Wir können die gleiche Zeit für die Implementierung von Funktionen aufwenden, aber unsere neu gewonnene Zeit nutzen, um Qualität und Langlebigkeit zu verbessern, während wir vielleicht sogar noch etwas schneller werden. Und die Punkte in deinem Backlog? Vielleicht kannst du diese jetzt beheben. Auch unsere Benutzer können KI nutzen. Automatische Übersetzung, intelligentere Automatisierungen, Spracherkennung, Bilderkennung und sogar das Schreiben von einmaligen Erweiterungen innerhalb von Softwareprodukten. Lass sie erleben, was wir gerade jetzt erleben. Wir als Softwareentwickler müssen uns darauf konzentrieren, robuste und freudschaffene Software zu liefern, die gute Schutzmechanismen bietet und unseren Benutzern ermöglicht, sich auf das Schwierige und Menschliche zu konzentrieren. --- # Problems with the current LLMs > Problems that I encountered programming a code llm tui app *Published on 2025-12-11 · en · https://codefionn.eu/problems-with-the-current-llms/* I recently developed a TUI for generating code via LLMs in my free time called [scriptschnell](https://github.com/codefionn/scriptschnell). This gave me some insight into what currently is lacking in current LLMs. ## Speed After using Cerebras and Groq (Groq with **q**) to a lesser extent, the speed that OpenAI's models gpt-5.1-codex or gpt-5.1-codex-mini provide is lacking. Code generation requires a lot of looking up current implementation details, creating or changing files and then validating the result. All this requires a lot of tokens (in my experience just searching through the codebase takes at least 20k tokens). Some code generation applications like [Windsurf use a significantly faster model](https://www.cerebras.ai/blog/case-study-cognition-x-cerebras) to speed up some tasks like exploring the codebase. ## Tool calls You can tell that many models are trained to use specific tool calls. E.g. when prompting Kimi K2 Instruct with just "ls" it tries to use a tool call called `shell` even though that doesn't exist.
Terminal session: the user types ls, the model calls a tool named shell, and the app answers with the error unknown tool "shell"
Kimi K2 Instruct reaching for a tool it has never been given
More specifically, it required a lot of trial and error to make LLM models write more complex programs for my golang sandbox tool call (e.g. building an application and extracting key errors with a summarize method). ## Built-in assumptions Model performance currently seems best when using built-in assumptions baked in during the training process. This is especially problematic when using newer external libraries than the training knowledge cut-off date. ## Context window woes The current context windows of state-of-the-art models are at least 128k tokens. But when some part of this is already used by the system prompt and tool call descriptions and then the investigation of the codebase, the context window is too little. The biggest problem here is that model performance seems to drop off a cliff when using large parts of the given context window.
A 128k token context window: system prompt, tool descriptions and at least 20k tokens of codebase exploration fill the start, the last quarter is unreliable, and felt quality drops off a cliff at three quarters
Where the context window goes, and where the model stops listening
E.g. when using Claude Opus 4.5, the latest, greatest and pricey model at the time of writing, it seems to love to ignore the codebase style when 3/4 of its context window is used up. The compaction of the context window to allow seamlessly endless sessions is also hard to steer to success. Transmitting information about code style and problems solved over the summarization boundary feels like a roll of the dice. Also, somehow telling the model that's only half the story leads to failure, because then the model begins going through the codebase again, using up precious tokens. ## Vision Vision still doesn't really work. Even simple tasks like creating an HTML file or SVG file from a screenshot make the model seem incompetent (e.g. creating an SVG from a company logo). ## Closing words I think we still have a long road ahead of us with the current state of transformer models. When you create agents for specific tasks, you have to optimize your program for the specific model family. We've come really far with the current LLM technology and I think a lot of "hacks" on top of the current implementations can still improve the current state significantly. ## Sources - [Case Study - Cognition x Cerebras](https://www.cerebras.ai/blog/case-study-cognition-x-cerebras) - [Context Length Alone Hurts LLM Performance Despite Perfect Retrieval](https://arxiv.org/abs/2510.05381) - [Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell](https://arxiv.org/abs/2406.14673) - [Vision language models are blind: Failing to translate detailed visual features into words](https://arxiv.org/abs/2407.06581) - [Evaluating LLMs at Detecting Errors in LLM Responses](https://arxiv.org/abs/2404.03602) --- # Probleme mit den aktuellen LLMs > Probleme, auf die ich bei der Programmierung einer Code-LLM-TUI-App gestoßen bin *Veröffentlicht am 2025-12-11 · de · https://codefionn.eu/probleme-mit-jetzigen-llms/* Ich habe in meiner Freizeit kürzlich eine TUI zur Code-Generierung über LLMs entwickelt namens [scriptschnell](https://github.com/codefionn/scriptschnell). Dies gab mir einige Einblicke darin, was aktuellen LLMs derzeit fehlt. ## Geschwindigkeit Nach der Nutzung von Cerebras und Groq (mit **q**) in geringerem Umfang, lässt die Geschwindigkeit, die OpenAI's Modelle gpt-5.1-codex oder gpt-5.1-codex-mini bieten, zu wünschen übrig. Die Code-Generierung erfordert viel Nachschlagen aktueller Implementierungsdetails, Erstellen oder Ändern von Dateien und anschließendes Validieren des Ergebnisses. All dies erfordert viele Tokens (nach meiner Erfahrung allein das Durchsuchen der Codebasis benötigt mindestens 20k Tokens). Einige Code-Generierungs-Anwendungen wie [Windsurf nutzen ein deutlich schnelleres Modell](https://www.cerebras.ai/blog/case-study-cognition-x-cerebras), um einige Aufgaben wie das Erkunden der Codebasis zu beschleunigen. ## Tool-Calls Man kann erkennen, dass viele Modelle darauf trainiert sind, spezifische Tool-Calls zu verwenden. Z.B. versucht Kimi K2 Instruct bei der Eingabe von nur "ls" einen Tool-Call namens `shell` zu verwenden, obwohl dieser nicht existiert.
Terminal-Sitzung: Der Nutzer tippt ls, das Modell ruft ein Tool namens shell auf, und die App antwortet mit dem Fehler unknown tool "shell"
Kimi K2 Instruct greift nach einem Tool, das es nie bekommen hat
Genauer gesagt erforderte es viele Versuche und Irrtümer, um LLM-Modelle dazu zu bringen, komplexere Programme für meinen Golang-Sandbox-Tool-Call zu schreiben (z.B. Erstellen einer Anwendung und Extrahieren von Schlüsselfehlern mit einer Summarize-Methode). ## Eingebaute Annahmen Die Modellleistung scheint derzeit am besten zu sein, wenn man eingebaute Annahmen verwendet, die während des Trainingsprozesses eingeprägt wurden. Dies ist besonders problematisch, wenn man neuere externe Bibliotheken verwendet als den Wissens-Cutoff-Zeitpunkt des Trainings. ## Probleme mit dem Kontextfenster Die aktuellen Kontextfenster sind bei State-of-the-Art-Modellen mindestens 128k groß. Aber wenn bereits ein Teil davon durch den System-Prompt und Tool-Beschreibungen belegt ist und anschließend die Untersuchung der Codebasis erfolgt, ist das Kontextfenster zu klein. Das größte Problem hierbei ist, dass die Modellleistung scheinbar abrupt einbricht, wenn große Teile des gegebenen Kontextfensters genutzt werden.
Ein Kontextfenster mit 128k Tokens: System-Prompt, Tool-Beschreibungen und mindestens 20k Tokens zum Erkunden der Codebasis füllen den Anfang, das letzte Viertel ist unzuverlässig, und die gefühlte Qualität bricht bei drei Vierteln ein
Wohin das Kontextfenster geht und ab wann das Modell nicht mehr zuhört
z.B. bei Claude Opus 4.5, dem neuesten, größten und teuersten Modell zum Zeitpunkt des Schreibens, scheint es zu lieben, den Code-Stil der Codebasis zu ignorieren, wenn 3/4 seines Kontextfensters verbraucht sind. Die Komprimierung des Kontextfensters, um nahtlos endlose Sitzungen zu ermöglichen, ist auch schwer erfolgreich zu steuern. Die Übertragung von Informationen über Code-Stil und gelöste Probleme über die Zusammenfassungs-Grenze fühlt sich wie ein Glücksspiel an. Auch das Modell nur die halbe Geschichte zu erzählen, führt zum Scheitern, da dann das Modell beginnt, die Codebasis erneut zu durchsuchen und wertvolle Tokens verbraucht. ## Vision Vision funktioniert immer noch nicht wirklich. Selbst einfachere Aufgaben wie das Erstellen einer HTML-Datei oder SVG-Datei aus einem Screenshot machen das Modell inkompetent wirken (z.B. Erstellen eines SVG aus einem Firmenlogo). ## Abschließende Worte Ich denke, wir haben mit dem aktuellen Zustand der Transformer-Modelle noch einen langen Weg vor uns. Wenn man Agenten für spezifische Aufgaben erstellt, muss man sein Programm für die spezifische Modellfamilie optimieren. Wir sind mit der aktuellen LLM-Technologie wirklich weit gekommen und ich denke, dass viele Hacks, die auf den aktuellen Implementierungen aufsetzen, den derzeitigen Zustand noch erheblich verbessern können. ## Quellen - [Case Study - Cognition x Cerebras](https://www.cerebras.ai/blog/case-study-cognition-x-cerebras) - [Context Length Alone Hurts LLM Performance Despite Perfect Retrieval](https://arxiv.org/abs/2510.05381) - [Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell](https://arxiv.org/abs/2406.14673) - [Vision language models are blind: Failing to translate detailed visual features into words](https://arxiv.org/abs/2407.06581) - [Evaluating LLMs at Detecting Errors in LLM Responses](https://arxiv.org/abs/2404.03602) --- # Why did software become so complex? > Why did software become so complex? Is it ECMAScript? *Published on 2025-07-19 · en · https://codefionn.eu/why-did-software-become-so-complex/* > Why did software become so complex? Is it ECMAScript?
Complex plane with an arrow from the origin to z = 3 + 2i, projected onto 3 on the real axis and 2i on the imaginary axis
Software isn't the only thing that got complex: z = 3 + 2i
## Change in expectations >> [Software is eating the world. - Marc Andreessen](https://www.wsj.com/articles/SB10001424053111903480904576512250915629460) Oh, hello. Did you just try to center a div horizontally and vertically today? Did Visual Studio decide that the error is your whole program, just to be restarted and the error simply not existing anymore? Did you have to restart your language server, because it had enough for now? So why did software become so complex? I think the answer lies in the changing times and expectations of software. When I got my first computer (probably about 2006-2007), there was still the saying >> save often, save early this is not expected behavior anymore and isn't part of the zeitgeist anymore. People expect auto save. Toggle a checkbox, change everything **immediately**. Otherwise, they will call your customer support. Software was an addon, now it is **required**. Schools managed their timetables and enrolled pupils with paperwork and a lot of organization. Parents can now look up their child's timetable and cancelled lessons, if a teacher called in sick. You probably use specialized software and not just excel tables or basic text editor. Before software was nice thing, that speed up your workflow (kinda like LLMs today) or made [big things possible](https://en.wikipedia.org/wiki/ENIAC), but today it exists, and there's no way around it. Your corporate files are immediately stored in a central repository and can be searched for information. This requires at least an intranet, file syncing, version management and authorization. You file your taxes with a computer (and maybe a smartphone?) over the internet. When editing a person in a CMS system, maybe just fetching the data and editing it isn't enough. The data may have to be live updated if someone else edits it at the same time concurrently. Before just people had to talk or write to each other to get things done as different groups, now your local specialties of software products have to communicate with each other too. ## Change in risk profiles >> [To express our gratitude, your next cup of coffee or late night snack is on us! - CrowdStrike](https://www.bbc.com/news/articles/ce58p0048r0o) Today your line of code can cause [someone to be mistangly be accused of fraud and false accounting](https://en.wikipedia.org/wiki/British_Post_Office_scandal), sometimes leading to someone taking their own life. The stock marked was a lot slower a few decades ago. Today milliseconds are the line between profit and loosing money. Software trades and [one wrong, untested configuration can bankrupt an entire company](https://en.wikipedia.org/wiki/Knight_Capital_Group) or [take a huge chunk of the digital world down](https://en.wikipedia.org/wiki/2024_CrowdStrike-related_IT_outages) and with it parts of the real world. ## Change in legal requirements >> [The internet is new land for all of us - Angela Merkel, 2013](https://www.zeit.de/digital/internet/2013-06/merkel-das-internet-ist-fuer-uns-alle-neuland) Politics in some parts of the world also discovered, that people can do pretty shitty stuff with software, too (or most times just want to spy on everything that moves). Software has to follow the law contrary to tech bro opinion (or they just buy a seat in government). In the EU you must have a privacy policy, informing users about the shitty things you do with their data. You have the right to be forgotten and download your data (including but not limited to moderation and internal comments on your person). ## Change in scope, everywhere Software has to be reliable, constantly updated, always evolving, never go down and still be somewhat affordable. This means software based on databases with a file system that runs in three different datacenters, logging, observability, end-to-end testing and live migrations. Software doesn't just have to work for a specific demographic, software must be designed for everyone and everything between 0 and at least a 100 years old (your grandma and the kids using that Tiktok thing). It has to work on Apple computers, Windows arm and x86-64, Linux, smartphones and maybe on a tv. Color palettes cannot be random, because some people may not be able to distinguished between a different set of colors. Your software must be able to show, talk and [braille](https://en.wikipedia.org/wiki/Braille). Software has nearly no limits and can be deployed everywhere. Software must be able to become old, we're talking decades and soon and maybe even now half a century. ## Closing words In summary, software become a big part of the real world and has to do a lot more real world things and deal with them. Of course your blog probably shouldn't be written with *Next.js*. --- # Warum ist Software so komplex geworden? > Warum ist Software eigentlich so komplex geworden? Ist es ECMAScript? *Veröffentlicht am 2025-07-19 · de · https://codefionn.eu/warum-ist-software-so-komplex-geworden/* > Warum wurde eigentlich Software so komplex? Ist es ECMAScript?
Komplexe Zahlenebene mit einem Pfeil vom Ursprung zu z = 3 + 2i, projiziert auf 3 auf der reellen Achse und 2i auf der imaginären Achse
Nicht nur Software ist komplex: z = 3 + 2i
## Wandel der Erwartungen >> [Software is eating the world. - Marc Andreessen](https://www.wsj.com/articles/SB10001424053111903480904576512250915629460) Oh, hallo. Hast du heute versucht, ein div horizontal und vertikal zu zentrieren? Hat Visual Studio entschieden, dass der Fehler dein ganzes Programm ist, nur um neu gestartet zu werden und der Fehler existiert einfach nicht mehr? Musstest du deinen Language Server neu starten, weil er genug hatte? Warum wurde Software also so komplex? Ich denke, die Antwort liegt in den sich wandelnden Zeiten und Erwartungen an Software. Als ich meinen ersten Computer bekommen habe (wahrscheinlich um 2006-2007), gab es noch den Spruch >> save often, save early das ist kein erwartetes Verhalten mehr und gehört nicht mehr zum Zeitgeist. Menschen erwarten automatisches Speichern. Checkbox umschalten, **sofort** alles ändern. Ansonsten rufen sie Ihren Kundensupport an. Software war ein Zusatz, jetzt ist sie **erforderlich**. Schulen verwalteten ihre Stundenpläne und meldeten Schüler mit Papier und viel Organisation an. Eltern können jetzt den Stundenplan ihres Kindes und ausgefallene Stunden nachsehen, wenn ein Lehrer sich krank gemeldet hat. Du verwendest wahrscheinlich spezialisierte Software und nicht nur Excel-Tabellen oder einen einfachen Texteditor. Früher war Software etwas Nettes, das deinen Workflow beschleunigte (ähnlich wie LLMs heute) oder [große Dinge möglich machte](https://en.wikipedia.org/wiki/ENIAC), aber heute existiert sie, und es gibt keinen Weg daran vorbei. Deine Unternehmensdateien werden sofort in einem zentralen Repository gespeichert und können nach Informationen durchsucht werden. Das erfordert mindestens ein Intranet, Dateisynchronisation, Versionsverwaltung und Autorisierung. Du reichst deine Steuern mit einem Computer (und vielleicht einem Smartphone?) über das Internet ein. Beim Bearbeiten einer Person in einem CMS-System reicht es vielleicht nicht aus, nur die Daten abzurufen und zu bearbeiten. Die Daten müssen möglicherweise live aktualisiert werden, wenn jemand anderes sie zur gleichen Zeit gleichzeitig bearbeitet. Früher mussten Menschen nur miteinander sprechen oder schreiben, um Dinge als verschiedene Gruppen zu erledigen, jetzt müssen die lokalen Spezialitäten von Softwareprodukten auch miteinander kommunizieren. ## Wandel der Risikoprofile >> [To express our gratitude, your next cup of coffee or late night snack is on us! - CrowdStrike](https://www.bbc.com/news/articles/ce58p0048r0o) Heute kann Ihre Codezeile dazu führen, dass [jemand fälschlicherweise des Betrugs und der falschen Buchführung beschuldigt wird](https://en.wikipedia.org/wiki/British_Post_Office_scandal), was manchmal dazu führt, dass jemand sich das Leben nimmt. Der Aktienmarkt war vor ein paar Jahrzehnten viel langsamer. Heute sind Millisekunden die Grenze zwischen Gewinn und Geldverlust. Software handelt und [eine falsche, ungetestete Konfiguration kann ein ganzes Unternehmen in den Bankrott treiben](https://en.wikipedia.org/wiki/Knight_Capital_Group) oder [einen großen Teil der digitalen Welt zum Erliegen bringen](https://en.wikipedia.org/wiki/2024_CrowdStrike-related_IT_outages) und mit ihr Teile der realen Welt. ## Wandel der rechtlichen Anforderungen >> [Das Internet ist für uns alle Neuland - Angela Merkel, 2013](https://www.zeit.de/digital/internet/2013-06/merkel-das-internet-ist-fuer-uns-alle-neuland) Die Politik in einigen Teilen der Welt hat auch entdeckt, dass Menschen mit Software ziemlich beschissene Sachen machen können (oder meistens nur alles ausspionieren wollen, was sich bewegt). Software muss sich an das Gesetz halten, entgegen der Meinung der Tech-Bros (oder die kaufen sich einfach einen Sitz in der Regierung). In der EU müssen Sie eine Datenschutzerklärung haben, die Benutzer über die beschissenen Dinge informiert, die Sie mit ihren Daten machen. Sie haben das Recht, vergessen zu werden und Ihre Daten herunterzuladen (einschließlich, aber nicht beschränkt auf Moderation und interne Kommentare zu Ihrer Person). ## Wandel des Umfangs, überall Software muss zuverlässig, ständig aktualisiert, immer weiterentwickelt sein, niemals ausfallen und trotzdem einigermaßen erschwinglich bleiben. Das bedeutet Software basierend auf Datenbanken mit einem Dateisystem, das in drei verschiedenen Rechenzentren läuft, Logging, Observability, End-to-End-Testing und Live-Migrationen. Software muss nicht nur für eine bestimmte Zielgruppe funktionieren, Software muss für alle und alles zwischen 0 und mindestens 100 Jahren (Ihre Oma und die Kinder, die dieses TikTok-Ding benutzen) entwickelt werden. Sie muss auf Apple-Computern, Windows ARM und x86-64, Linux, Smartphones und vielleicht auf einem Fernseher funktionieren. Farbpaletten können nicht zufällig sein, weil manche Menschen möglicherweise nicht zwischen verschiedenen Farben unterscheiden können. Deine Software muss anzeigen, sprechen und [braille](https://en.wikipedia.org/wiki/Braille) können. Software hat fast keine Grenzen und kann überall eingesetzt werden. Software muss alt werden können, wir sprechen von Jahrzehnten und bald und vielleicht sogar jetzt einem halben Jahrhundert. ## Schlusswort Zusammenfassend ist Software zu einem großen Teil der realen Welt geworden und muss viel mehr reale Dinge tun und mit ihnen umgehen. Natürlich sollte ein Blog wahrscheinlich nicht mit *Next.js* geschrieben werden. --- # Go is a great scripting language > Go is a great scripting language **TL;DR:** Go is a great scripting language if you require more than a shell language. *Published on 2025-05-11 · en · https://codefionn.eu/go-is-a-great-scripting-language/* I recently had to write a Dockerfile generator for caching automatic testing runs and incrementally caching the build without an actual incremental build in CICD.
Flow: XML project files go into main.go, run with go run, which writes a Dockerfile that the CI build runs as cached layers for dependencies, build and tests
The job: turn project files into a Dockerfile whose layers CI can cache
Previously this generator was written in bash and powershell. Required features for the choosen language were: - easy xml/general file parsing without external packages - not slow - fast startup - can be easily run with minimal effort The following languages/runtimes were in the running for this task: - Deno (TypeScript/ECMAScript runtime) - Go - Lua - Nushell - Python Deno, Lua and Python require external packages for parsing xml files. Deno would be easier, because external dependencies can simply be imported with just an URL. Nushell has problems parsing certain xml files. Go on the other hand has xml parsing builtin. Go is also simple to run with `go run main.go`, doesn't require auxillary files (like `packages.json` with npm), but can also be compiled and cached. Also, Go is trivally multi-threaded, which is a big problem in many interpreted languages. --- # Go ist eine gute Skriptsprache > Go ist eine gute Skriptsprache **TL;DR:** Go ist eine gute Skriptsprache, wenn du mehr als eine Shell-Sprache benötigst. *Veröffentlicht am 2025-05-11 · de · https://codefionn.eu/go-ist-eine-gute-skriptsprache/* Ich musste kürzlich einen Dockerfile-Generator schreiben für das Zwischenspeichern automatischer Test-Läufe und die inkrementelle Zwischenspeicherung des Builds ohne einen tatsächlichen inkrementellen Build in CICD.
Ablauf: XML-Projektdateien gehen in main.go, ausgeführt mit go run, das ein Dockerfile schreibt, das der CI-Build als zwischengespeicherte Schichten für Abhängigkeiten, Build und Tests ausführt
Die Aufgabe: aus Projektdateien ein Dockerfile mit zwischenspeicherbaren Schichten
Zuvor war dieser Generator in Bash und PowerShell geschrieben. Benötigte Features für die gewählte Sprache waren: - einfaches XML/allgemeines Datei-Parsing ohne externe Pakete - nicht langsam - schneller Start - kann einfach mit minimalem Aufwand ausgeführt werden Die folgenden Sprachen/Laufzeiten standen für diese Aufgabe zur Auswahl: - Deno (TypeScript/ECMAScript-Runtime) - Go - Lua - Nushell - Python Deno, Lua und Python benötigen externe Pakete für das Parsen von XML-Dateien. Deno wäre einfacher, weil externe Abhängigkeiten einfach mit nur einer URL importiert werden können. Nushell hat Probleme beim Parsen bestimmter XML-Dateien. Go hingegen hat XML-Parsing eingebaut. Go ist auch einfach mit `go run main.go` ausführbar, benötigt keine Hilfsdateien (wie `packages.json` bei npm), kann aber auch kompiliert und zwischengespeichert werden. Außerdem ist Go trivial multi-threaded, was ein großes Problem in vielen interpretierten Sprachen ist. --- # Rust Buildkit CICD Pipeline > A basic rust buildkit CICD pipeline with cross platform compilation *Published on 2025-02-26 · en · https://codefionn.eu/rust-builkit-cicd-pipeline/* The following files create an buildkit CICD pipeline with cross platform compilation support for `x86-64`, `aarch64` and `riscv64gc` for *Rust*. Also, the program is compiled with *musl*, so the resulting image is as small as possible. The tests output cobertura files (via `cargo-tarpaulin`) for use in your git-platform (e.g. Gitlab). Also, this whole pipeline has been run with *podman* in the background with *docker-buildx* installed but using the *podman* context, so it can be run without root privileges. You will have to modify the pipeline further to fit your program and upload the resulting docker images to a registry. Run the whole pipeline with: ```sh docker buildx bake default lint clippy test ``` Your `docker-bake.hcl`-file: ```hcl target "default" { platforms = [ "linux/amd64", "linux/arm64", "linux/riscv64", ] output = ["type=cacheonly"] } target "lint" { target = "lint" output = ["type=cacheonly"] } target "clippy" { target = "clippy" output = ["type=cacheonly"] } target "test" { target = "test-results" output = [ "type=local,dest=.", ] } ``` Your `Dockerfile`: ```Dockerfile # syntax=docker/dockerfile:1 # Script for converting Go's TARGETPLATFORM to Rust's target triple # (also comes with RUSTFLAGS settings) FROM --platform=${BUILDPLATFORM} docker.io/python:3.14-rc-alpine AS build-env ARG TARGETPLATFORM RUN < ./src/main.rs COPY Cargo.toml Cargo.lock ./ FROM toml-files AS fetch RUN . /env; cargo fetch --locked --target "$RUST_TARGET" # Copy all source files to the container now FROM fetch AS fetch-with-files COPY . . FROM fetch-with-files AS clippy RUN cargo clippy --all-targets --all-features -- -D warnings FROM fetch-with-files AS lint RUN cargo fmt --check FROM fetch-with-files AS test RUN cargo-tarpaulin --engine llvm --out Html --out Xml --output-dir /target/coverage FROM scratch AS test-results COPY --from=test /target/coverage/ /target/coverage/ FROM fetch-with-files AS build ARG TARGETPLATFORM RUN --mount=type=cache,id=cargo-build-${TARGETPLATFORM},dst=/src/target . /env; cargo build --release --target $RUST_TARGET RUN --mount=type=cache,id=cargo-build-${TARGETPLATFORM},dst=/src/target . /env; mkdir /app; cp -r ./target/$RUST_TARGET/release/* /app RUN ls FROM scratch AS final COPY --from=build /app/ /app/ CMD ["/app/my-app"] ``` Your `.dockerignore`: ```gitignore /.git /.gitignore /target **/target /docker-backe.hcl /Dockerfile /.vscode /docs /README.md ``` --- # Useful git stuff > Useful commands for using git *Published on 2024-12-14 · en · https://codefionn.eu/useful-git-stuff/* **git** has a vast feature set for those, that want to clean up their merge requests (before and after the review, but not during). First of: Use the command line tool. For merging other tools may be more useful, but please don't use anything more than that. 3rd party integrations are always different, learn the CLI once and be good at git forever. ## List of useful commands The list below is a list of useful, advanced git commands (so no just `git commit -a -m "msg"`) | Command | Explanation | |:---------------------------------------|:-------------------------------------------------------| | git commit --amend | Change the last commit (be sure you can change it on | | | the remote! | | git rebase --interactive <ref>~ | With this you can fixup your commit structure from | | | the COMMIT-REF to the latest commit on the branch | | git reset --soft <ref> | Remove all commits up to but not including <ref> | | | and add modified files in staging | | git commit --fix <ref> | Create a simple commit fixing something in a merge | | | request, so you can simply rebase it away in the | | | future! | | git add --patch <file> | Add only partial changes from a file | ## Things you should do ### git rerere Did you ever get annoyed by rebasing onto the main branch and having to resolve conflicts, which you have previously resolved? Enter [**git rerere**](https://git-scm.com/book/en/v2/Git-Tools-Rerere), the solution to your `git rebase` problems. --- # Nützliche Git-Befehle > Nützliche Befehle für die Verwendung von Git *Veröffentlicht am 2024-12-14 · de · https://codefionn.eu/nuetzliche-git-befehle/* **git** hat einen großen Funktionsumfang für diejenigen, die ihre Merge Requests aufräumen wollen (vor und nach dem Review, aber nicht währenddessen). Vorerst: Benutze das Kommandozeilen-Tool. Zum Mergen mögen andere Tools nützlicher sein, aber bitte benutze nicht mehr als das. Integrationen von Drittanbietern sind immer unterschiedlich, lerne die CLI einmal und sei für immer gut in Git. ## Liste nützlicher Befehle Die folgende Liste ist eine Liste nützlicher, fortgeschrittener Git-Befehle (also nicht nur `git commit -a -m "msg"`) | Befehl | Erklärung | |:---------------------------------------|:-------------------------------------------------------| | git commit --amend | Ändere den letzten Commit (sei sicher, dass du ihn auf | | | dem Remote ändern kannst! | | git rebase --interactive <ref>~ | Damit kannst du deine Commit-Struktur vom COMMIT-REF | | | bis zum neuesten Commit auf dem Branch korrigieren | | git reset --soft <ref> | Entferne alle Commits bis ausschließlich <ref> | | | und füge geänderte Dateien zum Staging-Bereich hinzu | | git commit --fixup <ref> | Erstelle einen einfachen Commit, der etwas in einem | | | Merge Request korrigiert, sodass du ihn in Zukunft | | | einfach wegrebasen kannst! | | git add --patch <file> | Füge nur teilweise Änderungen aus einer Datei hinzu | ## Dinge, die du tun solltest ### git rerere Hast du dich jemals darüber geärgert, auf den Main-Branch zu rebasen und Konflikte lösen zu müssen, die du bereits zuvor gelöst hast? Hier kommt [**git rerere**](https://git-scm.com/book/de/v2/Git-Tools-Rerere), die Lösung für deine `git rebase`-Probleme. --- # The PL series - In defense of dynamic type systems > The programming language series - In partial defense of dynamic type systems *Published on 2024-06-22 · en · https://codefionn.eu/the-pl-series-in-defense-of-dynamic-typesystems/* This is the programming language series on this blog. This series of blog articles is trying to explain certain programming language design decisions/paradigms. This article is about dynamic type systems and why they are a valid paradigm for certain applications. ## First of: What is a dynamic type system? Programming language like **ECMAScript** or **Python** famously don't use explicit types, but also shells like **(ba)sh** or **PowerShell**. In dynamically type systems you do not have the specify the type of objects and type checking is performed at runtime (e.g. primitive types). This typically allows you to freely modifiy structures used, like adding properties or functions. ## Consequences The goal of dynamic type systems is to allow the programmer to the create programs without thinking about (your own) types (at least that should be the goal here). But types still have to be checked, if you call functions manipulating lists or strings. This means that programming languages using a dynamic type system, typically also feature a heavy use of implicit type coercion (implicitly converting types). E.g. in ECMAScript you have `==`, which implicitly converts types and `===` which only checks equality of objects with the same type or `"" + 0` is evaluated to just `"0"`.
Node.js REPL: "" + 0 gives the string 0, 0 == "" is true, 0 === "" is false
Implicit coercion in Node.js: == converts, === doesn't
Also, a programmer has to write more tests, because the structure (objects, function signatures) has to be tested as well. Due to types being unknown before actually running a program here, programming languages using this type system are typically interpreted or have a compiler producing comparably slower programs or target a VM. Another consequence is that, it is harder to offer code completions because the code editor/LSP has to look into function bodies quite extensivly to guess the valid usage of a variable. Refactorings are also made harder, because you can't just statically type check your whole program for errors, because of the change. The programmer has to creatively employ the use of regex to search and destroy (replace) or waste countless hours using inferior search methods (not counting awk). ## Advantages Dynamic type systems allow a programming language to be, well, more dynamic. This allows you to: - retroactively fix mistakes in a console interpreter - sometimes fix the standard library (or [break it](https://github.com/mootools/mootools-core/issues/2402)) - polyfilling - not needing to express complex structural behaviour in types (but in documentation). ## My opion I prefer using statically typed languages for larger projects. But for very small programs and shell-like senarios, I prefer the dynamic nature of dynamic type systems. Especially the relatively poor LSP experience and refactoring hell using dynamically typed programming languages make them unsuited for larger systems. As a PHP programmer sometimes handling older PHP code, I know. Also: When using JSDoc with ECMAScript, you partially don't use a dynamic type system anymore. --- # Your language can improve correctness > Your programming language of choice can actually improve the correctness of your program *Published on 2024-04-10 · en · https://codefionn.eu/your-programming-language-can-improve-correctness/* *A/N:* Unlike other articles on this blog, it isn't continuously updated. Because this is a matter of opinion. Yes, you read right: the programming language of your choice can actually improve the correctness of your program. Not everything you choose is made equal. Tooling choices actually matter. ## Example Imagine two programs written in programming language A and another written in B: - A is a statically typed programming language
(simplified example: `let x: int = 0`) - B is a dynamically typed programming language, i.e. no types
(simplified example: `let x = 0`) Both programming languages have the following equal conditions: - Latency to executing the program (e.g. compilation step, startup of the interpreter) - Runtime speed - Automated testing And no, this is not an article against dynamically typed programming languages. ### How do you improve correctness? First of, correctness in this context means the following: - your program does fullfill perceived (by the developer) initial requirements - your program works in real-world deployments For program written in **A** you can do the following: 1. Let others critique your code 2. Write automated tests 3. Do tests by hand 4. Use the forced upon you type system For program written in **B** you can do the following: 1. Do the above 1-3
Two stacks: language A has code review, automated tests, manual tests and a type system, language B has the same first three and an empty slot where the type system would be
Same checks, minus one layer
The frequency of automated tests and manual tests can be improved with a smaller latency to executing the program and faster runtime performance, which most likely leads to increased program correctness. There's also a trade-off between a more strict type system and a vastly improved testing story. ### Comparision Programming language **A** does provide a mechanism to improve the correctness of your code: a static type system. **B** doesn't have that advantage and looses on that point. Due to all other features being equal, **A** is better at improving program correctness, because **B** cannot easily replace the need for better correctness with more tests. Of course, if it's too time-consuming for you to use said statically typed language due to skill issues or comfort, you may be better off programming in **B**. ## In practice Go vs. TypeScript First of, a collection of features in both languages, that could drive correctness. Summarizing **Go**: - A static type system - Fast compilation speed, even faster startup time of compiled programs - [Built-in support for writing unit tests](https://go.dev/doc/tutorial/add-a-test) Summarizing **TypeScript**: - A static type system - No built-in support for writing unit tests - [Hot module replacement](https://v2.vitejs.dev/guide/features.html#hot-module-replacement) (in some cases) **Go**'s type system is superior in correctness, because **TypeScript's** `any` can at any time break your beautifully (painfully) typed world. Due to **Go**'s built-in unit tests, small applications can more easily be tested. In small projects, **Go**'s compilation speed and amazing runtime speed, increases the frequency of automated and manual tests, which is less likely with **TypeScript**. On the other hand in bigger web dev projects, I would rate both languages equal, though my experience is lacking in this area (professional PHP, C#, ECMAScript, small-time TypeScript programmer here). Other bigger **TypeScript** applications cannot profit as much from the likes of HMR, meaning you're better of writing them in **Go**, notwithstanding other trade-offs. Use the right to tool for the **job**. Not the DX tool. (DX as in = the thing you are already the most comfortable with, before you used it). When you're not on the job, maybe this just helps you to explain some of your tooling problems. ### Closing words It was a hard decision to include HMR to '**TypeScript's** features', but it offsets compile time in (complicated) web dev environments. I didn't add testing frameworks, because simple console applications can't really use a built-in language feature for testing. This article btw. only cares about language features, which (may) improve program correctness. This article was written with the help of Lume, a 11ty-like Deno static webpage generator, and yes some TypeScript. --- # DDNS with unbound and kea > Dynamic DNS with unbound and kea for giving local devices DNS-names *Published on 2023-11-14 · en · https://codefionn.eu/ddns-with-unbound/* My [NixOS router](/nixos-router-with-j4125-and-intel-i225/) uses [unboud](https://nlnetlabs.nl/projects/unbound/about/) as a DNS server and [kea](https://www.isc.org/kea/) as the DHCPv4/6 server (currently the IPv6 DHCP isn't working). This article contains a script for listening to changes in *kea* DHCP-leases and writing the hostnames to unbound. This gives your local devices domain names. You might know that, *kea* also supports DDNS configuration. While that may be true, this only works, if the DNS server supports TSIG. The problem is that unbound doesn't support it. I looked into moving from unbound to a differnt DNS server (like *Bind9* or *Knot DNS*), but those are complicated to configure with NixOS, so I stayed with unbound. The solution: a bash script running as systemd service. The script itself requires that unbound can be remote controlled and the *inotify-tools* for listening to file changes. ```bash #!/usr/bin/env bash HOSTNAME="$(hostname).lan" function readFile() { if [[ "$2" == "A" ]] ; then cat "$1" | tail -n +2 | while IFS=, read -r address hwaddr client_id valid_lifetime expire subnet_id fqdn_fwd fqdn_rev hostname state user_context do echo "${address},${hostname}" done else cat "$1" | tail -n +2 | while IFS=, read -r address duid valid_lifetime expire subnet_id pref_lifetime lease_type iaid prefix_len fqdn_fwd fqdn_rev hostname hwaddr state user_context hwtype hwaddr_source do echo "${address},${hostname}" done fi } function readFileUnique() { readFile "$1" $2 | uniq | while IFS=, read -r address hostname do if [[ "${hostname}" == *.$HOSTNAME ]] ; then echo ${hostname} $2 ${address} unbound-control local_data ${hostname} $2 ${address} if [[ "$2" == "A" ]] ; then echo ${address} | while IFS=. read -r ip0 ip1 ip2 ip3 do unbound-control local_data ${ip3}.${ip2}.${ip1}.${ip0}.ip4.arpa. PTR ${hostname} unbound-control local_data ${ip3}.${ip2}.${ip1}.${ip0}.in-addr.arpa. PTR ${hostname} done fi fi done } function syncFile() { readFileUnique "$1" "$2" while inotifywait -e close_write,create "$1" ; do readFileUnique "$1" "$2" done } syncFile "/var/lib/kea/dhcp4.leases" A & syncFile "/var/lib/kea/dhcp6.leases" AAAA & wait ``` This scripts reads the DHCP4/6-Leases CSV database stored by *kea* and then reads the address and hostname from that table. *unbound-control* is called to set the `A` or `AAAA` record. If it is IPv4, we reverse the IP-address and store a PTR record for reverse IP-lookup (`dig -x YOUR.IP.GOES.HERE`). The wait at the end waits for the to forked `syncFile`-functions to finish. This script first reads all DHCP-leases and sets the DNS records accordingly and then waits for changes in the DHCP-leases with inotify. Hostnames used for creating these DNS-Records must end here with your hostname + local-domain, That's why, you have to make sure that your kea DHCPv4/6 has the following configuration option: ``` ddns-qualifying-suffix = "YOUR-HOSTNAME.lan" ``` In your systemd-Service set `PartOf`, `After` and `Wants` to `unbound.service`. This will ensure that, the script is started, after `unbound` and that the script is always restarted, when unbound itself is. `WantedBy` must be set to `multi-user.target`. The configuration in NixOS is: ```nix systemd.services.unbound-sync = { enable = true; path = with pkgs; [ unbound inotify-tools ]; script = '' function readFile() { if [[ "''\$2" == "A" ]] ; then cat "''\$1" | tail -n +2 | while IFS=, read -r address hwaddr client_id valid_lifetime expire subnet_id fqdn_fwd fqdn_rev hostname state user_context do echo "''\${address},''\${hostname}" done else cat "''\$1" | tail -n +2 | while IFS=, read -r address duid valid_lifetime expire subnet_id pref_lifetime lease_type iaid prefix_len fqdn_fwd fqdn_rev hostname hwaddr state user_context hwtype hwaddr_source do echo "''\${address},''\${hostname}" done fi } function readFileUnique() { readFile "''\$1" ''\$2 | uniq | while IFS=, read -r address hostname do if [[ "''\${hostname}" == *.${config.networking.hostName}.lan ]] ; then echo ''\${hostname} ''\$2 ''\${address} unbound-control local_data ''\${hostname} ''\$2 ''\${address} if [[ "''\$2" == "A" ]] ; then echo ''\${address} | while IFS=. read -r ip0 ip1 ip2 ip3 do unbound-control local_data ''\${ip3}.''\${ip2}.''\${ip1}.''\${ip0}.ip4.arpa. PTR ''\${hostname} unbound-control local_data ''\${ip3}.''\${ip2}.''\${ip1}.''\${ip0}.in-addr.arpa. PTR ''\${hostname} done fi fi done } function syncFile() { readFileUnique "''\$1" "''\$2" while inotifywait -e close_write,create "''\$1" ; do readFileUnique "''\$1" "''\$2" done } syncFile "/var/lib/kea/dhcp4.leases" A & syncFile "/var/lib/kea/dhcp6.leases" AAAA & wait ''; wants = [ "network-online.target" "unbound.service" ]; after = [ "network-online.target" "unbound.service" ]; partOf = [ "unbound.service" ]; wantedBy = [ "multi-user.target" ]; }; ``` --- # Some of PHP's problems > Some of PHP's problems that lead to technical difficulties *Published on 2023-11-12 · en · https://codefionn.eu/some-of-phps-problems/* First of: What even is PHP? PHP is a widely used programming language in the Web. Wordpress & plugins are built on PHP. Another cool thing about PHP is, that can just throw a PHP script onto a typical webserver and it will *just work*. The language's type system itself is a mix between ECMAScript, TypeScript and Java. [PHP 8.2 even deprecated dynamic properties](https://www.php.net/manual/en/migration82.deprecated.php) (though you can still opt-in on a class by class basis). It has a package manager called [composer](https://getcomposer.org/) and the package registry is [packagist](https://packagist.org/).
PHP logo
PHP
Composer logo
Composer
(logo color changes)
Packagist logo
Packagist
## Problems ### PHP tutorials are misleading PHP was created to make HTML more interactive. That's why every PHP file has to start with ` PHP Test

Hello, World

``` and [transform it to this](https://www.w3schools.com/php/php_superglobals_request.asp): ```php PHP Test Hello, ' $_REQUEST['name'] . '

'; ?> ``` That's cool. Now the webpage says **Hello, Fionn**, if the GET or POST parameter **name** is set to **Fionn**. The issue is that beginners might think: *Oh yeah, that's just like a template.* But PHP doesn't escape special HTML symbols, so you could do a XSS injection here. You have to escape the input parameter here with ``htmlentities``, like this: ```php PHP Test Hello, ' . htmlentities($_POST['name']) . '

'; ?> ``` Like in a video from [Programmers are also human](https://youtu.be/GzRfvwo1iNU?t=50) said: > The web1 wouldn't have existed without PHP and the web2 wouldn't be as > cautious of security issues. This is mainly a problem with existing beginner tutorials, but bleeds into actual production code. Also stay clear of the *HTML with special tags* stuff that PHP offers. Use an actual templating language. ### PHP's weird type coercion While ECMAScript developers might complain that `0.1 + 0.2 == 0.3` results in `false`, that is perfectly normal, because you're using [floating-point numbers](https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_point_in_modern_computers). In PHP [md5('240610708') == md5('QNKCDZO')](https://news.ycombinator.com/item?id=9484757) results in `true`. And not because of hash-collisions. This is because `md5('240610708')` is `0e462097431906509019562988736854` and `md5('QNKCDZO')` is `0e830400451993494058024219903391`. Notice that there are just numbers behind the `0e`? PHP notices too. And converts it. To a floating-point number. Both floating-point numbers are 0. This results in `0 == 0`, which is `true`. So PHP just implicitly converted a string to a float, even if both sides are strings. Debug this in production. I dare you. You can disable some PHP's type coercion with `declare(strict_types=1);`. Though, your coworkers might not like you using this and it won't fix the `md5('240610708') == md5('QNKCDZO')` issue, so use `===` (or `!==`) instead. There's another example of this, that I just found recently: ```php PHP_INT_MAX + 1 ``` On a 64-bit system that is `9.2233720368548E+18`. We went from an **int** to **float**. Just great. ### Functions as values Let's square all values in of an array. Example with arrow functions: ```php array_map(static fn($x) => $x * $x, [1, 2, 3]) ``` Example with anonymous functions: ```php array_map(function($x: int): int { return $x * $x; }, [1, 2, 3]); ``` Or do this: ```php function square(int $x): int { return $x * $x; } array_map('square', [1, 2, 3]) ``` So yeah, I would prefer ```php static fn($x) => square($x) ``` instead of that. Your LSP server and tooling will thank you. ### PHP's arrays [PHP provides arrays, which are actually ordered maps.](https://www.php.net/manual/en/language.types.array.php) So if you're expecting an actual array (a map with `0..n` keys), this contract is in comments only. This also created interesting problems which manifest in the standard library. Because sometimes you want the keys to be preserved and sometimes not. Example: ```php // [0 => 0, 1 => 1, 2 => 2, 3 => 3, 4 => 4] $ary = range(-1,4); // [0 => 0, 2 => 2, 4 => 4] $result = array_filter($ary, static fn($x) => $x % 2 === 0); ``` So you have to be cautious of that. It doesn't help that PHP calls that stuff an **array**. In PHP arrays are also not objects, which means you must call `array_map($fn, $ary)` instead of `$ary->map($fn)`. Arrays behave more like values like integers in other languages. If you modify them, you don't modify an object (with a pointer) but modify them in the current context (*stack*). Example: ```php $ary = [1, 2, 3]; function add_four($ary) { $ary[] = 4; } add_four($ary); // $ary is still [1, 2, 3] ``` You can do that with references: ```php $ary = [1, 2, 3]; function add_four(&$ary) { $ary[] = 4; } add_four($ary); // $ary is now [1, 2, 3, 4] ``` This is just a insignificant detail, but I thought it was worth mentioning. And did you know that `current`, `each`, `end`, `next` or `prev` can really make your coworkers day miserable? So don't use them. But PHP's arrays are also a happy accident, because they're great for writing JSON. ### Typesystem without commitment Modern PHP tries to have a better type-system story with each version (e.g. [type declarations in properties](https://www.php.net/manual/en/language.oop5.properties.php)). But it lacks significant features like templates (or something like it) and: - arrays (read: maps) cannot be specified except that it is an array - Generators (yield) cannot be specified further Variables can be declared on the fly, without any *complicated* syntax like (type|let|const|var) varname = val, but with just $varname = val. As PHP variables aren't typed, their type can always completely change with a new assignment. And yes some of that can be somewhat further specified with a good LSP and PHPDoc. ### Serverless PHP is what many call today *serverless* (in 1993!), though it is typically hosted on just one server and not globally replicated on a CDN. This creates several problems: - no caching in memory (you have use external caching services like APCu or memcached) - WebSocket support? Where? ([Ratchet](https://github.com/ratchetphp/Ratchet) apparently died and Openswoole is an uncommon extension) - Short application lifespan, which doesn't allow PHP's JIT to scale well even when using [OPcache](https://www.php.net/manual/en/intro.opcache.php) - PHP's ability to create global definitions changes the environment which makes optimizations over several requests more difficult Additionally the PHP interpreter isn't supported by *Big Tech* like ECMAScript (or Python), which means that there isn't the budget to do as many magic optimizations like V8 has. ### Performance Improving the performance of a PHP program as a PHP dev is hard due to several different reasons: - it's an interpreted language - use cases that aren't supported by the standard library are slow - no async-await support - no escape hatch like WebAssembly in ECMAScript I actually wrote a full-blown lexer, parser, interpreter and compiler for a small company-specific template language. That experience taught me, that writing readable code is sometimes the polar opposite of writing good performing PHP code. ### Small complaint corner about try catch This might out me as a Rust-guy, but try-catch statements are just the worst sometimes. In PHP you can always throw an exception and don't even have to care about that. At least in Java you can somewhat force the caller to know about your exception. ## Summary PHP is a weird language with messy and contradicting ideas, but easy to read. I sometimes just feel like developing something in C without the performance benefits. Currently, a real deal-breaker can be no async-await support (as in concurrency). I hope this changes. Though, recent changes like adding `Enumerations` with `match` statements really catapult the language forward. You might just be able to actually use that in production in half a decade (which isn't an entirely unrealistic timeframe). --- # What's the difference between compilers and interpreters? > What's actually the difference between compilers and interpreters? And why is it difficult to define? *Published on 2023-04-22 · en · https://codefionn.eu/whats-the-difference-between-compilers-and-interpreters/* This blog post will explain what compilers are and what interpreters are. Then the differences are outlined and why there's a blurry line between them. For more information about some of the terms used here, go to my compiler and interpreter lexicon. ## What's a compiler? A compiler transforms code into a different or even the same kind of code. Typically, compilers are defined by transforming human-readable source code into machine readable code. Examples for compilers are GCC, clang, javac or the TypeScript compiler. Even an ECMAScript minifier could be defined as a compiler. I personally would define something as a compiler, if it has deeper knowledge about the input code and the output is a different kind of code, that semantically does the same thing. ## What's an interpreter? An interpreter executes code immediatly. Technically, a virtual machine executing bytecode could also be counted as an interpreter. Though, typically the input is expected to be high-level, human-readable code. Examples for interpreters are V8, CPython or technically the JVM. ## So what's the difference? The typically definition is that a compiler transforms code into machine code, that will be executed later, and interpreters execute code immediatly. A compiler doesn't execute code, that mutates the runtime or the system, an interpreter does. ## And why is that so hard to define today? Interpreters like V8 can compile the code into bytecode or machine code, so that the execution of the code is faster (this is called JIT compilation). See How JavaScript works: Optimizing the V8 compiler for efficiency for more information. Likewise, compilers can execute code at runtime see constexpr in C++. Rust (or Cargo for that matter) can even execute not-constant-code at compile time, see Build Scripts or proc_macro. --- # Web security basics > Web security 101 *Published on 2023-04-16 · en · https://codefionn.eu/web-security-basics/* This article covers a few of the basics, that are required for a secure connection and what types of attacks could be prevented using them. These are only a few of many other security measures. ## Basic basics ### HTTPS - TLS End-to-end encryption between the server and the client using public-key cryptography. Certificates can be aquired without hassle with Let's Encrypt. > If not used: plaintext can be read (e.g. MITM attack) ### No standard passwords Even if you want to easily administer the website (of a client), do not ever use standard or built-in passwords. If you must have a somewhat secure standard login, use public-key cryptography. > If used: It will be found ### Authentication As sad as it is, somehow (nodejs) backend developers love to forget this. If not used: PR is much more important than basic security, right? > And yes, authentication is always one of the hardest parts to do right but > completely forgetting it, cannot be excused. ### Password hashing Hash every password you can hash and DO NOT store the cleartext password. If not used: An attacker with access to the database, can immediatly misuse the user logins. Keep your password hashing algorithms **up-to-date**, don't use MD5. ## Strict Transport Security (HSTS) HSTS Informs the client that the site should only be accessed using HTTPS. Even when a link to the website uses the `http://` schema, the client automatically uses `https://` instead. This should be in the section Basic basics, but reality is sadly different. If not used: Even if the server always forwards to HTTPs traffic, an attacker can force a regression attack, forcing the client to communicate with plaintext Example: ``` Strict-Transport-Security: max-age=63072000; includeSubDomains; Strict-Transport-Security: max-age=63072000; includeSubDomains; preload; ``` ## Content Security Policy (CSP) Restricts where resources (e.g. images, iframes, ECMAScript) can come from. If not used: Attackers could - if the Content-Security-Policy does not exist or is not restrictive enough - use a XSS attack, if they successfully inject HTML or ECMAScript-Code into your webpage. Example: ``` Content-Security-Policy: default-src 'self' ``` ## X-Frame-Options: DENY Do not allow your website to be included as an iframe in another webpage. Example: ``` X-Frame-Options: DENY ``` ## X-Content-Type-Options: nosniff Do not allow sniffing (check the Content-Type). If not used: An attacker could potentially use an upload form on your website and include malicious code in `.js`. This could also be combined with a XSS attack on your website (or on other websites). Example: ``` X-Content-Type-Options: nosniff ``` ## Cookies As cookies are used for authentication, they should be protected properly. ### HttpOnly If true, then the cookie cannot be read with ECMAScript. ### Secure If Secure is true, the client forces cookies to be only transmitted with HTTPS. If not used: If an attack can force a connection with HTTP (e.g. custom links) the cookie can be read. In older browsers the cookie could be overwritten with an insecure cookie by the client, if the server does not check for this, the cookie could still be read, even if it was initially Secure. ### SameSite SameSite can restrict the cookie to the first-party (current domain). - Strict: Cookie is only send when origin. This also means, that an user coming from a different website or the QR-Code reader will not be logged in. - Lax: Cookie is send to domains different from the origin, no third party cookies None: Same as Lax but with third party cookies --- # Useful Linux command line tools > An incomplete list of useful Linux command-line tools *Published on 2023-04-16 · en · https://codefionn.eu/useful-linux-command-line-tools/* This is an incomplete list of some useful Linux/\*BSD command-line tools:
Command Description
ncdu List size of directories and files
sed -i 's/old/new/' Find & replace with Regex
telnet host port Test cleartext network connection
openssl s_client -connect host:port Test encrypted network connection
vim/emacs (Most) Powerful text editor. Install COC plugins for the awesome vscode language server.
fd Awesome rust replacement for find
rg Awesome rust replacement for grep
bat Not a replacment for cat but an addition
netstat -tulpn Show open network ports
--- # The importance of randomness in authentication > Importance of secure random generators when generating authentication keys *Published on 2023-04-16 · en · https://codefionn.eu/the-importance-of-randomness-in-authentication/* Authentication keys are machine-generated character sequences used for password resets, account deletions, API keys and other use cases. The generated random character sequence must use a secure random generator to create the key, otherwise anyone can use them. ## Why? If the attacker knows the algorithm used to generate the authentication key, which uses pseudo random numbers based on known metrics such as time and therefor not cryptographically secure, it can be more easily brute forced. Take for example this time-based generating algorithm: ```js // Additional is a parameter for additional bytes to avoid collisions. Don't // be confused by this, its just to add to the productionness of this function. async function generateSecretKey(additional) { // Get current timestamp const now = new Date(); const now_timestamp = now.getTime(); return await hashSecretKey(now_timestamp, additional); } async function hashSecretKey(secret_key, additional) { if (!additional) { additional = ""; } // Create a hexadecimal array from it const hexdec_string = Number(secret_key).toString(16) + additional; const hexdec_array = Int16Array.from(hexdec_string); // Obfuscate the data with a hashing algorithm const byteHash = Array.from( new Uint8Array(await crypto.subtle.digest("SHA-256", hexdec_array)), ); const charHash = byteHash.map((b) => b.toString(16).padStart(2, "0")).join( "", ); return charHash; } ``` And the function to guess the hash: ```js // Guess the hash with a given estimated time when the hash was created async function guessHash(target_hash, time_created) { const start_time = (new Date()).getTime(); // Guess milliseconds between 2.5 min before up to 3.0 min after the estimated // creation time const start = time_created - 2500; const end = time_created + 3000; for (time = start; time <= end; ++time) { const hash = await hashSecretKey(time); // Imagine a http request here let found_hash = hash === target_hash; if (found_hash) { const end_time = (new Date()).getTime(); return { duration: end_time - start_time, requests: time - start + 1, }; } } return false; } ``` Pseudo random generators are weak against attacks guessing the seed or the original data. In this example it just particulary easy to guess the _API-Key_, because the time - which is kinda known - is used. ## Interactive example This example generates a hash and records the time with ± 1 Minutes, when it was generated. --- # Sway bar with Cmus and PulseAudio > Create a custom status bar command for sway/i3 *Published on 2023-04-16 · en · https://codefionn.eu/sway-bar-with-cmus-and-pulseaudio/* The following shows script to create a simple, text-based bar for sway. It shows the current track in cmus, the current kernel version, network default gateway, keyboard layout, sound volume, mounts, CPU utilisation and time. This works with pipewire, too (assuming you have the pipewire pulseaudio compatibility layer installed). ```bash #!/usr/bin/env bash export DELIM='|' CURRENT_SINK=$(pactl list short sinks \ | grep RUNNING \ | sed -e 's,^\([0-9][0-9]*\)[^0-9].*,\1,') CURRENT_NOW=$(pactl list sinks \ | grep '^[[:space:]]Volume:' \ | head -n $(( $CURRENT_SINK + 1 )) \ | tail -n 1 \ | sed -e 's,.* \([0-9][0-9]*\)%.*,\1,') function cmus_metadata_title() { local CURRENT_CMUS="$(cmus-remote -Q)" local TITLE="$(echo "$CURRENT_CMUS" | grep "^tag title " | cut -d ' ' -f 3-)" local ARTIST="$(echo "$CURRENT_CMUS" | grep "^tag artist " | cut -d ' ' -f 3-)" if [[ -z "$ARTIST" ]] ; then echo $TITLE elif [[ -z "$TITLE" ]] ; then echo $ARTIST else echo $ARTIST - $TITLE fi } CMUS_STATUS="$(cmus_metadata_title)" function not_empty_delim() { if ! [[ -z "$1" ]] ; then echo $DELIM fi } echo \ $CMUS_STATUS $(not_empty_delim "$CMUS_STATUS") \ Version: $(uname -r) $DELIM \ Gateway: $(ip route | grep ^default | cut -d ' ' -f 3-3) $DELIM \ Layout: $(swaymsg -t get_inputs \ | jq -r '[.[]|select(.type == "keyboard")][0] \ | .xkb_active_layout_name') $DELIM \ Volume: $CURRENT_NOW $DELIM \ $(lsblk -o MOUNTPOINT | grep -v '^$' | tail -n +2) $DELIM \ $(cat /proc/loadavg | cut -d ' ' -f 3-3) $DELIM \ $(date +'%Y-%m-%d %k:%M:%S') \ ``` And change the bar setting in `~/.config/sway/config` to ```bash bar { position top # When the status_command prints a new line to stdout, swaybar updates. # The default just shows the current date and time. status_command while true ; ~/.config/sway/bar.sh; do sleep 1; done #status_command waybar -l off #status_command swaybar font Noto Sans Mono 8 tray_output * icon_theme Adwaita colors { statusline #ffffff background #323232 inactive_workspace #32323200 #32323200 #5c5c5c } } ``` --- # Secure boot with NixOS and systemd > Creating secure boot keys and setup automatic kernel and bootloader signing in NixOS *Published on 2023-04-16 · en · https://codefionn.eu/secure-boot-with-nixos-and-systemd/* Note: This tutorial may not work on laptops, because their BIOS and secure boot implementation tends to be trash. Save the following in a script in `/etc/nixos/secureboot/setup.sh` ```bash #!/usr/bin/env bash cd "$(dirname "$0")" # From: https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface/Secure_Boot#Using_your_own_keys uuidgen --random > GUID.txt # Setup platform key openssl req -newkey rsa:4096 -nodes -keyout PK.key -new -x509 -sha256 -days 90000 -subj "/CN=codefionn Platform Key/" -out PK.crt openssl x509 -outform DER -in PK.crt -out PK.cer cert-to-efi-sig-list -g "$(< GUID.txt)" PK.crt PK.esl sign-efi-sig-list -g "$(< GUID.txt)" -k PK.key -c PK.crt PK PK.esl PK.auth # Allow removing platform key in user mode sign-efi-sig-list -g "$(< GUID.txt)" -c PK.crt -k PK.key PK /dev/null rm_PK.auth # Key exchange key openssl req -newkey rsa:4096 -nodes -keyout KEK.key -new -x509 -sha256 -days 90000 -subj "/CN=codefionn Key Exchange Key/" -out KEK.crt openssl x509 -outform DER -in KEK.crt -out KEK.cer cert-to-efi-sig-list -g "$(< GUID.txt)" KEK.crt KEK.esl sign-efi-sig-list -g "$(< GUID.txt)" -k PK.key -c PK.crt KEK KEK.esl KEK.auth # Signature Database key openssl req -newkey rsa:4096 -nodes -keyout db.key -new -x509 -sha256 -days 90000 -subj "/CN=codefionn Signature Database key/" -out db.crt openssl x509 -outform DER -in db.crt -out db.cer cert-to-efi-sig-list -g "$(< GUID.txt)" db.crt db.esl sign-efi-sig-list -g "$(< GUID.txt)" -k KEK.key -c KEK.crt db db.esl db.auth ``` Make the script executable with ```bash chmod 700 /etc/nixos/secureboot/setup.sh ``` and execute it with ```bash /etc/nixos/secureboot/setup.sh ``` This will created the necessary keys to sign the boot loader and Linux kernel for secure boot. In your NixOS configuration in `/etc/nixos/configuration.nix` add the following ``` # Sign all bootloaders boot.loader.systemd-boot.extraInstallCommands = '' export PATH="$PATH:${pkgs.sbsigntool}/bin" for i in /boot/efi/EFI/*/*.efi /boot/efi/EFI/*/*.EFI ; do sbsign --key "/etc/nixos/secureboot/db.key" --cert "/etc/nixos/secureboot/db.crt" "$i" --output "$i" done ''; ``` Execute ``` nixos-rebuild switch ``` to apply the changes. Notice that the `.efi` files are now signed. This will sign all bootloaders every time you update your NixOS installation. Please replace `/boot/efi/EFI` with your boot path (e.g. just `/boot/EFI`). Save the `db.esl` and `db.crt` (in the `/etc/nixos/secureboot` directory) onto a FAT filesystem (a USB-stick or your `/boot/EFI` directory). In your BIOS setup (typically launched with the `DEL` or `F1` key at boot) enable secure boot and append the authorized key `db.esl` or `db.crt` (maybe one of them will not be accepted by your BIOS setup). Save the changes and restart. --- # Proxmox Networking with NAT > Proxmox Networking with NAT (IP-Address translation) *Published on 2023-04-16 · en · https://codefionn.eu/proxmox-networking-with-nat/* Normally containers and virtual machines use the DHCP server of your router to get their IP address. If that's not possible in your network you might want to use NAT for creating a sub-network. This tutorial assumes your PROXMOX machine has the IP-address 192.168.178.10 and gateway 192.168.178.1, you can change it to whatever your current setup has (inspect with ip addr and ip route with the command line). This is basically Debian networking. Connect to your PROXMOX machine via shell/ssh. ## Configure network interfaces Edit /etc/network/interfaces with: ``` nano /etc/network/interfaces ``` First use either static or dhcp for determining the IP-address of your PROXMOX machine, also assign vmbr0 (the virtual network) an IP-Address range, here 10.10.4.1/24: ``` auto enp34s0 iface enp34s0 inet static address 192.168.178.10/24 gateway 192.168.178.1 auto vmbr0 iface vmbr0 inet static address 10.10.4.1/24 bridge-ports enp34s0 bridge-stp off bridge-fd 0 ``` Then disable the bridge-ports option and use IP-forwarding to transfer packets between enp34s0 and vmbr0: ``` auto vmbr0 iface vmbr0 inet static address 10.10.4.1/24 bridge-ports none bridge-stp off bridge-fd 0 post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s '10.10.4.0/24' -o enp34s0 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s '10.10.4.0/24' -o enp34s0 -j MASQUERADE ``` ## Configure DHCP-server Next, you have to create an DHCP-server for the vmbr0 interface. Type the following in the command line: ```bash apt install dnsmasq nano /etc/dnsmasq.conf ``` Make sure to set the following in your this file: ``` # Adds the proxmox as a domain address=/proxmox/192.168.178.10 # Hosts dnsmasq on vmbr0 interface=vmbr0 # The IP-adress range that should be used for the clients (virtual machines/containers): dhcp-range=10.10.4.50,10.10.4.150,12h # Just making sure dnsmasq knows the routers IP-Address dhcp-option=3,10.10.4.1 ``` ## Static IPs for certain MAC-addresses: You can also instruct dnsmasq to assign static IP-address for certain MAC addresses: ```bash nano /etc/dnsmasq.d/static-ips.conf ``` In this file you could write the following: ``` dhcp-host=B0:1A:38:B1:1B:D8,gitlab,10.10.4.10 ``` This is useful for making sure that a container has always the same IP-address. Reboot your machine with: ```bash reboot ``` and you should be good to go. ## Port-forwarding If you want to forward a port from your virtual network to your machine's IP-address, use iptables (here port 8080 from virtual is forwarded to the enp34s0 interface's port 8007): ``` auto vmbr0 iface vmbr0 inet static address 10.10.4.1/24 bridge-ports none bridge-stp off bridge-fd 0 post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s '10.10.4.0/24' -o enp34s0 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s '10.10.4.0/24' -o enp34s0 -j MASQUERADE post-up echo 1 > /proc/sys/net/ipv4/conf/enp34s0/forwarding post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/forwarding post-up iptables -t nat -A PREROUTING -p tcp -i enp34s0 --dport 8007 -j DNAT --to-destination 10.10.4.11:8080 post-up iptables -t nat -A FORWARD -p tcp -d 10.10.4.11 --dport 8080 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT post-down iptables -t nat -D PREROUTING -p tcp -i enp34s0 --dport 8007 -j DNAT --to-destination 10.10.4.11:8080 post-down iptables -t nat -D FORWARD -p tcp -d 10.10.4.11 --dport 8080 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT ``` That's about it. --- # PHP and MySQL Server with docker-compose > Create an apache server with php and mysql with docker-compose.yml *Published on 2023-04-16 · en · https://codefionn.eu/php-mysql-server-with-docker-compose/* This post shows how to create a `docker-compose.yml` file, which hosts an apache server (with PHP) and MySQL database and connect them correctly together. The directory of your project should look like this: ``` project-directory | +--docker-compose.yml +-+php | +-Dockerfile +-index.php ``` The `docker-compose.yml`: ```yaml version: '3.8' services: php-apache-environment: container_name: php-apache build: context: ./php dockerfile: Dockerfile depends_on: - db volumes: - ./php:/var/www/html/ - mysql-sock-volume:/var/run/mysqld ports: - 8000:80 db: container_name: db image: mysql restart: always environment: MYSQL_ROOT_PASSWORD: password MYSQL_DATABASE: exampledb MYSQL_USER: exampleuser MYSQL_PASSWORD: exampleuser volumes: - mysql-sock-volume:/var/run/mysqld volumes: mysql-sock-volume: ``` The `Dockerfile` in the _php_ directory: ``` FROM php:8.1-apache RUN docker-php-ext-install mysqli pdo pdo_mysql ``` Connect to the database in the `index.php`: ```php query('SELECT now()')->fetchAll()); ``` The webserver will be served on `http://localhost:8000`. --- # NixOS router with Intel J4125 and i225 > Using NixOS as an OS for an router with IPv4 and IPv6 *Published on 2023-04-16 · en · https://codefionn.eu/nixos-router-with-j4125-and-intel-i225/* I bought one of those cheap firewall-computers with an [Intel Celeron J4125 and i225 2.5G LAN chipset](https://youtu.be/IJhlqb4iGn4). Then used NixOS to operate it.
Front of the black fanless router with four Ethernet ports, ETH0 to ETH3, all plugged in, and a 12 V power jack
J4125 Router with i225 networking
J4125 router power consumption is about 9 W/h
J4125 Router power consumption
NixOS logo
NixOS
_enp1s0_ will be the port that is connected to a switch/home-router. _enp2s0_, _eno1_ and _enp4s0_ are the ports for other devices, which connect to this router. This configuration uses _kea_ as a DHCP-Server and the DNS-Server _unbound_. The router itself does encrypted DNS requests with [DoT](https://en.wikipedia.org/wiki/DNS_over_TLS). The configuration supports both IPv4 and IPv6. I have a basic settings file `/etc/nixos/settings.nix`: ```nix rec { allowedSSHKeys = [ "ssh-rsa XXXXXXXXXXX user@nixos" "ssh-rsa YYYYYYYYYYY user@nixos" ]; # Rename them so they're easier to handle eth1 = "enp1s0"; eth2 = "enp2s0"; eth3 = "eno1"; eth4 = "enp4s0"; wanInterface = "${eth1}"; lanInterfaces = [ "${eth2}" "${eth3}" "${eth4}" ]; # What IPs to use baseInternalIp = "192.168.9"; internalIp = "${baseInternalIp}.1"; baseInternalIp6 = "fd03:1234:fde0:0000"; internalIp6 = "${baseInternalIp6}::1"; internalDomain = "lan"; # Devices with reserved IPs and domains mac-nixos = "70:85:c2:93:c0:cb"; mac-raspberrypi = "e4:5f:01:98:9d:7b"; ip-nixos = "${baseInternalIp}.10"; ip6-nixos = "${baseInternalIp6}::0010"; ip-raspberrypi = "${baseInternalIp}.11"; ip6-raspberrypi = "${baseInternalIp6}::0011"; hostName = "fionn-router"; # All reserved domains an their IPs domain-and-ips = builtins.map (x: { domain = "${x.domain}.${internalDomain}"; ip = "${x.ip}"; }) [ { domain = "earth"; ip = "${ip-nixos}"; } { domain = "nixos"; ip = "${ip-nixos}"; } { domain = "raspberrypi"; ip = "${ip-raspberrypi}"; } { domain = "fionn-router"; ip = "${internalIp}"; } ]; } ``` This defines all basics we need for the router (so we don't always have to go into `/etc/nixos/configuration.nix` to tweak something). Then there's the `/etc/nixos/configuration.nix`: ```nix # Edit this configuration file to define what should be installed on # your system. Help is available in the configuration.nix(5) man page # and in the NixOS manual (accessible by running ‘nixos-help’). { config, pkgs, ... }: let settings = import ./settings.nix; in { imports = [ # Include the results of the hardware scan. ./hardware-configuration.nix ]; nix.settings.auto-optimise-store = true; # Bootloader. boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; boot.loader.efi.efiSysMountPoint = "/boot/efi"; # Reduce writes onto disk services.journald.extraConfig = '' Storage=volatile ''; networking.hostName = "${settings.hostName}"; # Define your hostname. # Allow NAT stuff ... boot.kernel.sysctl = { "net.ipv4.conf.all.forwarding" = true; "net.ipv6.conf.all.forwarding" = true; # source: https://github.com/mdlayher/homelab/blob/master/nixos/routnerr-2/configuration.nix#L52 # By default, not automatically configure any IPv6 addresses. "net.ipv6.conf.all.accept_ra" = 0; "net.ipv6.conf.all.autoconf" = 0; "net.ipv6.conf.all.use_tempaddr" = 0; # On WAN, allow IPv6 autoconfiguration and tempory address use. "net.ipv6.conf.${settings.wanInterface}.accept_ra" = 2; "net.ipv6.conf.${settings.wanInterface}.autoconf" = 1; }; # Configure network proxy if necessary # networking.proxy.default = "http://user:password@proxy:port/"; # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain"; # Set your time zone. time.timeZone = "Europe/Berlin"; # Select internationalisation properties. i18n.defaultLocale = "en_US.utf8"; # Allow unfree packages nixpkgs.config.allowUnfree = true; environment.variables = { EDITOR = "vim"; }; # List packages installed in system profile. To search, run: # $ nix search wget // Comment: Doesn't work (nix search) environment.systemPackages = with pkgs; [ wget traceroute dig iftop ethtool btop pciutils usbutils fd wakelan htop tmux ] ++ (with pkgs.vimPlugins; [ vim-nix ]); # Maybe use nano instead? programs.vim = { defaultEditor = true; package = ((pkgs.vim_configurable.override {}).customize { name = "vim"; vimrcConfig.packages.myplugins = with pkgs.vimPlugins; { start = [ vim-nix ]; opt = []; }; vimrcConfig.customRC = '' set nocompatible set expandtab set ts=2 set sw=2 syntax on ''; }); }; users.mutableUsers = false; users.users.root = { openssh.authorizedKeys.keys = settings.allowedSSHKeys; hashedPassword = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"; }; # Define a user account users.users.user = { openssh.authorizedKeys.keys = settings.allowedSSHKeys; isNormalUser = true; description = "Fionn Langhans"; extraGroups = [ "wheel" ]; hashedPassword = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"; packages = with pkgs; []; }; # List services that you want to enable: # Enable the OpenSSH daemon. services.openssh = { enable = true; passwordAuthentication = false; }; services.kea.dhcp4 = { enable = true; settings = { interfaces-config = { interfaces = [ "br0" ]; }; lease-database = { name = "/var/lib/kea/dhcp4.leases"; persist = true; type = "memfile"; }; option-data = [ { name = "domain-name-servers"; data = settings.internalIp; always-send = true; } { name = "routers"; data = settings.internalIp; } { name = "domain-name"; data = "${config.networking.hostName}.${settings.internalDomain}"; } ]; rebind-timer = 2000; renew-timer = 1000; valid-lifetime = 4000; subnet4 = [ { pools = [ { pool = "${settings.baseInternalIp}.100 - ${settings.baseInternalIp}.230"; } ]; subnet = "${settings.baseInternalIp}.0/24"; reservations = [ { hw-address = settings.mac-nixos; ip-address = settings.ip-nixos; } { hw-address = settings.mac-raspberrypi; ip-address = settings.ip-raspberrypi; } ]; } ]; }; }; services.kea.dhcp6 = { enable = true; settings = { interfaces-config = { interfaces = [ "br0" ]; }; lease-database = { name = "/var/lib/kea/dhcp6.leases"; persist = true; type = "memfile"; }; option-data = [ { name = "dns-servers"; data = settings.internalIp6; } { name = "unicast"; data = settings.internalIp6; } ]; preferred-lifetime = 3000; rebind-timer = 2000; renew-timer = 1000; valid-lifetime = 4000; subnet6 = [ { pools = [ { pool = "${settings.baseInternalIp6}::1000-${settings.baseInternalIp6}::EFFF"; } ]; subnet = "${settings.baseInternalIp6}::0/64"; reservations = [ { hw-address = settings.mac-nixos; ip-addresses = [ settings.ip6-nixos ]; } { hw-address = settings.mac-raspberrypi; ip-addresses = [ settings.ip6-raspberrypi ]; } ]; } ]; }; }; services.radvd = { enable = true; config = '' interface br0 { AdvSendAdvert on; prefix ${settings.baseInternalIp6}::/64 { AdvOnLink on; AdvAutonomous on; AdvRouterAddr on; }; route ${settings.internalIp6}/128 { }; RDNSS ${settings.internalIp6} { }; }; ''; }; # This is not really secure, but some games need it. services.miniupnpd = { enable = true; externalInterface = "${settings.wanInterface}"; internalIPs = [ "br0" ]; }; # DNS-Server services.unbound = { enable = true; settings = { server = { interface = [ "127.0.0.1" "${settings.internalIp}" "${settings.internalIp6}" ]; tls-system-cert = true; access-control = [ "0.0.0.0/0 refuse" "127.0.0.0/8 allow" "${settings.baseInternalIp}.0/24 allow" "${settings.baseInternalIp6}::0/64 allow" ]; prefer-ip6 = true; private-domain = [ "local" "${settings.internalDomain}" ]; private-address = [ "${settings.baseInternalIp}.0/24" "${settings.baseInternalIp6}::0/64" ]; unblock-lan-zones = true; insecure-lan-zones = true; local-zone = builtins.map (x: "\"${x.domain}.\" static") settings.domain-and-ips; local-data = builtins.concatLists (builtins.map (domain-and-ip: [ "\"${domain-and-ip.domain}. 3600 IN A ${domain-and-ip.ip}\"" ]) settings.domain-and-ips); local-data-ptr = builtins.map (x: "\"${x.ip} ${x.domain}\"") settings.domain-and-ips; }; forward-zone = [ { name = "."; forward-tls-upstream = true; forward-addr = [ "2620:fe::fe@853#quad9.net" "2606:4700:4700::1111@853#cloudflare-dns.com" "2001:4860:4860::8888@853#dns.google" "9.9.9.9@853#quad9.net" "1.1.1.1@853#cloudflare-dns.com" "8.8.8.8@853:dns.google" ]; } { name = "onion."; } ]; remote-control.control-enable = false; }; }; # Tailscale, because it's convenient services.tailscale = { enable = true; }; services.avahi = { enable = true; nssmdns = true; interfaces = [ "br0" "tailscale0" ]; }; services.syncthing = { enable = true; user = "user"; group = "users"; guiAddress = "${settings.internalIp}:8384"; dataDir = "/home/user"; devices = { codefionn = { addresses = []; id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"; }; raspberrypi = { addresses = []; id = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"; }; }; folders.default = { id = "default"; label = "Sync"; path = "/home/user/Sync"; devices = [ "codefionn" "raspberrypi" ]; }; }; # If you reboot the router often, this his handy systemd.services.startup-tune = { enable = true; path = with pkgs; [ beep kmod ]; preStart = "modprobe pcspkr"; script = "beep -f 1000 -l 50 -r 3 -d 1000 -n -d 100 -n -f 500 -d 1000 -l 50 -r 1"; wants = [ "network-online.target" ]; after = [ "network-online.target" ]; wantedBy = [ "multi-user.target" ]; }; # Open ports in the firewall. # networking.firewall.allowedTCPPorts = [ ... ]; # networking.firewall.allowedUDPPorts = [ ... ]; networking.firewall.checkReversePath = "loose"; networking.firewall.interfaces.br0 = { allowedTCPPorts = [ 53 22000 8384 ]; allowedTCPPortRanges = [ { from = 1714; to = 1764; } ]; allowedUDPPorts = [ 53 22000 21027 ]; allowedUDPPortRanges = [ { from = 1714; to = 1764; } ]; }; #---------- # Enables DHCP on each ethernet and wireless interface. In case of scripted networking # (the default) this is the recommended approach. When using systemd-networkd it's # still possible to use this option, but it's recommended to use it in conjunction # with explicit per-interface declarations with `networking.interfaces..useDHCP`. networking = { useDHCP = false; hosts = { "127.0.0.1" = [ "localhost" "${config.networking.hostName}" "${config.networking.hostName}.local" ]; }; interfaces = { "${settings.wanInterface}" = { useDHCP = true; # ... Or maybe define it statically }; br0 = { ipv4.addresses = [ { address = "${settings.internalIp}"; prefixLength = 24; } ]; ipv6.addresses = [ { address = "${settings.internalIp6}"; prefixLength = 64; } ]; }; }; bridges.br0 = { interfaces = settings.lanInterfaces; }; nat = { enable = true; enableIPv6 = true; externalInterface = "${settings.wanInterface}"; internalInterfaces = [ "br0" ]; }; networkmanager.enable = false; }; # Or disable the firewall altogether. # networking.firewall.enable = false; # This value determines the NixOS release from which the default # settings for stateful data, like file locations and database versions # on your system were taken. It‘s perfectly fine and recommended to leave # this value at the release version of the first install of this system. # Before changing this value read the documentation for this option # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html). system.stateVersion = "22.05"; # Did you read the comment? } ``` Sadly, my Intel AX210 wireless M.2 chipset doesn't work well with both Linux and this router. --- # Encrypted mosquitto server > Setting up an encrypted mosquitto server with a self-signed certificate *Published on 2023-04-16 · en · https://codefionn.eu/encrypted-mosquitto-server/* Below, certificates and keys will be created for TLS. Then the misquitto configuration will be edited. Your server will run on `localhost:8883` afterwards not `localhost:1883`. ## Create directory for certificates ```bash mkdir /etc/mosquitto/certs cd /etc/mosquitto/certs ``` ## Create CA Key Using password protection and 4096-bit RSA. ```bash openssl genrsa -des3 -out m2mqtt_ca.key 4096 Create CA Certificate openssl req -new -x509 -days 9000 -key m2mqtt_ca.key -out m2mqtt_ca.crt ``` ## Create Server Key Using no password and 4096-bit RSA. ```bash openssl genrsa -out m2mqtt_srv.key 4096 ``` ## Create Server Sign Request ```bash openssl req -new -out m2mqtt_srv.csr -key m2mqtt_srv.key -subj '/CN=localhost' ``` ## Create domain configuration domain.ext This is important, so the domain and certificate can be validated. ```bash authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE subjectAltName = @alt_names [alt_names] DNS.1 = localhost IP.1 = 127.0.0.1 IP.2 = ::1 ``` With cool command line stuff this is: ```bash cat > domain.ext << EOF authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE subjectAltName = @alt_names [alt_names] DNS.1 = localhost IP.1 = 127.0.0.1 IP.2 = ::1 EOF ``` ## Create Server Certificate ```bash openssl x509 -req -CA m2mqtt_ca.crt -CAkey m2mqtt_ca.key \ -in m2mqtt_srv.csr -out m2mqtt_srv.crt \ -days 9000 -CAcreateserial -extfile domain.ext ``` ## Edit /etc/mosquitto/mosquitto.conf Uncomment line with setting listener and add the following at the end of the line: ```bash 8883 ``` Uncomment line with setting cafile and add the following at the end of the line: ```bash /etc/mosquitto/certs/m2mqtt_ca.crt ``` Uncomment line with setting certfile and add the following at the end of the line: ```bash /etc/mosquitto/certs/m2mqtt_srv.crt ``` Uncomment line with setting keyfile and add the following at the end of the line: ```bash /etc/mosquitto/certs/m2mqtt_srv.key ``` These should be all changes you made just now: ```bash listener 8883 cafile /etc/mosquitto/certs/m2mqtt_ca.crt certfile /etc/mosquitto/certs/m2mqtt_srv.crt keyfile /etc/mosquitto/certs/m2mqtt_srv.key ``` Don't forget to add the CAfile m2mqtt\_ca.crt to your MQTT client! ## Restart the mosquitto server ```bash service mosquitto restart ``` --- # Nützliche Linux-Kommandozeilen-Tools > Eine unvollständige Liste nützlicher Linux-Kommandozeilen-Tools *Veröffentlicht am 2023-04-16 · de · https://codefionn.eu/neutzliche-linux-kommandozeilen-tools/* Dies ist eine unvollständige Liste einiger nützlicher Linux/\*BSD-Kommandozeilen-Tools:
Befehl Beschreibung
ncdu Größe von Verzeichnissen und Dateien auflisten
sed -i 's/alt/neu/' Suchen & Ersetzen mit Regex
telnet host port Klartext-Netzwerkverbindung testen
openssl s_client -connect host:port SSL/TLS-Verbindung testen
vim/emacs Gute Texteditoren
fd Nützlicher Rust-Erstz für find
rg Rust-Erstz für grep
bat Rust-Erstz für cat mit Syntax-Highlighting
netstat -tulpn Netzwerk-Ports anzeigen