Hypha: navigating Haskell source code for humans and machines

TL;DR I’m releasing a beta version of Hypha, an experimental CLI tool for Haskell development which can be useful to explore your project dependencies, read documentation offline without the need of generating Haddock files, and use LLM harnesses and tools without hammering Hackage or Hoogle. Think of Hypha as an offline-first, enhanced version of your Cabal store.

Albeit developed making extensive use of generative AI, Hypha itself can be used without any LLM: the data is yours, everything it caches is held locally, and it does not make calls to external AI services. Hypha can be used offline, and despite being designed for use together with an AI harness, it doesn’t require one.

In this blog post I’ll discuss the rationale behind this tool, what it does and how it does it.

An important disclaimer: Hypha was born as an experiment in AI-assisted development and as such it has been developed via heavy usage of LLMs, using a mix of proprietary state-of-the-art models but also open weights ones, to assess capabilities, strengths and weaknesses of each of them. In a sense, I have been mostly wearing the “Architect Hat” for Hypha (see my talk from the Haskell Foundation AI workshop). Hypha is currently in an early-beta stage, and while I have happily been using it for a few months now, it’s probably not free from bugs or inconsistencies. I’d be delighted to receive feedback and/or bug reports on the Hypha repo.

This post is deliberately written using a personal style as it reflects my own choices, both in style and form. For the readers who might desire to know, this post is 100% human-written!

The problem(s)

LLM interactions are mostly stateless and not cache friendly

Generative AI is a particularly divisive topic these days, with some companies and programming language communities embracing it, while others reject everything that an LLM has ever produced. Independently of which side you might be, the advent of AI harnesses like Claude Code, OpenCode or Pi created a lot of stress for websites, including Hackage.

Every time somebody asks an LLM “What’s the signature of mapConcurrentlyBounded?”¹ or “Can you find me a generalised version of this function?” there are two possibilities: either this information is already cached in the AI harness context, and can be served straight away (saving time and tokens) or it’s not. If there is a cache miss, the AI may:

• grep through the codebase for existing uses of that function, try to reconstruct the type signature from usage and return a first approximation of an answer;
• otherwise, try to find the exact source code for the requested function.

We might be in luck, with the source code already resident on our computer; that would be the case at least in a few scenarios:

• The package has been specified in your cabal.project as a source-repository-package (and thus Cabal checked out the relevant Git repo somewhere in the project’s dist-newstyle);
• The package has been downloaded from Hackage before, and it’s available compressed as a tarball inside Cabal’s cache;
• If you are a Nix user, the package would be somewhere in your Nix store.

In the scenarios above, the LLM might provide you with the right answer, but it’s a fairly non-deterministic process, because it varies according to the quality of the LLM one is using; weaker models might ignore the fact you want the signature for a specific version of a specific package, and just blindly give you any signature matching any version of the package. Similarly, if you are using an AI harness from within a sandbox (like you should!) the harness might not be able to find the cached information at all if the sandbox is blocking the cache path.

In unlucky cases, if we try to ask an LLM for the source code or the precise signature of mapConcurrentlyBounded, the AI could end up downloading the tarball of the async package from Hackage, unpacking it locally in a temporary directory, grepping it, finding the definition and returning it to the user. This is a lot of wasted effort, because if we ask the same question again in a separate session, chances are the temporary file has been pruned or simply the LLM has no memory of the past interaction, so the tarball ends up being downloaded afresh, hitting Hackage again.

The problem is made worse by the fact that sometimes (and I have been guilty of this in the past, as well) it’s so damn tempting to just point the AI harness to Hackage and then ask generic questions, and that creates unnecessary stress on a critical piece of infrastructure for the Haskell community.

Which version of async, anyway?

Another problem that we face when using LLMs to ask questions about our source code and its dependencies is that it might not be clear from the context which version of async we mean: if we paste in a prompt the Hackage link to async , we are implicitly assuming that we mean to explore the most recent version, which might not be true in the constraints of our project: maybe we are forced to use a specific subset of async versions which didn’t even have mapConcurrentlyBounded available!

What about token consumption?

As you can imagine, doing this kind of “stateless” interaction is not just wasteful in terms of HTTP traffic, but also in terms of token consumption; the LLM has to download the tarball, unpack it, understand its structure (at the very least grep for the requested information, and then read chunks of the source file to extract what’s needed), and there is very little room for caching this information given everything is done in an ad-hoc fashion.

What’s worse is that all this precious knowledge is sometimes lost the minute we wrap up the session (or our temporary directory is pruned): once we have done the hard work we cannot get more out of it (for example exploring reverse dependencies for a package, or figuring out when a function got removed between versions, or how a function is implemented internally and why).

What if a package is not on Hackage?

As said above, surprisingly that’s the easiest case, as the information would be stored in the dist-newstyle directory. But this relies on the LLM figuring out that the requested package is available as a source-repository-package in our cabal.project, which might not be immediately obvious and it might result in a few wasted calls to Hackage (bad) or web searches to figure things out. Modern LLMs are getting quite good at avoiding this, but not everybody is running expensive state-of-the-art models.

Given all the above, what do we do, then?

A quick tour of Hypha

Finally, we are ready to take a quick tour of Hypha; hopefully at the end of this tour you will have enough information to let you decide if you want to use it or not (and I hope you will!). If you’d like to try it yourself, see the installation instructions on the docs site and run cabal build all --dry-run to generate the Cabal build plan for your project.

Let’s start with a concrete example. Suppose we are working on a project, and we have a type like this:

data LogEntry
  = LogEntry
  { _le_payload   :: String
  , _le_timestamp :: UTCTime
  } deriving (Show, Eq, Ord)

We know our basic Haskell and we know sort exists, but here we want something a bit different: we don’t want to sort based on the Ord instance, but rather we would like to sort log entries on their timestamp, so ideally we would like to have a variant of sort which accepts a custom function to sort over a field.

We vaguely remember such function exists somewhere in base, but we don’t remember its name. Of course we could go on Hackage, find the definition of sort and look for similarly-named functions within that module. But that would make the tutorial much more boring! :-) Jokes aside, this is not about what you, a human, would do, it’s about creating the most efficient and easy-to-use path for an AI harness.

hypha lookup

What we can do instead is to ask hypha about the existence of such function:

> hypha lookup 'Ord b => (a -> b) -> [a] -> [a]'

actions:
  base/Data.List/sortOn: hypha symbol base/Data.List/sortOn
  base/GHC.Exts/sortWith: hypha symbol base/GHC.Exts/sortWith
  base/GHC.OldList/sortOn: hypha symbol base/GHC.OldList/sortOn
  containers/Data.Containers.ListUtils/nubOrdOn: hypha symbol containers/Data.Containers.ListUtils/nubOrdOn
result:
  providers:
    - mod: Data.List
      name: sortOn
      pkg: base
      sig: "Ord b => (a -> b) -> [a] -> [a]"
      tier: local-hoogle
    - mod: GHC.Exts
      name: sortWith
      pkg: base
      sig: "Ord b => (a -> b) -> [a] -> [a]"
      tier: local-hoogle
    - mod: GHC.OldList
      name: sortOn
      pkg: base
      sig: "Ord b => (a -> b) -> [a] -> [a]"
      tier: local-hoogle
    - mod: Data.Containers.ListUtils
      name: nubOrdOn
      pkg: containers
      sig: "Ord b => (a -> b) -> [a] -> [a]"
      tier: local-hoogle
  query: "Ord b => (a -> b) -> [a] -> [a]"

This is lookup’s main purpose: giving you a concrete entry point for discovering functions you might have already available transitively in your build plan but you don’t know where they are or how they are called. Internally, lookup queries Hoogle (offline first, remote once only on a cache miss), and the result of lookup is a list of candidates all satisfying the requested type signature.

What makes lookup useful is that it is not limited to just locally-cached packages in your project. This means you or an LLM can use lookup as the initial gateway to access relevant information, saving the time of looking things up in two different data sources, while protecting the integrity of the infrastructure by trying to be as cache-friendly as possible.

hypha symbol

Lo and behold, we have now found that sortOn exists and it seems to be the function we are looking for. The next step would be to get a bit more information about it, which we can do via the symbol command:

> hypha symbol base/Data.List/sortOn

actions:
  module_index: hypha module base/Data.List
  package_info: hypha package base
  view_source: hypha source base/Data.List/sortOn
result:
  defined_in:
    component: base
    module: Data.OldList
  haddock_raw: " Sort a list by comparing the results of a key function applied to each\n element.  @sortOn f@ is equivalent to @sortBy (comparing f)@, but has the\n performance advantage of only evaluating @f@ once for each element in the\n input list.  This is called the decorate-sort-undecorate paradigm, or\n Schwartzian transform.\n\n Elements are arranged from lowest to highest, keeping duplicates in\n the order they appeared in the input.\n\n >>> sortOn fst [(2, \"world\"), (4, \"!\"), (1, \"Hello\")]\n [(1,\"Hello\"),(2,\"world\"),(4,\"!\")]\n\n @since 4.8.0.0"
  kind: function
  module: Data.List
  name: sortOn
  package: base
  signature: "sortOn :: Ord b => (a -> b) -> [a] -> [a]"
  version: '4.16.4.0'

As you can see, the output includes two major “blocks”:

• An actions block, which are like breadcrumbs to suggest to an LLM where to go next (as we will see later);
• The result block, which includes useful information like haddock_raw, which allows answering some easy questions immediately (think if the question was “what does sortOn do?”).

If desired, it’s possible to select only a subset of fields from the result block via the --select option:

> hypha symbol base/Data.List/sortOn --select "version, haddock_raw"

actions:
  module_index: hypha module base/Data.List
  package_info: hypha package base
  view_source: hypha source base/Data.List/sortOn
result:
  haddock_raw: " Sort a list by comparing the results of a key function applied to each\n element.  @'sortOn' f@ is equivalent to @'sortBy' ('comparing' f)@, but has the\n performance advantage of only evaluating @f@ once for each element in the\n input list.  This is called the decorate-sort-undecorate paradigm, or\n <https://en.wikipedia.org/wiki/Schwartzian_transform Schwartzian transform>.\n\n Elements are arranged from lowest to highest, keeping duplicates in\n the order they appeared in the input.\n\n The argument must be finite.\n\n ==== __Examples__\n\n >>> sortOn fst [(2, \"world\"), (4, \"!\"), (1, \"Hello\")]\n [(1,\"Hello\"),(2,\"world\"),(4,\"!\")]\n\n >>> sortOn length [\"jim\", \"creed\", \"pam\", \"michael\", \"dwight\", \"kevin\"]\n [\"jim\",\"pam\",\"creed\",\"kevin\",\"dwight\",\"michael\"]\n\n ==== __Performance notes__\n\n This function minimises the projections performed, by materialising\n the projections in an intermediate list.\n\n For trivial projections, you should prefer using 'sortBy' with\n 'comparing', for example:\n\n >>> sortBy (comparing fst) [(3, 1), (2, 2), (1, 3)]\n [(1,3),(2,2),(3,1)]\n\n Or, for the exact same API as 'sortOn', you can use `sortBy . comparing`:\n\n >>> (sortBy . comparing) fst [(3, 1), (2, 2), (1, 3)]\n [(1,3),(2,2),(3,1)]\n\n @since base-4.8.0.0"
  version: '4.20.2.0'

hypha source

Following the same “stream of consciousness” that an LLM would also follow, the next step (depending on the original query, of course) would be to take a look at the source code for sortOn, which, as you might guess, is available via the source command:

> hypha source base/Data.List/sortOn

result:
  line: 1819
  module: GHC.Internal.Data.OldList
  package: ghc-internal
  path: /home/alfredo/.cache/hypha/source/ghc-internal-9.1003.0/src/GHC/Internal/Data/OldList.hs
  snippet: |
    -- the projections in an intermediate list.
    --
    -- For trivial projections, you should prefer using 'sortBy' with
    -- 'comparing', for example:
    --
    -- >>> sortBy (comparing fst) [(3, 1), (2, 2), (1, 3)]
    -- [(1,3),(2,2),(3,1)]
    --
    -- Or, for the exact same API as 'sortOn', you can use `sortBy . comparing`:
    --
    -- >>> (sortBy . comparing) fst [(3, 1), (2, 2), (1, 3)]
    -- [(1,3),(2,2),(3,1)]
    --
    -- @since base-4.8.0.0
    sortOn :: Ord b => (a -> b) -> [a] -> [a]
    sortOn f =
      map snd . sortBy (comparing fst) . map (\x -> let y = f x in y `seq` (y, x))

    -- | Construct a list from a single element.
    --
    -- ==== __Examples__
    --
    -- >>> singleton True
    -- [True]
    --
    -- >>> singleton [1, 2, 3]
    --[[1,2,3]]
    --
    -- >>> singleton 'c'
    -- "c"
    --
  symbol: sortOn

From here we can just read the implementation, query other ancillary functions etc.

hypha module

More often than not, discovering one function makes us curious about its neighbours: if Data.List has sortOn, what else is in there that we have been re-implementing by hand? LLMs have a tendency to hand-roll a lot of things, so this is a nice arrow to have in our quiver. We can use the module command for that:

> hypha module base/Data.List

actions:
  "++": "hypha symbol base/Data.List/++"
  List: hypha symbol base/Data.List/List
  head: hypha symbol base/Data.List/head
  last: hypha symbol base/Data.List/last
  module_index: hypha module base/Data.List
  package_info: hypha package base
  tail: hypha symbol base/Data.List/tail
result:
  exports:
    - name: List
    - name: "++"
    - name: head
    - name: last
    - name: tail
    - name: init
    - name: uncons
    - name: unsnoc
    - name: singleton
    - name: null
    - name: length
    - name: map
    - name: reverse
    - name: intersperse
    - name: intercalate
    - name: transpose
    - name: subsequences
    - name: permutations
    - name: foldl
    - name: "foldl'"
    - name: foldl1
    # ...

That by default gives us the version of base and Data.List which is specific to our build plan, but we can also ask for a specific version using this alternative syntax:

hypha module base-4.10.0.0/Data.List

This is especially nice as it can be used by an LLM to easily diff the resulting YAML to spot missing functions, deprecations, removals, etc.

hypha package

In the same vein it’s possible to have a holistic view on a whole package using the package command, which has the effect you might imagine:

> hypha package thread-supervisor-0.2.0.0

actions:
  Control.Concurrent.Supervisor: hypha module thread-supervisor/Control.Concurrent.Supervisor
  Control.Concurrent.SupervisorInternal: hypha module thread-supervisor/Control.Concurrent.SupervisorInternal
  Data.DelayedQueue: hypha module thread-supervisor/Data.DelayedQueue
  module_index_hint: "hypha module thread-supervisor/<Module>"
  reverse_deps: hypha deps thread-supervisor --reverse
  version_history: hypha versions thread-supervisor
result:
  exposed_modules:
    - Control.Concurrent.Supervisor
    - Control.Concurrent.SupervisorInternal
    - Data.DelayedQueue
  is_local: false
  name: thread-supervisor
  origin:
    kind: hackage
  version: '0.2.0.0'

Everything we have seen so far is also available as a Claude plugin which can be installed to use hypha directly inside a Claude Code session. For example, we might want to ask:

/hypha:hypha-lookup mapConcurrentlyBounded

And this is what you would see:

● Bash(cabal run -v0 hypha -- lookup mapConcurrentlyBounded 2>&1 | head -60)
  ⎿  actions:
       async/Control.Concurrent.Stream/mapConcurrentlyBounded: hypha symbol async/Control.Concurrent.Stream/mapConcurrentlyBounded
     result:
       providers:
         - mod: Control.Concurrent.Stream
           name: mapConcurrentlyBounded
           pkg: async
           sig: "mapConcurrentlyBounded :: Int -- ^ Maximum concurrency -> (a -> IO b) -- ^ Function to map over the input values -> [a] -- ^ List of input values -> IO [b] -- ^ List of output values"
           tier: cache
       query: mapConcurrentlyBounded

● mapConcurrentlyBounded found, tier cache:

  mapConcurrentlyBounded
    :: Int              -- ^ Maximum concurrency
    -> (a -> IO b)      -- ^ Function to map over the input values
    -> [a]              -- ^ List of input values
    -> IO [b]           -- ^ List of output values

  Package async, module Control.Concurrent.Stream. No haddock body beyond arg comments.

Serving interactive documentation

One of the side goals of the project was to fix an old pet peeve of mine, by providing an easily-accessible, offline-first UI to browse and navigate Haskell documentation, at least of those packages I use daily. Back when I was on Mac OS for development, I used Dash, which supported GHC Haskell (essentially, base) documentation alongside the ability to download so-called “docsets” for Haskell packages.

Unfortunately I couldn’t find an equally-satisfying alternative in Linux land; the closest I’m aware of is Zeal but, at the time of writing this blog post, it didn’t support documentation from Hackage. Even when I was using Dash, though, I wasn’t completely satisfied with it either, mostly because if you decided to download a docset for a package, the program would silently download revisions for each new version out there, making it difficult to do precise querying.

hypha server

For all these reasons, given the fact the hard work was already done for the AIs, I have decided to also bolt on a server which can be run locally in order to browse the documentation. You start it with a single command:

hypha server

The homepage gives you an immediate sense of what Hypha knows about your project. The sidebar is split into two groups:

• Project, for the packages that make up your own codebase, and
• Dependencies, for everything else in the build plan.

Notice that the Project group is not limited to libraries: in the screenshot you can see that Hypha tracks every component of the project, including its executables (hypha:exe:hypha and hypha:exe:hypha-mcp), and the same goes for sub-libraries. This works because Hypha does not stop at plan.json: for every unit in the plan whose source directory is known, it also reads the corresponding .cabal stanza, so it knows exactly which components exist and which modules each one exposes.

The “docset” hypha server exposes is not any collection of packages, but it’s an exact snapshot, version-by-version of all your project’s dependencies. This eliminated for me the all-too-frequent pattern of checking in the build plan which version of a package I was using, taking note of its version, searching for the package on Hackage, selecting the required version and finally browsing through the documentation.

Browsing the documentation

Clicking on a module opens its documentation page. The header shows the usual metadata, the module description is rendered in full, and on the right hand side you get a panel listing the symbols the module exports, grouped by kind, a bit like the module pages on Hackage, or like Dash. Every entry in the panel is interactive: clicking it brings up a symbol card with the signature and its Haddock documentation, and from there you can jump to the source:

The top search bar allows you to do fuzzy queries for packages and symbols:

If the same package re-exports the same symbol in multiple places, that is tracked by a little +N on the bottom of the search result (in the screenshot that’s the case for mapAccumL, and the exports are explicitly reported):

Every package in the plan is tagged with its provenance, i.e. where it came from, represented in the sidebar by a small single-letter badge:

• H for Hackage,
• D for a GHC distribution/boot library,
• L for a local package,
• S for a source-repository-package.

For packages that are published on Hackage the page also offers a “view on Hackage” link, so opening the latter is typically one click away:

It’s also possible to restrict the search results in the bar to consider only the symbols within a given package simply by pressing the small “in <pkg>” button next to the search bar. This is useful when multiple packages export heavily-overloaded function names (e.g. lookup), and it will return in the dropdown menu only symbols for the given package:

The plan-aware approach also covers dependencies that do not come from Hackage at all. If your project pins a package via a source-repository-package stanza in its cabal.project, Hypha reads the plan.json and learns exactly which git repository and which commit your dependency is pinned to. It means that it’s possible to browse not-on-Hackage dependencies as well:

Last but not least, from a symbol you can jump straight to its definition. If you have already built the Haddock documentation for a package (for example via documentation: True in your Cabal config), Hypha will happily serve that. Otherwise, Hypha tries to cheat by rendering documentation from the source code on the fly, to not pay the full price for building a huge tree of Haddock documentation offline; in that case the rendering is not as pretty as Haddock, and the source view is not hyperlinked like on Hackage:

Alternative documentation browsers

A legitimate question is whether or not having a tool which implements interactive documentation browsing is a useful addition to the ecosystem. Why can’t we just build the full Haddock documentation for all our local packages and be done with it? Conversely, what are the advantages of using hypha server compared to solutions like Flora?

Building offline Haddock documentation

The most obvious alternative to the Hypha server would be to build the offline Haddock documentation, for example by setting documentation: True in your Cabal config. That would certainly work and it’s a defensible approach; what you gain by using Hypha is that if you already have the offline documentation, the Hypha server would happily serve it, and on top of that you also get features like holistic search across all your transitive dependencies, whereas the typical offline-Haddock approach would be to open the documentation of your root package and then drill down to the individual dependencies by following the cross-package links in the type signatures. Even cabal haddock-project, which gives you a nice landing page, only indexes your local packages (tools like cabal-hoogle can get you a project-wide search, but at the price of a full documentation build).

Of course, if you haven’t built (or don’t want to build) the offline documentation then the Hypha server experience is somewhat less polished than the one you would get by using just Haddock; due to the fact that Hypha only parses the package sources but skips the rest of the compilation pipeline, TH splices won’t get evaluated and thus they are invisible to Hypha, and the source view is not hyperlinked like Haddock’s – if that means a lot to you, that is certainly an important difference to be aware of.

Flora

Flora is a great project that gives a powerful query system to search for a specific package; however, to the best of my knowledge Flora piggybacks on Hackage itself for browsing the documentation. Visiting a page like the one for containers shows valuable information like dependencies, dependents, the versions etc, but clicking on “Documentation” redirects to Hackage.

In a nutshell, I think every approach is valuable and defensible, and there is no silver bullet: hypha server shines if all we want is to get started quickly by having holistic search of all our project’s dependencies at the right versions and without having to compile anything, at the cost of the quality of documentation in some cases. Using Haddock offline is what gives you the best accuracy at the cost of a slightly less ergonomic experience for discovering functions by fuzzy searching from the root of the package. Flora is great if you want to search for a particular project or build artifact (like an executable), has a great API reference which makes it suitable for being used programmatically and supports search modifiers like depends: and exe: which Hackage doesn’t have.

Goals and related work

Before designing Hypha, I explored the solution space, and to my knowledge, these are the projects that existed last spring when I did my survey:

• hackage-doc-mcp
• haskell-docs-cli

Both projects have their own merits, and I encourage you to look into them! I don’t like to comment on the design and implementation of other packages, as I think the only constructive stance for open source projects is to be grateful that somebody took the time out of their lives to contribute back to the community.

Rather than saying why I didn’t use one or the other, perhaps it would be more interesting to say what I wanted in the first place, and why I have decided to design Hypha. I wanted a tool which was:

• Capable of figuring out which packages and which versions my project was using just by inspecting the plan.json available for each cabal project (at least after you run cabal build all --dry-run);
• Offline-first: it should be possible to work offline on an airplane or in a coffee shop without an internet connection;
• Cache-friendly: we shouldn’t hammer Hackage or any other web service with AI harnesses more than it’s strictly necessary, i.e. we should take good care of our open source infrastructure;
• Implemented using the CLI Printing Press approach rather than as an MCP server;
• Useful for machines (i.e. non-interactive) but also humans (i.e. I could make sense of the output of the tool even if hypha were to be called standalone);
• Helpful for interactively navigating documentation via a UI on top of what the CLI was already providing;
• A playground to experiment with AI-assisted development without being worried about violating any project’s ethos or contribution guidelines, or trying to shoehorn in features which projects might not want to support.

MCP vs CLIs

One of the most fundamental questions when building a piece of software which should be AI-first is to ask ourselves “How is the LLM going to consume it?” and, up until recently, the one and only answer was “exposed as an MCP server”. But what is exactly an MCP server?

MCP stands for Model Context Protocol and, as the documentation states, is “an open-source standard for connecting AI applications to external systems”. The super TL;DR is best explained with a concrete example. Suppose you want to connect your AI to your Postgres database, for some data analysis: how do you “teach” the AI how to connect to Postgres, how to query data, etc.? How do you do this remotely? By writing an MCP server you are essentially publishing (some or all of) the atomic operations that this service supports behind an API. So you could imagine that (naively, disregarding any consideration about security, SQL injection etc.) you could expose an operation like db_read(table, sql_query) which the AI could call and that your MCP server could implement however it wants. Now, the next time you ask the AI to query some data, internally the harness would make a request to the registered MCP server, calling the API corresponding to the db_read operation, and giving you back the result.

The advantage of using MCP is clear:

• It’s an open source standard, so it’s something that most harnesses and LLM websites support;
• It can be hosted anywhere and exposed remotely, so you can connect your favourite service and access the MCP server from within a web chat, for example.

The disadvantage of using an MCP server is that you have to build it in the first place, obviously, but you also pay a roundtrip price as the information has to travel back and forth using JSON-RPC 2.0, and it produces verbose results, so it is not immediately the most context/token friendly option if you really care about those things.

In the past few months, a new trend emerged: just use a CLI! If you think about it, it makes sense: LLMs have been trained on a ton of Bash code, so they know very well how to write Bash scripts and how to call Unix tools. In the example above, we could have a little CLI wrapping our blessed operations and just ask an AI harness to use it (for example: echo <my_sql_query> | my-cli db_read <table> -). This is obviously a very local-centric solution (we cannot natively “publish” the CLI) and is not standard (i.e. AIs wouldn’t know how to connect to your CLI natively). However, we can easily wrap our CLI in an MCP server, to get the best of both worlds: locally our AI harness can call the CLI directly, without paying the token cost for full JSON, but we can still get the benefit of the MCP server when it matters. This is the approach taken by the wonderful CLI Printing Press project, which gives you the ability to produce a CLI wrapping many public-facing services, including an MCP server and a cache to make things more efficient.

This is the route I went for Hypha as well; not using CLI Printing Press directly, but borrowing from its manifesto. The Hypha repository ships a hypha CLI, a hypha-mcp binary, and a Claude plugin that teaches Claude to query them.

Conclusion

We have covered the basics, and if you are still interested the Hypha documentation site has everything you need to get started. I hope I have made a convincing case on why Hypha might be useful, despite it having been mostly coded via an LLM. Even though it’s beta and there are still bugs to be fixed (and many more to be discovered) I have found it useful on more than one occasion, especially as a documentation browser. Please get in touch if you have any feedback and consider filing an issue for anything that you think should work but doesn’t!

---

1. For the entirety of this section I’m using async and the mapConcurrentlyBounded function it exposes as an example to discuss the problems highlighted below, but the narrative generalizes to any Haskell package.↩︎

by alfredo at October 29, 2026 12:00 AM

Running local development services

Intro

Nix developer shells are one of my favorite Nix features. They let you spin up a full environment with every tool you need. That way, you never have to worry about manually installing dependencies or messing up your system. They’re also a lifesaver for onboarding – new team members can get to work with one command:

nix develop

But real projects often need services like databases or message queues. You usually want to run these locally so you can play with data without affecting anyone else, and this is where Nix shells currently fall short. There’s been some progress in this area, but I think we can still do better.

In this tutorial, we’ll walk through a hands-on setup to bridge this gap. We’ll build a lightweight NixOS VM that runs battle-tested Nixpkgs services managed by systemd. To follow along, all you need to know is how to set up a basic Nix flake, use a Nix dev shell, and write simple NixOS modules. By the end, you’ll have a reproducible, production-grade service stack ready for local development with a single command.

The current landscape

Running backing services locally alongside an application isn’t a new challenge. We already have container orchestrators like Docker Compose and Podman Compose, as well as native process supervisors like Hivemind or dekit.

The Nix ecosystem takes this a step further by combining dependency provisioning and process management into a single workflow via tools like devenv or devbox. (Shout out to devshell as well, though its service support is currently limited to PostgreSQL.) These are great projects – I’m a devenv user myself – but I have two gripes with the current approach:

1. We’re reinventing the wheel

   Nixpkgs already has a massive library of service modules. Yet, tools like devenv end up re-implementing them from scratch. To me, that’s a waste of effort. You end up with subtle differences and manual porting between projects instead of just using the battle-tested definitions we already have.

2. Yet another process manager.

   devshell starts PostgreSQL with a shell script, Devbox uses Process Compose, and devenv even has a custom process manager. Don’t get me wrong, this does the job, but it’s one more thing to learn that only exists in development. That knowledge doesn’t help you when you’re actually running things in production. And then there’s the risk of eventually running into corner cases where the two environments behave in subtly different ways. That’s what Nix was supposed to fix, right?

My ideal solution? I want to use the exact same service modules from Nixpkgs and orchestrate them with a tool I can use in production, like systemd.

Running a NixOS VM

Nixpkgs makes it incredibly easy to spin up a local VM. Let’s explore how we can leverage this to host our development services.

For simplicity, we’ll assume that you’re on Linux. Support for macOS hosts is left as an exercise for the reader. ¹

First, let’s define a helper function to build the VM:

# vm-helper.nix
{ lib, ... }:
{
  nixosVM =
    {
      system ? "x86_64-linux",
      modules,
    }:
    (lib.nixosSystem { inherit system modules; }).config.system.build.vm;
}

The vm attribute contains a package with an executable that starts a QEMU virtual machine configured by modules.

The key here is that the modules argument accepts standard NixOS modules. This allows you to use the exact same configuration blocks you would use for a NixOS system.

Using the vm-helper.nix we just wrote, we can define a package that launches the VM:

# flake.nix
{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-26.05";
  outputs =
    { nixpkgs, ... }:
    let
      system = "x86_64-linux";
      inherit (import ./vm-helper.nix { inherit (nixpkgs) lib; }) nixosVM;
    in
    {
      packages.${system}.default = nixosVM {
        modules = [ ./dev-vm.nix ];
      };
    };
}

Configuring the guest VM

In dev-vm.nix, we define how the VM behaves. This file establishes the common configuration we want whenever we run a VM for local development. Since this is strictly for local use, we can strip away the extras: we’ll disable graphics, turn off the firewall, and even add a shortcut (Ctrl-k) to instantly shut down the VM:

# dev-vm.nix
{ pkgs, modulesPath, ... }:
{
  imports = [
    "${modulesPath}/profiles/qemu-guest.nix"
    "${modulesPath}/virtualisation/qemu-vm.nix"
  ];

  virtualisation.graphics = false;

  networking.firewall.enable = false;

  programs.bash.shellInit = ''
    bind -x '"\C-k": poweroff'
  '';

  services.getty.autologinUser = "root";

  system.stateVersion = "26.05";
}

Integrating with the development shell

Finally, we want to make launching this VM seamless. We can add a custom script directly to a devShell attribute so we don’t have to remember any long commands:

# flake.nix
{
  # ...
  outputs = { self, nixpkgs, ... }:
    let
      # ...
      pkgs = nixpkgs.legacyPackages.${system};
    in
    {
      # ...
      devShells.${system}.default = pkgs.mkShell {
        packages = [
          (pkgs.writeShellApplication {
            name = "vm";
            text = "exec run-nixos-vm";
            runtimeInputs = [ self.packages.${system}.default ];
          })
          pkgs.postgresql
          # ...
        ];
      };
    };
}

Now, whenever you’re in your devShell, just run vm. You’ll have a local NixOS VM running whatever services you need, managed exactly like they would be in production.

Adding services to our developer VM

With our basic VM in place, we can start adding the services we need.

To keep things clean, we will split the configuration into two modules:

1. production.nix: The standard service configuration. If you run NixOS in production, this can be the exact same file you deploy to your servers. (If you deploy elsewhere, like Kubernetes, treat this as your environment-agnostic baseline).
2. local.nix: Local development overrides (port forwards, relaxed authentication, dev tools).

Because the NixOS module system automatically merges declarations together, any setting defined in local.nix cleanly overlays onto production.nix without requiring us to touch the base service definition.

Here is the plan: we will define the service baseline, layer our local overrides on top, and then wire both modules into flake.nix.

1. Defining the base services (production.nix)

Let’s assume our stack requires PostgreSQL and FerretDB. We declare them in production.nix:

# production.nix
{ ... }:
{
  services.postgresql = {
    enable = true;
    ensureDatabases = [ "app-db" ];
    ensureUsers = [
      {
        name = "app-db";
        ensureDBOwnership = true;
      }
    ];
    enableTCPIP = true;
  };

  services.ferretdb = {
    enable = true;
    settings = {
      FERRETDB_LISTEN_ADDR = "0.0.0.0:27017";
    };
  };
}

Since NixOS uses systemd to manage services, we can easily extend these definitions if needed. For instance, if you want to run a custom setup script after the PostgreSQL database is up and running, you can simply hook into the postStart script:

# production.nix
{ ... }:
{
  # ...
  systemd.services.postgresql.postStart = "# Do something";
}

Since FerretDB can use PostgreSQL as its backend, a more realistic example would be extending the FerretDB systemd unit so that it depends on the Postgres service:

# production.nix
{ ... }:
{
  services.postgresql = {
    # ...
    ensureDatabases = [
      # ...
      "ferret"
    ];
    ensureUsers = [
      # ...
      {
        name = "ferret";
        ensureDBOwnership = true;
      }
    ];
  };
  services.ferretdb = {
    # ...
    settings = {
      # ...
      FERRETDB_HANDLER = "pg";
      FERRETDB_POSTGRESQL_URL = "postgres://ferret@localhost/ferret";
    };
  };

  systemd.services.ferretdb = {
    after = [ "postgresql.target" ];
    bindsTo = [ "postgresql.target" ];
  };
}

For more complex scenarios, you can even define custom systemd services that depend on PostgreSQL. This level of control is why leveraging NixOS modules – and the full power of systemd – is so much more effective than re-implementing services inside a separate development tool.

2. Layering local overrides (local.nix)

In production, you would lock down firewall rules, bind to internal networks, and require secure passwords. In local development, you usually want the opposite: host-accessible ports and hassle-free authentication.

Instead of hacking conditionals into our base setup, we use NixOS module composition. In local.nix, we forward ports through QEMU to our host and override PostgreSQL’s auth to trust local connections:

# local.nix
{ ... }:
{
  services.postgresql = {
    authentication = ''
      host    all    all   127.0.0.1/0    trust
    '';
  };

  virtualisation.forwardPorts = [
    {
      host.port = 5432;
      guest.port = 5432;
    }
    {
      host.port = 27017;
      guest.port = 27017;
    }
  ];
}

3. Wiring everything into the flake

Now that both modules are defined, we update flake.nix to supply them to our VM helper.

# flake.nix
{
  # ...
  outputs = { nixpkgs, ... }:
    # ...
    {
      packages.${system}.default = nixosVM { modules = [
        ./dev-vm.nix
        ./production.nix
        ./local.nix
      ]; };
    };
}

We can also add environment variables to our devShell so we can connect to Postgres and Ferret without having to remember long URLs:

# flake.nix
{
  # ...
  outputs =
    { nixpkgs, ... }:
    # ...
    {
      devShells.${system}.default = pkgs.mkShell {
        PG_URI = "postgres://app-db@localhost/app-db";
        FERRET_URI = "mongodb://127.0.0.1:27017";
        # ...
      };
    };
}

Now you’re ready to run the VM:

nix run .

Monitoring services on the VM

One of the best things about tools like Process Compose is how easily you can view service logs. While our VM approach is more robust, we still want that same level of visibility.

There are several ways to accomplish this. For a high-quality terminal experience, you can use a tool like lazyjournal. It provides a beautiful TUI for navigating systemd logs, giving you a “dashboard” feel for your VM’s services without leaving your terminal.

Add it to your local configuration and run lazyjournal in the console after starting the VM:

# local.nix
{ pkgs, ... }:
{
  # ...
  environment.systemPackages = [
    pkgs.lazyjournal
  ];
}

This is just one example of how you can keep extending your local development VM with NixOS module options to suit your needs.

Closing notes

Running a full NixOS VM sits at one extreme of the spectrum for managing development environments. With this approach, we trade runtime performance and iteration speed for full reproducibility. Compared to native process managers, a VM carries a higher resource footprint and takes a few extra seconds to boot. However, because this setup is intended for developer workstations, hardware overhead is rarely a bottleneck. Trading some spare RAM and a slightly slower initial launch for complete host isolation, zero configuration drift, and battle-tested systemd fidelity is well worth the cost.

The benefits don’t stop at local development. We can launch this exact same VM in CI to run reproducible integration tests.

While this post walked through a basic implementation, it could be refined and abstracted into a reusable library.

You can find the complete working code in the dev-services-nix-demo repository.

There’s more to explore, such as customizing the development VM to match your host environment. For example, we could set up SSH access to the VM without checking keys into version control. We will come back to that in a future post.

1. If you are on macOS, the example repository features a darwin branch. It adds extra virtualization complexity, so we suggest walking through the main branch on Linux first. Also, remember that on macOS you need to set up the Linux builder ↩

by José Luis Lafuente at October 29, 2026 12:00 AM

2026: Annus horribilis

I enjoy staring up at the stars at night and trying to wrap my tiny brain around the vast distances and time that their light has travelled. I find it relaxing to consider just how insignificant I am, and how brief and unimportant my existence is. I am hypnotised when watching the wind ripple through vast fields of reeds out in Suffolk and wondering what it is that figures out, at a subatomic level, how those reeds move, where each atom moves next. Where does the universe do its maths? Is the only point of the universe to run its own simulation? I think so.

I am extraordinarily lucky. For the last 25 years, it’s apparently been the case that I’m better than average at understanding computer programs. I can anticipate unintended consequences, I’m able to chart a course between all manner of constraints, and I can make sufficiently accurate and well-judged trade-offs, that I’ve never had any difficulties finding employment. A computer program is a manifestation of the approximate human understanding of a problem. I enjoy working with humans to understand the problems they’re trying to solve. I enjoy building up that understanding and those relationships, and then being able to carefully translate that to code.

I enjoy too the puzzle-solving aspect of programming. In some cases, the journey is more rewarding than the destination. Spending days slowly exploring a failing test, gathering information, teasing apart your faulty beliefs to finally arrive at an understanding of what’s wrong with the code. I used to take pride in the fact that I could do that, and the fact that every time I did, I learnt something new and for a brief moment, knew more than anyone else on Earth about some tiny piece of code. Which would probably get deleted in a week or two. Knowing a code-base inside out, or even just part of it, was a source of pride for me.

All of that’s now gone.

The stress caused by the changes ravaging the software industry this year have directly made me quite ill. Painful diseases from which it takes months to recover. We’re now well past the point where I can pretend there’s any part of writing code at which I can still out-perform the bots. There may still be one or two esoteric areas, but you have to go looking hard, and it’s exceptionally unlikely such areas will remain unconquered for long.

I’ve seen multiple companies adopt the view that from now on, actually looking at the code yourself is bad behaviour. The only acceptable interaction with the code-base is via the bot, and specs must be written by the bot, reviewed by the bot, and used by the bot to write the code, which is then itself reviewed by the bot. I’ve heard of people being told that asking each other questions, discussing, and just communicating, is inefficient. We should now only talk to the bot. What is the point of life if not to communicate with each other?

I came across Rikab Gambhir’s post on Terence Tao’s blog. This is a reaction to OpenAI’s dump of mathematical proofs in early October 2026. Rikab points out that most of the proofs in that dump are horribly written. As far as I know, there is no known commercial value to these proofs (particularly given the direction in which they’ve been proven), so it seems reasonable to me that a human would seek to prove them in order to prove them. Intrinsically linked to that is that the proof can be communicated to others, understood by others, and verified by others. You can’t build on the past if you can’t understand the past. OpenAI has no interest in these proofs beyond advertising.

Ultimately, this isn’t just about mathematics or physics – it’s about fundamentally reimagining what it means to be human. By outsourcing the burdens of curiosity, creativity, and understanding, we can finally focus on what matters most: generating value, maximizing productivity, and accelerating progress towards the ultimate goal of science: a complete repo of potentially true facts in a Library of Babel that nobody will bother reading.

(Incidentally, I’m aware that some people struggle with those who express themselves through irony, sarcasm, and acerbic wit. Personally I have no problem with it and do not believe it lessens their message nor the points being made.)

This reminded a little of the Arts and Crafts movement which started in the 1880s: “a reaction against a perceived decline in standards that the reformers associated with machinery and factory production”. The trouble is, although hand-made furniture can be beautiful to behold, to touch, and to use, it takes a long time to make, and consequently commands a high price tag. The market for hand-made furniture today is small, and at the luxury end of the market. I don’t think there will ever be a market for hand-made software, despite what we sometimes tell ourselves about the beauty of our own code.

It also prompted me to think about why was it ever important for me to understand my own code? Understanding my own code meant I could write more code that built upon it. It meant I could debug it and correct it. It meant that, when working within a team, I could explain my thinking to others, and theirs to me. But the company actually didn’t care, nor the investors, customers, or users. They care about fixing the bug, delivering the feature, or supporting the user. If that is best done by developers understanding each other’s code, then so be it. But if there are alternative ways of achieving those goals which don’t rely on understanding code, and could be faster, then they are to be explored. Unlike finding proofs to mathematical problems, the code itself is not the goal.

I wonder whether software is moving a bit closer to other engineering disciplines like mechanical engineering or civil engineering. I have no direct knowledge of such disciplines, but I’m guessing that at university they get taught various fundamental formulae, e.g. how to calculate the maximum loading of a truss bridge. In their professional careers, I imagine they never directly use such formulae and algorithms themselves, and instead rely on large commercial software packages that can do it all for them, quickly and reliably. Knowledge and experience certainly helps spot when a result looks unlikely, but ultimately you trust the software package. Maybe this is what software engineering will become: you trust the bot to write the code, and it’s helpful to know what to expect, but ultimately you’re playing a much more creative role, closer to the customer, and are directly commissioned by the customer. Having to listen to the customer would be something of an ultimate irony for an industry for which the term enshittification was coined.

All of which leads me to realise that it’s simply absurd to spend your life caring about code, and perhaps it ever was. The jobs I’ve had that I’ve enjoyed the most have been because of the people with whom I’ve worked, not because of the code. People matter. Friends and family matter. Being at peace with yourself, with your fragile, flailing, and fleeting existence, and loving those around you, and being loved, matter. Code does not.

I really enjoy spending time outside, especially in forests. I like finding huge old trees and thinking about what they’ve seen, standing for hundreds of years. Trees that were hundreds of years old before I was born, and will out-live me by hundreds of years. It makes me smile to think that all of us will be beaten by trees.

October 28, 2026 04:01 PM

Adopting Nickel in CodeCommons

Complex architectures with multiple services require a lot of configuration. There are a lot of knobs to be tuned, and that configuration needs to be persisted in configuration files. These come in a large variety of formats - widely accepted standards such as YAML or TOML, ecosystem-specific formats such as key-value pairs in the Apache data engineering world, or ad-hoc custom configuration formats. At the same time, errors in configuration files can lead to hard to detect and frustrating bugs: invalid or missing field values can lead to late crashes after minutes of start-up, silent fallbacks to unwanted defaults due to incomplete or unknown configuration error handling in external packages, etc.

Nickel is a programmable configuration language developed by Tweag that aims to provide safe and validated configurations with ease. You’ll learn more about the most important Nickel features later, but if you’re impatient, you can read more about it on Nickel’s website or in our Nickel series on this blog.

This blog post is aimed at developers who have never used Nickel before. I show how we used Nickel to improve the difficult configuration management of a massive data ingestion pipeline, as part of the CodeCommons project. This ingestion pipeline relies on multiple industry-standard services, but also on components developed for the Software Heritage (SWH) project, and thus comes with a whole zoo of configuration files formats - all the characteristics of an ideal Nickel testing ground.

You’ll first learn the most important facts about CodeCommons, the ingestion part of it and Nickel, then I’ll describe how we tied Nickel into the existing architecture and what difference it makes, and finally I’ll summarize our observations.

CodeCommons ingestion pipeline architecture and configuration

In the CodeCommons project, Modus Create and French research institutions aim to build a reference source code database for training code-generation large language models (LLMs) and for research on source code. The project builds on top of the Software Heritage archive, which is the world’s largest archive of open-source code. In the following, I’ll discuss in some detail the configuration of a key part: the source code data ingestion pipeline Modus Create is working on.

Service architecture and configuration

The CodeCommons ingestion pipeline mostly relies on the existing Software Heritage ingestion pipeline when it comes to its components, but is deployed on a scientific high-performance computing (HPC) cluster. CodeCommons uses the same data structure as Software Heritage, namely a Merkle Directed Acyclic Graph (used, for example, by Git), to represent archived source code repositories. In this view, ingestion of source code is all about adding unique nodes to this graph. After scheduling ingestion of a project in a Redis queue, loaders read projects from that queue and map new data into the graph data structure - see the following figure for an overview. The leaves of this graph are source code files, and the loaders write them to custom object storage. Intermediate nodes represent things such as releases, revisions, or directories, and the loaders write them to distinct topics in a Kafka journal. From the Kafka journal, a custom-written “deduplicator” component reads topics and exports the graph data into two sets of Optimized Row Columnar (ORC) files: one appropriate for CodeCommons use, and one that will be used to update the existing Software Heritage archive.

Service-specific configuration options that a user might want to tune differently from run to run, and deployment-related parameters were all summarized in a simple Bash script with lines of the form export CONFIG_OPTION=<value> that served as our principal configuration file. This setup led to quite some difficulties, as we’ll see.

Configuration challenges

One of the particular difficulties with this setup was the fact that service-specific configuration files contained at least three different “classes” of parameters:

• a lot of parameters that rarely, if ever, needed to be adjusted (e.g., batch sizes, timeouts, etc.)
• settings that users of the pipeline realistically might want to change from one pipeline run to the other (e.g., deployment environment, list of projects to ingest, etc.),
• and finally, configuration options that were known only at pipeline runtime (e.g., hostnames and ports of upstream services).

These classes were non-negotiable - if we didn’t want to adapt the service’s code, we had to live with the above. At the time, we made it work with the following approach:

• each service used a template configuration file with rarely adjusted settings filled in. This file was in the format natively expected by the service (e.g. key-value pairs for Kafka, YAML for Software Heritage services, etc.). For Redis, for example, this file looked like so:

  save 60 100000
  appendonly yes
  rdbcompression yes
  
  protected-mode no
  
  dir ${REDIS_DATA_DIR}
  port ${REDIS_PORT}
  

• settings that might change from pipeline run to pipeline run were stored in a global “Bash config file” config.sh with lines export PARAMETER=<VALUE>. A couple example lines from this file were:

  export IS_IN_CLUSTER=[0|1]
  export PYTHON_VENV=${SHAREDWORKDIR}/tools/swh/venv
  export KAFKA_NUM_NETWORK_THREADS=16
  

• settings that were known only at runtime were passed from service to service via exchange files that, again, contained relevant export PARAMETER=<VALUE> lines, such as REDIS_SLURM_NODELIST (which was, essentially, the hostname of the compute node running Redis)

Finally, just before running a service, all the above were combined into a final, complete configuration file using tools such as envsubst. This environment variable-based approach also allowed us to use configuration settings in the orchestration Bash scripts directly, e.g. to send a command to the cluster’s job scheduler or run it locally: For example, the value of REDIS_DEPLOYMENT was checked in the orchestration script, and depending on its value, the Redis server was started either on the machine the script ran on or on a compute node.

But this approach had multiple weak points. On the operational side, when running the pipeline, values were not validated on even a basic level, e.g. encoding a Boolean value wrongly as 0 instead of false, and local copies of the reference config.sh file became outdated quickly: A developer maintained a config file config_local.sh, but after a git pull a new mandatory config option might have been added to the template file, which silently invalidated the local configuration. And there was config_cluster.sh over on the HPC cluster, too… These issues were very common and often led to crashes very late in the pipeline, after minutes-long start-up of preceding services had completed.

When extending the configuration, sensible / allowed values needed to be documented manually and this documentation needed to be maintained - what was sensible / allowed changed with time. When a new developer had to set up a configuration from scratch, no editor tooling was available that automatically gave you information such as type or allowed values or facilitated life via auto-completion.

All these made this system difficult to use and to maintain. How could we do better?

Reigning in the chaos: nickelizing the CodeCommons ingestion pipeline

Nickel

Nickel is, at its core, a bit like JSON with functions, not unlike the Nix language upon which it was designed to be an improvement. But on top, Nickel offers “configuration safety” features: It features contracts and optional static typing to validate data. While types are checked by Nickel before the program is run, contracts enforce arbitrary assertions lazily at runtime, thus covering different use cases. Nickel also aims to help its users write configurations that can be easily factorized into reusable parts. These can then be combined into more complex configurations. A configuration written in Nickel can then be easily exported into standard configuration file formats such as YAML, TOML or JSON.

Different shells of Nickel

In our specific project, we could intervene with Nickel on multiple levels, each entailing a different level of complexity:

• we could generate our Bash configuration file config.sh with Nickel from a nickelized version config.ncl of the existing file and leave everything else as is,
• we could additionally replace service config templates with Nickel configuration equivalents: translate the Kafka / Redis / etc.-specific configuration files to Nickel and merge them with the principal config.ncl at pipeline runtime, filling in values only known at runtime upon exporting the service config in the specific format it requires
• finally, in addition to both of the above, we could generate the actual Bash scripts that run the services with Nickel: each Bash script would be a templated string in Nickel, and completed and exported to a Bash file at runtime when needed. This would remove the need for a Bash config file in the first place, because we wouldn’t need to rely on environment variables that are read by Bash any more.

We chose to adopt the first, and partially the second, in the project, but here, I will describe only the first, most straightforward change.

Generating the main Bash configuration file with Nickel

The main Bash configuration file consisted of a flat list of around 120 exported Bash variables, most of which were prefixed with the name of the component they pertained to, for example, a configuration setting for the number of loader workers was called SWH_LOADER_CONCURRENCY. Variables were grouped by component / scope.

Step 1: Grouping configuration options

To bring order into this, we structured the Nickel equivalent of this configuration as a nested record, which we could feed into an “output” field in the correct format. A record is very similar to a JSON object, only that Nickel allows additional operations such as merging records. The very basic nickelized version of our configuration thus looked like so (remember, there were many more settings than shown below):

# schema.ncl
{
  config = {
    swh_loader = {
      SWH_LOADER_CONCURRENCY = 4
    },
    swh_scheduler = {
      SWH_SCHEDULER_DEPLOYMENT = "local"
    },
    deduplicator = {
      DEDUPLICATOR_USE_NIX = 0
    },
    monitoring = {
      GRAFANA_INSTALL_DIR = "/some/dir"
    }
  }
}

In schema.ncl, we declared all the configuration settings that would appear in the final Bash config file. But this made the configuration neither safer nor much more user-friendly.

Step 2: Simple contracts

A first step towards a safer configuration was then adding basic contracts. A contract serves to validate data at runtime, and is much more flexible than a static type. For example, we can have a very basic contract String that checks at runtime whether a variable is a string, or we can have a more complex contract that checks whether a number is a valid port - a positive natural number within a certain range. For most of our settings, very basic built-in contracts already did a good job in making sure we didn’t use values that didn’t make sense:

• the above number of workers defined by SWH_LOADER_CONCURRENCY needed to fulfill std.number.PosNat (be a positive natural number),
• switches such as SWH_WINERY_ENABLED needed to fulfill the Bool contract,
• GRAFANA_INSTALL_DIR needed to be a String,
• etc.

A slightly less trivial contract type we used were enums: In the above example, you’ll see the string “local”, and another valid value would be “slurm” (to run the scheduler component on a Slurm node). The contract [| 'local, 'slurm |] then allowed only the values 'local and 'slurm for this and similar fields. When output as a string, these would turn into local and slurm (no single quotes) - just what we needed.

Nickel also allows you to write more complex custom contracts. We used such a custom contract to check for correct Boolean value encoding:

let BooleanNumber =
  std.contract.from_validator (match {
    0 => 'Ok,
    1 => 'Ok,
    value => 'Error { message = "Expected 0 or 1, got %{value}" }
  })

which let a field take a Boolean value encoded as 0 or 1, and made sure that everything else would result in an invalid configuration.

Finally, we could encode the fact that we wanted some settings to be non-optional by not assigning them a default value.

In the end, our example looked as follows:

# schema.ncl
{
  config = {
    swh_loader = {
      SWH_LOADER_CONCURRENCY | std.number.PosNat | default = 10
    },
    swh_scheduler = {
      SWH_SCHEDULER_DEPLOYMENT | [| 'local, 'slurm |]
    },
    deduplicator = {
      DEDUPLICATOR_USE_NIX | BooleanNumber
    },
    monitoring = {
      GRAFANA_INSTALL_DIR | String
    }
  }
}

Once a field is annotated with a contract or a type, the Nickel LSP will immediately be aware and validate the configuration. This also holds for autocompletion: Nickel’s LSP will know right away and propose you any new element such as a custom contract or a field. Running nickel export on the final configuration will then fail if a field has an invalid value. But of course Nickel can do more than just hierarchical records with contracts!

Step 3: Modularizing the configuration

In Nickel, you can merge two records using the & operator, and you can import a Nickel file into another Nickel file with import <some_file.ncl>. These two useful features allowed us to make further improvements: composing the final configuration from multiple files. For this step, the schema.ncl discussed above stayed largely as it was, only that we removed some defaults for values that we would set later in “overrides” specific for local or cluster deployments. We added the following modules that we finally all merged with schema.ncl:

• components.ncl: contained enable / disable switches for pipeline components,
• run_dirs.ncl: contained paths for temporary directories used during ingestion,
• software.ncl: contained paths to binaries required by the pipeline,
• deployment.ncl: contained switches that determine where certain components are deployed (login or compute node).

That way, a developer wanting to change an option knew directly where to navigate, and these files also reused common paths, prefixes and the like, while schema.ncl really only gave the schema and default values of fields that were rarely touched.

Finally, for launching a simple, uncustomized run of the ingestion pipeline, we asked the user to complete two short files:

• user_statics_template.ncl: fields that were user-specific, but were otherwise static across pipeline runs (e.g. the parent directory of all binary dependencies),
• local_template.ncl / cluster_template.ncl: filled pipeline environment and run-specific variables such as the data output directory, and the file with the list of origins to ingest.

With this setup, for “just running” the pipeline with sensible default parameters, the user didn’t have to complete a file with ~120 options any more, many of which shared common data, but only had to set a handful of mandatory, intuitive settings. And of course, a user could customize the configuration to their liking by adding overrides to their copy of {local,cluster}_template.ncl, without touching any other file.

That wraps up the concrete implementation! If you are interested in looking at the full configuration, you’ll find it on the ingestion pipeline’s GitLab repository.

Lessons learned and observations

A couple of factors came together that led to introducing the above improvements to the configuration using Nickel: the team working on the ingestion pipeline was struggling a lot with the configuration management; my (now former) colleague Yann, who is mostly behind Nickel, was looking for a project that could be a test case for real-world Nickel adoption; and last but not least, I was simply curious and wanted to learn something new and less “mainstream”. This section is thus a mix of observations through these three different lenses.

Impact on development workflow

Running the pipeline became much easier; getting a working configuration was a matter of minutes. This made a difference to new joiners and to engineers like me who had been working on that project for a while already, but had their SSD self-brick (twice!) and needed to start from scratch.

A big improvement over the existing workflow was the availability of an LSP server for Nickel. If installed and correctly hooked up to your IDE, writing a configuration in Nickel feels just as developer-friendly as working on the actual pipeline code with a good Python LSP. As mentioned earlier, the LSP validates your configuration on-the-fly and yells at you as soon as you make a mistake such as assigning a value of the wrong type or that doesn’t fulfill a certain contract to a field. Note that this also applies to non-trivial, custom contracts, such as, for example, making sure a list of users had unique user IDs. (There is a blog post on Nickel LSP’s contract evaluator, if you are interested.) Auto-completion is likewise available instantaneously, and of course the Nickel LSP supports formatting your code, too. This way, you get immediate feedback about the validity of your configuration, and changes are fast and comfortable to implement.

Adoption by other developers

It is always hard to push engineers to learn a new tool with the promise of replacing something that works with something that works better. Are we sure it will work better? Will it be worth the effort? Can I just get my work done and learn this new skill later?

In our case, it was possible to introduce many of the benefits of Nickel without initially forcing anyone to use it - for a while, we maintained two copies of the main configuration: the Nickel version and the original, plain Bash version. This was obviously a maintenance burden, which we softened by the mechanism described in the next section. In the meantime, though, we removed the Bash config file from version control for easier maintenance, and developers were then using the Nickel config to generate their personal configuration files. Validation of using Nickel came then notably by an engineer who joined the project after the first version of the Nickel config was finished. This engineer did not encounter major configuration issues, and was instead very glad to be able to spend their time smoothing out other rough edges!

Short detour: keeping team members up to date about configuration file changes

As mentioned before, for a while, we kept both the old Bash template config file and the Nickel config existing in parallel. Now what happened if a team member, who didn’t use the Nickel config yet, updated the Bash template and pipeline code with a mandatory config field? This change would be easy to miss for developers not using the Bash template any more. To avoid team members using outdated Nickel configurations, we set up an additional step in our GitLab CI pipeline: Whenever a commit was merged on the main branch that changed the Bash config file, an issue telling developers to update the Nickel config and to adapt any local configuration files was opened. The issue text linked to the specific MR that changed the config file.

This would actually have been a good idea way earlier - we often encountered the issue of local configuration file copies being out of date: engineers would copy the template config file to, say, config.local_simeon.sh, but then the upstream config.sh would change, rendering the engineer’s copy invalid. Missing a single small MR was then all it needed to get possibly hard to debug crashes when running the pipeline the next time. We imagine that this is a common problem, and even if you decide after having read this blog post that Nickel is not for you, perhaps you can still make use of this completely configuration language-neutral trick.

Learning curve

This is more of a subjective topic, but it’s difficult to report about introducing a new tool to an existing project without mentioning the learning curve.

Admittedly, it was a long-time goal of mine to learn Nickel, so the basic motivation was high already. Nevertheless, it turned out that getting to a first working (and useful!) version of the configuration was rather easy. After having grasped the basic idea of writing a nested record and the trivial, but painful task of copying and splitting up the existing configuration into useful sections, the only slightly difficult part was writing the custom code to output the correctly formatted Bash file:

let config_string = config
  |> flatten_merge
  |> std.record.to_array
  |> std.array.map (fun { field, value } => "export %{field}=\"%{escape_single_quotes value}\"")
  |> std.string.join ("\n")
  in [...]

As an imperative programmer by training who’s been functionally curious for a little while, that whopper was the one part of “real” coding that required some help from the pros. I had a rough idea of what to do, but the required functional way of thinking turned this into an actual effort. Yann again stepped in to help me set up the above pipeline that flattens the hierarchical record (which is the most difficult part), converts data structures, and finally constructs the correctly formatted lines. It was then relatively straight-forward to add basic types and mark fields as required, which yielded a big improvement over editing a Bash config file template by hand.

When I became more independent after a little while, the Nickel Discord channel with very responsive and helpful members was of great help. But if your use case requires a common configuration format supported natively by Nickel, then there’s a good chance you can get pretty far without writing a single recursive function!

Conclusion

In this tutorial / use case / experience report / blog post hybrid, I showed how we were able to significantly improve our configuration management using basic Nickel. I hope this helps you think about what a simple Nickel use in your own project could look like, and what gains and efforts you might expect. As always, if you have any questions, insights, or are curious about the CodeCommons project or our Nickel adoption, don’t hesitate to reach out in the Discord channel or via the CodeCommons ingestion pipeline GitLab project.

by Simeon Carstens at October 22, 2026 12:00 AM

A Tale of Three Months with AI: Guardrails, and How We Are More Alike Than We Wish to Admit

As I promised in A fresh leaf, I want to share my experiences with using LLMs for agentic development in the last three months. Summarised: I started as an initial sceptic, became quite enthusiastic, and am now moderately enthusiastic and guarded for the future. Allow me to take you through my journey.

Coming from a background of rigorous engineering with up-front thinking about program structures, algorithms, and types, it seemed incomprehensible to me how a jumped-up neural net could generate working code, let alone sensible systems. How to write understandable software has been hotly debated for decades, continuously shifting as new languages, type systems, and ecosystems introduce different trade-offs.

Early in my career, coming from academia, I was fixated on finding the “just-right” solution: elegant abstractions, strict correctness, and formal soundness. As problems grew larger and messier, and time became scarce, holding out for perfection became an obstacle. I found far more satisfaction in achieving something functional, with elegance as a welcome second. Having both overlap is fantastic; achieving the actual goal remains both a priority and a necessity.

This shift in perspective explains why LLMs, and agentic development, eventually proved so attractive to me. Expecting a model to generate a “just-right” architecture, code, or text only causes frustration. Conversely, it acts as an engine to reach concrete milestones in areas I was always curious about without having the time to do things “properly”, or simply eliminating the toil of writing boilerplate when I can describe my intent with less mechanical precision.

If your goal is simply to achieve something concrete, they are remarkably effective. That spans from something as simple as integrating an unfamiliar library to breaking ground in an unfamiliar problem domain where you lack the tools and vocabulary.

The greenfield honeymoon

#

My catalyst was a device sitting in my living room. After reports emerged that its successor was aggressively hostile to user privacy, I wanted to verify whether my own hardware behaved the same way. Since I had no background in firmware inspection or packet analysis on embedded devices, I was initially discouraged. Luckily, I noticed right then that I had a free trial month of Codex lying around, so I decided to see how far these so-called “amazing frontier” models could carry me.

I started out by talking in ChatGPT and asking questions on how to solve my problem.

This quickly became using Codex to work on my laptop, letting it download images, and running local scripts. The way of work was very interactive, prompt-based, and relatively unstructured and unguided.

I was amazed by how well the model worked when told to achieve a very specific goal and even more so when giving the context and constraints in a single prompt: as long as the prompt wasn’t too big.

When I pointed out errors, the models complimented me lavishly on the discovery and apologised profusely for the defect.¹ Yet that contrition yielded no durable change in their subsequent behaviour. It was just text, learned from reams of documents.

Asserting control

#

As soon as the codebase and program to detect any nefarious behaviour was growing, I noticed the performance of the models degrade.² Responses lost their initial speed, precision, and correctness. The model began overlooking interdependent updates that required lock-step coordination, shedding context established in earlier turns. In other words, it required more hand-holding, for which I had neither time, nor patience, nor appetite.

At that point, I also started working on some other ideas that had been lying dormant in my head. I used an agent to create a Terraform project to spin up a small timeseries-db backup in Google Cloud constrained such that it would stay within the free tier. Creating the provisioning scripts was the work of mere moments. It checked the exact sizing requirements for staying within the free tier including some caveats, whipped the Terraform code together, and added the Tailscale integration. All from a simple prompt of what I wanted to achieve and the fact that I wanted to be able to reconstruct the VM from a single command. That was awesome.

It was here that I first started suspecting a deeper link between models and humans: it all came down to how bounded the task was. Over the years, I noticed that when you ask a person an open-ended question, their answer is mostly influenced by their recent history, their emotional state, and whatever problem they were just wrestling with. Models work almost identically. Ask them a broad, open-ended question and you get a generic response shaped by their pre-trained weights and immediate context window. Getting either a human or a model to think outside that default box requires tight, explicit constraints, and careful prodding (or prompting).

One of the often touted best-practices I encountered in the beginning was to have a three-phase approach. Use one agent to plan the work interactively, producing a handover document. Then task a second agent with implementing that handover document unattended. Finally pass the control to a review agent, which gets the diff and the plan, and reviews it. The obvious improvement is to make the last two communicate and work in a loop until no more findings are established.

I first tried automating this division of work (planning, implementation, and review) by writing conventions as prose in AGENTS.md. As code grew and threads grew longer, however, prose instructions became less effective and easily ignored by the model. To take this workflow further, I wanted to combine the specialised strengths of different models: Codex as the fast, local workhorse for writing code, and Gemini as the thoughtful reviewer for architecture. To let them talk directly to each other, I built gemini-mcp.

Here I hit another quirk of model psychology: autonomous models possess such supreme self-certainty that review triggers require explicit programmatic conditions. A loose prose directive like “Ask Gemini for help when uncertain” generated precisely zero invocations; the model was never uncertain. Conversely, a deterministic circuit breaker (“Attempt to solve each problem a maximum of three times; if still failing, consult Gemini via MCP, implement its recommendations, and repeat”) reliably prompted Codex to escalate for second opinions.

This put me with an immediate security challenge: because I was using this setup to assess the privacy of the TV, I needed a classifier to prevent prompts between models from looking like offensive security probes, which could get my Google account suspended.

Scaffolding the classifier was quick work. Making it function reliably, however, involved dozens of subtle architectural choices that flew past too fast for me to properly evaluate. Steering the model through prose in AGENTS.md simply wasn’t giving me the control I needed. That was the moment I turned to systematic, deterministic tooling, starting with a simple directive: run make check after every modification to verify that the project was still coherent.

This worked so well that ever since I have been extending what is behind this check continuously. Relying on what I already knew worked well in larger projects with multiple contributors, I decided to use strict tools to check as much as possible. This meant automatic linting, formatting, and upgrades of the different types of files in the project: Markdown, code, YAML files, ADRs, formatting, modernisation of Go, and dependency upgrades. Moreover, I decided to take more out of the hands from the agents and not even bother them with formatting and easy to fix linting errors by automatically running formatting, automatic imports, and adding --fix to golangci-lint already as part of the check step to ensure all easy fixes were implemented before the agent even saw the result.³

I introduced this rigorous tracking in my first big vibe-coded project, semantic-editor. The whole point of the project was to let the agent worry about intent while delegating the nitty-gritty editing work to deterministic AST tools. I applied that same mindset to the repository itself, using strict checks for the markdown, the ADRs, and the schemas. This approach worked pretty well, and the project was progressing smoothly.

Around that time, another personal project came into my head. I had always felt sad that in this age of search engines and lately omniscient (and often wrong) LLMs, my kids didn’t have the experience to just browse an encyclopedia, guided by interest. For me the concrete example of this was Microsoft Encarta ‘97. It gave me endless browsing, learning of all kinds of things, and important to my parents (and now me), relatively safe because it was curated content.

As you may have guessed, I decided to vibe code the answer to this problem as well and make a viewer of Encarta data. Because in-repository decision tracking had operated satisfactorily, I relied on that established harness to guide development.

However, because this was my first GUI app, I sensed it needed another layer of constraints and decision tracking which I dubbed “GUI-invariants” as a specialised form of ADR for GUI decisions such as layout on a more generic level instead of specific decisions. Getting the agent to consider the invariants proved relatively simple by just adding it to the agent instructions. Letting it interpret them in a way was harder, because invariants are inherently generic. So they needed more concrete positive and negative examples and be more prescriptive.

For some reason I had a quirk that I wanted all text to be “properly” type-set such as in TeX. And why not? We are custom coding this project and I have tokens to burn, so let’s have at it. The agent was very helpful in this bounded context. A simple assignment, a starting example algorithm, the specific constraint of the SwiftUI infrastructure, and an oracle to determine whether it did it right. This allowed me to set a simple goal and I could walk away from it.

These three main projects and several side projects, all shared one important property: they were greenfield projects where I individually was the sole owner and decision maker.

In urban planning terms, greenfield land is effortless to traverse and develop, precisely because it is virgin ground: untouched, open, and unencumbered by history. Brownfield sites, by contrast, are contaminated swaths of land burdened by buried foundations, undocumented pipelines, and wrapped in a veneer of toxic legacy waste. Left unattended, software tends to follow an identical trajectory. When projects expand without discipline, the virgin meadow rapidly deteriorates into a muddy, hazardous swamp that engineers dread to touch.

Yet that decay is avoidable. Long-lived systems like the Linux kernel survive decades of churn not through bureaucratic upfront planning, but through uncompromising maintainer gatekeeping: strict subsystem boundaries, relentless review against established patterns, and clear commit rationale.⁴ We should learn the lesson and apply it to agentic programming. Because general purpose models lack this inherent institutional memory and intrinsic motivation, we must codify that gatekeeping directly into the environment: tracking architectural decisions, enforcing invariant suites, and maintaining deterministic mechanical guardrails.

Handled with that discipline, a codebase avoids becoming a toxic brownfield. Instead, it becomes a cultivated glasshouse: a structured, highly productive environment where automated checks supply continuous nourishment, and recorded context allows development to continue to flourish at speed.

From the green pasture into the mud

#

While agents seem to shine on greenfield projects that they manage themselves, which is why vibecoding works so well, they start falling flat when projects become larger or they need to be used in established projects. This is because the structure agents use to locate where to make changes mirrors the latent patterns ingrained during training.⁵ What a coincidence, this is also the same for human engineers.

My experience creating the initial draft of the dynamic-fault-injection filter for an open-source “Built on Envoy” project exposed this boundary. I had the foundational distribution formula and prior experience running it in a JVM-based sidecar; directing an agent to scaffold the initial Go plugin was straightforward. The friction began when integrating it upstream. Unlike self-owned projects where you can invent your own shortcuts, I needed to fit cleanly into the project’s established discipline: adhering to its battle-tested test patterns, documentation conventions, and repository layouts.

Anyone can staple a quick fix onto a codebase, and just that quickly another wart is added. The actual challenge is figuring out where the change truly belongs, and doing the painstaking rework so it looks like it was designed that way from the start. Occasional contributors face this constantly: deciding when to live with an existing pattern, and when to roll up your sleeves and rewrite surrounding code to make it fit properly.

Navigating that friction succeeded primarily because I received generous, precise guidance from the lead maintainer. That human feedback supplied the unspoken repository context and architectural intent that neither I nor the model could extract from the code alone.

I could then feed those maintainer insights directly back into prompts, giving the agent the concrete boundaries it needed to shape the implementation correctly.

However, sitting down quietly with a cup of tea and looking over the code, I realised the code organisation was still a bit off. That intuition was only available as latent skill in my head based on code scattered across the repository, and admittedly not applied consistently enough for an untrained eye like mine on the project. This is where concrete ADRs and contributing guides earn their keep: they turn what is locked in maintainers’ heads into explicit rules agents can actually read.

When tackling bigger projects, and more-insufficient usage limits, I started looking into other models and noticed that the model families tend to have similar behaviour.

To me these are the qualifications of the model families, that I have personal experience with. Even though they are a mathematical probabilistic bag of parameters, they seem to have emergent personalities:

Hyperactive Junior Engineer Qwen: The Qwen family (both 3.6 dense, MoE, and 3.8) generates code at blistering speed, rarely pausing to ask questions. It demands highly granular guidance. The 2-bit Saluki quantisation displays this impulsiveness even more acutely.

Expensive easily-distracted CS Professor Gemini: Extremely strong at reducing a problem description to its core and finding the academic consensus on the solution fit. It is particularly strong in Go and machine-learning topics, and great at explaining why models behave in certain ways. Especially at the architecture level.

However, it is expensive for routine coding, not that fast or good, and gets stuck in repetitive read loops, endlessly inspecting files without making changes:

Gemini 4.0 Argon is even worse for everyday work: a review of a small piece of text burns through 50% of my weekly Pro allowance. No matter how good it is, it is unusable in this way.

Dependable, rabbit-holing workhorse GPT: This is the family that is actually interesting, especially given the pricing strategies. In general, I found GPT to rabbit-hole into a particular solution direction faster and refuse to get out of it, where Gemini could step back. Sol 6.1 (and prior with 5.6 Terra) is the perfect reviewer and Luna (both 5.6 and 6) is the perfect workhorse. Luna is decently smart, but especially because it is virtually free. Both Sol and Luna have the tendency to double down on a chosen path and are very hard to get to think outside the box or take a step back.

Across all these model families, one shared quirk emerged: an almost infuriating default bias toward Python for writing quick scripts. Instructing an agent explicitly not to use Python invariably yielded TypeScript. This is clearly the gravitational pull of public training data, where Python and TypeScript examples dwarf everything else.⁶ It constantly left me longing for something like goshell: a single standalone Go file declaring its dependencies inline, executed directly without scaffolding an entire module.

AI companies love to shout about breakthroughs at the absolute peak of their frontier models. Yet what actually changed my day-to-day experience was not a dramatic leap in the raw intelligence of the top tier. The real shift was the surge in quality among the affordable models: the ones actually in reach for everyday work. A model like Sol 6.1 brings reasoning that used to belong exclusively to flagships, but at a price where you can afford to keep it running continuously in tight feedback loops. It is not that the absolute Pareto frontier has moved, but that the affordable Pareto frontier has moved. In that sense, practical agentic development has become far more of an economics game than a science game.⁷ Slashing the price of a monster flagship like Astra or Argon would be an easy way to win the “smartest model” contest, but subsidising those gargantuan inference costs would burn through cash at a staggering rate. The provider that wins the practical developer ecosystem won’t necessarily be the one with the cleverest paper, but the one that delivers reliable reasoning at a price where hundreds of daily iterations don’t break the bank.

Choosing the right model for the task at hand is an important part of being able to work in large code-bases. However, more is needed.

The Engineering harness

#

Looking back at my career on what made successful teams achieve their successes besides their social skills and trust among themselves, was the fact that they spend time on great tooling. Even the best of minds have a limit to how much they can keep in their RAM, how fast they can write new code, and how fast they can diligently review code.

They use tools to prevent discussions about code style by applying linters and auto formatting rules, anywhere from in a check command, a git hook, or in the PR reviews. This avoids noise.

They have an integral test-suite, that lets them move fast ahead with ideas, while mistakes are still caught. This integral test suite goes further than mere unit tests and includes tools that have been around in various variations.

It automatically tests for:

• Performance
  • Microbenchmarking
  • Profiling
• Security
  • Static analysis
  • Dynamic pentesting tools
  • Fuzzers
• Quality
  • Linting
  • Coverage and coverage trends
  • Property tests and mutation testing

Notice how far away this is from reams of prose. Humans with a limited span of attention and limited brain capacity cannot map their every action against a bookwork of architecture and procedural rules, it would be too time consuming. They use heuristics to drive their “online” work, and “offline” guard rails to prevent their oversights from hitting customer impact. These same measures apply with agentic coding.

Because agents are LLMs with extra rules, as long as you have compute power, and sufficient supply of well thought out jobs / issues, the agents can produce code faster and without bottlenecks. This increases the leverage, arm, or multiplier of how much agents benefit from having these guardrails in place.

Institutional Knowledge

#

Guardrails in the form of rigorous automated test suites, as introduced above, carry us only so far. They cannot remember the WHY: why do we have a certain test, why do we implement a feature, why did we choose a particular library.

In smaller, one-person greenfield projects, recording the decisions in simple ADRs was good enough to keep the agent bounded and give me a mechanism for manually pointing out to adhere to the ADRs. In sprawling brownfields and cultivated glasshouses alike, decisions come from many different directions; the institutional experience of the engineering corps is heterogeneous and in continuous flux.

This is what organisational theorists and software architects also refer to as “Institutional memory”. ⁸

In humans this knowledge is gradually ingrained into the engineer through experience. If you are lucky they stay on the projects long enough to provide continuity.

Agents have no such context at all. Actually, engineers are also fallible. They can forget. (Context compaction and fresh sessions) They can have a lapse of focus or being disturbed. (The “lost-in-the-middle” effect⁹.) They can move on to a new project. (Starting brand new chats.)

With agents, that departure happens constantly: every time you open a brand-new chat thread, every unrecorded insight simply evaporates into /dev/null (unless you count the provider retraining on your prompts ;-) ).

Thus we need a system that actually benefits both engineers and agents and so I extended the harness in two ways:

First: I introduced Research Questions, capturing the intent of any change or feature. Basically it represents the inbox to the project. Before an implementation starts, stakeholders and engineers discuss goals, trade-offs, and explore alternatives. This leads to what to build and more importantly what not to build. This is context that very rarely survives into issues, pull-requests, and/or commit messages and is therefore lost. By recording these deliberations in the research questions, it retains this information while at the same time also allows ADRs to stay small, concrete, and focussed, to stay kind to the limited context windows of Agents.

Secondly: Even with this mechanism in place, I noticed that there was still drift so I added a rule to check nightly or weekly to do a full reconciliation of changes against established ADRs and of code against Research Questions to see whether new data had come in or we actually already made decisions.

This now closes the loop in the harness:

• Agents drive change
• Linters, compilers, and tests protect the quality of the implementation and documentation.
• Reconciliation loops at various levels and scopes (code change, daily, weekly, monthly)

This highlights a fundamental parallel: the core engineering discipline remains identical for human engineers and autonomous agents; only the execution velocity differs. Improvements introduced for engineers benefit agents even more dramatically. We must heed what software engineering has demonstrated over the past half-century.

This entire harness stems from a core conviction: let the neural net focus on high-level reasoning and intent, while letting deterministic CPU tools handle the mechanical verification.

Compilers (especially with strong type checkers), linters, and test suites execute in milliseconds with absolute certainty. Burning expensive, probabilistic tokens on verification that a local binary handles in microseconds is wasteful in the extreme.

This same conviction also drives my preference for dense languages such as Haskell. Strict types, purity as a first principle, and a dense syntax help to keep functions small, isolated, and focused. It allows an engineer to hold a function in their working memory, reason about its behaviour, and ensure its correctness. From the outside it is enough to know that the types match. Afterwards, type signatures alone dictate how to compose functions, rarely requiring subsequent repairs due to unforeseen defects.

What is next: leveraging the speedup gains from agentic development and corporate fit

#

With deterministic guardrails governing both code quality and institutional memory, a curious dynamic emerges. The bottleneck shifts upwards from code generation to coordinating concurrent, sensible streams of work.

Additionally, with the approach of using ADR and Research Questions as outlined in this article, the institutional memory remains siloed in a single repository, whereas architectural principles, designs, and organisational conventions span the entire organisation and therefore multiple projects. The current approach still lacks a clean mechanism to unlock that higher-level institutional memory to the agent without flooding its context window with (potentially) irrelevant history.

These two challenges are where an active body of research is currently directed, and what I will focus on next.¹⁰

Conclusion

#

I like to end with something that has been cycling in my head all along and my AI quoted back to me when reviewing this conclusion:

software engineering has never been about syntax; it has always been about managing fallible, context-bounded intelligence, whether made of carbon or silicon.

(Gemini, when repeating back what I told it above.)

Software Programming has traditionally centred on syntax, algorithms, and code generation. In that narrow domain, modern models excel: they produce functional routines, wire up libraries, and spit out syntax with astonishing velocity. On that level, programming as we know it has become very niche.

Conversely, Software Engineering has always begun where raw programming ends.¹¹ It encompasses establishing architectural boundaries, aligning technical trade-offs with business priorities, and managing fallible, context-bounded intelligence across time.

Although agents are tireless programmers, they possess zero business context. They lack the knowledge whether a repository is a throwaway prototype that needs to ship by Friday or a core payment engine where a subtle edge case creates legal liability. Engineering transcends syntax because it demands commercial judgement: knowing when to accept technical debt, how to balance elegance against commercial deadlines, and which problems actually matter to the business. Institutional memory (captured in our ADRs and Research Questions) is simply the durable record of those business trade-offs. The agent can generate the syntax, but the engineer must supply the strategic context: the architectural boundaries, the business priorities, and the long-term intent that code alone can never express.

Ultimately, in the context of software engineering, human engineers and autonomous agents are far more alike than we might care to admit. Both are fallible, context-bounded intelligences prone to lapses in attention, recency bias, and unstated assumptions. The hard-won disciplines of decades of software engineering (clear boundaries, explicit contracts, automated invariant testing, and durable records of intent) were always meant to safeguard us from our own limitations. It turns out nicely that these exact same disciplines are also what allow AI to succeed. Hopefully this marks the time where we are able to bring these practices in daily use.

What we need to figure out next is how to leverage the immense speed gains agentic programming gives us while staying sane as engineers ourselves.

Postscript

#

This blog post is itself an example of that dynamic: making something of higher quality in the same amount of time. I spent about a day writing this post. Using LLMs in a targeted way allowed me to test different phrasing and let the model hunt for inconsistencies in my arguments. All text was hand-written (or rather, typed), then reviewed alongside Gemini to refine logical flow, prune meandering thoughts, and source primary references. I hope you enjoyed the journey.

---

1. M. Sharma et al., “Towards Understanding Sycophancy in Language Models”, arXiv:2310.13548, 2023; E. Perez et al., “Discovering Language Model Behaviors with Model-Written Evaluations”, arXiv:2212.09251, 2022. back

2. C.-P. Hsieh et al., “RULER: What’s the Real Context Size of Your Long-Context Language Models?”, arXiv:2404.06654, 2024. back

3. C. Sadowski et al., “Tricorder: Building a Program Analysis Ecosystem”, Proceedings of the 37th International Conference on Software Engineering (ICSE), pp. 598–608, 2015. back

4. M.M. Lehman, “Programs, Life Cycles, and Laws of Software Evolution”, Proceedings of the IEEE, 68(9), pp. 1060–1076, 1980; D.E. Perry and A.L. Wolf, “Foundations for the Study of Software Architecture”, ACM SIGSOFT Software Engineering Notes, 17(4), pp. 40–52, 1992; M.W. Godfrey and Q. Tu, “Evolution in Open Source Software: A Case Study of Linux”, IEEE International Conference on Software Maintenance (ICSM), pp. 131–142, 2000; G. Kroah-Hartman, “Linux Kernel Development Model”, The Linux Kernel Documentation, 2018; D. Spinellis and P. Avgeriou, “Evolution of the Unix System Architecture: An Exploratory Case Study”, IEEE Transactions on Software Engineering, 47(6), pp. 1134–1163, 2021. back

5. R. Hu et al., “Schrödinger’s Code Repository: Evaluating LLMs on Dynamic Codebases”, arXiv:2406.03444, 2024; R. Brooks, “Towards a theory of the cognitive processes in computer programming”, International Journal of Man-Machine Studies, 18(6), pp. 543–554, 1983. back

6. D. Kocetkov et al., “The Stack: 3 TB of Permissively Licensed Source Code”, Transactions on Machine Learning Research, 2022; F. Cassano et al., “MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation”, IEEE Transactions on Software Engineering, 49(7), pp. 3675–3691, 2023. back

7. M. Zaharia et al., “The Shift from Models to Compound AI Systems”, Communications of the ACM, 67(6), pp. 24–27, 2024; L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance”, arXiv:2305.05176, 2023. back

8. E.W. Stein and V. Zwass, “Actualizing Organizational Memory with Information Systems”, Information Systems Research, 6(2), pp. 85–117, 1995; P. Kruchten, H. Obbink, and J. Stafford, “The Past, Present, and Future for Software Architecture”, IEEE Software, 23(2), pp. 22–30, 2006. back

9. Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts”, Transactions of the Association for Computational Linguistics, 12, pp. 157–173, 2024. back

10. T.R. Sumers et al., “Cognitive Architectures for Language Agents”, Transactions on Machine Learning Research, 2024; S. Hong et al., “MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework”, ICLR, 2024. back

11. T. Winters, T. Manshreck, and H. Wright, Software Engineering at Google: Lessons Learned from Programming Over Time, O’Reilly Media, 2020; F.P. Brooks Jr., The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, 1975. back

October 20, 2026 09:00 AM

PenroseKiteDart User Guide

Introduction

(Updated June 2026 for PenroseKiteDart version 1.10)

PenroseKiteDart is a Haskell package with tools to experiment with finite tilings of Penrose’s Kites and Darts. It uses the Haskell Diagrams package for drawing tilings. As well as providing drawing tools, this package introduces tile graphs (Tgraphs) for describing finite tilings. (I would like to thank Stephen Huggett for suggesting planar graphs as a way to reperesent the tilings).

This document summarises the design and use of the PenroseKiteDart package.

PenroseKiteDart package is now available on Hackage.

The source files are available on GitHub at https://github.com/chrisreade/PenroseKiteDart.

There is a small art gallery of examples created with PenroseKiteDart here.

Index

1. About Penrose’s Kites and Darts
2. Using the PenroseKiteDart Package (initial set up).
3. Overview of Types and Operations
4. Drawing in more detail
5. Forcing in more detail
6. Advanced Operations
7. Other Reading

1. About Penrose’s Kites and Darts

The Tiles

In figure 1 we show a dart and a kite. All angles are multiples of (a tenth of a full turn). If the shorter edges are of length 1, then the longer edges are of length , where is the golden ratio.

Aperiodic Infinite Tilings

What is interesting about these tiles is:

It is possible to tile the entire plane with kites and darts in an aperiodic way.

Such a tiling is non-periodic and does not contain arbitrarily large periodic regions or patches.

The possibility of aperiodic tilings with kites and darts was discovered by Sir Roger Penrose in 1974. There are other shapes with this property, including a chiral aperiodic monotile discovered in 2023 by Smith, Myers, Kaplan, Goodman-Strauss. (See the Penrose Tiling Wikipedia page for the history of aperiodic tilings)

This package is entirely concerned with Penrose’s kite and dart tilings also known as P2 tilings.

Legal Tilings

In figure 2 we add a temporary green line marking purely to illustrate a rule for making legal tilings. The purpose of the rule is to exclude the possibility of periodic tilings.

If all tiles are marked as shown, then whenever tiles come together at a point, they must all be marked or must all be unmarked at that meeting point. So, for example, each long edge of a kite can be placed legally on only one of the two long edges of a dart. The kite wing vertex (which is marked) has to go next to the dart tip vertex (which is marked) and cannot go next to the dart wing vertex (which is unmarked) for a legal tiling.

Correct Tilings

Unfortunately, having a finite legal tiling is not enough to guarantee you can continue the tiling without getting stuck. Finite legal tilings which can be continued to cover the entire plane are called correct and the others (which are doomed to get stuck) are called incorrect. This means that decomposition and forcing (described later) become important tools for constructing correct finite tilings.

2. Using the PenroseKiteDart Package

You will need the Haskell Diagrams package (See Haskell Diagrams) as well as this package (PenroseKiteDart). When these are installed, you can produce diagrams with a Main.hs module. This should import a chosen backend for diagrams such as the default (SVG) along with Diagrams.Prelude.

    module Main (main) where
    
    import Diagrams.Backend.SVG.CmdLine
    import Diagrams.Prelude

For Penrose’s Kite and Dart tilings, you also need to import the PKD module and (optionally) the TgraphExamples module.

    import PKD
    import TgraphExamples

Then to ouput someExample figure

    fig::Diagram B
    fig = someExample

    main :: IO ()
    main = mainWith fig

Note that the token B is used in the diagrams package to represent the chosen backend for output. So a diagram has type Diagram B. In this case B is bound to SVG by the import of the SVG backend. When the compiled module is executed it will generate an SVG file. (See Haskell Diagrams for more details on producing diagrams and using alternative backends).

3. Overview of Types and Operations

Half-Tiles

In order to implement operations on tilings (decompose in particular), we work with half-tiles. These are illustrated in figure 3 and labelled RD (right dart), LD (left dart), LK (left kite), RK (right kite). The join edges where left and right halves come together are shown with dotted lines, leaving one short edge and one long edge on each half-tile (excluding the join edge). We have shown a red dot at the vertex we regard as the origin of each half-tile (the tip of a half-dart and the base of a half-kite).

The labels are actually data constructors introduced with type operator HalfTile which has an argument type (rep) to allow for more than one representation of the half-tiles.

    data HalfTile rep 
      = LD rep -- Left Dart
      | RD rep -- Right Dart
      | LK rep -- Left Kite
      | RK rep -- Right Kite
      deriving (Show,Eq)

Tgraphs

We introduce tile graphs (Tgraphs) which provide a simple planar graph representation for finite patches of tiles. For Tgraphs we first specialise HalfTile with a triple of vertices (positive integers) to make a TileFace such as RD(1,2,3), where the vertices go clockwise round the half-tile triangle starting with the origin.

    type TileFace  = HalfTile (Vertex,Vertex,Vertex)
    type Vertex    = Int  -- must be positive

The function

    makeTgraph :: [TileFace] -> Tgraph

then constructs a Tgraph from a TileFace list after checking the TileFaces satisfy certain properties (described below). We also have

    faces :: Tgraph -> [TileFace]

to retrieve the TileFace list from a Tgraph.

As an example, the fool (short for fool’s kite and also called an ace in the literature) consists of two kites and a dart (= 4 half-kites and 2 half-darts):

    fool :: Tgraph
    fool = makeTgraph [RD (1,2,3), LD (1,3,4)   -- right and left dart
                      ,LK (5,3,2), RK (5,2,7)   -- left and right kite
                      ,RK (5,4,3), LK (5,6,4)   -- right and left kite
                      ]

To produce a diagram, we simply draw the Tgraph

    foolFigure :: Diagram B
    foolFigure = draw fool

which will produce the diagram on the left in figure 4.

Alternatively,

    foolFigure :: Diagram B
    foolFigure = labelled drawj fool

will produce the diagram on the right in figure 4 (showing vertex labels and dashed join edges).

When any (non-empty) Tgraph is drawn, a default orientation and scale are chosen based on the lowest numbered join edge. This is aligned on the positive x-axis with length 1 (for darts) or length (for kites).

Tgraph Properties

Tgraphs are actually implemented as

    newtype Tgraph = Tgraph [TileFace]
                     deriving (Show)

but the data constructor Tgraph is not exported to avoid accidentally by-passing checks for the required properties. The properties checked by makeTgraph ensure the Tgraph represents a legal tiling as a planar graph with positive vertex numbers, and that the collection of half-tile faces are both connected and have no crossing boundaries (see note below). Finally, there is a check to ensure two or more distinct vertex numbers are not used to represent the same vertex of the graph (a touching vertex check). An error is raised if there is a problem.

Note: If the TileFaces are faces of a planar graph there will also be exterior (untiled) regions, and in graph theory these would also be called faces of the graph. To avoid confusion, we will refer to these only as exterior regions, and unless otherwise stated, face will mean a TileFace. We can then define the boundary of a list of TileFaces as the edges of the exterior regions. There is a crossing boundary if the boundary crosses itself at a vertex. We exclude crossing boundaries from Tgraphs because they prevent us from calculating relative positions of tiles locally and create touching vertex problems.

For convenience, in addition to makeTgraph, we also have

    makeUncheckedTgraph :: [TileFace] -> Tgraph
    checkedTgraph   :: [TileFace] -> Tgraph

The first of these (performing no checks) is useful when you know the required properties hold. The second performs the same checks as makeTgraph except that it omits the touching vertex check. This could be used, for example, when making a Tgraph from a sub-collection of TileFaces of another Tgraph.

Main Tiling Operations

There are three key operations on finite tilings, namely

    decompose :: Tgraph -> Tgraph
    force     :: Tgraph -> Tgraph
    compose   :: Tgraph -> Tgraph

Decompose

Decomposition (also called deflation) works by splitting each half-tile into either 2 or 3 new (smaller scale) half-tiles, to produce a new tiling. The fact that this is possible, is used to establish the existence of infinite aperiodic tilings with kites and darts. Since our Tgraphs have abstracted away from scale, the result of decomposing a Tgraph is just another Tgraph. However if we wish to compare before and after with a drawing, the latter should be scaled by a factor times the scale of the former, to reflect the change in scale.

We can, of course, iterate decompose to produce an infinite list of finer and finer decompositions of a Tgraph

    decompositions :: Tgraph -> [Tgraph]
    decompositions = iterate decompose

Force

Force works by adding any TileFaces on the boundary edges of a Tgraph which are forced. That is, where there is only one legal choice of TileFace addition consistent with the seven possible vertex types. Such additions are continued until either (i) there are no more forced cases, in which case a final (forced) Tgraph is returned, or (ii) the process finds the tiling is stuck, in which case an error is raised indicating an incorrect tiling. [In the latter case, the argument to force must have been an incorrect tiling, because the forced additions cannot produce an incorrect tiling starting from a correct tiling.]

An example is shown in figure 6. When forced, the Tgraph on the left produces the result on the right. The original is highlighted in red in the result to show what has been added.

Compose

Composition (also called inflation) is an opposite to decompose but this has complications for finite tilings, so it is not simply an inverse. (See Graphs,Kites and Darts and Theorems for more discussion of the problems). Figure 7 shows a Tgraph (left) with the result of composing (right) where we have also shown (in pale green) the faces of the original that are not included in the composition – the remainder faces.

Under some circumstances composing can fail to produce a Tgraph because there are crossing boundaries in the resulting TileFaces. However, we have established that

• If g is a forced Tgraph, then compose g is defined and it is also a forced Tgraph.

Try Results

It is convenient to use types of the form Try a for results where we know there can be a failure. For example, compose can fail if the result does not pass the connected and no crossing boundary check, and force can fail if its argument is an incorrect Tgraph. In situations when you would like to continue some computation rather than raise an error when there is a failure, use a try version of a function.

    tryCompose :: Tgraph -> Try Tgraph
    tryForce   :: Tgraph -> Try Tgraph

We define Try as a synonym for Either ShowS (which is a monad) in module Tgraph.Try.

type Try a = Either ShowS a

(Note ShowS is String -> String). Successful results have the form Right r (for some correct result r) and failure results have the form Left (s<>) (where s is a String describing the problem as a failure report).

The function

    runTry:: Try a -> a
    runTry = either error id

will retrieve a correct result but raise an error for failure cases. This means we can always derive an error raising version from a try version of a function by composing with runTry.

    force = runTry . tryForce
    compose = runTry . tryCompose

Elementary Tgraph and TileFace Operations

The module Tgraph.Prelude defines elementary operations on Tgraphs relating vertices, directed edges, and faces. We describe a few of them here.

When we need to refer to particular vertices of a TileFace we use

    originV :: TileFace -> Vertex -- the first vertex - red dot in figure 2
    oppV    :: TileFace -> Vertex -- the vertex at the opposite end of the join edge from the origin
    wingV   :: TileFace -> Vertex -- the vertex not on the join edge

A directed edge is represented as a pair of vertices.

    type Dedge = (Vertex,Vertex)

So (a,b) is regarded as a directed edge from a to b.

When we need to refer to particular edges of a TileFace we use

    joinE  :: TileFace -> Dedge  -- shown dotted in figure 2
    shortE :: TileFace -> Dedge  -- the non-join short edge
    longE  :: TileFace -> Dedge  -- the non-join long edge

which are all directed clockwise round the TileFace. In contrast, joinOfTile is always directed away from the origin vertex, so is not clockwise for right darts or for left kites:

    joinOfTile:: TileFace -> Dedge
    joinOfTile face = (originV face, oppV face)

In the special case that a list of directed edges is symmetrically closed [(b,a) is in the list whenever (a,b) is in the list] we can think of this as an edge list rather than just a directed edge list.

For example,

    internalEdges :: Tgraph -> [Dedge]

produces an edge list, whereas

    boundary :: Tgraph -> [Dedge]

produces single directions. Each directed edge in the resulting boundary will have a TileFace on the left and an exterior region on the right. The function

    dedges :: Tgraph -> [Dedge]

produces all the directed edges obtained by going clockwise round each TileFace so not every edge in the list has an inverse in the list.

Note 1: There is now a class HasFaces (introduced in version 1.4) which includes instances for both Tgraph and [TileFace] and others. This allows some generalisations. For example

    faces         :: HasFaces a => a -> [TileFace]
    internalEdges :: HasFaces a => a -> [Dedge]
    boundary      :: HasFaces a => a -> [Dedge] 
    dedges        :: HasFaces a => a -> [Dedge] 
    nullFaces     :: HasFaces a => a -> Bool

Note 2: There is now a class HasGraph (introduced in version 1.8) which includes instances for Tgraph as well as other types used in forcing. This allows some other generalisations. For example

    compose :: HasGraph a => a -> Tgraph
    decompose :: HasGraph a => a -> Tgraph
    recoverGraph :: HasGraph a => a -> Tgraph

Patches (Scaled and Positioned Tilings)

Behind the scenes, when a Tgraph is drawn, each TileFace is converted to a Piece. A Piece is another specialisation of HalfTile. Since version 1.10 this uses two two-dimensional vectors to represent the drawn edges of the half-tile starting from the origin, fixing its scale and orientation. The whole Tgraph then becomes a list of located Pieces called a Patch.

    type Piece = HalfTile [V2 Double]
    type Patch = [Located Piece]

Piece drawing functions can use the drawnEdges (two vectors) and the joinVector (which is just the sum of the drawn edge vectors). In particular (in the TileLib module) we have

    drawPiece :: Piece -> Diagram B
    drawjPiece :: Piece -> Diagram B
    fillPieceDK :: Colour Double -> Colour Double -> Piece -> Diagram B

where the first draws the drawnEdges (the non-join edges) of a Piece, the second does the same but adds a faint dashed line for the join edge, and the third takes two colours – one for darts and one for kites, which are used to fill the piece as well as using drawPiece.

Patch is an instance of class Transformable so a Patch can be scaled, rotated, and translated for example.

Vertex Patches

It is useful to have an intermediate form between Tgraphs and Patches, that contains information about both the location of vertices (as 2D points), and the abstract TileFaces. This allows us to introduce labelled drawing functions (to show the vertex labels) which we then extend to Tgraphs. We call the intermediate form a VPatch (short for Vertex Patch).

    type VertexLocMap = IntMap.IntMap (Point V2 Double)
    data VPatch = VPatch {vLocs :: VertexLocMap,  vpFaces::[TileFace]} deriving Show

and

    makeVP :: HasGraph a => a -> VPatch

calculates vertex locations using a default orientation and scale.

VPatch is made an instance of class Transformable so a VPatch can also be scaled, translated, and rotated for example.

One essential use of this intermediate form is to be able to draw a Tgraph with labels, transformed but without the labels themselves being transformed. We can simply convert the Tgraph to a VPatch, and transform that before drawing with labels.

    labelled draw (rotate someAngle (makeVP g))

We can also align a VPatch using vertex labels.

    alignXaxis :: (Vertex, Vertex) -> VPatch -> VPatch 

So if g is a Tgraph with vertex labels a and b we can align it on the x-axis with a at the origin and b on the positive x-axis (after converting to a VPatch), instead of accepting the default orientation.

    labelled draw (alignXaxis (a,b) (makeVP g))

Another use of VPatches is to share the vertex location map when drawing only subsets of the faces (see Overlaid examples in the next section).

4. Drawing in More Detail

Class Drawable

There is a class Drawable with instances Tgraph, VPatch, Patch. When the token B is in scope standing for a fixed backend then we can assume

    draw   :: Drawable a => a -> Diagram B  -- draws non-join edges
    drawj  :: Drawable a => a -> Diagram B  -- as with draw but also draws dashed join edges
    fillDK :: Drawable a => Colour Double -> Colour Double -> a -> Diagram B -- fills with colours

where fillDK clr1 clr2 will fill darts with colour clr1 and kites with colour clr2 as well as drawing non-join edges.

These are the main drawing tools. However they are actually defined for any suitable backend b so have more general types.

(Update Sept 2024) From version 1.1 onwards of PenroseKiteDart, these are

    draw ::   (Drawable a, OKBackend b) =>
              a -> Diagram b
    drawj ::  (Drawable a, OKBackend) b) =>
              a -> Diagram b
    fillDK :: (Drawable a, OKBackend b) =>
              Colour Double -> Colour Double -> a -> Diagram b

where the class OKBackend is a check to ensure a backend is suitable for drawing 2D tilings with or without labels.

In these notes we will generally use the simpler description of types using B for a fixed chosen backend for the sake of clarity.

The drawing tools are each defined via the class function drawWith using Piece drawing functions.

    class Drawable a where
        drawWith :: (Piece -> Diagram B) -> a -> Diagram B
    
    draw = drawWith drawPiece
    drawj = drawWith drawjPiece
    fillDK clr1 clr2 = drawWith (fillPieceDK clr1 clr2)

To design a new drawing function, you only need to implement a function to draw a Piece, (let us call it newPieceDraw)

    newPieceDraw :: Piece -> Diagram B

This can then be elevated to draw any Drawable (including Tgraphs, VPatches, and Patches) by applying the Drawable class function drawWith:

    newDraw :: Drawable a => a -> Diagram B
    newDraw = drawWith newPieceDraw

Class DrawableLabelled

Class DrawableLabelled is defined with instances Tgraph and VPatch, but Patch is not an instance (because this does not retain vertex label information).

    class DrawableLabelled a where
        labelColourSize :: Colour Double -> Measure Double -> (Patch -> Diagram B) -> a -> Diagram B

So labelColourSize c m modifies a Patch drawing function to add labels (of colour c and size measure m). Measure is defined in Diagrams.Prelude with pre-defined measures tiny, verySmall, small, normal, large, veryLarge, huge. For most of our diagrams of Tgraphs, we use red labels and we also find small is a good default size choice, so we define

    labelSize :: DrawableLabelled a => Measure Double -> (Patch -> Diagram B) -> a -> Diagram B
    labelSize = labelColourSize red

    labelled :: DrawableLabelled a => (Patch -> Diagram B) -> a -> Diagram B
    labelled = labelSize small

and then labelled draw, labelled drawj, labelled (fillDK clr1 clr2) can all be used on both Tgraphs and VPatches as well as (for example) labelSize tiny draw, or labelCoulourSize blue normal drawj.

Further drawing functions

There are a few extra drawing functions built on top of the above ones. The function smart is a modifier to add dashed join edges only when they occur on the boundary of a Tgraph

    smart :: HasGraph a => (VPatch -> Diagram B) -> a -> Diagram B

So smart vpdraw g will draw dashed join edges on the boundary of g before applying the drawing function vpdraw to the VPatch for g. For example the following all draw dashed join edges only on the boundary for a Tgraph g

    smart draw g
    smart (labelled draw) g
    smart (labelSize normal draw) g

When using labels, the function rotating allows a Tgraph to be drawn rotated without rotating the labels.

    rotating :: HasGraph a => Angle Double -> (VPatch -> b) -> a -> b
    rotating angle vpdraw = vpdraw . rotate angle . makeVP

So for example,

    rotating (90@@deg) (labelled draw) g

makes sense for a Tgraph g. Of course if there are no labels we can simply use

    rotate (90@@deg) (draw g)

Similarly aligning allows a Tgraph to be aligned on the X-axis using a pair of vertex numbers before drawing.

    aligning :: HasGraph a => (Vertex,Vertex) -> (VPatch -> b) -> a -> b
    aligning (a,b) vpdraw = vpdraw . alignXaxis (a,b) . makeVP

So, for example, if Tgraph g has vertices a and b, both

    aligning (a,b) draw g
    aligning (a,b) (labelled draw) g

make sense. Note that the following two examples are wrong. Even though they type check, they re-orient g without repositioning the boundary joins.

    smart (labelled draw . rotate angle) g      -- WRONG
    smart (labelled draw . alignXaxis (a,b)) g  -- WRONG

Instead use

    smartRotating angle (labelled draw) g
    smartAligning (a,b) (labelled draw) g

where

    smartRotating :: HasGraph a =>  Angle Double -> (VPatch -> Diagram B) -> a -> Diagram B
    smartAligning  :: HasGraph a => (Vertex,Vertex) -> (VPatch -> Diagram B) -> a -> Diagram B

are defined using

    smartOn :: HasGraph a => a -> (VPatch -> Diagram B) -> VPatch -> Diagram B

Here, smartOn g vpdraw vp uses the given vp for drawing boundary joins and drawing faces of g (with vpdraw) rather than converting g to a new VPatch. This assumes vp has locations for vertices in g.

Overlaid examples (location map sharing)

The function

    drawForce :: Tgraph -> Diagram B

will (smart) draw a Tgraph g in red overlaid (using <>) on the result of force g as in figure 6. Similarly

    drawPCompose  :: Tgraph -> Diagram B

applied to a Tgraph g will draw the result of a partial composition of g as in figure 7. That is a drawing of compose g but overlaid with a drawing of the remainder faces of g shown in pale green.

Both these functions make use of sharing a vertex location map to get correct alignments of overlaid diagrams. In the case of drawForce g, we know that a VPatch for force g will contain all the vertex locations for g since force only adds to a Tgraph (when it succeeds). So when constructing the diagram for g we can use the VPatch created for force g instead of starting afresh. Similarly for drawPCompose g the VPatch for g contains locations for all the vertices of compose g so compose g is drawn using the VPatch for g instead of starting afresh.

The location map sharing is done with

    subFaces :: HasFaces a => 
                a -> VPatch -> VPatch

so that subFaces fcs vp is a VPatch with the same vertex locations as vp, but replacing the faces of vp with fcs. [Of course, this can go wrong if the new faces have vertices not in the domain of the vertex location map so this needs to be used with care. Any errors would only be discovered when a diagram is created.]

For cases where labels are only going to be drawn for certain faces, we need a version of subFaces which also gets rid of vertex locations that are not relevant to the faces. For this situation we have

    restrictTo:: HasFaces a => 
                 a -> VPatch -> VPatch

which filters out un-needed vertex locations from the vertex location map. Unlike subFaces, restrictTo checks for missing vertex locations, so restrictTo fcs vp raises an error if a vertex in fcs is missing from the keys of the vertex location map of vp.

5. Forcing in More Detail

The force rules

The rules used by our force algorithm are local and derived from the fact that there are seven possible (internal) vertex types as depicted in figure 8.

Our rules are shown in figure 9 (omitting mirror symmetric versions). In each case the TileFace shown yellow needs to be added in the presence of the other TileFaces shown with the red edge on the boundary.

Main Forcing Operations

To make forcing efficient we convert a Tgraph to a BoundaryState to keep track of boundary information of the Tgraph, and then calculate a ForceState which combines the BoundaryState with a record of awaiting boundary edge updates (an update map), and an UpdateGenerator. Then each face addition is carried out on a ForceState, converting back when all the face additions are complete. It makes sense to apply force (and related functions) to a Tgraph, a BoundaryState, or a ForceState, so we define a class Forcible with instances Tgraph, BoundaryState, and ForceState.

This allows us to define

    force :: Forcible a => a -> a
    tryForce :: Forcible a => a -> Try a

The first will raise an error if a stuck tiling is encountered. The second uses a Try result which produces a Left report for failures and a Right a for successful result a.

There are several other operations related to forcing including

    stepForce :: Forcible a => Int -> a -> a
    tryStepForce  :: Forcible a => Int -> a -> Try a

    addHalfDart, addHalfKite :: Forcible a => Dedge -> a -> a
    tryAddHalfDart, tryAddHalfKite :: Forcible a => Dedge -> a -> Try a

The first two force (up to) a given number of steps (=face additions) and the other four add a half dart/kite on a given boundary edge.

Update Generators

An update generator is used to calculate which boundary edges can have a certain update. There is an update generator for each force rule, but also a combined (all update) generator. The force operations mentioned above all use the default all update generator (defaultAllUGen) but there are more general (with) versions that can be passed an update generator of choice. For example

    forceWith :: Forcible a => UpdateGenerator -> a -> a
    tryForceWith :: Forcible a => UpdateGenerator -> a -> Try a

We can also define

    wholeTiles :: Forcible a => a -> a
    wholeTiles = forceWith wholeTileUpdates

where wholeTileUpdates is an update generator that just finds boundary join edges to complete whole tiles.

In fact UpdateGenerators are functions that take a BoundaryState and a focus (list of boundary directed edges) to produce an update map. Each Update is calculated as either a SafeUpdate (where two of the new face edges are on the existing boundary and no new vertex is needed) or an UnsafeUpdate (where only one edge of the new face is on the boundary and a new vertex needs to be created for a new face).

    type UpdateGenerator = BoundaryState -> [Dedge] -> Try UpdateMap
    type UpdateMap = Map.Map Dedge Update
    data Update = SafeUpdate TileFace 
                | UnsafeUpdate (Vertex -> TileFace)

Completing (executing) an UnsafeUpdate requires a touching vertex check to ensure that the new vertex does not clash with an existing boundary vertex. Using an existing (touching) vertex would create a crossing boundary so such an update has to be blocked.

Forcible Class Operations

The Forcible class operations are higher order and designed to allow for easy additions of further generic operations. They take care of conversions between Tgraphs, BoundaryStates and ForceStates. The first two are designed to create functions that return the same Forcible type as the input.

    class Forcible a where
      tryFSOp :: (ForceState -> Try ForceState) -> a -> Try a
      tryChangeBoundary :: (BoundaryState -> Try BoundaryChange) -> a -> Try a
      tryInitFS :: a -> Try ForceState

For example, given any f:: ForceState -> Try ForceState , then f can be generalised to work on any Forcible using tryFSOp f. This is used to define both tryForce and tryStepForce.

Similarly given any f:: BoundaryState -> Try BoundaryChange , then f can be generalised to work on any Forcible using tryChangeBoundary f. This is used to define tryAddHalfDart and tryAddHalfKite.

Note that the type BoundaryChange contains a resulting BoundaryState, the single TileFace that has been added, a list of edges removed from the boundary (of the BoundaryState prior to the face addition), and a list of the (3 or 4) boundary edges affected around the change that require checking or re-checking for updates.

The class function tryInitFS will create an initial ForceState for any Forcible. If the Forcible is already a ForceState it will do nothing. Otherwise it will calculate updates for the whole boundary using defaultAllUGen.

The update generator is assumed to be defaultAllUGen but this can be changed using

    tryFSOpWith :: Forcible a => UpdateGenerator -> (ForceState -> Try ForceState) -> a -> Try a

so, for example, we defined

    tryForceWith ugen = tryFSOpWith ugen tryForce

Efficient chains of forcing operations.

Note that (force . force) does the same as force, but we might want to chain other force related steps in a calculation.

For example, consider the following combination which, after decomposing a Tgraph, forces, then adds a half dart on a given boundary edge (d) and then forces again.

    combo :: Dedge -> Tgraph -> Tgraph
    combo d = force . addHalfDart d . force . decompose

Since decompose produces a Tgraph, the instances of force and addHalfDart d will have type Tgraph -> Tgraph so each of these operations, will begin and end with conversions between Tgraph and ForceState. We would do better to avoid these wasted intermediate conversions working only with ForceStates and keeping only those necessary conversions at the beginning and end of the whole sequence.

This can be done using tryFSOp. To see this, let us first re-express the forcing sequence using the Try monad, so

    force . addHalfDart d . force

becomes

    tryForce <=< tryAddHalfDart d <=< tryForce

Note that (<=<) is the Kliesli arrow which replaces composition for Monads (defined in Control.Monad). (We could also have expressed this right to left sequence with a left to right version tryForce >=> tryAddHalfDart d >=> tryForce). The definition of combo becomes

    combo :: Dedge -> Tgraph -> Tgraph
    combo d = runTry . (tryForce <=< tryAddHalfDart d <=< tryForce) . decompose

This has no performance improvement, but now we can pass the sequence to tryFSOp to remove the unnecessary conversions between steps.

    combo :: Dedge -> Tgraph -> Tgraph
    combo d = runTry . tryFSOp (tryForce <=< tryAddHalfDart d <=< tryForce) . decompose

The sequence actually has type Forcible a => a -> Try a but when passed to tryFSOp it specialises to type ForceState -> Try ForseState. This ensures the sequence works on a ForceState and any conversions are confined to the beginning and end of the sequence, avoiding unnecessary intermediate conversions.

A limitation of forcing

To avoid creating touching vertices (or crossing boundaries) a BoundaryState keeps track of locations of boundary vertices. At around 35,000 face additions in a single force operation the calculated positions of boundary vertices can become too inaccurate to prevent touching vertex problems. In such cases it is better to use

    recalibratingForce :: Forcible a => a -> a
    tryRecalibratingForce :: Forcible a => a -> Try a

These work by recalculating all vertex positions at 20,000 step intervals to get more accurate boundary vertex positions. For example, 6 decompositions of the kingGraph has 2,906 faces. Applying force to this should result in 53,574 faces but will go wrong before it reaches that. This can be fixed by calculating either

    recalibratingForce (decompositions kingGraph !!6)

or using an extra force before the decompositions

    force (decompositions (force kingGraph) !!6)

In the latter case, the final force only needs to add 17,864 faces to the 35,710 produced by decompositions (force kingGraph) !!6.

6. Advanced Operations

Guided comparison of Tgraphs

Asking if two Tgraphs are equivalent (the same apart from choice of vertex numbers) is a an np-hard problem. However, we do have an efficient guided way of comparing Tgraphs. In the module Tgraph.Rellabelling we have

    sameGraph :: (Tgraph,Dedge) -> (Tgraph,Dedge) -> Bool

The expression sameGraph (g1,d1) (g2,d2) asks if g2 can be relabelled to match g1 assuming that the directed edge d2 in g2 is identified with d1 in g1. Hence the comparison is guided by the assumption that d2 corresponds to d1.

It is implemented using

    tryRelabelToMatch :: (Tgraph,Dedge) -> (Tgraph,Dedge) -> Try Tgraph

where tryRelabelToMatch (g1,d1) (g2,d2) will either fail with a Left report if a mismatch is found when relabelling g2 to match g1 or will succeed with Right g3 where g3 is a relabelled version of g2. The successful result g3 will match g1 in a maximal tile-connected collection of faces containing the face with edge d1 and have vertices disjoint from those of g1 elsewhere. The comparison tries to grow a suitable relabelling by comparing faces one at a time starting from the face with edge d1 in g1 and the face with edge d2 in g2. (This relies on the fact that Tgraphs are connected with no crossing boundaries, and hence tile-connected.)

The above function is also used to implement

    tryFullUnion:: (Tgraph,Dedge) -> (Tgraph,Dedge) -> Try Tgraph

which tries to find the union of two Tgraphs guided by a directed edge identification. However, there is an extra complexity arising from the fact that Tgraphs might overlap in more than one tile-connected region. After calculating one overlapping region, the full union uses some geometry (calculating vertex locations) to detect further overlaps.

Finally we have

    commonFaces:: (Tgraph,Dedge) -> (Tgraph,Dedge) -> [TileFace]

which will find common regions of overlapping faces of two Tgraphs guided by a directed edge identification. The resulting common faces will be a sub-collection of faces from the first Tgraph. These are returned as a list as they may not be a connected collection of faces and therefore not necessarily a Tgraph.

Empires and SuperForce

In Empires and SuperForce we discussed forced boundary coverings which were used to implement both a superForce operation

    superForce:: Forcible a => a -> Forced a

and operations to calculate empires.

We will not repeat the descriptions here other than to note that

    forcedBoundaryECovering:: Tgraph -> [Forced Tgraph]

finds boundary edge coverings after forcing a Tgraph. That is, forcedBoundaryECovering g will first force g, then (if it succeeds) finds a collection of (forced) extensions to force g such that

• each extension has the whole boundary of force g as internal edges.
• each possible addition to a boundary edge of force g (kite or dart) has been included in the collection.

(possible here means – not leading to a stuck Tgraph when forced.) There is also

    forcedBoundaryVCovering:: Tgraph -> [Forced Tgraph]

which does the same except that the extensions have all boundary vertices internal rather than just the boundary edges. In both cases the result is a list of explicitly forced Tgraphs (discussed next).

Combinations and Explicitly Forced

We introduced a new type Forced (in v 1.3) to enable a forcible to be explictily labelled as being forced. For example

    forceF    :: Forcible a => a -> Forced a 
    tryForceF :: Forcible a => a -> Try (Forced a)
    forgetF   :: Forced a -> a

This allows us to restrict certain functions which expect a forced argument by making this explicit.

    composeF :: HasGraph a => Forced a -> Forced Tgraph

The definition makes use of theorems established in Graphs,Kites and Darts and Theorems that composing a forced Tgraph does not require a check (for connectedness and no crossing boundaries) and the result is also forced. This can then be used to define efficient combinations such as

    compForce:: (Forcible a, HasGraph a) => a -> Forced Tgraph      -- compose after forcing
    compForce = composeF . forceF

    allCompForce:: (Forcible a, HasGraph a) => a -> [Forced Tgraph] -- iterated (compose after force) while not emptyTgraph
    maxCompForce:: (Forcible a, HasGraph a) => a -> Forced Tgraph   -- last item in allCompForce (or emptyTgraph)

Note that BoundaryState, ForceState as well as Tgraph and Forced versions of these are all instances of class HasGraph.

Tracked Tgraphs

The type

    data TrackedTgraph = TrackedTgraph
       { tgraph  :: Tgraph
       , tracked :: [[TileFace]] 
       } deriving Show

has proven useful in experimentation as well as in producing artwork with darts and kites. The idea is to keep a record of sub-collections of faces of a Tgraph when doing both force operations and decompositions. A list of the sub-collections forms the tracked list associated with the Tgraph. We make TrackedTgraph an instance of class Forcible by having force operations only affect the Tgraph and not the tracked list. The significant idea is the implementation of

    decomposeTracked :: TrackedTgraph -> TrackedTgraph

Decomposition of a Tgraph involves introducing a new vertex for each long edge and each kite join. These are then used to construct the decomposed faces. For decomposeTracked we do the same for the Tgraph, but when it comes to the tracked collections, we decompose them re-using the same new vertex numbers calculated for the edges in the Tgraph. This keeps a consistent numbering between the Tgraph and tracked faces, so each item in the tracked list remains a sub-collection of faces in the Tgraph.

The function

    drawTrackedTgraph :: [VPatch -> Diagram B] -> TrackedTgraph -> Diagram B

is used to draw a TrackedTgraph. It uses a list of functions to draw VPatches. The first drawing function is applied to a VPatch for any untracked faces. Subsequent functions are applied to VPatches for the tracked list in order. Each diagram is beneath later ones in the list, with the diagram for the untracked faces at the bottom. The VPatches used are all restrictions of a single VPatch for the Tgraph, so will be consistent in vertex locations. When labels are used, there is also a drawTrackedTgraphRotating and drawTrackedTgraphAligning for rotating or aligning the VPatch prior to applying the drawing functions.

Note that the result of calculating empires (see Empires and SuperForce ) is represented as a TrackedTgraph. The result is actually the common faces of a forced boundary covering, but a particular element of the covering (the first one) is chosen as the background Tgraph with the common faces as a tracked sub-collection of faces. Hence we have

    empire1, empire2 :: Tgraph -> TrackedTgraph
    
    drawEmpire :: TrackedTgraph -> Diagram B

Figure 10 was also created using TrackedTgraphs.

7. Other Reading

Previous related blogs are:

• Diagrams for Penrose Tiles – the first blog introduced drawing Pieces and Patches (without using Tgraphs) and provided a version of decomposing for Patches (decompPatch).
• Graphs, Kites and Darts intoduced Tgraphs. This gave more details of implementation and results of early explorations. (The class Forcible was introduced subsequently).
• Empires and SuperForce – these new operations were based on observing properties of boundaries of forced Tgraphs.
• Graphs,Kites and Darts and Theorems established some important results relating force, compose, decompose.

by readerunner at October 16, 2026 05:15 PM

QuickCheck: shrinking like tuples, writing like applicative

The QuickCheck documentation for Arbitrary says that

There is no generic arbitrary implementation included because we don't know how to make a high-quality one.

Though in practice I've found that very often it suffices to write one that uses the fact that Gen is an Applicative. For instance, this type

data Foo = Foo
    { tata :: Int
    , titi :: Double
    }

would get an implementation that looks like this

instance Arbitrary Foo where
    arbitrary = Foo <$> arbitrary <*> arbitrary

When it comes to writing the function for shrinking Foo it is possible to rely on a generic function and the documentation says that

The default implementation returns the empty list, so will not try to shrink the value. If your data type has no special invariants, you can enable shrinking by defining shrink = genericShrink, but by customising the behaviour of shrink you can often get simpler counterexamples.

Based on some manual testing it seems that genericShrink generates something like this for the type Foo above

instance Arbitrary Foo where
    shrink (Foo tata_ titi_) = Foo <$> shrink tata_ <*> shrink titi_

which means that if shrink tata_ generates a list of m elements and shrink titi_ generates a list of n elements, then shrink foo will generate a list of m×n elements. It's clear that the size can become problematic with more fields. It also means that it's very easy to get an empty list – if shrink v results in an empty list, then shrink foo is also empty. Oh, and shrink Nothing is []!

A colleague pointed out that tuples have a rather efficient implementation of shrink that grows linearly with the number of elements and doesn't reduce to the empty list as easily. I could use it for Foo like this

instance Arbitrary Foo where
    shrink (Foo tata_ titi_) = uncurry Foo <$> shrink (tata_, titi_)

However, that doesn't scale that well to more fields. Up to a point I can use list comprehensions, but it gets a bit difficult to read, I find. On top of that, the largest tuple with an Arbitrary instance has 10 elements, and then I'll have to either define more or use inner tuples.

Wouldn't it be nice to write shrink in something that looks Applicative, but behaves like tuples?

Here's one approach to this.

First a type to hold information about what's being shrunk, or in this case waned

data Wane a = Wane a [a]
    deriving (Generic, Show)

instance Functor Wane where
    fmap f (Wane a as) = Wane (f a) (f <$> as)

Then a couple of functions to bring values in and out, unWane to get the list of values out and wane to wrap a value in Wane

unWane :: Wane a -> [a]
unWane (Wane _ as) = as

wane :: (Arbitrary a) => a -> Wane a
wane a = Wane a (shrink a)

Finally a way to combine two Wane values is needed, and here I was inspired by the implementation of Arbitrary for (a,b)

infixl 4 \*/

(\*/) ::  Wane (a -> b) -> Wane a -> Wane b
(\*/) (Wane f fs) (Wane a as) =  Wane (f a) ([f a' | a' <- as] <> [f' a| f' <- fs])

It's now possible to write shrink for Foo like this

instance Arbitrary Foo where
    shrink (Foo tata_ titi_) = unWane $ Foo <$> wane tata_ \*/ wane titi_

That scales to more fields just as nicely as applicative style, and the result matches what the variant using tuples produces (same elements, but the order is different).

Tags: haskell quickcheck

October 13, 2026 09:21 PM

Emacs salmagundi, 2026-10-11

Combining lists of compilation commands

To my delight I found that emacs-multi-compile will combine the of all matches compilation commands!

I added this to my setup

(add-to-list 'multi-compile-alist
               '((mes/is-in-git-project) . (("git clean -n" "git clean -ndx" (project-root (project-current)))
                                            ("git clean -f" "git clean -fdx" (project-root (project-current))))))

and those commands are now shown together with the ones I defined for Haskell project (see my previous post) when both match.

Recognising a git project

To add the git commands to multi-compile-list I needed a function to determine if the current buffer is in a git working tree. I failed to find something fitting so I ended up adding this little function.

(defun mes/is-in-git-project ()
  "Check if we're currently in a git project."
  (when-let* ((proj-dir (project-root (project-current))))
    (vc-git-root proj-dir)))

Taming compilation mode buffers

I'm using display-buffer-alist to control how Emacs places buffers. For compilation mode I had the following actions.

(display-buffer-reuse-mode-window display-buffer-in-side-window)
(mode compilation-mode)
(window-height . 0.25)
(side . bottom)
(post-command-select-window . t)

That last command made switching buffers with consult-buffer a bit irritating as previewing would interrupt the buffer switching by selecting the compilation window. However, I really want the compilation window selected after a compilation. Or rather, as I realised after reflecting a bit, I want the compilation window selected if there are errors.

Compilation mode has a way of running functions when a compilation finishes so I defined two functions and added them to compilation-finish-functions.

(defun mes/close-successful-compilation (buf msg)
  "Close compilation buffer after success."
  (when (and (buffer-live-p buf)
             (with-current-buffer buf (eq major-mode 'compilation-mode))
             (string-match "finished" msg))
    (run-with-timer 3 nil
                    (lambda ()
                      (kill-buffer buf)))))

(defun mes/select-failed-compilation (buf msg)
  "Select compilation buffer after failed compilation."
  (when (and (buffer-live-p buf)
             (with-current-buffer buf (eq major-mode 'compilation-mode))
             (string-match "abnormally" msg))
    (select-window (get-buffer-window buf))))

  (add-to-list 'compilation-finish-functions #'mes/close-successful-compilation)
  (add-to-list 'compilation-finish-functions #'mes/select-failed-compilation)

With that a successful compilation is closed automatically after 3s and on a failed compilation the compilation buffer is selection.

Tags: emacs

October 13, 2026 07:46 PM

Graphs, Kites and Darts – and Theorems

(This is a revised version Oct 2026 with corrections for lemma 1 and proof of lemma 3).

We continue our exploration of properties of Penrose’s aperiodic tilings with kites and darts using Haskell and Haskell Diagrams.

In this blog we discuss some interesting properties we have discovered concerning the , , and operations along with some proofs.

Index

1. Quick Recap (including operations , , on Tgraphs)
2. Composition Problems and a Compose Force Theorem (composition is not a simple inverse to decomposition)
3. Perfect Composition Theorem (establishing relationships between , , )
4. Multiple Compositions (extending the Compose Force theorem for multiple compositions)
5. Proof of the Compose Force Theorem (showing is total on forced Tgraphs)

1. Quick Recap

Haskell diagrams allowed us to render finite patches of tiles easily as discussed in Diagrams for Penrose tiles. Following a suggestion of Stephen Huggett, we found that the description and manipulation of such tilings is greatly enhanced by using planar graphs. In Graphs, Kites and Darts we introduced a specialised planar graph representation for finite tilings of kites and darts which we called Tgraphs (tile graphs). These enabled us to implement operations that use neighbouring tile information and in particular operations , , and .

For ease of reference, we reproduce the half-tiles we are working with here.

Figure 1 shows the right-dart (RD), left-dart (LD), left-kite (LK) and right-kite (RK) half-tiles. Each has a join edge (shown dotted) and a short edge and a long edge. The origin vertex is shown red in each case. The vertex at the opposite end of the join edge from the origin we call the opp vertex, and the remaining vertex we call the wing vertex.

If the short edges have unit length then the long edges have length (the golden ratio) and all angles are multiples of (a tenth turn) with kite halves having two 2s and a 1, and dart halves having a 3 and two 1s. This geometry of the tiles is abstracted away from at the graph representation level but used when checking validity of tile additions and by the drawing functions.

There are rules for how the tiles can be put together to make a legal tiling (see e.g. Diagrams for Penrose tiles). We defined a Tgraph (in Graphs, Kites and Darts) as a list of such half-tiles which are constrained to form a legal tiling but must also be connected with no crossing boundaries (see below).

As a simple example consider kingGraph (2 kites and 3 darts round a king vertex). We represent each half-tile as a TileFace with three vertex numbers, then apply makeTgraph to the list of ten Tilefaces. The function makeTgraph :: [TileFace] -> Tgraph performs the necessary checks to ensure the result is a valid Tgraph.

kingGraph :: Tgraph
kingGraph = makeTgraph 
  [LD (1,2,3),RD (1,11,2),LD (1,4,5),RD (1,3,4),LD (1,10,11)
  ,RD (1,9,10),LK (9,1,7),RK (9,7,8),RK (5,7,1),LK (5,6,7)
  ]

To view the Tgraph we simply form a diagram (in this case 2 diagrams horizontally separated by 1 unit)

  hsep 1 [labelled drawj kingGraph, draw kingGraph]

and the result is shown in figure 2 with labels and dashed join edges (left) and without labels and join edges (right).

The boundary of the Tgraph consists of the edges of half-tiles which are not shared with another half-tile, so they go round untiled/external regions. The no crossing boundary constraint (equivalently, locally tile-connected) means that a boundary vertex has exactly two incident boundary edges and therefore has a single external angle in the tiling. This ensures we can always locally determine the relative angles of tiles at a vertex. We say a collection of half-tiles is a valid Tgraph if it constitutes a legal tiling but also satisfies the connectedness and no crossing boundaries constraints.

Our key operations on Tgraphs are , , and which are illustrated in figure 3.

Figure 3 shows the kingGraph with its decomposition above it (left), the result of forcing the kingGraph (right) and the composition of the forced kingGraph (bottom right).

Decompose

An important property of Penrose dart and kite tilings is that it is possible to divide the half-tile faces of a tiling into smaller half-tile faces, to form a new (smaller scale) tiling.

Figure 4 illustrates the decomposition of a left-dart (top row) and a left-kite (bottom row). With our Tgraph representation we simply introduce new vertices for dart and kite long edges and kite join edges and then form the new faces using these. This does not involve any geometry, because that is taken care of by drawing operations.

Force

Figure 5 illustrates the rules used by our operation (we omit a mirror-reflected version of each rule).

In each case the yellow half-tile is added in the presence of the other half-tiles shown where the red edge is on the boundary. The yellow half-tile is forced because, by the legal tiling rules and the seven possible vertex types, there is no choice for adding a different half-tile on the red edge.

We call a Tgraph correct if it represents a tiling which can be continued infinitely to cover the whole plane without getting stuck, and incorrect otherwise. Forcing involves adding half-tiles by the illustrated rules round the boundary until either no more rules apply (in which case the result is a forced Tgraph) or a stuck tiling is encountered (in which case an incorrect Tgraph error is raised). Hence is a partial function but total on correct Tgraphs.

Compose: This is discussed in the next section.

2. Composition Problems and a Theorem

Compose Choices

For an infinite tiling, composition is a simple inverse to decomposition. However, for a finite tiling with boundary, composition is not so straight forward. Firstly, we may need to leave half-tiles out of a composition because the necessary parts of a composed half-tile are missing. For example, a half-dart with a boundary short edge or a whole kite with both short edges on the boundary must necessarily be excluded from a composition. Secondly, on the boundary, there can sometimes be a problem of choosing whether a half-dart should compose to become a half-dart or a half-kite. This choice in composing only arises when there is a half-dart with its wing on the boundary but insufficient local information to determine whether it should be part of a larger half-dart or a larger half-kite.

In the literature (see for example 1 and 2) there is an often repeated method for composing (also called inflating). This method always make the kite choice when there is a choice. Whilst this is a sound method for an unbounded tiling (where there will be no choice), we show that this is an unsound method for finite tilings as follows.

Clearly composing should preserve correctness. However, figure 6 (left) shows a correct Tgraph which is a forced queen, but the kite-favouring composition of the forced queen produces the incorrect Tgraph shown in figure 6 (centre). Applying our function to this reveals a stuck tiling and reports an incorrect Tgraph.

Our algorithm (discussed in Graphs, Kites and Darts) detects dart wings on the boundary where there is a choice and classifies them as unknowns. Our composition refrains from making a choice by not composing a half dart with an unknown wing vertex. The rightmost Tgraph in figure 6 shows the result of our composition of the forced queen with the half-tile faces left out of the composition (the remainder faces) shown in green. This avoidance of making a choice (when there is a choice) guarantees our composition preserves correctness.

Compose is a Partial Function

A different composition problem can arise when we consider Tgraphs that are not decompositions of Tgraphs. In general, is a partial function on Tgraphs.

Figure 7 shows a Tgraph (left) with its sucessful composition (centre) and the half-tile faces that would result from a second composition (right) which do not form a valid Tgraph because of a crossing boundary (at vertex 6). Thus composition of a Tgraph may fail to produce a Tgraph when the resulting faces are disconnected or have a crossing boundary.

However, we claim that is a total function on forced Tgraphs.

Compose Force Theorem

Theorem: Composition of a forced Tgraph produces a valid Tgraph.

We postpone the proof (outline) for this theorem to section 5. Meanwhile we use the result to establish relationships between , , and in the next section.

3. Perfect Composition Theorem

In Graphs, Kites and Darts we produced a diagram showing relationships between multiple decompositions of a dart and the forced versions of these Tgraphs. We reproduce this here along with a similar diagram for multiple decompositions of a kite.

In figure 8 we show separate (apparently) commuting diagrams for the dart and for the kite. The bottom rows show the decompositions, the middle rows show the result of forcing the decompositions, and the top rows illustrate how the compositions of the forced Tgraphs work by showing both the composed faces (black edges) and the remainder faces (green edges) which are removed in the composition. The diagrams are examples of some commutativity relationships concerning , and which we will prove.

It should be noted that these diagrams break down if we consider only half-tiles as the starting points (bottom right of each diagram). The decomposition of a half-tile does not recompose to its original, but produces an empty composition. So we do not even have in these cases. Forcing the decomposition also results in an empty composition. Clearly there is something special about the depicted cases and it is not merely that they are wholetile complete because the decompositions are not wholetile complete. [Wholetile complete means there are no join edges on the boundary, so every half-tile has its other half.]

Below we have captured the properties that are sufficient for the diagrams to commute as in figure 8. In the proofs we use a partial ordering on Tgraphs (modulo vertex relabelling) which we define next.

Partial ordering of Tgraphs

If and are both valid Tgraphs and consists of a subset of the (half-tile) faces of we have

which gives us a partial order on Tgraphs. Often, though, is only isomorphic to a subset of the faces of , requiring a vertex relabelling to become a subset. In that case we write

which is also a partial ordering and induces an equivalence of Tgraphs defined by

in which case and are isomorphic as Tgraphs.

Both and are monotonic with respect to meaning:

We also have is monotonic, but only when restricted to correct Tgraphs. Also, when restricted to correct Tgraphs, we have is non decreasing because it only adds faces:

and is idempotent (forcing a forced correct Tgraph leaves it the same):

Composing perfectly and perfect compositions

Definition: A Tgraph composes perfectly if all faces of are composable (i.e there are no remainder faces of when composing).

We note that the composed faces must be a valid Tgraph (connected with no crossing boundaries) if all faces are included in the composition because has those properties. Clearly, if composes perfectly then

In general, for arbitrary where the composition is defined, we only have

Definition: A Tgraph is a perfect composition if composes perfectly.

Clearly if is a perfect composition then

(We could use equality here because any new vertex labels introduced by will be removed by ). In general, for arbitrary ,

Lemma 1: is a perfect composition if has the following 2 properties:

1. every half-kite with a boundary join has either a half-dart or a whole kite on the short edge, and
2. every half-dart with a boundary join has a half-kite on the short edge,

(Proof outline:) Firstly note that unknowns in (= ) can only come from boundary joins in . The properties 1 and 2 guarantee that has no unknowns (the boundary dart wings can be classified as ace or jack vertices). Since every face of has come from a decomposed face in , there can be no faces in that will not recompose, so will compose perfectly to .

(Note: a perfect composition may have unknowns even though its decomposition has none.)

It is easy to see two special cases:

1. If is wholetile complete then is a perfect composition.

   Proof: Wholetile complete implies no boundary joins which implies properties 1 and 2 in lemma 1 which implies is a perfect composition.

2. If is a decomposition then is a perfect composition.

   Proof: If is a decomposition, then every half-dart has a half-kite on the short edge which implies property 2 of lemma 1. Also, any half-kite with a boundary join in must have come from a decomposed half-dart since a decomposed half-kite produces a whole kite with no boundary kite join. So the half-kite must have a half-dart on the short edge which implies property 1 of lemma 1. The two properties imply is a perfect composition.

We note that these two special cases cover all the Tgraphs in the bottom rows of the diagrams in figure 8. So the Tgraphs in each bottom row are perfect compositions, and furthermore, they all compose perfectly except for the rightmost Tgraphs which have empty compositions.

In the following results we make the assumption that a Tgraph is correct, which guarantees that when is applied, it terminates with a correct Tgraph. We also note that preserves correctness as does (provided the composition is defined).

Lemma 2: If is a forced, correct Tgraph then

(Proof outline:) The proof uses a case analysis of boundary and internal vertices of . For internal vertices we just check there is no change at the vertex after using figure 12 (plus an extra case for the forced star). For boundary vertices we check the local context cases shown in figure 9.

This shows two cases for a kite origin, two cases for a kite opp, four cases for a kite wing, and four cases for a dart origin. The only case for a dart wing is one of the two kite opp cases and there are no dart opp cases as these cannot be on the boundary of a forced Tgraph. Actually figure 9 has a repeated case which is both a dart origin and a kite wing, but there are also 3 additional cases when we consider mirror images of those shown. Since there is no local change of the context in each case, and since this is true for all boundary vertices in any forced Tgraph, there can be no non-local change either. (We omit the full details).

Lemma 3: If is a perfect composition and a correct Tgraph, then

(Proof outline:) First we establish that when is a perfect composition

The difference between and lies in faces round the boundary dart wings of the former which have come from decomposing at boundary joins of . Since is a perfect composition, can have no unknowns, so forcing will add all faces at these dart wings and the result will be the same as

Applying to both sides (which are forced Tgraphs) gives

Next we establish that for any wholetile complete we have

The proof uses induction on force steps applied to where we can assume the context is wholetile complete. For each force rule we establish that contains both the new halftile and its other half. [A separate proof is needed to show that the ordering of applying force rules makes no difference to a final correct Tgraph (apart from vertex relabelling)].

Combining these results, for perfect composition we have

and for the right hand side we have

as required.

Lemma 4 If composes perfectly and is a correct Tgraph then

Proof: Assume composes perfectly and is a correct Tgraph. Since is non-decreasing (with respect to on correct Tgraphs)

and since is monotonic

Since composes perfectly, the left hand side is just , so

and since is monotonic (with respect to on correct Tgraphs)

For the opposite direction, we substitute for in lemma 3 to get

Then, since , we have

Apply to both sides (using monotonicity)

For any for which the composition is defined we have so we get

Now apply to both sides and note to get

Combining this with (*) above proves the required equivalence.

Theorem (Perfect Composition): If composes perfectly and is a correct Tgraph then

Proof: Assume composes perfectly and is a correct Tgraph. By lemma 4 we have

Applying to both sides, gives

Now by lemma 2, with , the right hand side is equivalent to

which establishes the result.

Corollaries (of the perfect composition theorem):

1. If is a perfect composition and a correct Tgraph then

   

   Proof: Let (so ) in the theorem.

   [This result generalises lemma 2 because any correct forced Tgraph is necessarily wholetile complete and therefore a perfect composition, and .]

2. If is a perfect composition and a correct Tgraph then

   

   Proof: Apply to both sides of the previous corollary and note that

   

   provided the composition is defined, which it must be for a forced Tgraph by the Compose Force theorem.

3. If is a perfect composition and a correct Tgraph then

   

   Proof: Apply to both sides of the previous corollary noting is monotonic and idempotent for correct Tgraphs

   

   From the fact that is non decreasing and and are monotonic, we also have

   

   Hence combining these two sub-Tgraph results we have

   

   

It is important to point out that if is a correct Tgraph and is a perfect composition then this is not the same as composes perfectly. It could be the case that has more faces than and so could have unknowns. In this case we can only prove that

As an example where this is not an equivalence, choose to be a star. Then its composition is the empty Tgraph (which is still a pefect composition) and so the left hand side is the empty Tgraph, but the right hand side is a sun.

Perfectly composing generators

The perfect composition theorem and lemmas and the three corollaries justify all the commuting implied by the diagrams in figure 8. However, one might ask more general questions like: Under what circumstances do we have (for a correct forced Tgraph )

Definition A generator of a correct forced Tgraph is any Tgraph such that and .

We can now state that

Corollary If a correct forced Tgraph has a generator which composes perfectly, then

Proof: This follows directly from lemma 4 and the perfect composition theorem.

As an example where the required generator does not exist, consider the rightmost Tgraph of the middle row in figure 10. It is generated by the Tgraph directly below it, but it has no generator with a perfect composition. The Tgraph directly above it in the top row is the result of applying which has lost the leftmost dart of the Tgraph.

We could summarise this section by saying that can lose information which cannot be recovered by a subsequent and, similarly, can lose information which cannot be recovered by a subsequent . We have defined perfect compositions which are the Tgraphs that do not lose information when decomposed and Tgraphs which compose perfectly which are those that do not lose information when composed. Forcing does the same thing at each level of composition (that is it commutes with composition) provided information is not lost when composing.

4. Multiple Compositions

We know from the Compose Force theorem that the composition of a Tgraph that is forced is always a valid Tgraph. In this section we use this and the results from the last section to show that composing a forced, correct Tgraph produces a forced Tgraph.

First we note that:

Lemma 5: The composition of a forced, correct Tgraph is wholetile complete.

Proof: Let where is a forced, correct Tgraph. A boundary join in implies there must be a boundary dart wing of the composable faces of . (See for example figure 4 where this would be vertex 2 for the half dart case, and vertex 5 for the half-kite face). This dart wing cannot be an unknown as the half-dart is in the composable faces. However, a known dart wing must be either a large kite centre or a large dart base and therefore internal in the composable faces of (because of the force rules) and therefore not on the boundary in . This is a contradiction showing that can have no boundary joins and is therefore wholetile complete.

Theorem: The composition of a forced, correct Tgraph is a forced Tgraph.

Proof: Let for some forced, correct Tgraph , then is wholetile complete (by lemma 5) and therefore a perfect composition. Let , so composes perfectly (). By the perfect composition theorem we have

We also have

Applying to both sides, noting that is monotonic and the identity on forced Tgraphs, we have

Applying to both sides, noting that is monotonic, we have

By (**) above, the left hand side is equivalent to so we have

but since we also have ( being non-decreasing)

we have established that

which means is a forced Tgraph.

This result means that after forcing once we can repeatedly compose creating valid Tgraphs until we reach the empty Tgraph.

We can also use lemma 5 to establish the converse to a previous corollary:

Corollary If a correct forced Tgraph satisfies:

then has a generator which composes perfectly.

Proof: By lemma 5, is wholetile complete and hence a perfect composition. This means that composes perfectly and it is also a generator for because

5. Proof of the Compose Force theorem

Theorem (Compose Force): Composition of a forced Tgraph produces a valid Tgraph.

Proof: For any forced Tgraph we can construct the composed faces. For the result to be a valid Tgraph we need to show no crossing boundaries and connectedness for the composed faces. These are proved separately by case analysis below.

Proof of no crossing boundaries

Assume is a forced Tgraph and that it has a non-empty set of composed faces (we can ignore cases where the composition is empty as the empty Tgraph is valid). Consider a vertex v in the composed faces of and first take the case that v is on the boundary of . We consider local contexts for a vertex v on a forced Tgraph boundary where the composition is non-empty as shown in figure 11.

In each case v is shown as a red dot, and the composition is shown filled yellow. The cases for v are shown in rows: the first row is for dart origins, the second row is for kite origins, the third row is for kite wings, and the last rows is for kite opps. The dart wing cases are a subset of the kite opp cases, so not repeated, and dart opp vertices are excluded because they cannot be on the boundary of a forced Tgraph. We only show left-hand versions, so there is a mirror symmetric set for right-hand versions.

It is easy to see that there are no crossing boundaries of the composed faces at v in each case. Since any boundary vertex of any forced Tgraph (with a non-empty composition) must match one of these local context cases around the vertex, we can conclude that a boundary vertex of cannot become a crossing boundary in .

Next take the case where v is an internal vertex of .

Figure 12 shows relationships between the forced Tgraphs of the 7 (internal) vertex types (plus a kite at the top right). The red faces are those around the vertex type and the black faces are those produced by forcing (if any). Each forced Tgraph has its composition directly above with empty compositions for the top row. We note that a (forced) star, jack, king, and queen vertex remains an internal vertex in the respective composition so cannot become a crossing boundary vertex. A deuce vertex becomes the centre of a larger kite and is no longer present in the composition (top right). That leaves cases for the sun vertex and ace vertex (=fool vertex). The sun Tgraph (sunGraph) and fool Tgraph (fool) consist of just the red faces at the respective vertex (shown top left and top centre). These both have empty compositions when there is no surrounding context. We thus need to check possible forced local contexts for sunGraph and fool.

The fool case is simple and similar to a duece vertex in that it is never part of a composition. [To see this consider inverting the decomposition arrows shown in figure 4. In both cases we see the half-dart opp vertex (labelled 4 in figure 4) is removed].

For the sunGraph there are only 7 local forced context cases to consider where the sun vertex is on the boundary of the composition.

Six of these are shown in figure 13 (the missing one is just a mirror reflection of the fourth case). Again, the relevant vertex v is shown as a red dot and the composed faces are shown filled yellow, so it is easy to check that there is no crossing boundary of the composed faces at v in each case. Every forced Tgraph containing an internal sun vertex where the vertex is on the boundary of the composition must match one of the 7 cases locally round the vertex.

Thus no vertex from can become a crossing boundary vertex in the composed faces and since the vertices of the composed faces are a subset of those of , we can have no crossing boundary vertex in the composed faces.

Proof of Connectedness

Assume is a forced Tgraph as before. We refer to the half-tile faces of that get included in the composed faces as the composable faces and the rest as the remainder faces. We want to prove that the composable faces are connected as this will imply the composed faces are connected.

As before we can ignore cases where the set of composable faces is empty, and assume this is not the case. We study the nature of the remainder faces of . Firstly, we note:

Lemma (remainder faces)

The remainder faces of are made up entirely of groups of half-tiles which are either:

1. Half-fools (= a half dart and both halves of the kite attached to its short edge) where the other half-fool is entirely composable faces, or
2. Both halves of a kite with both short edges on the () boundary (so they are not part of a half-fool) where (at most) the origin is in common with composable faces, or
3. Whole fools where (at most) the shared kite origin in common with composable faces.

These 3 cases of remainder face groups are shown in figure 14. In each case the possible border in common with composable faces is shown yellow and the red edges are necessarily on the boundary of (the black boundary could be on the boundary of or shared with another reamainder face group). [A mirror symmetric version for the first group is not shown.] Examples can be seen in e.g. figure 13 where the first Tgraph has four examples of case 1, and two of case 2, the second has six examples of case 1 and two of case 2, and the fifth Tgraph has an example of case 3 as well as four of case 1. [We omit the detailed proof of this lemma which reasons about what gets excluded in a composition after forcing. However, all the local context cases are included in figure 15 (left-hand versions), where we only show those contexts where there is a non-empty composition.]

We note from the (remainder faces) lemma that the common boundary of the group of remainder faces with the composable faces (shown yellow in figure 14) is at most a single vertex in cases 2 and 3. In case 1, the common boundary is just a single edge of the composed faces which is made up of 2 adjacent edges of the composable faces that constitute the join of two half-fools.

This means each (remainder face) group shares boundary with exactly one connected component of the composable faces.

Next we establish that if two (remainder face) groups are connected they must share boundary with the same connected component of the composable faces. We need to consider how each (remainder face) group can be connected with a neighbouring such group. It is enough to consider forced contexts of boundary dart long edges (for cases 1 and 3) and boundary kite short edges (for case 2). The cases where the composition is non-empty all appear in figure 15 (left-hand versions) along with boundary kite long edges (middle two rows) which are not relevant for the proof.

We note that, whenever one group of the remainder faces (half-fool, whole-kite, whole-fool) is connected to a neighbouring group of the remainder faces, any common boundary (shared edges and vertices) with the compososable faces is also connected. The combined common boundary forms either 2 adjacent composed face boundary edges (= 4 adjacent edges of the composable faces), or a composed face boundary edge and one of its end vertices, or a single composed face boundary vertex.

It follows that any connected collection of the remainder face groups shares boundary with a unique connected component of the composable faces. Since the collection of composable and remainder faces together is connected ( is connected) the removal of the remainder faces cannot disconnect the composable faces. For this to happen, at least one connected collection of remainder face groups would have to be connected to more than one connected component of composable faces.

This establishes connectedness of any composition of a forced Tgraph, and this completes the proof of the Compose Force theorem.

References

[1] Martin Gardner (1977) MATHEMATICAL GAMES. Scientific American, 236(1), (pages 110 to 121). http://www.jstor.org/stable/24953856

[2] Grünbaum B., Shephard G.C. (1987) Tilings and Patterns. W. H. Freeman and Company, New York. ISBN 0-7167-1193-1 (Hardback) (pages 540 to 542).

by readerunner at October 13, 2026 12:27 PM

A fresh leaf

It has been a very long time since I posted a new article. The reasons for this are quite commonplace. Starting a family and building a career captured both my interest and energy, leaving little of either for sharing.

Obviously, I never stopped reading or learning; however, the vehicle for sharing changed to one with less friction: Twitter. Educating my teammates and colleagues satisfied my need to teach and share for a long time. It felt like this was the place where I could contribute. I found so many great articles and ideas that I simply wanted to amplify them, feeling unable to add to their eloquence. Ultimately, when family and career demands grew further, I stopped sharing completely. (Twitter going down the drain also didn’t help.)

Fast-forward a few years, and I found myself confronted with burnout. I was and still am lucky enough to recover at my own pace, take the time to get back on my feet, and reflect on the road that brought me here.

The burnout, while not nice to experience, also has been a great boon. It allowed me to reinvent myself, re-establish my sense of self, and plot a new course. Essentially, it meant going back to the roots of doing what I enjoy: learning, creating, and sharing.

All this time, ideas have been floating around my head continuously, ranging from academic research topics and work tasks to household improvements and side projects.

For a long time, I was quite sceptical about using LLMs for programming. My experience stemmed from early research agents around 2018–2020. I shared a common sense of AI fatigue in the news. Some enthusiastic colleagues tried to convince me how great agentic programming with these “new frontier” models was and never really succeeded since I had such an underwhelming experience with LLMs in the past.

Then, three things converged to change my mind: genuine skin in the game, zero skills to investigate on my own, and free access to a frontier model.

I saw Gamers Nexus: “216,000,000 Spy TVs | The LG Smart TV Problem” on Hacker News about LG TVs allegedly containing spyware. I own an LG TV, an earlier model than the article, and wanted to know whether my device was affected as well. I had no experience with this type of digging into systems so I didn’t know where to start. Now I had skin in the game and really wanted to figure this out. Coincidentally, I got a link from someone to an OpenAI page and realised I created an account there once because my password manager suggested a login. Curious, I logged in and saw that I had a month of free usage! So why not use it for the investigation?

Long story short, I started by using ChatGPT to explain to me how I should approach this investigation. The approach sounded reasonable. It quickly escalated to using Codex with many agents. Along the way I made many mistakes blowing through five-hour usage limits, etc. But the agents were finding results. Not the ones from the article, but close. (Maybe more on that later.) This was amazing: here was a tool that could do what I could not, and all I needed to do was nudge it forward, left, or right. I was hooked.

In a month, I went from an AI/LLM-sceptic to a guarded optimist. Doing several projects, I discovered how to control agents, the effects of harnesses, their strengths (and weaknesses…), and the tendencies of the different model families. I went from using ACP in Zed to just using Codex and Antigravity on the CLI, only rarely inspecting the code manually. Steering the code and code quality happens with test-coverage, mutation-coverage, and linting rules.

This is a journey I feel I want to share. For that I needed to redo the site. I didn’t want to relearn modern Hugo or figure out all the best themes out there and how to configure them; I just wanted results.

There was another impetus. I have a long-standing pet peeve with how my blog looked before, and how sites in general look. Specifically for longer reads. I love the visual of text typeset in LaTeX (specifically the Knuth–Plass line-breaking algorithm), and now Typst, but I cannot stand how most text looks in browsers, especially with text-align: justify.

The solution was simple. Clone my old site repository, point Codex at it, explain what I wanted to do, and it got cracking. It asked me a few questions on what I liked, what design I wanted, and came back with a few themes. I added my text layout criteria and we went forward implementing it. (It was also straightforward to let it port the algorithm to Go, as I previously had it port the code to Swift for my KnowledgeBase Viewer.)

The result is what you are reading now. I look forward to sharing more.

October 08, 2026 08:00 PM

First official release of hs-bindgen

Well-Typed and the entire hs-bindgen team—there are quite a few of us, see below—are delighted to announce hs-bindgen 1.0, the first official release. In case you missed the announcement of the alpha release, hs-bindgen is a tool to automatically generate Haskell bindings from C headers.

In this blog post we will start with a very brief intro to hs-bindgen, but we will not repeat everything that we discussed in the first two blog posts that announced the first and second alpha releases. Instead, we will highlight a few of the major changes; for a full list (which is quite long!), please see the changelog.

You can find hs-bindgen on Hackage.

Introduction

Consider this small C header:

#include <stddef.h>

typedef size_t length;

typedef struct str {
  char* bytes;
  length len;
} str;

/**
 * Construct \ref str from null-terminated input
 */
str new_string(const char* str);

When we run hs-bindgen on this example

$ hs-bindgen-cli preprocess example.h --omit-field-prefixes # .. more args

it will generate a number of Haskell modules, which we summarize below:

newtype Length = Length{
    unwrap :: CSize
  }

data Str = Str{
    bytes :: Ptr CChar
  , len   :: Length
  }

-- .. bunch of instances ..

{-| Construct 'Str' from null-terminated input

    __C declaration:__ @new_string@
    .. other Haddocks elided, here and elsewhere ..
-}
new_string ::
     PtrConst CChar -- ^ __C declaration:__ @str@
  -> IO Str
new_string = {- .. -}

Some notable features:

• C names are translated into Haskell names, modifying them to match Haskell’s naming rules where needed.
• With --omit-field-prefixes the Haskell code relies on DuplicateRecordFields; without that option it would have prefixed every field (lengthUnwrap, strBytes, etc.).
• No Haskell definition is generated for size_t, defined in stddef.h; instead, it is mapped to the existing CSize type by means of a binding specification for the standard C library. Binding specifications make binding generation compositional: bindings generated for one library can be reused in another.
• The length typedef is translated to a Haskell newtype, but the str typedef around the str struct is not: the latter exists in C only for syntactic convenience.
• Function new_string returns a struct by value, which is not natively supported by Haskell’s FFI, and so hs-bindgen will also generate the necessary C wrappers.
• All Haskell declarations get Haddock comments linking them to the corresponding C declaration; if that C declaration has associated Doxygen comments, those are also translated.

In the remainder of this blog post we will highlight some of the improvements we’ve made since the alpha release. As you will see, there are a lot of details to get right!

FFI types

Consider a C function

void foo(MyType x)

The obvious translation into Haskell is

foreign import {- .. -} foo_wrapper :: MyType -> IO ()

However, this is valid only if GHC can see that MyType is a type supported by the Haskell FFI; in particular, this means that GHC must be able to unwrap that MyType newtype until it gets a base type from a fixed set of permitted foreign types.

This is hard to ensure in general, especially if MyType is not newly generated, but comes from another library (previously generated by hs-bindgen, or handwritten). In the alpha release we resolved this by reducing every argument to a base type, but this resulted in code that was less portable than it could have been; for example, we might use Int32 instead of MyType.

We now solve this in a better way: given a Haskell type corresponding to a C type, a binding specification can (and should) specify an “FFI type” for that Haskell type: an FFI type is a pair of a module M and type F such that F is legal to appear in a foreign import in a context where M is in scope. For example, Foreign.C.CSize is an FFI type, because CSize can appear in a foreign import if Foreign.C is imported: Foreign.C exports all the necessary newtype constructors.

The translation from MyType to its FFI type (which might be MyType itself) is now also more general; MyType just needs to have an instance of the HasFFIType type class:

class HasFFIType a where
  type FFIType a :: Type

  toFFIType   :: a -> FFIType a
  fromFFIType :: FFIType a -> a

The FFIType type instance should line up with the FFI type specified in the binding spec. In many cases, toFFIType and fromFFIType can simply be coerce.

As an example, if MyType’s FFI type is CSize, then hs-bindgen will generate code

foreign import {- .. -} foo_wrapper :: CSize -> IO ()

foo :: MyType -> IO ()
foo x = foo_wrapper (toFFIType x)

In general the bindings generated by hs-bindgen are still specific to the specified target platform (the host, by default), but the introduction of FFI types reduces an avoidable source of non-portability.

Unnamed declarations

In Haskell, we might define a binary tree with values in the leaves as

data BTree a = Leaf a | Branch (BTree a) (BTree a)

An analogous definition in C might look something like this:

enum tag {LEAF, BRANCH};

struct BTree {
  enum tag constr;
  union {
    int value;
    struct {
      struct BTree* left;
      struct BTree* right;
    } branch;
  };
};

The union is anonymous: in C, its fields are accessed as if they are fields of the parent (struct BTree). In addition, the inner struct is unnamed, although the field of the union in this case does have a name (branch).

Haskell does not have anonymous records or anonymous sums, and so we must choose names. We have a policy in hs-bindgen to never generate numbered names (“struct 1”), and instead pick names based on surrounding clues, to avoid the generated code changing in unpredictable ways when the C headers change. In this case, the union is named after its first field and its parent: BTree_anon'value; similarly the struct is named after its parent and the name of the field: BTree_anon'value_branch. The anon' infix avoids name clashes with other generated names (C names cannot contain ticks).

newtype Tag = Tag CUInt
pattern LEAF, BRANCH :: Tag

data BTree = BTree{
    constr     :: Tag
  , anon'value :: BTree_anon'value
  }

newtype BTree_anon'value = ..

instance HasField "value"  BTree CInt
instance HasField "branch" BTree BTree_anon'value_branch

instance HasField "value"  (Ptr BTree) (Ptr CInt)
instance HasField "branch" (Ptr BTree) (Ptr BTree_anon'value_branch)

data BTree_anon'value_branch = BTree_anon'value_branch{
    left  :: Ptr BTree
  , right :: Ptr BTree
  }

These names are perhaps a bit awkward, but in most cases code never has to deal with them, because hs-bindgen generates “indirect” HasField instances that make it possible to treat the fields of the union as if they are fields of the parent, just like in C. Moreover, we have these instances also for pointers; this means that when using the OverloadedRecordDot extension, we can write Haskell code that processes these C values in a very natural way:¹

toBTree :: Ptr C.BTree -> IO (BTree CInt)
toBTree bt = do
    tag <- peek bt.constr
    case tag of
      C.LEAF ->
        Leaf <$> peek bt.value
      C.BRANCH -> do
        left  <- peek bt.branch.left
        right <- peek bt.branch.right
        Branch <$> toBTree left <*> toBTree right

Improved record-dot support

Although support in GHC for overloaded record update is sadly not yet as polished as overloaded record access, it can nonetheless be quite helpful when dealing with C code that uses a lot of unions: unions don’t have a natural representation in Haskell, and hs-bindgen therefore encodes them as opaque datatypes.

To continue the example from the previous section, we can construct a value of type BTree as

import HsBindgen.Runtime.Union qualified as Union

btree1 :: C.BTree
btree1 = C.BTree{
      constr     = C.BRANCH
    , anon'value = Union.zero{
          branch = C.BTree_anon'value_branch{
              left  = nullPtr
            , right = nullPtr
            }
        }
    }

If we take advantage of the indirect fields described in the previous section, we can improve this further:

import HsBindgen.Runtime.Struct qualified as Struct

btree2 :: C.BTree
btree2 = Struct.zero{
     constr = C.BRANCH
   , branch = Struct.zero{
         left  = nullPtr
       , right = nullPtr
       }
   }

Note that OverloadedRecordUpdate currently also requires RebindableSyntax; hs-bindgen-runtime provides a module HsBindgen.Runtime.Overloading that restores the standard environment. See also Enabling record dot syntax in the manual.

Macros

Preprocessor macros are used extensively in C development, and so for many applications it is important they too get translated to Haskell. Macro handling in hs-bindgen has been significantly improved since the alpha. Most of those changes are internal, though they will become user-facing once we release hs-bindgen-as-a-library, at which point it will also be possible to substitute your own macro languages, replacing hs-bindgen’s default c-expr.

The full list of improvements can be found in the changelog; here we will mention just a few. First, we can now parse declarations that use a mixture of macros we can parse and macros that we cannot. For example, given

#define DeviceId int
#define MUST_CHECK __attribute__((warn_unused_result))

int upgradeFirmware(DeviceId did) MUST_CHECK;

where the definition of DeviceId is parsed by hs-bindgen but the definition of MUST_CHECK is not, we now generate

newtype DeviceId = DeviceId CInt
upgradeFirmware :: DeviceId -> IO CInt

Second, detection of ambiguous macros (macros that do not have a unique expansion throughout the translation unit) is improved. For example, given

#define A 5
#define B A
#define B A

#define X 1
#define Y X
#undef X
#define X 2
#define Y X

we generate bindings for A and B

a :: CInt
a = 5

b :: CInt
b = a

but not for X and Y.

Finally, macros can now be defined as part of the input to hs-bindgen (--hash-define FOO 1). Previously such definitions had to be given twice, once to hs-bindgen and once to the C compiler via ghc-options: -optc-DFOO, which was easy to get wrong and could be hard to debug.

Conclusions

Although Haskell’s foreign function interface is good, writing bindings to large C libraries is nonetheless a laborious process. The “fire and forget” approach that hs-bindgen offers takes care of a lot of the legwork. We’re pretty proud of the fact that hs-bindgen can handle almost everything that a C header might throw at it.

However, for many applications the bindings generated by hs-bindgen will be a starting point only: they are a direct translation of the C code, and are therefore very low-level. We are working on a binding-combinators library that makes this process easier, but ultimately such high-level bindings are then still written by hand, albeit assisted by a library. The next big goal for hs-bindgen is automatic generation of high-level bindings, guided by pluggable heuristics.

It has taken a lot of effort to get here, and we thank Anduril for sponsoring this work. Within Well-Typed, the hs-bindgen team includes or at one point included Armando Santos, Dominik Schrempf, Edsko de Vries, Finley McIlwaine, Joris Dral, Matthias Heinzel, Oleg Grenrus, Sam Derbyshire and Travis Cardwell. We have also benefited from numerous contributions from external contributors in the form of issues and PRs, for which we are grateful!

hs-bindgen 1.0 is available on Hackage now; give it a spin and let us know if you have any problems!

---

1. This assumes a prescriptive binding spec to tell hs-bindgen to mark the tag as closed, causing hs-bindgen to generate a COMPLETE pragma. See #2101.↩︎

by edsko at October 08, 2026 12:00 AM

How to set up and run a Nixpkgs team

Motivation and background

Nixpkgs is the largest open source software distribution in the world, maintained by an ever-growing global community with almost a thousand active contributors every month. In Nixpkgs, a package maintainer ensures that the latest version of a given piece of software is packaged and operational. Most people contributing to Nixpkgs are volunteers, and if you meet some of them at NixCon or any of the numerous meetups, you can tell they’re passionate about building a healthy software ecosystem.

But volunteering obviously has limits. As a result, there is usually only one committer who ever looks at a pull request, including those touching critical packages, such as programs in the NixOS boot chain. And for an outside observer, it can be hard to gauge whether the software distributed is trustworthy. Without peer review, there is a constant risk that changes might introduce security vulnerabilities – or simply degrade code quality and add friction to the next person’s contribution.

I’m part of a group at Tweag that has been working on targeted improvements of supply chain security for Nixpkgs since mid-2025. That naturally involves reviewing pull requests. At some point it seemed sensible to establish a Tweag-branded team directly inside Nixpkgs, to get notified of package updates. That way, we thought, users and other maintainers would know who’s taking care of those packages.

However, when we raised the request with the then-newly-founded Nixpkgs core team, it prompted a thoughtful re-evaluation of the team policy: corporate-affiliated teams are no longer permitted, and team members must be individuals, with or without corporate affiliation. From a distance, such a turn of events may appear annoying at best. But actually, it was the opposite: everyone involved immediately agreed that it was the right course of action. Still, I ended up positively surprised by the outcome.

Setting up the team

To kick it off, we drafted an outline of objectives and procedures and posted it as a Nixpkgs issue. Our primary goal was straightforward: prevent malicious changes from reaching the repository. We wouldn’t merge anything ourselves, but would block merging in case of problems.

Fortunately, any Nixpkgs committer has permission to create GitHub teams under the Nixpkgs organisation. So we did exactly that, leveraging the fact that the in-code team definitions sync automatically from GitHub—a piece of automation built by my former colleague Silvan Mosberger (@infinisil) in 2025.

To manage daily reviews transparently, we set up a project board alongside a custom script to keep it updated. This allows others to track what we’re reviewing and step in to help. We maintain a public list of tracked packages, while keeping our internal review checklist in a private location to make it ever so slightly harder for bad actors to game our checks. The only thing left to do was adding the new team as maintainers of the relevant packages.

An open, topic-oriented team means anyone can join. We decided to require commit access for team membership, since that already entails substantial community reputation and a high level of trust. And we’ve learned that people are genuinely keen to help improve Nixpkgs if given the opportunity! One volunteer joined the team right away, and another a few months later. Team members regularly post feedback, and we established a dedicated Matrix channel to discuss best practices and review standards.

Since then we’ve rotated some members, but keep going at a steady pace. Between March and September 2026, we reviewed more than 250 pull requests, that is, everything that touched the packages in question.

Is Nixpkgs more secure now? We haven’t discovered outright malicious code or active zero-day vulnerabilities. That said, we did catch some genuine mistakes (such as an incorrectly applied patch) and corrected some code quality issues. And we’re on the watch.

Things to keep in mind

Starting a team in Nixpkgs proved surprisingly straightforward. I was initially hesitant about opening the team to everyone, as I expected that the work – reading through code and checking details – might be too boring or mundane for volunteers to do in their free time. To my surprise, some individuals were eager to join and contribute.

When running a community team, I recommend keeping a few practical considerations in mind:

1. People you don’t know may want to join, so be prepared to welcome outside interest and integrate new reviewers.

   Exercise judgement, but lean on existing trust networks and procedures.

2. Volunteer availability fluctuates, so expect that you will need to send gentle reminders to get things moving forward.

   Try to find out early on what works best for the team.

3. Everyone is at their best if they can focus on one thing at a time.

   Be ready to provide support: organise tasks, facilitate exchange, and keep everyone in the loop.

Take responsibility!

If you want to contribute public good to Nixpkgs, setting up a team is one of the most effective steps you can take. It is easy to start, fosters constructive discussions, and brings in fresh ideas from passionate people facing the same challenges. If you are interested in security reviews specifically or want to see how we operate, take a look at our Nixpkgs issue and consider joining the effort.

by Aleksandr Bantev at October 08, 2026 12:00 AM

Maybe a better way to track the release history of a Git repo

While I was still at ZipRecruiter, our Git monorepo was developing an annoying problem: it had too many refs.

Whenever there was a deployment of any of the applications in the repo, to staging or to production, the deployment tool would create a tag object with a name like this:

refs/release/stg.www.our-app-2019-09-06T114410.954-0700-treesha-179dab6fd11bee5195f0f8e31cea3edfb86ae23b

The tag contained the name of the target tier of the deployment (stg here, for “staging”), the app name and class of hosts on which it ran, www.our-app in this case, the date and time, and the hash of the tree that was deployed. This made a complete record of what was deployed, where, and when.

After we had been doing this for a while, there were thousands of these tag objects. Git is well-designed for handling thousands of commits, but not for handling thousands of refs. Perhaps this has changed, but at the time, Git would store refs in a file called packed-refs, which, for each ref, contained an entry with the ref's name and the ID of the object to which it referred.

(As with all things Git, the actual situation is somewhat more complicated. These additional ramifications don't affect the following discussion.)

Any operation that required resolving a ref name to its object had to search through the packed-refs file, which was taking an appreciable amount of time, far more than was desirable for an operation that everyone did many times a day. This affected not just local operations but operations on the Git server as well, and it was clear that it was only going to get slower as time went on. Also, the outputs of commands like git fetch and git branch were becoming increasingly cluttered with the names of tag objects whose only purpose was to record release history.

It seemed reasonable to me record the deployment history in the repository, but using thousands of tag objects clearly had serious drawbacks. In addition to the performance problems, the metadata itself was purely a convention in the tag name format, not legible to Git itself. There was a tree ID there, but nothing enforcing the rule that the tag should refer to a commit with that tree.

Also, the tag objects themselves seemed poorly-designed for the purpose. One good reason for storing history data in the respoitory is to render it immutable. But tags aren't immutable! They can be deleted and new ones can be inserted at any time, with any names desired.

And more important: consider the most natural sort of question that someone might want answered by the release history:

When did we last deploy the www.our-app app to production?

The only way to answer this question was to get the full list of all tags, filter it for prod.www.our-app, then sort by date and take the last item.

Or similarly: What if the most recent staging deployment had turned out to be bad, and we wanted to immediately revert to the version before that? Again, we (or the reversion process) would have had to filter and sort a list of thousands of tags, then pick out the second-last one.

I proposed this alternative in 2019. It stores the deployment history of each app as a series of unnamed but related Git objects, using Git's own history mechanism, so it's simple, efficient, and meshes well with Git's commands.

I don't think it was ever adopted, and it might turn out to have serious drawbacks I haven't foreseen. But it might be useful to someone in a similar situation.

---

Maintaining an audit and reversion trail without thousands of refs

Currently, each time a new version of an app is deployed, we create a ref:

refs/release/stg.www.our-app-2019-09-06T114410.954-0700-treesha-179dab6fd11bee5195f0f8e31cea3edfb86ae23b

As these refs proliferate, many operations in the Git repo become slower and slower.

We can record this information in a way that is at least as useful and which doesn't clutter up the ref list.

Current practice

At present, the release procedure creates a ref something like this:

• A new release is created at commit H

• A new tag object is created, pointing to H, something like

  git tag -a refs/release/stg.www.our-app-2019-09-.... H

Every new tag object has a name in the ref database. This adds a name for every release. The structure is something like this:

One ref object per release, each pointing to its own head commit

Alternative proposal

• Each (tier, app) pair is represented by a linked list of release commits
• The head of the list is the most recent release
• One ref points to the head of the list
• Older releases can be found by following the links back in time

Main point of this proposal: There is only one ref for each (tier, app) pair, no matter how many releases have been done.

It will look like this:

A chain of metainfo commits, each with the previous metainfo commit as first parent and the actual release commit as second parent

The one ref (which might or might not be an actual ref object) is named stg.www.our-app. That points not to the actual release commit, but to a commit that contains metainformation, such as the treesha. The metainformation could be stored in any of several ways. (See below.) The exact time of the release doesn't need to be stored explicitly because it is exactly the time at which the metainfo commit was created.

The metainfo commit has two parent commits. One is the actual release commit, the one that, in the current system, is the target of the named ref object. The other parent is the metainfo commit from the previous release.

The metainfo commits are chained together exactly the same way Git normally chains commit history, so the usual Git commands will work on the release history in reasonable and useful ways. In particular, the command git log stg.www.our-app will produce a listing of the release history.

Technical details

Creating this sort of two-parent commit is easy. Suppose the ref stg.www.our-app points to the current metainfo commit. And suppose we want to release new commit H. We do:

(Write some useful commit message into /tmp/message.$$.)

git commit-tree -p refs/release/stg.www.our-app -p H \
    H^{tree} < /tmp/message.$$

This creates a new metainfo commit with two parents, given by the arguments of the -p options. The first parent is the metainfo commit for the current release and the second is new commit H.

The git commit-tree command creates the new metainfo commit and writes its SHA on standard output. Say this SHA is N.

git push origin N:refs/release/stg.www.our-app

The release ref now points to the metainfo commit for the new release.

Usage

This will be easier to use than what we have now, because Git's command set was designed to manage a list of commits linked in this way; this is exactly how Git normally expects to represent history. This natural fit with Git's command set shows up in many places.

You can fetch a complete release history for any project with a single fetch:

git fetch origin refs/release/stg.www.our-app:HISTORY

After this, git log HISTORY will list the entire release history in reverse chronological order. All the usual git-log options make sense to filter or search this list. For example, if we want to find the last release before a certain date, we can use:

git log -1 --until=date HISTORY

If we want a forward-chronological list of release commit SHAs with their release dates, we could use something like:

    git log --reverse --format="%P %cd" HISTORY

At all times:

• The current release commit is accessible at HISTORY^2.
• The metainfo commit from n releases ago is at HISTORY~n.
• The release commit from n releases ago is at HISTORY~n^2.

Notice: if M is a metainfo commit that describes a release, M^ is the metainfo commit for the release before that, and M^{2} is the exact commit that was actually released. We don't have to record the tree IDs in the ref names; the tree that was released in release M is M^{2}^{tree}. These notations will work as expected in all git commands; see git-rev-parse.

Because Git naturally understands this way of organizing history, it will be easy to locate a particular previous release for rollback. When we roll back, we can roll back to the exact same commit as before: the new metainfo commit and the old one can share the exact same release commit object.

We can see the differences between the last two releases, without knowing their exact dates, with something like:

    git diff HISTORY~1^2 HISTORY^2

The metainfo commit is a full commit with a full set of Git commit metadata, such as a creation date. It has a commit message where we can record any unstructured text that we want. We could also require the commit message to be in TOML format, or we could use the tooling provided by Git to store semi-structured information in the commit message; see git-interpret-trailers.

If the built-in commit structure is not big enough for everything we want to record, the metainfo commit also has an associated tree where we could record any amount of structured information. In the example above I had its tree be identical to the release commit's tree, which might be convenient. (That way, if you wanted to see the files that were released, you could look in either the release commit or the metainfo commit; this would make some of the commands simpler.) But the tree doesn't have to be the same. We could also add metainformation files to this tree, or we could commit an entirely unrelated tree with nothing in it other than structured metainformation.

Performance?

Ploni Almoni asks about the performance issues of tracing out a linked list of commits. But this is exactly the same mechanism that Git normally uses to record history. When we do git log, Git is tracing a linked list of commits in exactly the way suggested here. If any operation in Git is going to be fast, it will be this one.

Git currently takes a few seconds to trace the 275,000-commit history of our monorepo. Tracing the similarly-structured history of a few hundred release commits is not going to be an issue.

---

I wrote this originally, but my only copy was in the form of a PDF. I had Claude convert the PDF to Markdown, which is the native format accepted by my blog, and the two diagrams to SVG. I then substantially edited the text and the diagrams. Along the way Claude pointed out three minor errors, which I corrected. I also took Claude's advice on how to adjust the colors from the original to make them more legible to color-blind readers.

Every word was written by my brain using my fingers.

by Mark Dominus (mjd@plover.com) at October 06, 2026 12:00 PM

Recent addenda to articles 202609: Errors, omissions, and retractions

I was going to start with “it has been a while since we had one of these” but then I saw that the previous one started with “Wow, has it really been 7 months since I did one of these?” and that was in December 2023. Maybe I can make it a more regular thing. Anyway blogging itself has become a more regular thing of late so there is quite a lot to report just from last month.

• In 2007 I described Naomi Wolf as “very smart”. I have retracted this description.

• My article from January about crooked policitians and crab cakes now mentions that "Johnny Doc” was released from prison on the orders of Donald J. Trump.

• In 2022 I published an article about the etymology of a purported Dutch phrase, which ended with “This article, which I wrote at 3:00 in the morning, is filled with many errors, including some that I would not have made if it had been daytime. Please disbelieve what you have read, and await a correction.” I have decided that it is uncorrectable, and retracted the whole thing.

• Dan Luu pointed out an important omission from my July article following up on Steph Curry's 2015–16 crushing of the 3PFG record.

• In 2021 I argued for an interesting replacement for the usual engineering interview process. The coming of LLM technology has rendered my suggestion completely impractical.

This article was written by an opaque and incomprehensible stoachastic process, namely me.

by Mark Dominus (mjd@plover.com) at October 03, 2026 02:51 PM

87: Edward Kmett

For the new Haskell Interlude, we sat down with Edward Kmett. Ed is the Founder and Chief Scientist of Positron AI. More importantly, he's a legend in the Haskell community for authoring many, many popular Haskell packages, first and foremost lens - and generally turning up the abstraction level to 11. We talk about design principles for Haskell libraries, category theory, cache-oblivious algorithms and his current work involving Haskell, C++, and FPGAs. Ed talks fast, so strap in!

by Haskell Podcast at October 02, 2026 01:00 PM

Memory profiling of large Haskell applications with ghc-debug

ghc-debug is a debugging tool for performing precise heap analysis of Haskell programs (for more background, check out our previous post improving ghc-debug and the Case Study: Debugging a Haskell space leak). While doing some memory profiling for Mercury, we took the opportunity to make some much needed improvements and quality of life fixes to both the ghc-debug set of libraries and the ghc-debug-brick terminal user interface.

The 0.8 releases focused on improving responsiveness and usability on very large memory profiles. In particular, ghc-debug now streams intermediate profile results, giving immediate feedback for huge heap traversals.

To summarise the improvements to ghc-debug-brick, it now:

• shows partial results, and allows heap traversals to be stopped at any moment without losing existing results;
• implements history navigation, allowing you to switch between heap profile views; and
• improves the decoding of stack frames and shows stack annotations.

Streaming of intermediate results

The ghc-debug-brick TUI is the starting point for any memory profiling with ghc-debug. It provides useful heap traversal features, for example finding closures with certain properties (via an expressive filter system), summarising all live heap objects, or analysing thunk origins.

Most of these features need to traverse the full heap to provide a conclusive result. A summary of all live heap objects cannot be complete until the full heap has been traversed. This works fine for smaller heaps, but what if the program heap is really big? Say, more than half of system RAM? The full heap traversal will take a long time, or even worse never finish, as it may use more memory than the system can provide.

To make ghc-debug usable with large heaps, we added support for streaming of results as they come in. No more waiting around: seconds after you start the heap traversal, the first results are shown on the screen. While you wait for more results, you can interact with the profile in a limited way. If you think you’ve gathered enough data, you can stop the heap traversal by pressing ESC and then explore retainers and fields, or start a different heap traversal.

Note that a stopped traversal can’t be resumed, and you have to start from scratch to find more results. There are currently no plans to add support for resuming a stopped traversal, as this would be complex to implement and can quickly leak a lot of memory when accumulating multiple partial queries.

All ghc-debug-brick heap traversals support this form of streaming/incremental results. While ghc-debug can be slow because the heap traversals are slow, it now feels much more responsive and more rewarding to work with large heaps.

History navigation

Heap traversals are expensive and take time, but ghc-debug-brick had only one view: the result of the last heap traversal. This was a constant source of frustration when analysing huge heaps, making it tedious to make progress. We improved this by adding a history for finished (and partial) heap traversals. You can view the history by pressing Ctrl+r, or via the command picker, and see the results of previous queries.

The history can be navigated with Alt+Left Arrow to go back in history or Alt+Right Arrow to go forward again.

Stack decoding

We improved the stack frame decoding logic of ghc-debug considerably, adding support for underflow frames, fixing InfoTable decoding and tidying up the rendering. Similar to other heap closures, stack frames have source locations (info table provenance entries) attached, which help with identifying where a particular stack frame or heap object was allocated.

Stack frames themselves can be retainers of heap objects, so improving ghc-debug’s support for stack frame decoding was much-needed.

Stack annotations

We added support for stack annotations in supported GHC versions to ghc-debug. These allow the user to annotate the native Haskell execution stack with additional metadata, such as additional context about the current location in the source program. Naturally, we taught ghc-debug how to decode stack annotation frames and display them as well.

As stack annotations are themselves merely heap objects, they are represented as SomeStackAnnotation closures that you can further drill into. Unfortunately, the annotations themselves can’t be displayed simply via the usual displayStackAnnotation rendering function, as this might evaluate thunks and change the Haskell heap.

Stack annotations help to understand complicated computation flows by adding user-defined instrumentation to the RTS stack. They were crucial in identifying multiple bugs in ghc-debug’s stack frame decoding logic.

Conclusion

We hope that the improvements to ghc-debug and ghc-debug-brick will aid the workflows of anyone looking to perform detailed inspections of the heap of their Haskell processes. The improvements should especially benefit anyone who works with large heaps. The new features have all been released to Hackage in ghc-debug-brick-0.8.

This work has been performed in collaboration with Mercury. Mercury have a long-term commitment to the scalability and robustness of the Haskell ecosystem and are supporting the development of memory profiling tools to aid with these goals.

Well-Typed are always interested in projects and looking for funding to improve GHC and other Haskell tools. Please contact info@well-typed.com if we might be able to work with you!

---

1. Frame 0 is the top of the stack, while Frame 23 is the bottom, indicated by the STOP_FRAME which is always the first stack frame on the GHC RTS stack.↩︎

by hannes at October 02, 2026 12:00 AM

Yet Another AI Security OSS Externality

Yet Another Rant About AI Security OSS Externality

IMPORTANT DISCLOSURE: This blog post is written in my personal capacity reflecting on the recent Apache Spark patch releases (3.5.9, 4.0.4, and 4.1.3). These are not the views of the Apache Software Foundation (ASF), the Apache Spark project, or any of my employers past or present. Both the ASF and the AI research lab have been provided an opportunity to comment on an earlier draft of this blog post and correct any information they see as incorrect and while many individuals have provided feedback in their personal capacity, there is not (as of yet) any official comment from either.

AI has fundamentally changed the volume of security reports in all projects, especially OSS projects where the barrier to analysis is even lower. We need to revisit how we handle security disclosures in the age of AI-driven pull requests and security auditing. Open Source Software (OSS) has long been for the community, and while some of us have jobs which give us time to work on open source, it's most often largely a volunteer effort for the public good, with the ASF focused on Software For the Public Good since it was incorporated in 1999. Security is of the utmost importance and requires greater care in collaboration to avoid accidental disclosure.  For security teams, when working with OSS projects, please be flexible on release timelines when issues are not actively being exploited. If we have to move our release schedule to meet your planned disclosure date, it's more likely to cause us to ship more security issues rather than fewer.

While some labs have early access programs, in my experience they're not working. Despite there being an agreement for early access to one of the labs' "frontier security models", we only got access to a scan in August. While a team at that same lab, that we'll call Team X, is, seemingly, using their frontier models from at least February to report security bugs and issue deadlines for resolution and disclosure.

The old model of "here's a bug, you have ~90 days to fix your issues before we publish how to exploit your code" came about because software vendors, mostly closed source but some open too, were in the habit of ignoring security reports (or being openly hostile to them). Previously security reports took a human to make, and we'd get a few every month, but now with AI our security report inbox is looking more like the promotions tab in my Gmail. Additionally, more bug fixes are landing later in the release candidate cycle from AI driven PRs, so these previously reasonable hard security disclosure deadlines are making it more likely we ship worse security bugs while we try and play catch up. To that end, I want to call on all frontier AI labs to put more funding into mitigating the impact of their tools on the OSS projects which are vital to their success.

To put it in “corporate� the key asks from this entire post are fairly simple:

• Frontier AI labs should provide real usable access to their frontier models for OSS

• AI labs should be careful in teams using their models in house for security in advance of their own programs for “trustedâ€� access.

• More flexibility on disclosure dates is needed given the impacts of AI

• A pony with attached coffee cup holder^ (and unlimited coffee) (jk)

The core of the issue remains that reviewer bandwidth is the scarcest thing in large open source projects. While tools like CodeRabbit are amazing (big shout-out to them for their OSS plans), they still don't capture everything. Thanks to the increasing frequency of  attempted (and sometimes successful) malicious PRs and takeovers of OSS projects, we can't just leave it to the robots to argue amongst themselves. Maintainer burnout is a real security risk that we as an industry are not prepared to handle.

While these suggestions are only a starting point, the big AI labs can do a number of things to mitigate some of the downsides. One is providing usable non-time locked free accounts to the people whose software they used. For some of the bigger projects, the AI labs have staff who are able to do things like manage releases, but they're often too busy working on internal issues to have the time to work upstream. I think giving engineers something akin to Google's "20% time" (which some jokingly called 120% time) to give back to the projects they're using could help. 

I want to double down on the "usable" part. While Anthropic has done better than any of the other AI labs at trying to provide free tooling for OSS developers, their effort still doesn’t do enough. In trying to use my Anthropic provided OSS maintainer account on a recent Spark security issue I tripped its safety guards and eventually ran out of tokens*. In practice, I've found paid inference on open weight models** to be more effective at addressing security issues. Most OSS maintainers, including myself, don't yet have usable direct free access to the frontier security models, meaning we're pushed into a reactive mode rather than pro-active. 

For me these issues came into focus during the recent ASF Spark 3 and 4 patch releases, available everywhere fine bits can be downloaded. Most impactful was a security issue reported by Team X where because of a combination of a lack of communication in the initial disclosure and lack of flexibility on the surprise deadline once the release managers became aware, we had to scramble for extra reviewers on complex separate security issues.

Team X reported security issues in the Apache Spark project around February 25th of this year. This report was (thankfully) accompanied by a human-reviewed proposed patch and no specific deadline in the initial report, which to be fair is better than we would get with a frontier model alone. So far, so good.

However, out of band, Team X then gave a vendor who distributes and contributes to Apache Spark (not my current employer) 90 days to fix this, or Team X would disclose the issue publicly. While the vendor requested an initial extension to line up with the planned OSS release schedule, which Team X granted, the deadline was not communicated to the broader OSS group, including myself, until much later.

While working on the most recent releases, we got roughly 30 post-first-line filter security reports in the week before the first patch release candidate, a few of which turned out to be actionable. Part way through the release cycle, I became aware of the deadline from Team X and reached out to the head of security at the lab to further extend the deadline so we could address the outstanding security issues. Some of these issues were blocked on knowledge of specialized libraries I personally don't have. Initially Team X was unwilling to extend the deadline any further, but when I explained how our 72-hour release voting normally works, they were willing to grant one extra week if the vote failed. During this negotiation, the amazing ASF security folks found a reviewer for the complex issue I was wrangling. We were able to get known issues fixed and the release candidate vote thankfully passed, but that outcome was far from certain.

If we had not been able to find reviewers for the encryption fix, we would have had to ship a partial release with a known broken encryption issue. At the end of the day, Team X has still not published the disclosures as of the end of September, strongly suggesting that they could have given us more time. As a release manager, this is of course frustrating and had personal impacts on my vacation coffee consumption being more necessary rather than enjoyed, but the impacts could have been much worse (not just missing out on limoncello).

I am not particularly anti-AI as far as things go. Having worked on making AI tools to help people appeal health insurance denials, I think there are potential positive cases where AI can help people.

I'd also love to give a shout-out to my point of contact at the frontier AI lab for helping unravel the game of telephone and getting me that extra week.

I don't think this problem is unique to Team X, the process is broken and compounding the existing reviewer constraint. Reviewer bandwidth has almost always been the biggest bottleneck in OSS projects, and the decreasing cost to generate code will continue to push against this bottleneck. When issues are discovered in OSS projects that are not actively being exploited, requiring releases from the OSS project with a fixed timeline is counterproductive. Release cycles are likely going to keep getting longer (see how big the latest Linux RCs are), and we should allow time for this. For the humans driving the robots, when there is a public interest-reason like an active exploit, not just a conference or PR opportunity, for providing a disclosure deadline, make sure to include that in your initial report whenever possible.

I'd like to end by reiterating my request to all of the frontier AI labs: please offer truly usable free plans for OSS developers, be flexible with your disclosure deadlines when no active exploits exist. In addition, there are some amazing engineers at these labs, if you give your staff time to participate in OSS maintenance tasks it could make a big difference. There's a strong history of industry and OSS working together in both ASF and other projects from Spark to PyTorch to the Linux kernel. That and a pony with a coffee cup holder and a coffee, but I'll take more tokens in place of a pony (I still want the coffee though :p).

If you're coming to Glasgow for Community over Code, I'll be around and looking to catch up with other OSS folks for their views on how we can try and make the externalities from AI less terrible, including chatting with folks at the responsible AI round table, and talking about how to make PySpark even faster. For SF based folks I'll be at QConSF in November talking on using AI powered transpilation to make UDFs faster. And for folks regardless of location I'm going to be doing a virtual talk end of October on dealing with AI security reports (I'll post on LI and blog when the registration link is available).

P.S.

I am well aware this is a bit of a “do as I say not as I did� situation as I did report some Yahoo! Mail security vulnerabilities purely through side channels back in University almost two decades ago. But I was young and did not have the resources of a multi-billion dollar company behind me, and I would argue the security implications of unencrypted e-mail passwords are a bit different than data processing software with schema inference driven class instantiation vulnerabilities.

* I try to use the account for other OSS work, not just security. It is more successful for non-security work.

** I don't currently have enough vRam to run these on my own hardware, but if anyone has a few hundred spare RTX5090s or DGXs and a few 20a circuits give me a call :p 

^ Did you know that's a real thing? I mean not the unlimited coffee, but they make coffee cup holders for saddles. In practice since I live in a city and do not have space for a pony let alone the skills, I ride a Tiger 900 (without coffee cup holder) and an LX150 (with coffee cup holder). If the AI labs are reading this: coffee cup holder for motorcycle with unlimited coffee is an acceptable substitute 😛

by Holden Karau (noreply@blogger.com) at September 29, 2026 06:20 PM

How we could turn Edinburgh into a real cycling city

 

A series by award-winning engineer Corey Boyle in the Edinburgh Inquirer. Well reasoned and well written. Some of the cited facts blew me away. ("Road traffic is the single biggest killer of 5 to 29 year olds on earth, and kills around 500 children every day. In 2024, 161 people, three of them children, died on Scotland’s roads, up 13% in a year. Twelve were in Edinburgh and the Lothians.") Spotted on edi.bike, via the Edinburgh Minute.

Corey Boyle’s three-part, evidence-rich series for the Edinburgh Inquirer on pushing the city towards a more sustainable future wrapped up this week with some practical approaches that could move the needle.

Boyle also had a piece on World Car Free Day published in The Scotsman this week.

As always, it’s been refreshing hearing a new and well-informed voice calling for change - links to all three chapters are below, on the Inquirer’s SubStack:

Part 1 â€” ğŸŒ� Even ‘the world’s worst cycle lane’ works

Part 2 â€” 🚗 Fewer commuters drive here than in any other Scottish city

Part 3 â€” 🚲 How we could turn Edinburgh into a real cycling city

Here's a bit from the intro.

The car was sold to us as freedom. For most of us, it has become the opposite, without us even realising.

There is a name for that. Psychologists call it motonormativity, or “car brainâ€�: the unconscious habit of excusing from driving the harm we would never accept from anything else. Once you see it, you see it everywhere.

Dr Patrick Miner at the University of Edinburgh has helped calculate the human cost. The result? The car has killed somewhere between 60 and 80 million people, around the same as both world wars combined, and it injures roughly 102 million more every year.

Over three parts, I want to take apart the car-centric society we accept as normal, confront Edinburgh’s disgrace, its poverty, and show that a cycling city, where the bike is how most people make most journeys, is the healthier, fairer alternative.

Caption to the photo above.

The same 69 people, and the road each takes: one bus, 69 bikes, 60 cars. Sixty, because that is how many an average 69 people fill: Credit: We Ride Australia, 2012, recreating a City of Münster original 

by Philip Wadler (noreply@blogger.com) at September 28, 2026 10:15 AM

Stormtoops and Freikorps Again: A timely Time Warp tribute in memory of Tim Curry

 

A brilliant, and sadly timely, parody. Spotted via Boing Boing.

by Philip Wadler (noreply@blogger.com) at September 28, 2026 08:50 AM

L-systems generate weevils, pizza toppings, and matriarchal lineages

The Fibonacci sequence is so-called because in Western mathematics is first appears in connection with a problem from Fibonacci's 1202 book Liber Abaci. Fibonacci asks us to consider a rabbit farm where a pair of baby rabits grows to adulthood in one month, and an adult pair of rabbits gives birth to a new pair of babies after a month. Representing an adult pair with a and a baby pair with b, we could write this as:

    b → a
    a → ab

Starting with one pair of baby rabbits, and performing the indicated substitutions month by month, we get:

    b
    a
    ab
    aba
    abaab
    abaababa
    abaababaabaab
    abaababaabaababaababa
    abaababaabaababaababaabaababaabaab
    abaababaabaababaababaabaababaabaababaababaabaababaababa
    …

The total number of symbols (and pairs of rabbits) each month gives us the Fibonacci sequence, as does the total number of as, or the total number of bs, in each line.

A structure like this one, in which there are symbols, in this case a and b, that are repeatedly replaced with strings of symbols, is called an “L-system”, and this L-system in particular is the “Fibonacci L-system”.

Yesterday I happened to mention Zyzzyva, which is a genus of weevils that I can only assume was named by a biologist who wanted to be last in the dictionary. When I see a pattern like this I instantly think of the Fibonacci L-system. We can use the Fibonacci L-system to generate a sequence of increasingly long weevil names:

    Yva
    Zva
    Zyva
    Zyzva
    Zyzzyva
    Zyzzyzyzva
    Zyzzyzyzzyzzyva
    Zyzzyzyzzyzzyzyzzyzyzva
    Zyzzyzyzzyzzyzyzzyzyzzyzzyzyzzyzzyva
    Zyzzyzyzzyzzyzyzzyzyzzyzzyzyzzyzzyzyzzyzyzzyzzyzyzzyzyzva

Or pizza toppings:

    eroni
    proni
    peroni
    peproni
    pepperoni
    peppepeproni
    peppepeppepperoni
    peppepeppeppepeppepeproni
    peppepeppeppepeppepeppeppepeppepperoni
    peppepeppeppepeppepeppeppepeppeppepeppepeppeppepeppepeproni

Or family matriarchs:

    Grandma
    Grandmam
    Grandmamma
    Grandmammamam
    Grandmammamammamma
    Grandmammamammammamammamam
    Grandmammamammammamammamammammamammamma
    Grandmammamammammamammamammammamammammamammamammammamammamam

I had to cheat a little here because there's an a in the middle that I didn't substitute. If we don't skip that we get the unpronounceable “Grmammandmammamammamma”.

A quick computer search suggests several other variations, including both “mammaries” and “titties”, but to prevent this article from devolving into juvenile sniggering, I'm going to cut the scroll here.

Amazingly, there was a previous article on a closely related topic.

I tried to get Claude to help me with this article, but it said the premise was too stupid so I had to do it myself.

by Mark Dominus (mjd@plover.com) at September 25, 2026 04:01 PM

The road to epsilon-zero: “Shortlex” isn't actually a single order

Previously:

1. Ordinal numbers and basic set theory
2. Ordinals as nim-heaps
3. Nim always ends, even with infinite ordinals
4. Infinite Nim as a coin-moving game
5. Coin-moving games with no coins
6. Productive programs and well-founded orders
7. Shortlex order also orders sequences of numbers

Something tricky happened a couple of articles ago that I wish now I'd explained more carefully. The recent articles talk about “shortlex order”. But “shortlex order” isn't really just one order. It's actually a method for turning one order into another, fancier one.

The definition I gave said:

The rule to compare two strings in shortlex order is:

1. If the strings are different lengths, the shorter one comes first.
2. If they're the same length, compare them lexicographically.

But there's something hiding in that “lexicographically”. To compare strings lexicographically, you have to compare them character-by-character, and that means you already have to have some idea of the order of the characters themselves. If we don't notice that “sort these into lexicographic order” is ambigous, it's only because a particular standard ordering of the characters is implicit.

But lexicographic and shortlex orderings aren't restricted to strings of characters. They can be used to extend an ordering on any kind of thing to an ordering on sequences of that-kind-of-thing. If the things are characters, they extend the ordering of the characters to an ordering on strings, which are sequences of characters. But the things don't have to be characters.

In the previous article we took the ordinary ordering on finite numbers (note, not characters) and used the shortlex constructin to produce an ordering on sequences of finite numbers:

Replace any one sequence of [finite number] coordinates with one that is earlier in shortlex order.

The phrase “in shortlex order” there is implicitly assuming the usual ordering of finite numbers. If we had decided to start with a different ordering of finite numbers, the shortlex extension to sequences would be different.

A of couple of episodes ago we saw an ordering of the positive numbers that put all the odd numbers before all the even numbers. Since that basic ordering puts before , the shortlex extension of it would put before . And because the basic ordering also put before the shortlex extension would put before .

Digression

For wacky fun, put the list of words in the dictionary into a lexicographic order that is extended from a different alphabetical order than the usual one!

For example, suppose the alphabet was in the order QWERTY UIOP ASDF GHJKL ZXCV BNM. The dictionary would start with “queer”, then all the other QU words up through “quandaries”, finish out the Q section with a few non-Qu words like “Qantas”, and then continue with the W words. The last word in the dictionary wouldn't be “zyzzyva”, it would be “mnemonics”.

Is your kid having trouble learning to alphabetize? Try alphabetizing this random list into QWERTY order to help yourself feel sympathetic:

criteria dapple duskiest flapjack gnawing immerses irk puppet surveyor sweetens

Remember, ‘c’ is no longer the third letter of the alphabet, it's now 22nd.

Here they are in QWERTY order instead:

irk immerses puppet sweetens surveyor duskiest dapple flapjack gnawing criteria

Shortlex isn't an order, it's a method for elaborating a simpler order

Say we have some set of things, and an order relation on , which we'll call .

We use the symbol “” rather than “” to emphasize that this is not necessarily the usual ordering. We still write to mean that comes before in whatever order represents, and analogously we can write means the same: comes after .

The notation means the set of all finite sequences of elements of . For example, when is a character set, is the set of strings over that character set. Elements of are not at all the same kind of things as elements of . Elements of are something-or-others, maybe melons or something — let's say melons, melons are fun — and if melon is rounder than melon . Elements of are sequences of melons.

If and are two sequences, you can't sensibly ask whether or , since it doesn't make to say that one sequence is rounder than another. is meaningful for comparing melons, not sequences.

We can't use to order elements of . But we can lexicographicize to make a new ordering that makes sense for . To lexicographically compare two elements of , that's two finite sequences of elements of , whatever they are, do this:

1. Compare the corresponding elements of each sequence, using , starting with the first elements
2. If two sequence elements are different at some posision, the sequence with the -earlier element comes earlier
3. If the two elements are the same, move on to the next corresponding elements and continue similarly.
4. If one sequence ends before the other, then that one comes earlier.
   If both end at the same time, the two sequences were exactly the same, so neither comes earlier.

This turns into a lexicographic elaboration of that makes it work on sequences. This is the same transformation that turns an ordering on characters into the derived ordering for strings.

We can turn into an ordering for sequences in another way, the shortlex way, producing an order I'll write as :

1. If the sequences are different lengths, the shorter one comes first.
2. If they're the same length, compare them lexicographically as above.

This is the same transformation that turns the regular ordering on digits () into the derived ordering for ordinary base-10 numerals.

I thought we'd get to today, but it seems not. Here's a preview of the next article, though: We're not going to stop with , we're going to go on to and .

Claude wanted to help with this article, but I wouldn't let it.

by Mark Dominus (mjd@plover.com) at September 25, 2026 12:14 AM

Real-Time Telemetry with Eventlog Live

Well-Typed are happy to announce Eventlog Live, a program that streams real-time telemetry from any Haskell application to any observability platform that supports the OpenTelemetry protocol, such as Grafana Cloud, Prometheus, or local viewers such as otel-tui and otel-desktop-viewer.

In this blog post, we’ll show a variety of ways to use Eventlog Live. For each showcase, we’ll link to the relevant instructions using Eventlog Live version 0.10.0.0 as well as the script that was used to generate the showcase. If you would like to use Eventlog Live, we recommend following the up-to-date instructions in the README of the latest release of Eventlog Live.

Heap Profiles by Closure Type

The Grafana Heap Profiles dashboard, which shows a heap profile broken down by closure type (-hT) as well as a variety of metrics for memory usage. For detailed instructions, see Eventlog Live – The Basic Version. This showcase uses the oddball-with-pipe example with the Docker Compose setup in demo/docker-compose.yml.

The Grafana Heap Profiles dashboard, which shows a heap profile broken down by closure type (-hT) as well as a variety of metrics for memory usage. For detailed instructions, see Eventlog Live – The Basic Version. This showcase uses the oddball-with-pipe example with the Docker Compose setup in demo/docker-compose.yml.

Eventlog Live

Eventlog Live analyses your application’s eventlog and exports it via the OpenTelemetry protocol. Eventlog Live is lightweight, runs side-by-side with your application in only a few megabytes of memory, and requires no instrumentation other than the -threaded and -rtsopts build options.¹ All you need is somewhere to send your data. For instance, if you have a Grafana Cloud² account, adding telemetry to your application is as easy as:

# Create a pipe for the eventlog
EVENTLOG_PIPE="/tmp/eventlog.pipe"
mkfifo "${EVENTLOG_PIPE}"

# Start your application
your-application +RTS -l -ol"${EVENTLOG_PIPE}" -hT --eventlog-flush-interval=1 -RTS &

# Start eventlog-live-otlp
OTEL_SERVICE_NAME="your-application"                                              \
OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)"                         \
OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp-gateway-<YOUR_REGION>.grafana.net/otlp" \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic%20<YOUR_AUTHORIZATION_HEADER>"    \
eventlog-live-otlp --eventlog-file="${EVENTLOG_PIPE}" -hT --eventlog-flush-interval=1

Out of the box, Eventlog Live supports -hT and -hi heap profiles³ as well as a variety of memory usage metrics, cost-centre stack profiles, log messages, productivity, detailed thread & capability usage traces. Each of these features is showcased below using an example Grafana dashboard. These dashboards can be found in the repository under demo-grafana-cloud/grafana-dashboards and demo/config/grafana-dashboards. Click any showcase for more information.

Heap Profiles by Closure Type

Heap Profiles by Info Table

The Grafana Heap Profiles dashboard, which shows a variety of metrics for memory usage and a heap profile broken down by info table (-hi). For detailed instructions, see Eventlog Live with Heap Profiling by Info Table. This showcase uses the oddball-with-hi example with the Docker Compose setup in demo/docker-compose.yml.

Cost-Centre Stack Profiles

The Grafana Cost-Centre Stack Profiles dashboard, which shows the cost-centre stack samples collected by GHC's built-in profiler in a flame graph. For detailed instructions, see Eventlog Live with Cost-Centre Stack Profiling. This showcase uses the jumpy-jump-with-cost-centre-profiler example with the Docker Compose setup in demo/docker-compose.yml.

Logs

The Grafana Logs dashboard, which shows log messages emitted using traceEventIO, traceMarkerIO, and threadLabel. This showcase uses the oddball-with-pipe example with the Docker Compose setup in demo/docker-compose.yml.

Logs as Markers

The Grafana Heap Profiles dashboard, which shows dotted red lines on each visualisation at the times that correspond to the markers, emitted using traceMarkerIO, that match the regular expression Summing \d+ numbers. This showcase uses the oddball-with-hi example with the Docker Compose setup in demo/docker-compose.yml.

Productivity

The Grafana Threads dashboard, which shows the cumulative productivity per capability, as well as the overall productivity over a sliding window. This showcase uses the oddball-with-pipe example with the Docker Compose setup in demo/docker-compose.yml.

Thread & Capability Usage

The Grafana Heap Profiles dashboard, which shows a variety of metrics for memory usage and a heap profile broken down by info table (-hi). For detailed instructions, see Eventlog Live with Heap Profiling by Info Table. This showcase uses the oddball-with-hi example with the Docker Compose setup in demo/docker-compose.yml.

The Grafana Cost-Centre Stack Profiles dashboard, which shows the cost-centre stack samples collected by GHC's built-in profiler in a flame graph. For detailed instructions, see Eventlog Live with Cost-Centre Stack Profiling. This showcase uses the jumpy-jump-with-cost-centre-profiler example with the Docker Compose setup in demo/docker-compose.yml.

The Grafana Logs dashboard, which shows log messages emitted using traceEventIO, traceMarkerIO, and threadLabel. This showcase uses the oddball-with-pipe example with the Docker Compose setup in demo/docker-compose.yml.

The Grafana Heap Profiles dashboard, which shows dotted red lines on each visualisation at the times that correspond to the markers, emitted using traceMarkerIO, that match the regular expression Summing \d+ numbers. This showcase uses the oddball-with-hi example with the Docker Compose setup in demo/docker-compose.yml.

The Grafana Threads dashboard, which shows the cumulative productivity per capability, as well as the overall productivity over a sliding window. This showcase uses the oddball-with-pipe example with the Docker Compose setup in demo/docker-compose.yml.

The Tempo trace explorer, which shows a detailed trace of the scheduling of one thread, which includes the durations for each running/blocked segment as well as the reason for each time the thread becomes blocked. These traces are not enabled by default, as their volume easily overwhelms most endpoints, and must be enabled using a configuration file. This showcase uses the spectral-norm example with the Docker Compose setup in demo/docker-compose.yml.

Eventlog Live’s OTLP exporter is configured using OpenTelemetry’s environment variables. The output of eventlog-live-otlp --help has comprehensive documentation on its support for these environment variables. By default, Eventlog Live aggregates telemetry data over 1 second intervals and exports every 30 seconds, but these intervals can be configured for each telemetry signal using configuration files.

Dynamic Control with Eventlog Socket

If you instrument your application with eventlog-socket, you’ll be able to start/stop profiling while your application is running using a REST API, which you can connect, e.g., to buttons on your Grafana dashboard, as we’ve done on the example Heap Profiles dashboard.

Dynamic Control with Eventlog Socket

Using the buttons on the Grafana Heap Profiles dashboard. When “Stop” is pressed, the heap profile flatlines. When “Start” is pressed, the heap profile resumes. The “Census” button can be used to request a single heap census. For detailed instructions, see Eventlog Live with Eventlog Socket. This showcase uses the oddball-with-hT example with the Docker Compose setup in demo/docker-compose.yml.

Using the buttons on the Grafana Heap Profiles dashboard. When “Stop” is pressed, the heap profile flatlines. When “Start” is pressed, the heap profile resumes. The “Census” button can be used to request a single heap census. For detailed instructions, see Eventlog Live with Eventlog Socket. This showcase uses the oddball-with-hT example with the Docker Compose setup in demo/docker-compose.yml.

Eventlog Socket lets your application write its eventlog over sockets and uses the other direction of communication for control messages. Eventlog Socket’s control protocol is generic. It isn’t bound to any specific application or library. Any library can register its own control commands under its own namespace,⁴ using Haskell IO actions as callbacks:

registerMyEventlogSocketSupport = do
  myPackageNamespace <- registerNamespace "my-package-name"
  registerCommand myPackageNamespace (CommandId 1) myCommandCallback1
  registerCommand myPackageNamespace (CommandId 2) myCommandCallback2

Any command that your application registers with Eventlog Socket can be invoked using Eventlog Live’s REST API.

For security reasons, Eventlog Socket’s control protocol and Eventlog Live’s REST API are hidden behind the +control feature flag, which is disabled by default. If you develop a library that (optionally) uses Eventlog Socket’s control protocol, we recommend using the same +control feature flag.

Call-Stack Profiles with GHC Stack Profiler

If you instrument your application with ghc-stack-profiler, you’ll be able to use its lightweight call-stack profiler.

Call-Stack Profiles with GHC Stack Profiler

The Grafana Call-Stack Profiles dashboard, which shows the call-stack samples collected by ghc-stack-profiler in a flame graph. For detailed instructions, see Eventlog Live with GHC Stack Profiler. This showcase uses the jumpy-jump-with-ghc-stack-profiler example with the Docker Compose setup in demo/docker-compose.yml.

GHC Stack Profiler samples GHC’s runtime call-stack, rather than a virtual cost-centre stack, and writes these samples to the eventlog. Based on our benchmarks, GHC Stack Profiler has virtually no overhead when it’s not running and we observed 2-8% overhead while it was running, compared to GHC’s built-in cost-centre stack profiler, where we observed 54-128% overhead while it wasn’t running and 98-136% overhead while it was running.

If you’ve also instrumented your application with Eventlog Socket, you’ll be able to start/stop GHC Stack Profiler while your application is running using the same REST API, which you can connect, e.g., to buttons on your Grafana dashboard, as we’ve done on the example Call-Stack Profiles dashboard.

GHC Stack Profiler can also be used without Eventlog Live, e.g., by exporting the profiles to speedscope. We’ll discuss GHC Stack Profiler in detail in an upcoming blog post.

Local Viewers

Eventlog Live does not yet include a built-in telemetry viewer, as it was primarily developed to monitor long-running processes in production. We are currently developing a built-in telemetry viewer, but the prototype is not yet ready for publication. In the meantime, some developers may feel that using either Grafana Cloud or the Docker Compose setup in demo/docker-compose.yml is too heavyweight for local development. Fortunately, the OpenTelemetry ecosystem has several lightweight telemetry viewers, such as otel-tui and otel-desktop-viewer, which are general purpose telemetry viewers that use a text and browser-based UI, respectively.

Eventlog Live with otel-tui

This showcase uses the oddball-with-otel-tui example with otel-tui version 0.7.4. This example creates a screen session that combines otel-tui with oddball and eventlog-live-otlp.

Eventlog Live with otel-desktop-viewer

This showcase uses the oddball-with-otel-desktop-viewer example with otel-desktop-viewer version 0.5.0.

This showcase uses the oddball-with-otel-tui example with otel-tui version 0.7.4. This example creates a screen session that combines otel-tui with oddball and eventlog-live-otlp.

This showcase uses the oddball-with-otel-desktop-viewer example with otel-desktop-viewer version 0.5.0.

Future Work

We are currently developing a built-in telemetry viewer that is purpose-built for GHC’s telemetry, with the intention of making Eventlog Live more useful for short development sessions. We’re also planning to create a library for writing structured log messages to the eventlog, expand the analyses supported by Eventlog Live, and expand the Eventlog Socket protocol to permit command arguments. Moreover, we’re always open to suggestions and feature requests from the community.

Acknowledgements

This work has been performed in collaboration with Mercury, who have a long-term commitment to the scalability and robustness of the Haskell ecosystem. Well-Typed are always interested in projects and looking for funding to improve Haskell tools. Please contact info@well-typed.com if we might be able to work with you!

---

1. If your application is built with GHC 9.2 or older, it must also be built with -eventlog. Since GHC 9.4, the eventlog is enabled by default.↩︎

2. The repository contains a self-contained demo for using Eventlog Live with Grafana Cloud. Unfortunately, as Grafana Cloud changes its user interface frequently, that demo may not be fully up-to-date.↩︎

3. Eventlog Live is developed and tested with the -hT and -hi heap profile breakdowns, but the -hm/-hd/-hy/-he heap profile breakdowns should work. The -hc/-hb/-hr heap profile breakdowns require additional support, which is planned but not yet implemented.↩︎

4. We strongly recommend using your package name.↩︎

by wen at September 24, 2026 12:00 AM

Tricorder MCP server: feedback your agent won't forget

A little over two months ago, we released the first version of Tricorder, a Haskell development tool built for developers of the modern era, where responsive and frictionless developer experience is king, and AI usage in coding is on the rise. Since then, we’ve shipped a steady stream of improvements based on user feedback. One of our most recent releases is tricorder-mcp, a Model Context Protocol (MCP) server that lets AI agents use Tricorder effectively on larger code bases.

While Tricorder is primarily developed for human use, it doubles as a bridge to GHCi for AI agents. This article showcases tricorder-mcp and how it makes agentic development with Tricorder more reliable.

For the uninitiated: what is Tricorder?

Tricorder is a CLI program that runs a lightweight daemon per project, each keeping a GHCi session warm in the background. Instead of having your agent cold-call cabal build and wait through a slow compile after every edit, it can query that daemon for the most up-to-date build state. Tricorder uses GHCi to recompile incrementally as files change, keeping type errors, warnings, test results, and eval comments available on demand. The daemon does the slow part once and keeps it warm; the agent just asks what’s true right now.

Tricorder exposes the build state through both a programmatic CLI interface and a more human-friendly TUI interface that you might just leave open in another terminal tab while working on your project.¹

Context matters

We’ve offered a ready-to-use agent skill for some time now, and if you’ve tried it, you might agree it works really well. However, longer sessions reveal its limitations, as the agent forgets how to use it properly. This is especially the case when working in large codebases, where a lot of code needs to be brought into the context window for the agent to do its job effectively.

The reason is the dreaded context window: once the agent decides it needs a skill, it reads the skill definition once and reuses that original read definition for the rest of the session. It is not automatically re-read unless you prompt the agent to do so. That original read is just more content sitting in the context window, and like everything else there, it’s subject to compaction and context bloat. The agent never forgets the skill exists, but it can lose track of how to use it.

MCP tools don’t have this problem. An agent’s harness resends the full list of available tools—alongside skills and built-in tools—on every single turn, as concise descriptions the agent invokes directly with the right arguments. There’s no one-time read to lose to compaction, so tools stay just as reliable in a two-hour session as in a two-minute one.

In effect, the agent doesn’t actually know how to use the tricorder CLI program directly; it’s the MCP server exposing it as a tool that handles that, turn after turn, without the instructions ever degrading.²

Words have meaning

The skill works by telling the agent how to invoke tricorder: a page of prose describing subcommands, flags, and output formats that the agent has to read, remember, and reproduce correctly on every call. Forget --wait, or get a flag name slightly wrong, and the agent doesn’t fail loudly. It just gets a stale or subtly wrong answer, or gives up on Tricorder entirely and pulls out cabal directly, and carries on.

tricorder-mcp replaces that with typed tools: Status, TestResults, Restart, and others, each coupled with a structured schema. tricorder-mcp better surfaces the difference between “Tricorder found build errors” and “Tricorder itself failed to run”. The status, test-results, and eval-comments commands exit non-zero when they’re reporting errors or failing tests, and the server still returns that as a normal result rather than an error, so the agent doesn’t have to guess what a non-zero exit means.

Show and tell

To better illustrate how easy it is for an agent to use Tricorder with this MCP server, let’s use it in some actual work!

Picking a simple enough task straight off of Tricorder’s own issue board: the Tricorder daemon should quit after a period of inactivity. Currently, the daemon will just hang around after you’re done with it until you happen to remember to kill it. This is a bit of a problem, seeing as GHCi can gobble up a sizeable chunk of memory after a few :adds and :reloads.

I will use Claude Code for this demonstration. The full conversation is available in a public Gist.

Our initial prompt looks like this:

❯ Currently, the Tricorder daemon will remain running in the background forever
once started, until the user themselves calls `tricorder stop` or otherwise
kills the process. It would instead be preferable if the daemon would shut
down by itself after a certain amount of idle time.
...

I prompt Claude to make the Tricorder daemon shut down after a configurable period of inactivity. I also ask it to use the Tricorder MCP server to check the build and test status along the way, and I make sure to mention some edge-cases that it needs to address in its implementation.

Claude provides us an overview of its plan, and begins exploring.

● I'll start by loading the Tricorder MCP tools and checking the current build
  status, then explore the daemon/session architecture.

As Claude explores the repo to figure out how to achieve its goal, it runs Tricorder through the MCP server, observing the build process for any issues. It can view the build status itself, and as soon as the build is finished it can view test results as well.

  Read 1 file, called tricorder 5 times, ran 2 shell commands

After making a series of changes, Claude can then check in with Tricorder again to observe the build process. Since Tricorder is running in the background, the build is probably ready for Claude to check by the time it gets around to doing so. Claude doesn’t have to wait around for cabal build to finish.

● Good. Now let's build and check for type errors via Tricorder MCP.

  Read 1 file, called tricorder 5 times, ran 2 shell commands

Claude can perform various actions using Tricorder as it goes along:

• Check test results:

  ● tricorder - test_results (MCP)(wait: true)
    └  atelier-core:test:atelier-test  passed (214) (410ms)
         test/Driver.hs
           Config
             extractNestedConfig
               should use default value for non-object Values:              OK
               should fetch top-level property:                             OK
               ...
  

• Inspect daemon logs:

  ● tricorder - log_contents (MCP)
    └  [INFO] Starting tricorder v0.2.1.0 (ac7251c)
       [INFO] Starting session 1
       [INFO] GHCi started (session #1): 0 diagnostics
       [INFO] Running 4 test suite(s)
       ...
  

• View module sources:

  ● tricorder - source (MCP)(modules: ["Data.Time.Units"])
    └  {-# LANGUAGE DeriveDataTypeable #-}
       {-# LANGUAGE GeneralizedNewtypeDeriving #-}
       -- |This module defines types for many useful time periods, as well as
       -- mechanisms for converting between them.
       module Data.Time.Units(
               TimeUnit(..)
             , Attosecond
             , Femtosecond
             , Picosecond
  

• Restart the daemon entirely, if the need arises:

  ● tricorder - restart (MCP)
    └
  

A lot of these operations also support narrowing down the returned result to reduce the size of the context window filled by Tricorder’s output.

After a little bit of back and forth, checking the build along the way, Claude considers its task complete.

● Implementation is complete and verified (0 diagnostics, all 4 test suites
  passing including the new tests). Summary of changes:

  - Tricorder.Session.IdleTimeout (new): IdleTimeout newtype, defaults to 300s.
  - Session.Config / Session: added idle_timeout_seconds config field
    (default 300), threaded into Session and re-evaluated on every loadSession call
    (i.e. on config reload), per your request.
  ...

Further refining Claude’s solution from here is as simple as ever, but thoroughly sped up by Tricorder’s snappy build loop and output.

This is a simple example of how you can use Tricorder through the Tricorder MCP server with Claude or another coding agent. Admittedly, this task could’ve been achieved without Tricorder just as easily, or by using the Tricorder skill, but I hope this walkthrough demonstrates the various ways an agent can use Tricorder to optimise its work.

Try it out!

If you don’t have Tricorder itself already, check out Tricorder’s README on how to get started.

Want to try it out with an AI agent like Claude, Copilot, Codex, Cursor or Grok? Check the guide on using Tricorder with coding agents for a full suite of plugins that manage the MCP server binary for you, and can pick between the MCP server and the skill intelligently.

To use Tricorder’s MCP server directly without a plugin, see the official installation instructions for tricorder-mcp.

Have any feedback or suggestions for Tricorder or tricorder-mcp? Let us know either on the issue board or on Tricorder’s Discord channel!

The road ahead

Next up for Tricorder is further improving and refining the tool’s outputs. Much care has been given to the output of tricorder ui so far, for human usage, but there is still a fair amount of work left to improve the regular CLI interface in tricorder status, tricorder test-results, et al.

Keep up with Tricorder by starring the repo on GitHub or through the Discord channel.

1. For more detail, you can watch the in-depth Tricorder presentation given at the Haskell Foundation’s “Haskell and AI” workshop. ↩

2. I am grossly simplifying the potential uses for MCP servers in this article. They can be used for many different things, especially when they define resources and prompts on top of the tools we describe here. At full scope, it’s helpful to think of MCP servers as AI-oriented APIs over resources that have a well-defined set of operations you can perform. ↩

by Victor Nascimento Bakke at September 24, 2026 12:00 AM

Noting another house repair chore victory

Yesterday I reported about how much I despise home repair, how I feel I'm really bad at it, how it sometimes goes terribly wrong for me, and how my work diary had demonstrated that a seeming failure was catually a glorious success.

I included this picture of our (formerly!) leaky laundry tub, and it reminded me of something else.

When we first moved into the house, more than four years ago now, that black outflow hose was unsecured. If you weren't careful — and sometimes even if you were — it would thrash around while the washer was emptying, and sometimes spray water everywhere.

In the first week or two after we moved in, there was a long list of minor problems like that. I found some store online that was selling a fix for the thrashing hose. The fix, which you can see in the picture, is a stiff but flexible rod with a collar at each end. You affix the collars around the top and bottom ends of the hose, then bend the flexible rod to force the hose to point in the direction you want. Then the hose can't thrash.

I observed a while back that the worst projects are also the ones that are easiest to remember. These are the ones that drag on for months, requiring endless adjustments, different approaches, and a lot of thought and toil. Or they're the ones that fail completely leaving some unfinished damage that reminds me every day that what I did didn't work.

Here's an example in the other direction. I didn't know how to solve this problem, but I found a solution, I executed it, and because the solution has worked flawlessly for more than four years I completely forgot about it until today when I was looking at this photo.

The work diary can help with this sort of thing. If I had a rollup page for laundry room issues I would be able to open it and see a list of all the things I had gotten right over the years, instead of forgetting them and only remembering the stuff that went wrong.

Claude advised that it would be better to write “hub” instead of “tub”, but I opted not to take its advice.

Addendum

Another example that occurs to me: in our house it's my job to unclog toilets. How many times over the years has a toilet clogged and I unfussily unclogged it with the plunger? Too many to count. And there have been other occasions when the plunger didn't work, so I would shrug, and go downstairs to get the toilet auger. I don't remember these many episodes because to me they were no big deal.

I should try to remember that no matter how incompetent I feel in general, just owning a toilet auger puts me way ahead of a lot of other people.

by Mark Dominus (mjd@plover.com) at September 23, 2026 01:46 PM

Haskell ecosystem activities report: June–August 2026

This is the thirty-second edition of our Haskell ecosystem activities report, which describes the work Well-Typed are doing on GHC, Cabal, HLS and other parts of the core Haskell toolchain. The current edition covers roughly the months of June, July and August 2026.

You can find the previous editions collected under the haskell-ecosystem-report tag.

Sponsorship

We offer Haskell Ecosystem Support Packages to provide commercial users with support from Well-Typed’s experts while investing in the Haskell community and its technical ecosystem including through the work described in this report. To find out more, read our announcement of these packages in partnership with the Haskell Foundation. We need funding to continue this essential maintenance work!

Many thanks to our Haskell Ecosystem Supporters: Standard Chartered, Channable and QBayLogic, as well as to our other clients who also contribute to making this work possible: Anduril, Juspay and Mercury; and to the HLS Open Collective for supporting HLS release management.

Highlights

Known-key names and reinstallable base

We have long been working to decouple GHC from base, to make it easier to upgrade the compiler independently from the (standard) libraries. A key restriction that prevented disentangling base from GHC was GHC’s treatment of known entities. These are things in the standard libraries that GHC needs to know something about (e.g. Show for deriving, Monad for desugaring do notation, etc). This is detailed in section 3.3 of a recent blog post by Simon Peyton Jones.

Rodrigo and Sam recently overhauled GHC’s treatment of known entities, in collaboration with Simon (!15899), finally landing the patch in August. This allowed known entities to be defined completely outside of the compiler for the first time, without hard-coding the precise module that defines them.

As part of this change, it was also necessary to rework some implementation details of Typeable, namely the KindRep datatype. Sam opened CLC proposal #433 to get approval on this change, which required an accompanying change to the binary package, which Rodrigo implemented in #244.

AArch64 code generation and sub-word handling

Andreas has been hunting down a slew of correctness bugs in the AArch64 native code generator, mostly related to the handling of sub-word values. Issues include sub-word handling across C calls (#27430), in atomic loads/stores (#27533), in conversions (#27537), in if conditionals (#27538), in %i2f64 (#27539) and the use of invalid immediates (#27550). These were all fixed in !16255, with a lot of the testing infrastructure being provided by Sam’s overhaul of test-primops (!43).

Coercion zapping

Extremely poor compiler performance for programs involving significant type-level computation has long plagued the community (#8095, recently re-reported as #27336). One major cause for this is blow-ups in coercion sizes due to the representation of accumulated type family reduction steps in Core, GHC’s typed intermediate language.

This problem resisted many attempts over the years, with promising approaches being brought down by performance regressions on certain types of programs (e.g. singleton-heavy programs). However, the future is bright for type-level wizards: MR !16378 implementing coercion zapping promises to bring the story to a satisfying conclusion.

This MR has an unusual history. In July, Sylvain Henry put up a proof-of-concept patch that was generated using Claude Fable. This patch was a carefully balanced “best of” previous approaches over the years, combining elements from MRs by Ben (!611), Adam (!6476), Sam (!7787) and Richard Eisenberg (!7909), each of which made significant progress on some programs but struggled to overcome significant compiler performance regressions on other programs.

Sam took over this patch and rewrote it from the ground up, doing a deep dive into GHC’s type family rewriter, and finding several related optimisation opportunities (e.g. !16479).

The net result is a patch that significantly improves compilation speed of type-family-heavy programs without regressing elsewhere. This patch is under final review and we hope to land it soon. Performance results from GHC’s testsuite:

Test case	Compiler allocations	Compilation time
T8095	768 MB ⇝   128 MB (−83%)	2.432s ⇝ 0.160s (−93%)
T9872a	1 656 MB ⇝ 1 406 MB (−15%)	0.971s ⇝ 0.494s (−49%)
T12545	698 MB ⇝   425 MB (−39%)	0.648s ⇝ 0.387s (−40%)
T13386	428 MB ⇝    35 MB (−92%)	0.259s ⇝ 0.104s (−60%)
T27336	499 MB ⇝    36 MB (−93%)	1.188s ⇝ 0.110s (−91%)
CoOpt_Singletons	634 MB ⇝   596 MB ( −6%)	0.484s ⇝ 0.360s (−26%)
LargeRecord	1 227 MB ⇝   967 MB (−21%)	0.872s ⇝ 0.617s (−29%)
FamAppCachePerf	496 MB ⇝   192 MB (−61%)	0.675s ⇝ 0.191s (−72%)

Exception annotations

Ben’s GHC proposal #330 and accompanying CLC proposal #164 proposed a deep rework of exception backtraces, providing several different backtrace mechanisms (e.g. HasCallStack, profiling cost centres, IPE).

However, the implementation of these proposals remained patchy, as detailed in Edsko’s recent blog post.

The team made several follow-up improvements to mechanisms for attaching information to exceptions:

• Ben fixed WhileHandling dropping the ExceptionContext of the exception it is handling (!16275, #27456).
• Sam made sure SomeException does not drop the ExceptionContext of the exception it is storing (!16274, #27455), with accompanying CLC proposal #432.
• Wolfgang added rethrowSTM (!16501), incorporated the re-export into the stm submodule (!16616), and upstreamed it to stm (#100), with accompanying CLC proposal #436.
• Zubin made annotateSTM use catchSTM# rather than catch# (!16508), with accompanying CLC proposal #437.

Trees That Grow

Over a long period, GHC has been gradually migrating to the Trees That Grow representation for its internal AST datatypes, and improving the modularity of the AST within the GHC codebase (#19218, #21592). Once complete, this will make it possible to expose the AST in a package that does not depend on the rest of the ghc library, and unblocks doing the same for the parser. As well as improving modularity, this will make it easier to use in tools that parse Haskell syntax, and may unlock future simplifications to template-haskell.

Alex and Rodrigo have been working on the next steps in that process, reorganising the codebase so that there will be no modules in the Language.Haskell.* namespace depending on modules in the GHC.* namespace. This included decoupling Language.Haskell.Syntax from GHC.Hs.Doc (!15601) and from GHC.Types.SourceText (!15573), representing string literals in the AST with Text rather than FastString (#21628, !12846), and decoupling ModuleNames used in the AST from those in GHC (!16546). As part of the original migration strategy, various orphan instances were introduced, which Alex has been cleaning up (#27755, !16299, !16610).

GitLab infrastructure work

Keeping Haskell’s infrastructure available has been increasingly challenging over the past couple of months due to the continued rise in LLM-driven crawling. See for comparison the Creepy crawlies blog post by Konstantin Ryabitsev, detailing the challenges faced by the git hosting infrastructure for the Linux kernel.

The GHC GitLab instance relies on Anubis to reduce crawler traffic. In early August, Magnus traced a fresh surge in load to a crawler that was bypassing Anubis entirely. Blocking it cut the load on the GitLab instance dramatically.

Other important infrastructure items:

• Magnus and Ben upgraded the instance’s PostgreSQL database, after which Ben carried the GitLab upgrade itself through to the latest release in a single fourteen-hour session. Besides clearing a long-standing backlog item, this made it possible to prioritise traffic from logged-in users.

• Magnus upgraded the host running the GitLab container, and applied patches to the GitLab instance (e.g. to close certain security vulnerabilities).

GHC

Releases

• Andreas posted an informal GHC release status update on Discourse in June. GHC release progress is officially tracked on the GHC release status page.
• Zubin worked on 9.14.2, cutting a first release candidate at the end of July, and a second release candidate in September. This work fixed a significant number of bugs, many of them blockers for the next major release, 10.0, which Zubin will prepare in due course.
• Magnus prepared release candidates for 9.12.5.
• Andreas continued work backporting patches to the 9.10.4 branch.

Frontend

• Magnus renamed ZonkAny to UnusedType and added pretty-printing logic for it (!16212, #27390), so that it no longer leaks confusingly in HLS output.

• Magnus refactored the error reporting code for ExplicitLevelImports (!16195).

• Adam marked various language extension flags as deprecated (!16145), implementing part of the categorisation of language extensions (#27329).

• Sam relaxed representation-polymorphism restrictions on newtype constructor patterns (!16342, #20363).

• Sam fixed a couple of issues with mis-oriented coercions produced by GHC (!16474, #27639, !16207, #27374).

• Sam fixed a crash in the Data.Data instance for HsCtxt (!16142, #27359).

• Sam reviewed a patch adding explanations for unsolved Typeable constraints (!16351).

Core & simplifier

• Andreas stopped the specialiser looping on recursive dictionaries in interestingDict (!16559, #27705).

• Zubin landed various fixes for “Entered absent arg” bugs (!16519, #27627, !16503, #27626), along with fixes for the interaction of absent fillers with speculative evaluation of dictionaries (!16219), following an extensive investigation of the crash reported in #25924.

• Following on from Simon Peyton Jones’ earlier refactor of GHC.Core.Utils.mkTick (!15494), which Sam helped with, Sam and Magnus tied up several loose ends:

  • preserving tick ordering in tickTickedExpr (!16601), fixing a regression in which source notes were moved past each other;

  • allowing more ticks in argument position in rebuild_app (!16558);

  • stopping ticks being dropped around variables of type IO () (!16123), fixing #27182 and #27225;

  • stopping mkTick in Core Prep from breaking ANF (!16198).

Driver

• Rodrigo made some improvements to the structure and caching of downsweep (!16330, #27053, #27461).

  During this work, Rodrigo and Sam found an opportunity to clean up some legacy Backpack logic, establishing that implicitRequirementsShallow can never reach a HoleUnit (!16346).

• Rodrigo deduplicated and shared Modules in interface files (!16216, !16213, #27401).

• Hannes shared UnitInfos across the UnitEnv using a UnitIndex and an external unit database cache (!16115), and arranged for the UnitIndexCache to be reused after initialising multiple home units (!16598, #27748).

• Hannes added the backendInfoTableMapValidity backend predicate (!16234).

• Hannes fixed the order of PackageDBFlags for the interactive home unit (!16591, #27640).

• Hannes diagnosed the quadratic behaviour of ghc-pkg list (#27275).

• Zubin carried out a substantial overhaul of the loader (!16493, !16582, !16583) to allow more control over unloading stale linkables, which is useful for HLS to efficiently evaluate expressions as files change in the IDE (#27606).

• Zubin made UniqDFM.alter preserve insertion order (!16385).

• Wolfgang finished up a patch sorting Usages by fingerprint, to ensure determinism (!15484).

Code generation

• Andreas fixed common block elimination silently dropping a procedure’s info table (!16592, #27722).

GHCi and bytecode

• Wolfgang added --show-byte-code, the counterpart of --show-iface for bytecode files (!16386, #26909).

RTS

• Duncan continued his sustained work to overhaul RTS I/O managers and the scheduler. One of the main goals is to bring the non-threaded RTS in line with the threaded one (e.g. #27335), and to improve maintainability across I/O manager implementations. As part of this effort, Duncan landed a collection of I/O manager refactors (!16217), preparing the ground for fixing idle GC and deadlock detection on the non-threaded RTS. Duncan also brought the POSIX and Win32 ticker implementations more in line (!16023).

• Andreas fixed pointer arithmetic in the RTS linker’s flushInstructionCacheRISCV64 (!16419, #27569).

Build system

• Sam made Hadrian check for cabal-install >= 3.12 upfront (!16132, #27317), rather than failing obscurely later on.

• David fixed the ghc-internal .def file name (!16190), removed unused wrapper scripts from the Windows bindist (!16138), and made Hadrian avoid response files when the command line is short enough (!16134), as response files were impeding developer workflows (#27230).

• David stopped ghc-toolchain throwing an exception when candidate executables are not found (!16173, #27369).

Testsuite

• Sam made the test driver normalise call-stack line numbers when referring to locations in libraries (!16247, #27387), so that tests are no longer sensitive to irrelevant internal changes.

• Andreas made the process009 test use SIGKILL (!16429).

• Zubin made various improvements to the JUnit output, most importantly showing diffs in the GitLab summary (!16309) and reporting fragile failures as skipped (!16250). He also landed assorted testsuite fixes (!16413).

CI and infrastructure

• Magnus worked on keeping nightly CI passing (!16296, #27127), adding a retry when fetching test metrics (!16281).

• David made the lint-changelog job check for the no-changelog label in the script rather than in rules (!16177), which avoids having to re-run the entire pipeline when applying the no-changelog label.

• Zubin arranged for tool messages to be reported as errors or warnings depending on the tool’s exit code, rather than always as errors (!16170, #27370).

process library

• Zubin fixed link errors on certain macOS configurations by allowing posix_spawn_file_actions_addchdir to be missing when linking against process on Darwin (#377, #376).

Cabal

Releases

• The 3.18.1.0 release was managed by Artem Pelenitsyn, with release engineering help from Mikolaj (#12104, #12128, #12016, #12010, #12017, #12108, #12113, #12182).

Recompilation checking

• Sam stopped Cabal from writing setup-config twice (#11949), fixing access-denied failures (#11619).

• Sam made cabal-install fall back to the external setup method on a Cabal-library mismatch (#11950), fixing stuck rebuilds of build-type: Custom packages (#11942).

• Sam allowed preserving the local build artifacts across minor Cabal version bumps without causing recompilation or errors (#12289).

• Tobias switched pre-processor recompilation checks to use file monitoring (#12021, with a preparatory refactoring in #12065). This fixes an issue with build tools not being re-run when their version changes (#11411).

Haskell Language Server

• Andrea added whole-project loading (#5002), as described in his recent blog post. This builds on the groundwork Andrea laid in hie-bios: new load modes for upfront loading of a cradle or project (#508) and the componentsToLoad field (#515), both reviewed and merged by Hannes.

• Hannes and Zubin pushed Guibou’s module-to-filename resolution over the finish line and applied further optimisations to put HLS’s import resolution on par with GHC’s (#4600).

• Zubin made HLS unload linkables when any transitive dependency changes (#5035), and stopped HLS unloading all bytecode whenever new bytecode is generated (#5033).

• Hannes set the FileCache hook before typechecking (#5030), fixing #4675.

• Hannes ensured the DocMap is never computed with stale data (#4936).

• Hannes tracked down multi-component session failures when expanding response files (#5004), fixed by whole-project loading.

• Hannes substantially improved HLS’s error messages (#5029).

Google Summer of Code projects

• Magnus, Hannes and Andreas mentored the 2026 GSoC project by Enrico Maria De Angelis, adding back the case-split plugin to HLS (#5014).

• Zubin and Hannes mentored the 2026 GSoC project by Vidit Odedra, adding go-to-definition for third-party dependencies to HLS (#4973, #5025).

Haskell syntax highlighting

• Hannes transferred the VS Code syntax highlighting extension to the Haskell community (#256), fixing a mishap that had caused the extension to disappear from the VS Code marketplace, breaking syntax highlighting for many users.

Runtime observability

Haskell Debugger

Rodrigo continued to lead development of the Haskell Debugger (hdb), an interactive step-through debugger for Haskell, with contributions from Andrea and Hannes. It also gained a logo, designed by George Thomas (#326).

• Hannes and Rodrigo added support for the future release of GHC 10.2 (#332, #359).

• hdb has two mechanisms for listing threads, decoding stacks and collecting exception information: external compiler commands where GHC supports them, and a fallback that inspects the heap directly on older GHCs. Rodrigo separated the two into distinct modules selected by a Cabal version conditional, rather than interleaving them (#356).

• Rodrigo made the singleThread flag of a continue request optional, as the Debug Adapter Protocol specifies (#38).

• Hannes made the stack frame decoding logic use the source locations that StackAnnotation carries from ghc-internal-9.1402 onwards (#333).

• Andrea made each stack frame report the variables bound in its own scope (#355, #160).

Live monitoring

Wen continued to develop eventlog-live, which allows GHC’s eventlog to be monitored as a program runs, streaming telemetry data from the runtime system to an external observability platform. One focus was on improving the OpenTelemetry integration.

Releases 0.6.0.0 through 0.9.0.0 came out this quarter. See the eventlog-live changelog for more details.

Stack and IPE profiling

Hannes and Wen worked on packages supporting stack profiling functionality, with a release of ghc-stack-profiler 0.4.0.0 (changelog) and ghc-stack-annotations 0.2.0.0 (changelog).

Wen rewrote ipedb’s LSM-tree database API (#18) and released 0.2.0.0/0.2.0.1 with other improvements (see the changelog).

by adam, alex, andrea, andreask, ben, davide, duncan, hannes, magnus, mikolaj, rodrigo, sam, tobias, wen, wolfgang, zubin at September 18, 2026 12:00 AM

GHC 10.0.1-alpha1 is now available

GHC 10.0.1-alpha1 is now available

wz1000 - 2026-09-18

The GHC developers are very pleased to announce the availability of the first alpha prerelease of GHC 10.0.1. Binary distributions, source distributions, and documentation are available at downloads.haskell.org and via GHCup.

GHC 10.0 will bring a number of new features and improvements, including:

• The default language edition is now GHC2024 (GHC Proposal #632, #26039).

• Type and Constraint are now completely distinct types, just as much as Int and Bool (#24279).

• The new QualifiedStrings language extension (GHC Proposal #723) allows string literals to be qualified using module qualification syntax, so that M."my string" desugars toM.fromString "my string".

• The treatment of static forms has been simplified, implementing GHC Proposal #732 (#26556). The free term variables of (static e) must now be bound at top level.

• -fpolymorphic-specialisation is now (re)enabled by default (#23559),

• Bytecode can now be written to disk with -fwrite-byte-code and packaged into bytecode libraries with -bytecodelib, which the interpreter can use to satisfy package dependencies (#26298).

• Executables can now be properly statically linked with the new -fully-static and -static-external flags (#26434).

• SIMD support in the AArch64 native code generator (#26536).

• A new poll I/O manager, which is now the default in the single-threaded RTS on all POSIX platforms except macOS (!9677).

• Version 2 of the GHC Jobserver Protocol for GHC’s -jsem jobserver on Linux and other POSIX platforms (#25087, #27253). This is sadly a necessary breaking change. -jsem requires cabal-install 3.18 or later, and is otherwise ignored with a warning, falling back to sequential compilation.

• Changes to base:

  • GHC.Num.{BigNat,Integer,Natural} are no longer exposed (CLC #359) and GHC internals in GHC.Num are deprecated (CLC #360).

  • Data.List.init and Data.List.last now carry an x-partial warning (CLC #292).

• The minimum bootstrap compiler is now GHC 9.10, LLVM versions up to 23 are supported (#26813, #27764), and the Debian 9, Debian 10, Ubuntu 18.04 and Ubuntu 20.04 bindists have been dropped in favour of a new Debian 13 bindist (#25876).

• … and many more

A full accounting of changes can be found in the release notes. See the migration guide for guidance on migrating programs to this release. As always, GHC’s release status, including planned future releases, can be found on the GHC Wiki status.

Given the many changes in this major release, we would very much appreciate testing on downstream workloads.

GHC development is sponsored by:

• Juspay
• QBayLogic
• Channable
• Haskell Foundation
• Serokell
• Well-Typed
• Tweag
• Dotcom-Monitor
• LoadView
• Web Hosting Buddy
• Find My Electric
• Standard Chartered
• UpCloud
• Mercury

We would like to thank these sponsors and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release.

As always, do give this release a try and open a ticket if you see anything amiss.

by ghc-devs at September 18, 2026 12:00 AM

Restoring type class support in Liquid Haskell

Over the last months, I’ve participated in a Google Summer of Code internship at Tweag in collaboration with Haskell.org, with the goal of bringing type class support back to Liquid Haskell. In this post, I will share the major insights and contributions that resulted from this project, walking through the main breaking points that I identified and their solutions, before briefly discussing the current state of the project and the limitations that emerged while expanding the test suite.

What is type class support, anyway?

Ask a Haskell programmer what makes a Monoid instance well-behaved, and the answer will certainly involve laws: <> should be associative and mempty should act as an identity. In Haskell, it is common practice to describe structures, many of them implemented as type classes, in terms of the properties that should hold for them.

However, the responsibility of checking that the instances are well-behaved usually falls on the developer. This is where Liquid Haskell (LH) can be really convenient, as it will delegate to an SMT solver the task of proving that the properties hold for instances of a class.

That feature used to exist back when LH targeted GHC 8, but as the library moved on to GHC 9, it broke and its test suite was disabled. During my GSoC 2026 project, I worked on bringing it back; the work is documented in this PR. I am thrilled to say that type classes are once again supported by LH, and that the test suite has been re-enabled.

Most of the machinery for working with type classes isn’t new. It was built during the GHC 8 era, and credit goes to its original authors: Yiyun Liu, James Parker and Niki Vazou. What the restoration required was a proper integration with the new GHC 9 API, and making targeted adjustments that account for recent developments that have taken place in LH since then.

When a class becomes a proof obligation

Before we dive into the work that was done, let’s take a look at an example (taken from All.hs in the test suite):

class Semigroup a where
    {-@ mappend :: a -> a -> a @-}
    mappend :: a -> a -> a

class Semigroup a => VSemigroup a where
    {-@ lawAssociative :: v:a -> v':a -> v'':a -> {mappend (mappend v v') v'' == mappend v (mappend v' v'')} @-}
    lawAssociative :: a -> a -> a -> ()

data PNat = Z | S PNat

instance Semigroup PNat where
  mappend Z     n = n
  mappend (S m) n = S (mappend m n)

instance VSemigroup PNat where
  lawAssociative Z     _ _ = ()
  lawAssociative (S p) m n = lawAssociative p m n

There are a few things worth noticing here. The laws that describe the behaviour of a class are no longer an agreement invisible to the user, but through VSemigroup, we actually have a way of allowing the user to provide a proof to the specification declared as a method.

For PNat, the body of lawAssociative serves as the proof that the law holds for that instance. The recursive call on the first argument carries out the induction: LH verifies the base case directly and the recursive call on the first argument tells the solver which induction hypothesis to use for the successor case.

In order to accomplish this, LH mirrors what GHC already does: while GHC compiles classes into record types, LH will generate a specification of the respective dictionary; GHC turns methods into record selectors, and LH lifts them into the refinement logic; and instances are elaborated into evidence by GHC, whereas LH will check the class’s laws for a concrete type.

What broke in the GHC 8 to 9 migration

In order to do its work, Liquid Haskell hooks into GHC as a plugin: It consumes GHC’s intermediate representation, calls its API, and even asks GHC to elaborate expressions on its behalf. This closeness between the two makes their integration seamless, but also puts pressure on the LH maintainers to keep up with the changes made to the compiler.

In particular, the release of the compiler’s version 9 brought many changes to LH, but the feature to check type classes was left behind, and thus the corresponding tests were guarded for earlier GHC versions:

executable typeclass-pos
    ...

    -- Fails on GHC 9.0.1
    if impl(ghc < 9)
      other-modules: ...

So, the first thing to do was to run the broken tests and identify the errors. The exploration was not as straightforward, since often an error would only show up after solving a different error at an earlier stage in the pipeline.

But looking back, errors can be classified in three buckets:

• Name resolution: Refinements that mention type class methods failed because their names weren’t visible to the resolver.
• Elaboration of expressions: Methods (or laws) elaborated through GHC came back incomplete, especially in subclasses; e.g. unbound symbols and wrong number of binders.
• Measure generation: The generated measures had dictionary constructors of the wrong shape, producing arity errors and mismatched types in the constraints.

The way I approached these problems was to use the behavior of GHC 8 as an oracle: I traced each error to find the pieces of code where the behaviour diverged from the working implementation, and re-aligned the two.

I will address these issues in the same order in the sections that follow.

Teaching the resolver about class methods

After the migration to GHC 9, Liquid Haskell introduced a significant change: the name resolution module. This new phase in the compilation serves as an entry point through which all names used by LH have to go. However, this new pipeline did not account for type class methods.

If you tried to run a test involving type classes after the introduction of name resolution, even doing something as simple as mentioning an imported class method in a specification, LH would throw an error. For example, this module imports check and uses it in the specification of a class law:

{-@ LIQUID "--typeclass" @-}
module Example where

import Check (MyCheck, check)

class MyCheck a => MyVCheck a where
    {-@ lawValid :: v:a -> {check v} @-}
    lawValid :: a -> ()

When building this, Liquid Haskell could not resolve check in the specification:

Unknown logic name `check`
Cannot resolve name
  |
7 |     {-@ lawValid :: v:a -> {check v} @-}
  |                             ^^^^^

The way to solve this consists of making LH aware of the existence of such class methods during the name resolution phase. If the lookup in LH’s environment fails, we can leverage GHC’s environment and check if there is a match for the name we are interested in. If that is the case, the name can be registered in the logic, as long as the entry’s parent is a class (thus making sure it is a class method).

For the time being, we assume that if the user wants to use class methods in specifications, they will also want to activate type class checking. Therefore we show an error message inviting the user to enable type class support:

Unknown logic name `check`
This is a class method, but typeclass support is disabled.
Enable it with {-@ LIQUID "--typeclass" @-}.

Where did dictionaries go?

When compiling code that uses a type class method, GHC inserts the dictionary that corresponds to the instance on which it is being called. Similarly, in order to reach a superclass method, it is necessary to first extract the parent dictionary. GHC stores that parent dictionary as a field of the subclass dictionary and generates a selector to retrieve it. For example, $p1VSemigroup is a compiler-generated selector¹ for the Semigroup dictionary stored inside a VSemigroup dictionary, so GHC produces mappend ($p1VSemigroup vsemigroupDict) x y instead of mappend vsemigroupDict x y. This applies only to subclasses, since only they have superclass dictionaries.

For LH to check a specification referencing these methods, it hands the refinement to GHC to obtain a Core expression. Therefore, GHC decides what dictionaries to insert at each location. These dictionaries are the evidence LH needs: they carry the implementations of methods and, for subclasses, the dictionaries of their superclasses. However, there were some slight changes in how those dictionaries are given: in GHC 8, the dictionaries used to be inlined in the application, whereas GHC 9 uses let-bindings to pass those arguments.

This was a breaking change, because the previous code was expecting abstract syntax trees with a different shape. In particular, the non-inlined dictionary bindings were ignored, discarding important evidence in the elaborated expressions. The expressions therefore referred to free variables, resulting in errors such as “Unbound symbol”.

The fix, then, was to not discard that evidence when elaborating the expressions. As with the rest of the restoration, I used GHC 8 as the source of truth and mimicked the behaviour it previously had: to inline the evidence instead.

Once the elaborated Core expression is returned by GHC, a function will traverse the tree to find evidence bindings (which can be easily identified via the GHC API) and substitute their occurrences by their definitions. This produces a valid expression with no unbound symbols.

Counting fields on dictionary constructors

I previously mentioned how classes are transformed into record types: every class declaration gives rise to the specification of a record that describes what the shape of the dictionary is, its fields and their respective types. That description is consumed in several places, most noticeably during measure generation².

To use these record types, LH must determine the number of fields each record has and reconstruct the shape of the corresponding GHC dictionary. This must account for fields storing superclass dictionaries.

GHC 9 had some changes on what fields were returned by the GHC API, and the expected arities started disagreeing in a few places because superclass fields were missing from the information LH received. This caused a series of errors and panics: “GHC gave back more/less binders than I expected”, “Liquid Type Mismatch”, “GHC and Liquid specifications have different numbers of fields”, and so on.

Previously, different phases had different counting rules. The solution required finding the correct rule to decide the arity of constructors, and applying the adjustment throughout the code base. In particular, it was necessary to always count superclass dictionaries as fields for class dictionaries, by including the data constructor’s theta information – the type class constraints that accompany the constructor. This ensures that all constraints, especially subclass constraints, are correctly included.

This was a targeted fix that only executes when working with type classes, correctly solving the previous errors.

Known limitations

With the original tests passing again, I started adding new ones to cover some scenarios that were missing. That surfaced a few remaining gaps that were already present in GHC 8. Tickets were opened for each one, with small code examples to reproduce the errors:

• Issue #2727: For a class such as class M m where ret :: a -> m a, where m is a higher-kinded parameter, m will be elaborated as if it had kind Type instead of Type -> Type. This triggers a kind error during the verification process.
• Issue #2743: Some special classes, such as Eq, Ord, and the numeric classes, are embedded directly in the SMT solver. Their methods are translated into SMT primitives instead of using the definitions supplied by their instances. As a result, LH assumes that these instances have the standard behaviour of their classes and cannot verify laws against their actual implementations.
• Issue #2744: A class is lifted into the LH logic only when at least one of its methods carries an LH specification. Without one, its methods cannot be used in refinements, and its instances are not checked against class specifications. It could be worth eliminating this requirement to make type class support easier to use, or documenting better how it is needed.
• Issue #2745: When a specification uses a class method whose constraints cannot be satisfied, LH reports an unbound dictionary symbol instead of a clear error resembling GHC’s error messages.

These are the immediately identified issues. As type class support remains experimental, we should expect some other issues to be uncovered still by more tests to come.

Conclusion

We now have a working test suite that runs the type class group on GHC 9: all previously working tests pass again, unchanged, showing that the restoration was done properly. A few more tests were added to check some specific critical points in the pipeline. Negative tests were included to verify that LH fails when that is expected behaviour.

The user documentation was updated to match the implementation. Comments were added as well, to explain some design choices and non-obvious steps in the code.

The whole restoration fits in about 150 lines of code, a small number because none of it is new machinery. Every change followed the same approach: find the place where GHC 9 and Liquid Haskell’s pipeline had drifted apart, and re-align them, trying to stay as close as possible to the GHC 8 implementation.

I want to thank my mentor, Facundo Domínguez, for his guidance and for steering me in the right direction as each new error came up; to the Haskell.org committee and Google Summer of Code for making the project possible; and to Tweag, for their work on Liquid Haskell and for hosting this post.

1. The exact name is an implementation detail which may vary and is internal to GHC. ↩

2. Measures are functions that refine the types of data constructors and lift them to the logic. Generating a measure for a data constructor (or a record type) requires identifying the shape of that constructor. This is important for type classes because of the transformation they undergo during compilation into record types. ↩

by Juan Pablo Yamamoto at September 17, 2026 12:00 AM