New paper: “Algebraic Effects Meet Hoare Logic in Cubical Agda”, by myself, Zhixuan Yang, and Nicolas Wu, will be published at POPL 2024.
Zhixuan has a nice summary of it here.
The preprint is available here.
I've just learned that Oddbins, a British chain of discount wine and liquor stores, went out of business last year. I was in an Oddbins exactly once, but I feel warmly toward them and I was sorry to hear of their passing.
In February of 2001 I went into the Oddbins on Canary Wharf and asked for bourbon. I wasn't sure whether they would even sell it. But they did, and the counter guy recommended I buy Woodford Reserve. I had not heard of Woodford before but I took his advice, and it immediately became my favorite bourbon. It still is.
I don't know why I was trying to buy bourbon in London. Possibly it was pure jingoism. If so, the Oddbins guy showed me up.
Thank you, Oddbins guy.
I've recently needed to explain to nontechnical people, such as my chiropractor, why the recent ⸢AI⸣ hype is mostly hype and not actual intelligence. I think I've found the magic phrase that communicates the most understanding in the fewest words: talking dog.
These systems are like a talking dog. It's amazing that anyone could train a dog to talk, and even more amazing that it can talk so well. But you mustn't believe anything it says about chiropractics, because it's just a dog and it doesn't know anything about medicine, or anatomy, or anything else.
For example, the lawyers in Mata v. Avianca got in a lot of trouble when they took ChatGPT's legal analysis, including its citations to fictitious precendents, and submitted them to the court.
“Is Varghese a real case,” he typed, according to a copy of the exchange that he submitted to the judge.
“Yes,” the chatbot replied, offering a citation and adding that it “is a real case.”
Mr. Schwartz dug deeper.
“What is your source,” he wrote, according to the filing.
“I apologize for the confusion earlier,” ChatGPT responded, offering a legal citation.
“Are the other cases you provided fake,” Mr. Schwartz asked.
ChatGPT responded, “No, the other cases I provided are real and can be found in reputable legal databases.”
It might have saved this guy some suffering if someone had explained to him that he was talking to a dog.
The phrase “stochastic parrot” has been offered in the past. This is completely useless, not least because of the ostentatious word “stochastic”. I'm not averse to using obscure words, but as far as I can tell there's never any reason to prefer “stochastic” to “random”.
I do kinda wonder: is there a topic on which GPT can be trusted, a noncanine analog of butthole sniffing?
I did not make up the talking dog idea myself; I got it from someone else. I don't remember who.
Safe coercions in GHC are a very powerful feature. However, they are not perfect; and already many years ago I was also thinking about how we could make them more expressive.
In particular such things like "higherorder roles" have been buzzing. For the record, I don't think Proposal #233 is great; but because that proposal is almost four years old, I don't remember why; nor I have tangible counterproposal either.
So I try to recover my thoughts.
I like to build small prototypes; and I wanted to build a small language with zerocost coercions.
The first approach, I present here, doesn't work.
While it allows model coercions, and very powerful ones, these coercions are not zerocost as we will see. For language like GHC Haskell where being zerocost is nonnegotiable requirement, this simple approach doesn't work.
The small "formalisation" is in Agda file https://gist.github.com/phadej/5cf29d6120cd27eb3330bc1eb8a5cfcc
We start by defining syntax. Our language is "simple": there are types
A, B = A > B  function type, "arrow"
coercions
co = refl A  reflexive coercion
 sym co  symmetric coercions
 arr co₁ co₂  coercion of arrows built from codomain and domain
 type coercions
and terms
f, t, s = x  variable
 f t  application
 λ x . t  lambda abstraction
 t ▹ co  cast
Obviously we'd add more stuff (in particular, I'm interested in expanding coercion syntax), but these are enough to illustrate the problem.
Because the language is simple (i.e. not dependent), we can define typing rules and small step semantics independently.
There is nothing particularly surprising in typing rules.
We'll need a "welltyped coercion" rules too though, but these are also very straighforward
Coercion Typing: Δ ⊢ co : A ≡ B

Δ ⊢ refl A : A ≡ A
Δ ⊢ co : A ≡ B

Δ ⊢ sym co : B ≡ A
Δ ⊢ co₁ : C ≡ A
Δ ⊢ co₂ : D ≡ B

Δ ⊢ arr co₁ co₂ : (C > D) ≡ (A > B)
Terms typing rules are using two contexts, for term and coercion variables (GHC has them in one, but that is unhygienic, there's a GHC issue about that). The rules for variables, applications and lambda abstractions are as usual, the only new is the typing of the cast:
Term Typing: Γ; Δ ⊢ t : A
Γ; Δ ⊢ t : A
Δ ⊢ co : A ≡ B

Γ; Δ ⊢ t ▹ co : B
So far everything is good.
But when playing with coercions, it's important to specify the reduction rules too. Ultimately it would be great to show that we could erase coercions either before or after reduction, and in either way we'll get the same result. So let's try to specify some reduction rules.
Probably the simplest approach to reduction rules is to try to inherit most reduction rules from the system without coercions; and consider coercions and casts as another "type" and "elimination form".
An elimination of refl would compute trivially:
t ▹ refl A ~~> t
This is good.
But what to do when cast's coercion is headed by arr
?
t ▹ arr co₁ co₂ ~~> ???
One "easy" solution is to etaexpand t, and split the coercion:
t ▹ arr co₁ co₂ ~~> λ x . t (x ▹ sym co₁) ▹ co₂
We cast an argument before applying it to the function, and then cast the result. This way the reduction is type preserving.
But this approach is not zerocost.
We could not erase coercions completely, we'll still need some indicator that there were an arrow coercion, so we'll remember to etaexpand:
t ▹ ??? ~~> λ x . t x
Treating coercions as another type constructor with cast operation being its elimination form may be a good first idea, but is not good enough. We won't be able to completely erase such coercions.
Another idea is to complicate the system a bit. We could "delay" coercion elimination until the result is scrutinised by another elimination form, e.g. in application case:
(t ▹ arr co₁ co₂) s ~~> t (s ▹ sym co₁) ▹ co₂
And that is the approach taken in Safe Zerocost Coercions for Haskell, you'll need to look into JFP version of the paper, as that one has appendices.
(We do not have space to elaborate, but a key example is the use ofnth
in ruleS_KPUSH
, presented in the extended version of this paper.)
The rule S_Push
looks some what like:
 S_Push
(t ▹ co) s ~~> t (s ▹ sym (nth₁ co)) ▹ nth₂ co
where we additionally have nth
coercion constructor to decompose coercions.
Incidentally there was, technically is, a proposal to remove decomposition rule, but it's a wrong solution to the known problem. The problem and a proper solution was kind of already identified in the original paper
We could similarly imagine a lattice keyed by classes whose instance definitions are to be respected; with such a lattice, we could allow the coercion ofMap Int v
toMap Age v
precisely whenInt
’s andAge
’sOrd
instances correspond.
The original paper also identified the need for higherorder roles. And also identified that
This means that Monad
instances could be defined only for types
that expect a representational parameter.
which I argue should be already required for Functor
(and traverseBia
hack with unlawful Mag
would still work if GHC had unboxed representational coercions, i.e. GADTs with bakedin representational (not only nominal) coercions).
There also the mention of unidirectional Coercible
, which people asked about later and recently:
Such unidirectional version of Coercible
amounts to explicit
inclusive subtyping and is more complicated than our current symmetric system.
It is fascinating that authors were able to predict the relevant future work so well. And I'm thankful that GHC got Coercible
implemented even it was already known to not be perfect. It's useful nevertheless. But I'm sad that there haven't been any results of future work since.
A few days after I published Hackage revisions in Nix I got a comment from
Wolfgang W that the next release of Nix will have a callHackageDirect
with
support for specifying revisions.
The code in PR #284490 makes callHackageDirect
accept a rev
argument. Like
this:
haskellPackages.callHackageDirect { pkg = "openapi3"; ver = "3.2.3"; sha256 = "sha2560F16o3oqOB5ri6KBdPFEFHB4dv1z+Pw6E5f1rwkqwi8="; rev = { revision = "4"; sha256 = "sha256a5C58iYrL7eAEHCzinICiJpbNTGwiOFFAYik28et7fI="; }; } { }
That's a lot better than using overrideCabal
!
Last year, at the end of winter, we wrote our impressions of the trends and evolution of infrastructure and configuration management after attending FOSDEM and CfgMgmtCamp. We’re at it again, but with Kubecon this year, the biggest cloud native computing conference.
If you’ve never heard of cloud native computing before, it has a number of definitions online, but the simplest one is that it’s mostly about Kubernetes.
Kubecon is a huge event with thousands of attendees. The conference spanned several levels of the main convention center in Paris, with a myriad of conference rooms and a whole floor for sponsor booths. FOSDEM already felt huge compared to academic conferences, but Kubecon is even bigger.
Although the program was filled with appealing talks, we ended up spending most of our time chatting with people and visiting booths, something you can’t do as easily online.
The very first morning, as we were walking around and waiting for the caffeine to kick in, we immediately spotted a Nix logo on someone’s sweatshirt. No better way to start the day than to meet with fellow Nix users!
And Nix was in general a great entry point for conversations at this year’s Kubecon. We expected it to still be an outsider at an event about containerdriven technology, but the problems that “containers as a default packaging unit” can’t solve were so present that Nix’s value proposition is attuned to what everyone had on their minds anyway. In other words, Nix is now known enough as to serve as a conversation starter: the company might not use it, but many people have heard about it and were very interested to hear insights from big contributors like Tweag, the Modus Create OSPO.
Many security products were represented. Securing cloudnative applications is indeed a difficult matter, as their many layers and components expose a large attack surface.
For one, the schedule included a security track, with a fair share of talks being about SBOMs tying back into the problem of containers that ship an opaque content without inventory. Nix, as a solution to this problem, was a great conversation starter here as well, especially for fellow Nixer Matthias, who can talk for hours about how Nix is the best (and maybe only) technology for automatically deriving complete SBOMs of a piece of software, in a trustworthy manner. Our own NLNetfunded project genealogos, which does exactly that, is recently getting a lot of interest.
Besides the application code and what goes in it, another focus was avoiding misconfiguration of the cloud infrastructure layer, of the Kubernetes cluster, and anything else going into container images. Many companies propose SaaS combinations of static linters scanning the configuration files directly with various policy rules, heuristics and dynamic monitoring of secure cloud native applications. Our configuration language Nickel was very relevant here: one of its raisons d’être is to provide efficient tools (types, contracts and a powerful LSP) to detect and fix misconfigurations as early as possible. We had cool conversations around writing custom security policies as Nickel contracts with the new LSP background contract checking (introduced in 1.5) reporting noncompliance live in the editor — in that light, contracts are basically a lightweight way to program an LSP.
IDPs were a hot topic at Kubecon. Tweag’s mission, as the OSPO of a leading software development consultancy, is to improve developer experience across the software lifecycle, which makes IDPs a natural topic for us.
An IDP is a platform  usually a web interface in practice  which glues several developer tools and services together and acts as the central entry point for most developer workflows. It’s centered around the idea of selfservice, not unlike the console of cloud providers, but configurable for your exact use case and open to integrate tools across ecosystem boundaries. We already emphasized this emerging new abstraction in last year’s post. Example use cases are routinely deploying new infrastructure with just a few clicks, rather than requiring to sync and to send messages backandforth to the DevOps team. IDPs don’t replace other tools but offer a unified interface, usually with customized presets, to interact with repositories, internal data and infrastructure.
Backstage, an opensource IDP developed by Spotify, had its own subconference at Kubecon. Several products are built on top of it as well: it’s not really a readytouse solution but rather the engine to build a custom IDP for your company, which leaves room for turnkey offers. We feel that such integrated, centralized and simpletouse services may become a standard in the future: think of how much GitHub (or an equivalent) is a central part of our modern workflow, but also of how many things it frustratingly can’t do (in particular infrastructure).
Many Kubernetesbased MLOps companies propose services to make it easy to deploy and manage scalable AI models in the cloud.
In the other direction, with the advent of generative AI, there was no doubt that we would see AIbased products to ease the automation of infrastructurerelated tasks. We attended a demo of a multiagent system which integrates with e.g. Slack, where you can ask a bot to perform endtoend tasks (which includes interacting with several systems, like deploying something to the cloud, editing a Jira ticket and pushing something to a GitHub repo) or ask highlevel questions, such as “which AWS users don’t have MFA enabled”.
It’s hard to tell from a demo how solid this would be in a real production system. I also don’t know if I would trust an AI agent to perform tasks without proper validation from a human (though there is a mode where confirmation is required before applying changes). There are also security concerns around having those agents run somewhere with write access to your infrastructure.
Putting those important questions aside, the demo was still quite impressive. It makes sense to automate those small boring tasks which usually require you to manually interact with several different platforms and are often quite mechanical indeed.
We attended a BirdsofaFeather session on Open Source Program Offices (OSPO). While the small number of participants was a bit disappointing (between 10 and 15, compared to the size of the conference), the small group discussions were still engrossing, and we were pleased to meet people from other OSPOs as well as engineers wanting to push for an OSPO in their own company.
The generally small size OSPOs (including from very large and influential tech companies) and their low maturity from a strategic point of view was surprising to us. Many OSPOs seem to be stuck in tactical concerns, managing license and IP issues that can occur when developers open up companyowned repos. In such a situation, all OSPO members are fully occupied by the large number of requests they get. But the most interesting questions: how to share benefits and costs by working efficiently with open source communities, how to provide strategic guidance and support, and how to gain visibility in communities of interest were only addressed by few. A general concern seemed to be generally a lack of understanding by upper management about the real strategic power that an OSPO can provide. From that perspective, Tweag, although a pink unicorn as a consulting OSPO, is quite far on the maturity curve with concrete strategical and technical firepower through technical groups, and our opensource portfolio (plus the projects that we contribute to but aren’t ours).
Kubecon was a great experience, and we’re looking forward to the next one. We are excited about the advent of Internal Developer Platforms and the concept of selfserving infrastructure, which are important aspects of developer experience.
On the technological side, the cloudnative world seems to be dominated by Kubernetes with Helm charts and YAML, and Docker, while the technologies we believe in and are actively developing are still outsiders in the space (of course they aren’t a full replacement for what currently exists, but they could fill many gaps). I’m thinking in particular about Nix (and more generally about declarative, hermetic and reproducible builds and deployments) and Nickel (better configuration languages and management tools). But, conversation after conversation, conference after conference, we’re seeing more and more interests in new paradigms, sometimes because those technologies are best equipped  by far  to solve problems that are on everyone’s radar (e.g. software traceability through SBOMs with Nix) thanks to their different approach.
Recently I came up with a criteria for a good warning to have in a compiler:
If compiler makes a choice, or has to deal with some complication, it may well tell about that.
That made me think about warnings I implemented into GHC over the years. They are fine.
Let us first understand the criteria better. It is better explained by an example which triggers few warnings:
First warning is Wnameshadowing
:
Shadow.hs:3:11: warning: [Wnameshadowing]
This binding for ‘x’ shadows the existing binding
bound at Shadow.hs:2:11

3  let x = 'y' in x
 ^
When resolving names (i.e. figuring out what textual identifiers refer to) compilers have a choice what to do with duplicate names. The usual choice is to pick the closest reference, shadowing others. But it's not the only choice, and not the only choice GHC does in similarish situations. e.g. module's toplevel definition do not shadow imports; instead an ambiguous name error is reported. Also \ x x > x
is rejected (treated as a nonlinear pattern), but \x > \x > x
is accepted (two separate patterns, inner one shadows). So, in a way, Wnameshadowing
reminds us what GHC does.
Another warning in the example is Wunusedbinds
:
Shadow.hs:2:11: warning: [Wunusedlocalbinds]
Defined but not used: ‘x’

2  foo = let x = 'x' in
 ^
This a kind of warning that compiler might figure out in the optimisation passes (I'm not sure if GHC always tracks usage, but IIRC GCC had some warnings triggered only when optimisations are on). When doing usage analysis, compiler may figure out that some bindings are unused, so it doesn't need to generate code for them. At the same time it may warn the user.
Let go through few of the numerous warnings GHC can emit.
Woverflowedliterals
causes a warning to be emitted if a literal will overflow. It's not strictly a compiler choice, but a choice nevertheless in base
's fromInteger
implementations. For most types ^{1} the fromInteger
is a total function with rollover behavior: 300 :: Word8
is 44 :: Word8
. It could been chosen to not be total too, and IMO that would been ok if fromInteger
were used only for desugaring literals.
Wderivingdefaults
: Causes a warning when both DeriveAnyClass
and GeneralizedNewtypeDeriving
are enabled and no explicit deriving strategy is in use. This a great example of a choice compiler makes. I actually don't remember which method GHC picks then, so it's good that compiler reminds us that it is good idea to be explicit (using DerivingStrategies
).
Wincompletepatterns
warns about places where a patternmatch might fail at runtime. This a complication compiler has to deal with. Compiler needs to generate some code to make all pattern matches complete. An easy way would been to always implicitly default cases to all pattern matches, but that would have performance implications, so GHC checks patternmatch coverage, and as a sideproduct may report incomplete pattern matches (or Winaccesiblecode
) ^{2}.
Wmissingfields
warns you whenever the construction of a labelled field constructor isn’t complete, missing initialisers for one or more fields. Here compiler needs to fill the missing fields with something, so it warns when it does.
Worphans
gets an honorary mention. Orphans cause so much incidental complexity inside the compiler, that I'd argue that Worphans
should be enabled by default (and not only in Wall
).
Wmissingimportlists
warns if you use an unqualified import
declaration that does not explicitly list the entities brought into scope. I don't think that there are any complications or choices compiler needs to deal with, therefore I think this warning should been left for style checkers. (I very rarely have import lists for modules from the same package or even project; and this is mostly a style&convenience choice).
Wprepositivequalifiedmodule
is even more of an arbitrary style check. With Wmissingimportlists
it is generally accepted that explicit import lists are better for compatibility (and for GHCs recompilation avoidance). Whether you place qualified
before or after the module name is a style choice. I think this warning shouldn't exist in GHC. (For the opposite you'd need a style checker to warn if ImportQualifiedPost
is enabled anywhere).
Note, while Wtabs
is also mostly a style issue, but the compiler has to make a choice how to deal with them. Whether to always convert tabs to 8 spaces, convert to next 8 spaces boundary, require indentation to be exactly the same spaces&tabs combination. All choices are sane (and I don't know which one GHC makes), so a warning to avoid tabs is justified.
Compatibility warnings are usually good also according to my criteria. Often it is the case that there is an old and a new way of doing things. Old way is going to be removed, but before removing it, it is deprecated.
Wsemigroup
warned about Monoid
instances without Semigroup
instances. (A warning which you shouldn't be able to trigger with recent GHCs). Here we could not switch to new hierarchy immediately without breaking some code, but we could check whether the preconditions are met for awhile.
Wtypeequalityoutofscope
is somewhat similar. For now, there is some compatibility code in GHC, and GHC warns when that fallback code path is triggered.
One of the warning I added is Wmissingkindsignatures
. For long time GHC didn't have a way to specify kind signatures until StandaloneKindSignatures
were added in GHC8.10. Without kind signatures GHC must infer kind of a data type or type family declaration. With kind signature it could just check against given kind (which is a technically a lot easier). So while the warning isn't actually implemented so, it could be triggered when GHC notices it needs to infer a kind of a definition. In the implementation the warning is raised after the typechecking phase, so the warning can include the inferred kind. However, we can argue that when inference fails, GHC could also mention that the kind signature was missing. Adding a kind signature often results in better kind errors (c.f. adding a type signature often results in a better type error when something is wrong).
The Wmissingpolykindsignatures
warning seems like a simple restriction of above, but it's not exactly true. There is another problem GHC deals with. When GHC infers a kind, there might be unsolved metakind variables left, and GHC has to do something to them. With PolyKinds
extension on, GHC generalises the kind. For example when inferring a kind of Proxy
as in
GHC infers that the kind is k > Type
for some k
and with PolyKinds
it generalises it to type Proxy :: forall {k}. k > Type
. Another option, which GHC also may do (and does when PolyKinds
are not enabled) is to default kinds to Type
, i.e. type Proxy :: Type > Type
. There is no warning for kind defaulting, but arguable there should be as defaulted kinds may be wrong. (Haskell98 and Haskell2010 don't have a way to specify kind signatures; that is clear design deficiency; which was first resolved by KindSignatures
and finally more elegantly by StandaloneKindSignatures
).
There is defaulting for type variables, and (in some cases) GHC warns about them. You probably have seen Defaulting the type variable ‘a0’ to type ‘Integer’
warnings caused by Wtypedefaults
. Adding Wkinddefaults
to GHC makes sense, even only for uniformity between (types of) terms and types; or arguably nowadays it is a sign that you should consider enabling PolyKinds
in that module.
The warning criteria also made me think about the following: the error hints are by necessity imprecise. If compiler knew exactly how to fix an issue, maybe it should just fix it and instead only raise a warning.
GHC has few of such errors. For example when using a syntax guarded by an extension. It can be argued (and IIRC was recently argued in discussions around GHC language editions) that another design approach would be simply accept new syntax, but just warn about it. The current design approach where extensions are "feature flags" providing some forward and backward compatibility is also defendable.
Conversely, if there is a case where compiler kindofknows what the issue is, but the language is not powerful enough for compiler to fix the problem on its own, the only solution is to raise an error. Well, there is another: (find a way to) extend the language to be more expressive, so compiler could deal with the currently erroneous case. Easier said than done, but in my opinion worth trying.
An example of above would be Wmissingbinds
. Currently writing a type signature without a corresponding binding is a hard error. But compiler could as well fill it in with a dummy one, That would complement Wmissingmethods
and Wmissingfields
. Similarly for types, a standalone kind signature tells the compiler already a lot about the type even without an actual definition: the rest of the module can treat it as an opaque type.
Another example is briefly mentioned making moduletoplevel definitions shadow imports. That would make adding new exports (e.g. to implicitly imported Prelude
) less affecting. While we are on topic of names, GHC could also report early when imported modules have ambiguous definitions, e.g.
doesn't trigger any warnings. But if you try to use Lazy.unpack
you get an ambiguous occurrence error. GHC already deals with the complications of ambiguous names, it could as well have an option to report them early.
If compiler makes a choice, or has to deal with some complication, it may well tell about that.
Seems like a good criteria for a good compiler warning. As far as I can tell most warnings in GHC pass it; but I found few "bad" ones too. And also identified at least one warningworthy case GHC doesn't warn about.
With XNegativeLiterals
and Natural
, fromInteger
may result in runtime error though, for example:
<interactive>:6:1: warning: [Woverflowedliterals] Literal 1000 is negative but Natural only supports positive numbers *** Exception: arithmetic underflow
Using [fmaxpmcheckmodels
] we could almost turn off GHCs patternmatch coverage checker, which will make GHC consider (almost) all pattern matches as incomplete. So Wincompletepatterns
is kind of an example of a warning which is powered by an "optional" analysis is GHC.↩︎
Avi Press is interviewed by Joachim Breitner and Andres Löh. Avi is the founder of Scarf, which uses Haskell to analyze how open source software is used. We’ll hear about the kind of shitstorm telemetry can cause, when correctness matters less than fearless refactoring and how that can lead to statically typed Stockholm syndrome.
Avi Press is interviewed by Joachim Breitner and Andres Löh. Avi is the founder of Scarf, which uses Haskell to analyze how open source software is used. We’ll hear about the kind of shitstorm telemetry can cause, when correctness matters less than fearless refactoring and how that can lead to statically typed Stockholm syndrome.
PenroseKiteDart is a Haskell package with tools to experiment with finite tilings of Penrose’s Kites and Darts. It uses the Haskell Diagrams package for drawing tilings. As well as providing drawing tools, this package introduces tile graphs (Tgraphs
) for describing finite tilings. (I would like to thank Stephen Huggett for suggesting planar graphs as a way to reperesent the tilings).
This document summarises the design and use of the PenroseKiteDart package.
PenroseKiteDart package is now available on Hackage.
The source files are available on GitHub at https://github.com/chrisreade/PenroseKiteDart.
There is a small art gallery of examples created with PenroseKiteDart here.
Index
In figure 1 we show a dart and a kite. All angles are multiples of (a tenth of a full turn). If the shorter edges are of length 1, then the longer edges are of length , where is the golden ratio.
What is interesting about these tiles is:
It is possible to tile the entire plane with kites and darts in an aperiodic way.
Such a tiling is nonperiodic and does not contain arbitrarily large periodic regions or patches.
The possibility of aperiodic tilings with kites and darts was discovered by Sir Roger Penrose in 1974. There are other shapes with this property, including a chiral aperiodic monotile discovered in 2023 by Smith, Myers, Kaplan, GoodmanStrauss. (See the Penrose Tiling Wikipedia page for the history of aperiodic tilings)
This package is entirely concerned with Penrose’s kite and dart tilings also known as P2 tilings.
In figure 2 we add a temporary green line marking purely to illustrate a rule for making legal tilings. The purpose of the rule is to exclude the possibility of periodic tilings.
If all tiles are marked as shown, then whenever tiles come together at a point, they must all be marked or must all be unmarked at that meeting point. So, for example, each long edge of a kite can be placed legally on only one of the two long edges of a dart. The kite wing vertex (which is marked) has to go next to the dart tip vertex (which is marked) and cannot go next to the dart wing vertex (which is unmarked) for a legal tiling.
Unfortunately, having a finite legal tiling is not enough to guarantee you can continue the tiling without getting stuck. Finite legal tilings which can be continued to cover the entire plane are called correct and the others (which are doomed to get stuck) are called incorrect. This means that decomposition and forcing (described later) become important tools for constructing correct finite tilings.
You will need the Haskell Diagrams package (See Haskell Diagrams) as well as this package (PenroseKiteDart). When these are installed, you can produce diagrams with a Main.hs module. This should import a chosen backend for diagrams such as the default (SVG) along with Diagrams.Prelude
.
module Main (main) where
import Diagrams.Backend.SVG.CmdLine
import Diagrams.Prelude
For Penrose’s Kite and Dart tilings, you also need to import the PKD
module and (optionally) the TgraphExamples
module.
import PKD
import TgraphExamples
Then to ouput someExample
figure
fig::Diagram B
fig = someExample
main :: IO ()
main = mainWith fig
Note that the token B
is used in the diagrams package to represent the chosen backend for output. So a diagram has type Diagram B
. In this case B
is bound to SVG by the import of the SVG backend. When the compiled module is executed it will generate an SVG file. (See Haskell Diagrams for more details on producing diagrams and using alternative backends).
In order to implement operations on tilings (decompose
in particular), we work with halftiles. These are illustrated in figure 3 and labelled RD
(right dart), LD
(left dart), LK
(left kite), RK
(right kite). The join edges where left and right halves come together are shown with dotted lines, leaving one short edge and one long edge on each halftile (excluding the join edge). We have shown a red dot at the vertex we regard as the origin of each halftile (the tip of a halfdart and the base of a halfkite).
The labels are actually data constructors introduced with type operator HalfTile
which has an argument type (rep
) to allow for more than one representation of the halftiles.
data HalfTile rep
= LD rep  Left Dart
 RD rep  Right Dart
 LK rep  Left Kite
 RK rep  Right Kite
deriving (Show,Eq)
We introduce tile graphs (Tgraph
s) which provide a simple planar graph representation for finite patches of tiles. For Tgraph
s we first specialise HalfTile
with a triple of vertices (positive integers) to make a TileFace
such as RD(1,2,3)
, where the vertices go clockwise round the halftile triangle starting with the origin.
type TileFace = HalfTile (Vertex,Vertex,Vertex)
type Vertex = Int  must be positive
The function
makeTgraph :: [TileFace] > Tgraph
then constructs a Tgraph
from a TileFace
list after checking the TileFace
s satisfy certain properties (described below). We also have
faces :: Tgraph > [TileFace]
to retrieve the TileFace
list from a Tgraph
.
As an example, the fool
(short for fool’s kite and also called an ace in the literature) consists of two kites and a dart (= 4 halfkites and 2 halfdarts):
fool :: Tgraph
fool = makeTgraph [RD (1,2,3), LD (1,3,4)  right and left dart
,LK (5,3,2), RK (5,2,7)  left and right kite
,RK (5,4,3), LK (5,6,4)  right and left kite
]
To produce a diagram, we simply draw
the Tgraph
foolFigure :: Diagram B
foolFigure = draw fool
which will produce the diagram on the left in figure 4.
Alternatively,
foolFigure :: Diagram B
foolFigure = labelled drawj fool
will produce the diagram on the right in figure 4 (showing vertex labels and dashed join edges).
When any (nonempty) Tgraph
is drawn, a default orientation and scale are chosen based on the lowest numbered join edge. This is aligned on the positive xaxis with length 1 (for darts) or length (for kites).
Tgraphs are actually implemented as
newtype Tgraph = Tgraph [TileFace]
deriving (Show)
but the data constructor Tgraph
is not exported to avoid accidentally bypassing checks for the required properties. The properties checked by makeTgraph
ensure the Tgraph
represents a legal tiling as a planar graph with positive vertex numbers, and that the collection of halftile faces are both connected and have no crossing boundaries (see note below). Finally, there is a check to ensure two or more distinct vertex numbers are not used to represent the same vertex of the graph (a touching vertex check). An error is raised if there is a problem.
Note: If the TilFace
s are faces of a planar graph there will also be exterior (untiled) regions, and in graph theory these would also be called faces of the graph. To avoid confusion, we will refer to these only as exterior regions, and unless otherwise stated, face will mean a TileFace
. We can then define the boundary of a list of TileFace
s as the edges of the exterior regions. There is a crossing boundary if the boundary crosses itself at a vertex. We exclude crossing boundaries from Tgraph
s because they prevent us from calculating relative positions of tiles locally and create touching vertex problems.
For convenience, in addition to makeTgraph
, we also have
makeUncheckedTgraph :: [TileFace] > Tgraph
checkedTgraph :: [TileFace] > Tgraph
The first of these (performing no checks) is useful when you know the required properties hold. The second performs the same checks as makeTgraph
except that it omits the touching vertex check. This could be used, for example, when making a Tgraph
from a subcollection of TileFace
s of another Tgraph
.
There are three key operations on finite tilings, namely
decompose :: Tgraph > Tgraph
force :: Tgraph > Tgraph
compose :: Tgraph > Tgraph
Decomposition (also called deflation) works by splitting each halftile into either 2 or 3 new (smaller scale) halftiles, to produce a new tiling. The fact that this is possible, is used to establish the existence of infinite aperiodic tilings with kites and darts. Since our Tgraph
s have abstracted away from scale, the result of decomposing a Tgraph
is just another Tgraph
. However if we wish to compare before and after with a drawing, the latter should be scaled by a factor times the scale of the former, to reflect the change in scale.
We can, of course, iterate decompose
to produce an infinite list of finer and finer decompositions of a Tgraph
decompositions :: Tgraph > [Tgraph]
decompositions = iterate decompose
Force works by adding any TileFace
s on the boundary edges of a Tgraph
which are forced. That is, where there is only one legal choice of TileFace
addition consistent with the seven possible vertex types. Such additions are continued until either (i) there are no more forced cases, in which case a final (forced) Tgraph
is returned, or (ii) the process finds the tiling is stuck, in which case an error is raised indicating an incorrect tiling. [In the latter case, the argument to force
must have been an incorrect tiling, because the forced additions cannot produce an incorrect tiling starting from a correct tiling.]
An example is shown in figure 6. When forced, the Tgraph
on the left produces the result on the right. The original is highlighted in red in the result to show what has been added.
Composition (also called inflation) is an opposite to decompose
but this has complications for finite tilings, so it is not simply an inverse. (See Graphs,Kites and Darts and Theorems for more discussion of the problems). Figure 7 shows a Tgraph
(left) with the result of composing (right) where we have also shown (in pale green) the faces of the original that are not included in the composition – the remainder faces.
Under some circumstances composing can fail to produce a Tgraph
because there are crossing boundaries in the resulting TileFaces
. However, we have established that
g
is a forced Tgraph
, then compose g
is defined and it is also a forced Tgraph
.It is convenient to use types of the form Try a
for results where we know there can be a failure. For example, compose
can fail if the result does not pass the connected and no crossing boundary check, and force
can fail if its argument is an incorrect Tgraph
. In situations when you would like to continue some computation rather than raise an error when there is a failure, use a try version of a function.
tryCompose :: Tgraph > Try Tgraph
tryForce :: Tgraph > Try Tgraph
We define Try
as a synonym for Either String
(which is a monad) in module Tgraph.Try
.
type Try a = Either String a
Successful results have the form Right r
(for some correct result r
) and failure results have the form Left s
(where s
is a String
describing the problem as a failure report).
The function
runTry:: Try a > a
runTry = either error id
will retrieve a correct result but raise an error for failure cases. This means we can always derive an error raising version from a try version of a function by composing with runTry
.
force = runTry . tryForce
compose = runTry . tryCompose
The module Tgraph.Prelude
defines elementary operations on Tgraph
s relating vertices, directed edges, and faces. We describe a few of them here.
When we need to refer to particular vertices of a TileFace
we use
originV :: TileFace > Vertex  the first vertex  red dot in figure 2
oppV :: TileFace > Vertex  the vertex at the opposite end of the join edge from the origin
wingV :: TileFace > Vertex  the vertex not on the join edge
A directed edge is represented as a pair of vertices.
type Dedge = (Vertex,Vertex)
So (a,b)
is regarded as a directed edge from a to b. In the special case that a list of directed edges is symmetrically closed [(b,a) is in the list whenever (a,b) is in the list] we can think of this as an edge list rather than just a directed edge list.
For example,
internalEdges :: Tgraph > [Dedge]
produces an edge list, whereas
graphBoundary :: Tgraph > [Dedge]
produces single directions. Each directed edge in the resulting boundary will have a TileFace
on the left and an exterior region on the right. The function
graphDedges :: Tgraph > [Dedge]
produces all the directed edges obtained by going clockwise round each TileFace
so not every edge in the list has an inverse in the list.
The above three functions are defined using
faceDedges :: TileFace > [Dedge]
which produces a list of the three directed edges going clockwise round a TileFace
starting at the origin vertex.
When we need to refer to particular edges of a TileFace
we use
joinE :: TileFace > Dedge  shown dotted in figure 2
shortE :: TileFace > Dedge  the nonjoin short edge
longE :: TileFace > Dedge  the nonjoin long edge
which are all directed clockwise round the TileFace
. In contrast, joinOfTile
is always directed away from the origin vertex, so is not clockwise for right darts or for left kites:
joinOfTile:: TileFace > Dedge
joinOfTile face = (originV face, oppV face)
Behind the scenes, when a Tgraph
is drawn, each TileFace
is converted to a Piece
. A Piece
is another specialisation of HalfTile
using a two dimensional vector to indicate the length and direction of the join edge of the halftile (from the originV
to the oppV
), thus fixing its scale and orientation. The whole Tgraph
then becomes a list of located Piece
s called a Patch
.
type Piece = HalfTile (V2 Double)
type Patch = [Located Piece]
Piece
drawing functions derive vectors for other edges of a halftile piece from its join edge vector. In particular (in the TileLib
module) we have
drawPiece :: Piece > Diagram B
dashjPiece :: Piece > Diagram B
fillPieceDK :: Colour Double > Colour Double > Piece > Diagram B
where the first draws the nonjoin edges of a Piece
, the second does the same but adds a dashed line for the join edge, and the third takes two colours – one for darts and one for kites, which are used to fill the piece as well as using drawPiece
.
Patch
is an instances of class Transformable
so a Patch
can be scaled, rotated, and translated.
It is useful to have an intermediate form between Tgraph
s and Patch
es, that contains information about both the location of vertices (as 2D points), and the abstract TileFace
s. This allows us to introduce labelled drawing functions (to show the vertex labels) which we then extend to Tgraph
s. We call the intermediate form a VPatch
(short for Vertex Patch).
type VertexLocMap = IntMap.IntMap (Point V2 Double)
data VPatch = VPatch {vLocs :: VertexLocMap, vpFaces::[TileFace]} deriving Show
and
makeVP :: Tgraph > VPatch
calculates vertex locations using a default orientation and scale.
VPatch
is made an instance of class Transformable
so a VPatch
can also be scaled and rotated.
One essential use of this intermediate form is to be able to draw a Tgraph
with labels, rotated but without the labels themselves being rotated. We can simply convert the Tgraph
to a VPatch
, and rotate that before drawing with labels.
labelled draw (rotate someAngle (makeVP g))
We can also align a VPatch
using vertex labels.
alignXaxis :: (Vertex, Vertex) > VPatch > VPatch
So if g
is a Tgraph
with vertex labels a
and b
we can align it on the xaxis with a
at the origin and b
on the positive xaxis (after converting to a VPatch
), instead of accepting the default orientation.
labelled draw (alignXaxis (a,b) (makeVP g))
Another use of VPatch
es is to share the vertex location map when drawing only subsets of the faces (see Overlaid examples in the next section).
There is a class Drawable
with instances Tgraph
, VPatch
, Patch
. When the token B
is in scope standing for a fixed backend then we can assume
draw :: Drawable a => a > Diagram B  draws nonjoin edges
drawj :: Drawable a => a > Diagram B  as with draw but also draws dashed join edges
fillDK :: Drawable a => Colour Double > Colour Double > a > Diagram B  fills with colours
where fillDK clr1 clr2
will fill darts with colour clr1
and kites with colour clr2
as well as drawing nonjoin edges.
These are the main drawing tools. However they are actually defined for any suitable backend b
so have more general types
draw :: (Drawable a, Renderable (Path V2 Double) b) =>
a > Diagram2D b
drawj :: (Drawable a, Renderable (Path V2 Double) b) =>
a > Diagram2D b
fillDK :: (Drawable a, Renderable (Path V2 Double) b) =>
Colour Double > Colour Double > a > Diagram2D b
where
type Diagram2D b = QDiagram b V2 Double Any
denotes a 2D diagram using some unknown backend b
, and the extra constraint requires b
to be able to render 2D paths.
In these notes we will generally use the simpler description of types using B
for a fixed chosen backend for the sake of clarity.
The drawing tools are each defined via the class function drawWith
using Piece
drawing functions.
class Drawable a where
drawWith :: (Piece > Diagram B) > a > Diagram B
draw = drawWith drawPiece
drawj = drawWith dashjPiece
fillDK clr1 clr2 = drawWith (fillPieceDK clr1 clr2)
To design a new drawing function, you only need to implement a function to draw a Piece
, (let us call it newPieceDraw
)
newPieceDraw :: Piece > Diagram B
This can then be elevated to draw any Drawable
(including Tgraph
s, VPatch
es, and Patch
es) by applying the Drawable
class function drawWith
:
newDraw :: Drawable a => a > Diagram B
newDraw = drawWith newPieceDraw
Class DrawableLabelled
is defined with instances Tgraph
and VPatch
, but Patch
is not an instance (because this does not retain vertex label information).
class DrawableLabelled a where
labelColourSize :: Colour Double > Measure Double > (Patch > Diagram B) > a > Diagram B
So labelColourSize c m
modifies a Patch
drawing function to add labels (of colour c
and size measure m
). Measure
is defined in Diagrams.Prelude with predefined measures tiny
, verySmall
, small
, normal
, large
, veryLarge
, huge
. For most of our diagrams of Tgraph
s, we use red labels and we also find small
is a good default size choice, so we define
labelSize :: DrawableLabelled a => Measure Double > (Patch > Diagram B) > a > Diagram B
labelSize = labelColourSize red
labelled :: DrawableLabelled a => (Patch > Diagram B) > a > Diagram B
labelled = labelSize small
and then labelled draw
, labelled drawj
, labelled (fillDK clr1 clr2)
can all be used on both Tgraph
s and VPatch
es as well as (for example) labelSize tiny draw
, or labelCoulourSize blue normal drawj
.
There are a few extra drawing functions built on top of the above ones. The function smart
is a modifier to add dashed join edges only when they occur on the boundary of a Tgraph
smart :: (VPatch > Diagram B) > Tgraph > Diagram B
So smart vpdraw g
will draw dashed join edges on the boundary of g
before applying the drawing function vpdraw
to the VPatch
for g
. For example the following all draw dashed join edges only on the boundary for a Tgraph g
smart draw g
smart (labelled draw) g
smart (labelSize normal draw) g
When using labels, the function rotateBefore
allows a Tgraph
to be drawn rotated without rotating the labels.
rotateBefore :: (VPatch > a) > Angle Double > Tgraph > a
rotateBefore vpdraw angle = vpdraw . rotate angle . makeVP
So for example,
rotateBefore (labelled draw) (90@@deg) g
makes sense for a Tgraph g
. Of course if there are no labels we can simply use
rotate (90@@deg) (draw g)
Similarly alignBefore
allows a Tgraph
to be aligned using a pair of vertex numbers before drawing.
alignBefore :: (VPatch > a) > (Vertex,Vertex) > Tgraph > a
alignBefore vpdraw (a,b) = vpdraw . alignXaxis (a,b) . makeVP
So, for example, if Tgraph g
has vertices a
and b
, both
alignBefore draw (a,b) g
alignBefore (labelled draw) (a,b) g
make sense. Note that the following examples are wrong. Even though they type check, they reorient g
without repositioning the boundary joins.
smart (labelled draw . rotate angle) g  WRONG
smart (labelled draw . alignXaxis (a,b)) g  WRONG
Instead use
smartRotateBefore (labelled draw) angle g
smartAlignBefore (labelled draw) (a,b) g
where
smartRotateBefore :: (VPatch > Diagram B) > Angle Double > Tgraph > Diagram B
smartAlignBefore :: (VPatch > Diagram B) > (Vertex,Vertex) > Tgraph > Diagram B
are defined using
restrictSmart :: Tgraph > (VPatch > Diagram B) > VPatch > Diagram B
Here, restrictSmart g vpdraw vp
uses the given vp
for drawing boundary joins and drawing faces of g
(with vpdraw
) rather than converting g
to a new VPatch
. This assumes vp
has locations for vertices in g
.
The function
drawForce :: Tgraph > Diagram B
will (smart) draw a Tgraph g
in red overlaid (using <>
) on the result of force g
as in figure 6. Similarly
drawPCompose :: Tgraph > Diagram B
applied to a Tgraph g
will draw the result of a partial composition of g
as in figure 7. That is a drawing of compose g
but overlaid with a drawing of the remainder faces of g
shown in pale green.
Both these functions make use of sharing a vertex location map to get correct alignments of overlaid diagrams. In the case of drawForce g
, we know that a VPatch
for force g
will contain all the vertex locations for g
since force only adds to a Tgraph
(when it succeeds). So when constructing the diagram for g
we can use the VPatch
created for force g
instead of starting afresh. Similarly for drawPCompose g
the VPatch
for g
contains locations for all the vertices of compose g
so compose g
is drawn using the the VPatch
for g
instead of starting afresh.
The location map sharing is done with
subVP :: VPatch > [TileFace] > VPatch
so that subVP vp fcs
is a VPatch
with the same vertex locations as vp
, but replacing the faces of vp
with fcs
. [Of course, this can go wrong if the new faces have vertices not in the domain of the vertex location map so this needs to be used with care. Any errors would only be discovered when a diagram is created.]
For cases where labels are only going to be drawn for certain faces, we need a version of subVP
which also gets rid of vertex locations that are not relevant to the faces. For this situation we have
restrictVP:: VPatch > [TileFace] > VPatch
which filters out unneeded vertex locations from the vertex location map. Unlike subVP
, restrictVP
checks for missing vertex locations, so restrictVP vp fcs
raises an error if a vertex in fcs
is missing from the keys of the vertex location map of vp
.
The rules used by our force algorithm are local and derived from the fact that there are seven possible vertex types as depicted in figure 8.
Our rules are shown in figure 9 (omitting mirror symmetric versions). In each case the TileFace
shown yellow needs to be added in the presence of the other TileFace
s shown.
To make forcing efficient we convert a Tgraph
to a BoundaryState
to keep track of boundary information of the Tgraph
, and then calculate a ForceState
which combines the BoundaryState
with a record of awaiting boundary edge updates (an update map). Then each face addition is carried out on a ForceState
, converting back when all the face additions are complete. It makes sense to apply force
(and related functions) to a Tgraph
, a BoundaryState
, or a ForceState
, so we define a class Forcible
with instances Tgraph
, BoundaryState
, and ForceState
.
This allows us to define
force :: Forcible a => a > a
tryForce :: Forcible a => a > Try a
The first will raise an error if a stuck tiling is encountered. The second uses a Try
result which produces a Left string
for failures and a Right a
for successful result a
.
There are several other operations related to forcing including
stepForce :: Forcible a => Int > a > a
tryStepForce :: Forcible a => Int > a > Try a
addHalfDart, addHalfKite :: Forcible a => Dedge > a > a
tryAddHalfDart, tryAddHalfKite :: Forcible a => Dedge > a > Try a
The first two force (up to) a given number of steps (=face additions) and the other four add a half dart/kite on a given boundary edge.
An update generator is used to calculate which boundary edges can have a certain update. There is an update generator for each force rule, but also a combined (all update) generator. The force operations mentioned above all use the default all update generator (defaultAllUGen
) but there are more general (with) versions that can be passed an update generator of choice. For example
forceWith :: Forcible a => UpdateGenerator > a > a
tryForceWith :: Forcible a => UpdateGenerator > a > Try a
In fact we defined
force = forceWith defaultAllUGen
tryForce = tryForceWith defaultAllUGen
We can also define
wholeTiles :: Forcible a => a > a
wholeTiles = forceWith wholeTileUpdates
where wholeTileUpdates
is an update generator that just finds boundary join edges to complete whole tiles.
In addition to defaultAllUGen
there is also allUGenerator
which does the same thing apart from how failures are reported. The reason for keeping both is that they were constructed differently and so are useful for testing.
In fact UpdateGenerator
s are functions that take a BoundaryState
and a focus (list of boundary directed edges) to produce an update map. Each Update
is calculated as either a SafeUpdate
(where two of the new face edges are on the existing boundary and no new vertex is needed) or an UnsafeUpdate
(where only one edge of the new face is on the boundary and a new vertex needs to be created for a new face).
type UpdateGenerator = BoundaryState > [Dedge] > Try UpdateMap
type UpdateMap = Map.Map Dedge Update
data Update = SafeUpdate TileFace
 UnsafeUpdate (Vertex > TileFace)
Completing (executing) an UnsafeUpdate
requires a touching vertex check to ensure that the new vertex does not clash with an existing boundary vertex. Using an existing (touching) vertex would create a crossing boundary so such an update has to be blocked.
The Forcible
class operations are higher order and designed to allow for easy additions of further generic operations. They take care of conversions between Tgraph
s, BoundaryState
s and ForceState
s.
class Forcible a where
tryFSOpWith :: UpdateGenerator > (ForceState > Try ForceState) > a > Try a
tryChangeBoundaryWith :: UpdateGenerator > (BoundaryState > Try BoundaryChange) > a > Try a
tryInitFSWith :: UpdateGenerator > a > Try ForceState
For example, given an update generator ugen
and any f:: ForceState > Try ForceState
, then f
can be generalised to work on any Forcible
using tryFSOpWith ugen f
. This is used to define both tryForceWith
and tryStepForceWith
.
We also specialize tryFSOpWith
to use the default update generator
tryFSOp :: Forcible a => (ForceState > Try ForceState) > a > Try a
tryFSOp = tryFSOpWith defaultAllUGen
Similarly given an update generator ugen
and any f:: BoundaryState > Try BoundaryChange
, then f
can be generalised to work on any Forcible
using tryChangeBoundaryWith ugen f
. This is used to define tryAddHalfDart
and tryAddHalfKite
.
We also specialize tryChangeBoundaryWith
to use the default update generator
tryChangeBoundary :: Forcible a => (BoundaryState > Try BoundaryChange) > a > Try a
tryChangeBoundary = tryChangeBoundaryWith defaultAllUGen
Note that the type BoundaryChange
contains a resulting BoundaryState
, the single TileFace
that has been added, a list of edges removed from the boundary (of the BoundaryState
prior to the face addition), and a list of the (3 or 4) boundary edges affected around the change that require checking or rechecking for updates.
The class function tryInitFSWith
will use an update generator to create an initial ForceState
for any Forcible
. If the Forcible
is already a ForceState
it will do nothing. Otherwise it will calculate updates for the whole boundary. We also have the special case
tryInitFS :: Forcible a => a > Try ForceState
tryInitFS = tryInitFSWith defaultAllUGen
Note that (force . force)
does the same as force
, but we might want to chain other force
related steps in a calculation.
For example, consider the following combination which, after decomposing a Tgraph
, forces, then adds a half dart on a given boundary edge (d
) and then forces again.
combo :: Dedge > Tgraph > Tgraph
combo d = force . addHalfDart d . force . decompose
Since decompose:: Tgraph > Tgraph
, the instances of force
and addHalfDart d
will have type Tgraph > Tgraph
so each of these operations, will begin and end with conversions between Tgraph
and ForceState
. We would do better to avoid these wasted intermediate conversions working only with ForceState
s and keeping only those necessary conversions at the beginning and end of the whole sequence.
This can be done using tryFSOp
. To see this, let us first reexpress the forcing sequence using the Try
monad, so
force . addHalfDart d . force
becomes
tryForce <=< tryAddHalfDart d <=< tryForce
Note that (<=<
) is the Kliesli arrow which replaces composition for Monads (defined in Control.Monad). (We could also have expressed this right to left sequence with a left to right version tryForce >=> tryAddHalfDart d >=> tryForce
). The definition of combo
becomes
combo :: Dedge > Tgraph > Tgraph
combo d = runTry . (tryForce <=< tryAddHalfDart d <=< tryForce) . decompose
This has no performance improvement, but now we can pass the sequence to tryFSOp
to remove the unnecessary conversions between steps.
combo :: Dedge > Tgraph > Tgraph
combo d = runTry . tryFSOp (tryForce <=< tryAddHalfDart d <=< tryForce) . decompose
The sequence actually has type Forcible a => a > Try a
but when passed to tryFSOp
it specialises to type ForceState > Try ForseState
. This ensures the sequence works on a ForceState
and any conversions are confined to the beginning and end of the sequence, avoiding unnecessary intermediate conversions.
To avoid creating touching vertices (or crossing boundaries) a BoundaryState
keeps track of locations of boundary vertices. At around 35,000 face additions in a single force
operation the calculated positions of boundary vertices can become too inaccurate to prevent touching vertex problems. In such cases it is better to use
recalibratingForce :: Forcible a => a > a
tryRecalibratingForce :: Forcible a => a > Try a
These work by recalculating all vertex positions at 20,000 step intervals to get more accurate boundary vertex positions. For example, 6 decompositions of the kingGraph
has 2,906 faces. Applying force
to this should result in 53,574 faces but will go wrong before it reaches that. This can be fixed by calculating either
recalibratingForce (decompositions kingGraph !!6)
or using an extra force
before the decompositions
force (decompositions (force kingGraph) !!6)
In the latter case, the final force
only needs to add 17,864 faces to the 35,710 produced by decompositions (force kingGraph) !!6
.
Tgraph
sAsking if two Tgraph
s are equivalent (the same apart from choice of vertex numbers) is a an npcomplete problem. However, we do have an efficient guided way of comparing Tgraph
s. In the module Tgraph.Rellabelling
we have
sameGraph :: (Tgraph,Dedge) > (Tgraph,Dedge) > Bool
The expression sameGraph (g1,d1) (g2,d2)
asks if g2
can be relabelled to match g1
assuming that the directed edge d2
in g2
is identified with d1
in g1
. Hence the comparison is guided by the assumption that d2
corresponds to d1
.
It is implemented using
tryRelabelToMatch :: (Tgraph,Dedge) > (Tgraph,Dedge) > Try Tgraph
where tryRelabelToMatch (g1,d1) (g2,d2)
will either fail with a Left report
if a mismatch is found when relabelling g2
to match g1
or will succeed with Right g3
where g3
is a relabelled version of g2
. The successful result g3
will match g1
in a maximal tileconnected collection of faces containing the face with edge d1
and have vertices disjoint from those of g1
elsewhere. The comparison tries to grow a suitable relabelling by comparing faces one at a time starting from the face with edge d1
in g1
and the face with edge d2
in g2
. (This relies on the fact that Tgraph
s are connected with no crossing boundaries, and hence tileconnected.)
The above function is also used to implement
tryFullUnion:: (Tgraph,Dedge) > (Tgraph,Dedge) > Try Tgraph
which tries to find the union of two Tgraph
s guided by a directed edge identification. However, there is an extra complexity arising from the fact that Tgraph
s might overlap in more than one tileconnected region. After calculating one overlapping region, the full union uses some geometry (calculating vertex locations) to detect further overlaps.
Finally we have
commonFaces:: (Tgraph,Dedge) > (Tgraph,Dedge) > [TileFace]
which will find common regions of overlapping faces of two Tgraph
s guided by a directed edge identification. The resulting common faces will be a subcollection of faces from the first Tgraph
. These are returned as a list as they may not be a connected collection of faces and therefore not necessarily a Tgraph
.
In Empires and SuperForce we discussed forced boundary coverings which were used to implement both a superForce
operation
superForce:: Forcible a => a > a
and operations to calculate empires.
We will not repeat the descriptions here other than to note that
forcedBoundaryECovering:: Tgraph > [Tgraph]
finds boundary edge coverings after forcing a Tgraph
. That is, forcedBoundaryECovering g
will first force g
, then (if it succeeds) finds a collection of (forced) extensions to force g
such that
force g
as internal edges.force g
(kite or dart) has been included in the collection.(possible here means – not leading to a stuck Tgraph
when forced.) There is also
forcedBoundaryVCovering:: Tgraph > [Tgraph]
which does the same except that the extensions have all boundary vertices internal rather than just the boundary edges.
Combinations such as
compForce:: Tgraph > Tgraph  compose after forcing
allCompForce:: Tgraph > [Tgraph]  iterated (compose after force) while not emptyTgraph
maxCompForce:: Tgraph > Tgraph  last item in allCompForce (or emptyTgraph)
make use of theorems established in Graphs,Kites and Darts and Theorems. For example
compForce = uncheckedCompose . force
which relies on the fact that composition of a forced Tgraph
does not need to be checked for connectedness and no crossing boundaries. Similarly, only the initial force
is necessary in allCompForce
with subsequent iteration of uncheckedCompose
because composition of a forced Tgraph
is necessarily a forced Tgraph
.
The type
data TrackedTgraph = TrackedTgraph
{ tgraph :: Tgraph
, tracked :: [[TileFace]]
} deriving Show
has proven useful in experimentation as well as in producing artwork with darts and kites. The idea is to keep a record of subcollections of faces of a Tgraph
when doing both force operations and decompositions. A list of the subcollections forms the tracked list associated with the Tgraph
. We make TrackedTgraph
an instance of class Forcible
by having force operations only affect the Tgraph
and not the tracked list. The significant idea is the implementation of
decomposeTracked :: TrackedTgraph > TrackedTgraph
Decomposition of a Tgraph
involves introducing a new vertex for each long edge and each kite join. These are then used to construct the decomposed faces. For decomposeTracked
we do the same for the Tgraph
, but when it comes to the tracked collections, we decompose them reusing the same new vertex numbers calculated for the edges in the Tgraph
. This keeps a consistent numbering between the Tgraph
and tracked faces, so each item in the tracked list remains a subcollection of faces in the Tgraph
.
The function
drawTrackedTgraph :: [VPatch > Diagram B] > TrackedTgraph > Diagram B
is used to draw a TrackedTgraph
. It uses a list of functions to draw VPatch
es. The first drawing function is applied to a VPatch
for any untracked faces. Subsequent functions are applied to VPatch
es for the tracked list in order. Each diagram is beneath later ones in the list, with the diagram for the untracked faces at the bottom. The VPatch
es used are all restrictions of a single VPatch
for the Tgraph
, so will be consistent in vertex locations. When labels are used, there is also a drawTrackedTgraphRotated
and drawTrackedTgraphAligned
for rotating or aligning the VPatch
prior to applying the drawing functions.
Note that the result of calculating empires (see Empires and SuperForce ) is represented as a TrackedTgraph
. The result is actually the common faces of a forced boundary covering, but a particular element of the covering (the first one) is chosen as the background Tgraph
with the common faces as a tracked subcollection of faces. Hence we have
empire1, empire2 :: Tgraph > TrackedTgraph
drawEmpire :: TrackedTgraph > Diagram B
Figure 10 was also created using TrackedTgraph
s.
Previous related blogs are:
Piece
s and Patch
es (without using Tgraphs) and provided a version of decomposing for Patches (decompPatch
).Forcible
was introduced subsequently).force
, compose
, decompose
.I thought about this because of yesterday's article about the person who needed to count the 3colorings of an icosahedron, but didn't try constructing any to see what they were like.
Around 2015 Katara, then age 11, saw me writing up my long series of articles about the Cosmic Call message and asked me to explain what the mysterious symbols meant. (It's intended to be a message that space aliens can figure out even though they haven't met us.)
I said “I bet you could figure it out if you tried.” She didn't believe me and she didn't want to try. It seemed insurmountable.
“Okay,” I said, handing her a printed copy of page 1. “Sit on the chaise there and just look at it for five minutes without talking or asking any questions, while I work on this. Then I promise I'll explain everything.”
She figured it out in way less than five minutes. She was thrilled to discover that she could do it.
I think she learned something important that day: A person can accomplish a lot with a few minutes of uninterrupted silent thinking, perhaps more than they imagine, and certainly a lot more than if they don't try.
I think there's a passage somewhere in Zen and the Art of Motorcycle Maintenance about how, when you don't know what to do next, you should just sit with your mouth shut for a couple of minutes and see if any ideas come nibbling. Sometimes they don't. But if there are any swimming around, you won't catch them unless you're waiting for them.
The GHC developers are happy to announce the availability of GHC 9.6.5. Binary distributions, source distributions, and documentation are available on the release page.
This release is primarily a bugfix release addressing some issues found in the 9.6 series. These include:
process
library to 1.6.19.0 to avoid a potential
command injection vulnerability on Windows for clients of this library. This isnâ€™t
known to affect GHC itself, but allows users who depend on the installed
version of the process
to avoid the issue.hsc2hs
wrapper using flags from the
compiler build environment (#24050).fasmshortcutting
optimisation with O2
as it is known
to result in unsoundess and incorrect runtime results in some cases (#24507).LDFLAGS
into account when configuring a linker (#24565).A full accounting of changes can be found in the release notes. As some of the fixed issues do affect correctness users are encouraged to upgrade promptly.
We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, WellTyped, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose ongoing financial and inkind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of opensource contributors whose work comprise this release.
As always, do give this release a try and open a ticket if you see anything amiss.
Enjoy!
Zubin
I will be presenting a summary of the content in this post live on an upcoming episode of The Haskell Unfolder (scheduled for April 16th, 2024, 1830 UTC). I encourage you to join the live stream and submit any questions you might have after reading this post!
The Haskell Unfolder Episode 23: specialisation
Overloaded functions are common in Haskell, but they come with a cost. Thankfully, the GHC specialiser is extremely good at removing that cost. We can therefore write highlevel, polymorphic programs and be confident that GHC will compile them into very efficient, monomorphised code. In this episode, we’ll demystify the seemingly magical things that GHC is doing to achieve this.
Specialization is an optimization technique used by GHC to eliminate the performance overhead of adhoc polymorphism and enable other powerful optimizations. However, specialization is not free, since it requires more work by GHC during compilation and leads to larger executables. In fact, excessive specialization can result in significant increases in compilation cost and executable size with minimal runtime performance benefits. For this reason, GHC pessimistically avoids excessive specialization by default and may leave relatively lowcost performance improvements undiscovered in doing so.
Optimistic Haskell programmers hoping to take advantage of these missed opportunities are thus faced with the difficult task of discovering and enacting an optimal set of specializations for their program while balancing any performance improvements with the increased compilation costs and executable sizes. Until now, this dance was a clunky one involving desperately wading through GHC Core dumps only to come up with a precarious, inefficient, unmotivated set of pragmas and/or GHC flags that seem to improve performance.
In this twopart series of posts, I describe the recent work we have done to improve this situation and make optimal specialization of Haskell programs more of a science and less of a dark art. In this first post, I will
In the next post of the series, I will
The intended audience of this post includes intermediate Haskell developers who want to know more about specialization and adhoc polymorphism in GHC, and advanced Haskell developers who are interested in systematic approaches to specializing their applications in ways that minimize compilation cost and executable sizes while maximizing performance gains.
This work was made possible thanks to Hasura, who have supported many of WellTyped’s successful initiatives to improve tooling for commercial Haskell users.
In Haskell, an adhoc polymorphic or overloaded function is one whose type
contains class constraints. For example, this f
is an overloaded function:
For some type a
such that Ord a
and Num a
instances are provided, f
takes two values of type a
and evaluates to another a
.
Importantly, unlike type arguments, those class constraints are not erased at
runtime! Actually, they will be passed to f
just like any other value
argument, meaning f
at runtime is more like:
How does the definition of f
change to represent this? And what do these
ord_a
and num_a
values look like? This is how it works:
<
in the body of f
) become record selectors that are
applied to the dictionaries to look up the appropriate definitions.Thus, f
at runtime is more like:
f :: Ord a > Num a > a > a > a f ord_a num_a x y = if (<) ord_a x y then (+) num_a x y else () num_a x y
The previouslyinfix class operators are now applied in prefix position to select the appropriate definitions out of the dictionaries, which are then applied to the arguments.
We can see this for ourselves by compiling the definition of f
in a module
F.hs
and emitting the intermediate representation (in GHC’s Core language):
The O
flag enables optimizations, and the
ddumpds
flag tells GHC to dump the Core representation of
the program after desugaring, before optimizations. The other flags make the
output more readable.
For a comprehensive introduction to GHC Core and the flags GHC accepts for viewing it, check out The Haskell Unfolder Episode 9: GHC Core.
The above command will output the following Core for f
:
The if
has been transformed into a case
(Core has no if
construct). The
$dOrd
and $dNum
arguments are the Ord a
and Num a
instance dictionaries,
respectively. The <
operator is applied in prefix position (as are all
operators in Core) to the $dOrd
dictionary to get the appropriate
implementation of <
, which is further applied to x
and y
. The 
and +
operators in the branches of the case
are similar.
The extra allocations required to pass these implicit dictionary arguments and apply selectors to them do result in a measurable overhead, albeit one that is insignificant for most intents and purposes. As we will see, the real cost of adhoc polymorphism comes from the optimizations it prevents rather than the overhead it introduces.
In this context, specialization refers to the removal of
adhoc polymorphism. When we specialize an overloaded expression e :: C a => S a
, we create a new binding eT :: S T
, where T
is some concrete type
for which a C T
instance exists. Here eT
is the specialization of e
at (or to) type T
.
For example, we can manually create a specialization of f
at type Int
. The
source definition stays exactly the same, only the type changes:
At the Core level, the dictionaries that were passed as value arguments to
f
are now used directly in the body of fInt
. If we add the definition of
fInt
to our example module and compile it as we did before, we get the
following output:
f = \ @a $dOrd $dNum x y > case < $dOrd x y of { False >  $dNum x y; True > + $dNum x y } fInt = \ x y > case < $fOrdInt x y of { False >  $fNumInt x y; True > + $fNumInt x y }
fInt
no longer accepts dictionary arguments, and instead references the global
Ord Int
and Num Int
dictionaries directly. In fact, this definition of
fInt
is exactly what the GHC specializer would create if it decided to
specialize f
to Int
. We can see this for ourselves by manually instructing
GHC to do the specialization using a SPECIALIZE
pragma.
Our whole module is now:
module F where
{# SPECIALIZE f :: Int > Int > Int #}
f :: (Ord a, Num a) => a > a > a
f x y =
if x < y then
x + y
else
x  y
fInt :: Int > Int > Int
fInt x y =
if x < y then
x + y
else
x  y
And the ddumpds
Core output becomes:
fInt
= \ x y >
case < $fOrdInt x y of {
False >  $fNumInt x y;
True > + $fNumInt x y
}
$sf
= \ x y >
case < $fOrdInt x y of {
False >  $fNumInt x y;
True > + $fNumInt x y
}
f = \ @a $dOrd $dNum x y >
case < $dOrd x y of {
False >  $dNum x y;
True > + $dNum x y
}
The GHC generated specialization is named $sf
(all specializations that GHC
generates are prefixed by $s
). Note that our specialization (fInt
) and the
GHC generated specialization ($sf
) are exactly equivalent!
The above transformation really is all that the GHC specializer does to our programs. It may not be immediately clear why this optimization is a meaningful optimization at all. That is because specialization is an enabling optimization: The real benefit comes from the optimizations that it enables later in the pipeline, such as inlining.
Inlining is the replacement of defined (toplevel or letbound) variables with
their definitions. Although f
and its specialization $sf
look similar, the
key difference is that f
includes calls to “unknown” functions passed as part
of the dictionary arguments, while $sf
includes calls to “known” functions
contained in the $fOrdInt
and $fNumInt
dictionaries. Since GHC has access to
the definitions of those dictionaries and the contained functions, they can be
inlined, exposing yet more opportunities for optimization.
We can see this in action by comparing the fully optimized bindings of our
example module to those just after desugaring. To do this, compile using the
same command as above but add the ddumpsimpl
flag,
which tells GHC to dump the Core at the end of the Core optimization pipeline
(also add fforcerecomp
to force recompilation, since
we haven’t changed the code since our last compilation):
The dumped output is:
==================== Desugar (after optimization) ==================== Result size of Desugar (after optimization) = {terms: 57, types: 37, coercions: 0, joins: 0/0} fInt = \ x y > case < $fOrdInt x y of { False >  $fNumInt x y; True > + $fNumInt x y } $sf = \ x y > case < $fOrdInt x y of { False >  $fNumInt x y; True > + $fNumInt x y } f = \ @a $dOrd $dNum x y > case < $dOrd x y of { False >  $dNum x y; True > + $dNum x y } ==================== Tidy Core ==================== Result size of Tidy Core = {terms: 44, types: 29, coercions: 0, joins: 0/0} fInt = \ x y > case x of { I# x1 > case y of { I# y1 > case <# x1 y1 of { __DEFAULT > I# (# x1 y1); 1# > I# (+# x1 y1) } } } f = \ @a $dOrd $dNum x y > case < $dOrd x y of { False >  $dNum x y; True > + $dNum x y }  Local rules for imported ids  "USPEC f @Int" forall $dNum $dOrd. f $dOrd $dNum = fInt
The output of the desugaring pass is in the “Desugar (after optimization)” section, while the fully optimized output is in the “Tidy Core” section. The name “Desugar (after optimization)” only means it is the desugared Core output after GHC’s simple optimizer has run. The simple optimizer only does very lightweight, pure transformations to the Core program. We will still refer to the Core output of this stage as “unoptimized”.
During the full optimization pipeline, GHC identified the equivalence between
fInt
and $sf
and decided to remove $sf
. The fully optimized binding for
fInt
is unboxing the Int
s (pattern matching on the I#
constructor) and
using efficient primitive operations (<#
, #
, +#
), while the fully
optimized binding for f
is the same as the unoptimized binding. The optimizer
simply couldn’t do anything with those opaque dictionaries in the way!
At the bottom of the output is the rewrite rule that the
SPECIALIZE
pragma created, which will cause any calls of
f
known to be at type Int
to be rewritten as applications of fInt
. This is
what allows the rest of the program to benefit from the specialization. The rule
simply discards the dictionary arguments $dNum :: Num Int
and $dOrd :: Ord Int
, which is safe because of global typeclass coherence: any dictionaries
passed explicitly must have originally come from the same global instances.
In summary, by replacing the opaque dictionary arguments to f
with references
to the concrete Ord Int
and Num Int
dictionaries in fInt
, GHC was able to
do a lot more optimization later in the pipeline.
In our example module, we manually instructed GHC to generate a specialization
of f
at Int
using a SPECIALIZE
pragma. In reality, we
often rely on GHC to figure out what specializations are necessary and generate
them for us automatically. GHC needs to be careful though, since specialization
requires the creation and optimization of more bindings, which increases
compilation costs and executable sizes
GHC uses several heuristics to avoid excessive automatic specialization by default. The heuristics are very pessimistic, which means GHC can easily miss valuable specialization opportunities that programmers may wish to manually address. This is precisely the manual effort that our recent work aims to assist, so before we go any further it’s important that we understand exactly when and why GHC decides specialization should (or should not) happen.
GHC will only potentially attempt automatic specialization in exactly one scenario: An overloaded call at a concrete, statically known type is encountered (we’ll refer to such calls as “specializable” calls from now on). This means that automatic specialization will only ever be triggered at call sites, not definition sites. Even in this scenario, there are other factors to consider which the following example will demonstrate.
Let’s add a binding foo
to our example module F.hs
from above:
foo
makes a specializable call to f
at the concrete type Integer
, so we
might expect automatic specialization to happen. However, the inliner beats the
specializer to the punch here, which is evident in the
ddumpsimpl
output:
$wfoo
= \ ww ww1 >
case integerLt ww ww1 of {
False > integerSub ww ww1;
True > integerAdd ww ww1
}
foo = \ ds > case ds of { (ww, ww1) > $wfoo ww ww1 }
Instead of specializing, GHC decided to eliminate the call entirely by inlining
f
, thus exposing other optimization opportunities (such as
worker/wrapper) which GHC took advantage of. This is
intended, since f
is so small and GHC knows that inlining it is very cheap and
likely worth the performance outcomes.
Another way we can observe the inlining decision by GHC here is via the
ddumpinlinings
flag, which causes GHC to dump the
names of any bindings it decides to inline. Compiling our module with
results in output indicating that GHC did decide to inline f
:
Inlining done: F.f
GHC prefers inlining over specialization, when possible, since inlining eliminates calls and doesn’t require creation of new bindings. However, excessive inlining is often even more dangerous than excessive specialization. So, even when a specializable call is deemed too costly to inline, GHC will still attempt to specialize it.
We can artificially create such a scenario in our example by adjusting what GHC
calls the “unfolding use threshold”. An “unfolding” is, roughly, the definition
of a binding that GHC uses when it decides to inline or specialize calls to that
binding. The unfolding use threshold governs the maximum effective
size^{1} of unfoldings that GHC will inline, and it can be
manually adjusted using the
funfoldingusethreshold flag. Let’s set the
unfolding use threshold to 1, essentially making GHC think all inlining is very
expensive, and check the ddumpsimpl
output:
ghc F.hs O fforcerecomp ddumpsimpl funfoldingusethreshold=1
As we can see, GHC did specialize the call:
...
f_$sf1
= \ x y >
case integerLt x y of {
False > integerSub x y;
True > integerAdd x y
}
foo = \ ds > case ds of { (ww, ww1) > f_$sf1 ww ww1 }
 Local rules for imported ids 
"SPEC f @Integer" forall $dOrd $dNum. f $dOrd $dNum = f_$sf1
...
The name of the specialization (f_$sf1
) and the rewrite rule indicate that GHC
did successfully automatically specialize the overloaded call to f
.
Interestingly, the Core terms for foo
and its specialization f_$sf
are
alphaequivalent to the terms we arrived at when GHC
inlined the call and applied worker/wrapper
instead^{2}, with the specialization playing the same
role as the worker.
We have now discussed two prerequisites for automatic specialization of a call:
In fact, for specializable calls which occur in the definition module of the overloaded binding (as was the case in our previous example), these are the only prerequisites. When the overloaded binding is imported from another module (as is most often the case), there are additional prerequisites which we’ll discuss now.
INLINABLE
pragmaGHC performs separate compilation (as opposed to whole program compilation),
compiling one Haskell module at a time. When GHC compiles a module, it produces
not only compiled code in an object file, but also an interface file (with
suffix .hi
) . The interface file contains information about the module that
GHC might need to reference when compiling other modules, such as the names and
types of the bindings exported by the module. If certain criteria are met, GHC
will include a binding’s unfolding in the module’s interface file so that it can
be used later for crossmodule inlining or specialization. Such unfoldings are
referred to as exposed unfoldings.
Now, you might reasonably wonder: If unfoldings are used to do these powerful optimizations, why does GHC only expose unfoldings which meet some criteria? Why not expose all unfoldings? The reason is that during compilation, GHC holds the interfaces of every module in the program in memory. Thus, to keep GHC’s own default performance and memory usage reasonable, module interfaces need to be as small as possible while still producing welloptimized programs. One way that GHC achieves this is by limiting the size of unfoldings that get included in interface files so that only small unfoldings are exposed by default.
There’s another wrinkle here that impacts crossmodule specialization: Even if GHC decides to expose an overloaded binding’s unfolding, and a specializable call to that binding occurs in another module, GHC will still never automatically specialize that call unless it has been given explicit permission to create the specialization. Such explicit permission can only be given in one of the following ways:
INLINABLE
or
INLINE
pragma.fspecializeaggressively
flag
while compiling the calling module.Let’s explore this fact by continuing with our example. Move foo
, which makes
a specializable call to f
, to another module Foo.hs
that has
funfoldingusethreshold
set to 1 to fool
the inliner as before:
{# OPTIONS_GHC funfoldingusethreshold=1 #}
module Foo where
import F
foo :: (Integer, Integer) > Integer
foo (x, y) = f x y
Also remove everything from F.hs
except f
, for good measure:
Since f
is so small, we might expect GHC to expose its unfolding in the F.hi
module interface by default. If we compile with just
we get the object file F.o
and the interface file F.hi
. We can determine
whether GHC decided to expose the unfolding of f
by viewing the contents of
the interface file using GHC’s showiface
option:
Specific information for each binding in the module is listed towards the bottom
of the output. The GHC Core of any exposed unfoldings will be displayed under
their respective bindings. In this case, the information for f
looks like
this:
bcb4b04f3cbb5e6aa2f776d6226a0930
f :: (Ord a, Num a) => a > a > a
[]
It only includes the type, no unfolding! This is because at GHC’s default
optimization level of O0
, the
fomitinterfacepragmas
and
fignoreinterfacepragmas
flags are
enabled which prevent unfoldings (among other things) from being included in and
read from the module interfaces. Recompile with optimizations enabled and check
the module interface again:
This time, GHC did expose the unfolding:
152dd20f273a86bea689edd6a298afe6 f :: (Ord a, Num a) => a > a > a [..., Unfolding: Core: <vanilla> \ @a ($dOrd['Many] :: Ord a) ($dNum['Many] :: Num a) (x['Many] :: a) (y['Many] :: a) > case < @a $dOrd x y of wild { False >  @a $dNum x y True > + @a $dNum x y }]
Remember, we still haven’t given GHC explicit permission to specialize calls to
f
across modules, so we should expect the fully optimized Core of Foo.hs
to
still include the overloaded call to f
. Let’s check:
The dumped Core includes:
$wfoo = \ ww ww1 > f $fOrdInteger $fNumInteger ww ww1 foo = \ ds > case ds of { (ww, ww1) > $wfoo ww ww1 }
Indeed, GHC applied the worker/wrapper transformation
to foo
, but was not able to specialize the call to f
, despite it meeting our
previously discussed prerequisites for automatic specialization.
There is a warning flag in GHC that can notify us of such a case:
Wallmissedspecializations
. Compile
Foo.hs
again, including this flag:
This will output the following warning:
Foo.hs: warning: [Wallmissedspecialisations]
Could not specialise imported function ‘f’
Probable fix: add INLINABLE pragma on ‘f’
If we do what the warning says by adding an INLINABLE
pragma on f
, and dump the core of Foo.hs
, we’ll see that automatic
specialization succeeds:
$sf
= \ x y >
case integerLt x y of {
False > integerSub x y;
True > integerAdd x y
}
foo = \ ds > case ds of { (ww, ww1) > $sf ww ww1 }
 Local rules for imported ids 
"SPEC/Foo f @Integer" forall $dOrd $dNum. f $dOrd $dNum = $sf
Removing the INLINABLE
pragma on f
and instead enabling
fspecializeaggressively
has the same
result.
We have now covered all the major prerequisites for automatic specialization. To summarize them, here is a decision graph illustrating the various ways that an arbitrary function call can trigger automatic specialization:
Now that we fully understand how, why, and when the GHC specializer works, we can move on to discussing the real problems that result from its behavior. Most of this discussion will be left for the next post in this series, but before concluding, I want to introduce something I call “the specialization spectrum”.
Specialization is a very valuable compiler optimization, but I’ve mentioned many times throughout this post that excessive specialization can be a bad thing. This prompts a question: How do we know if we are appropriately benefitting from specialization? The meaning of “appropriately” here depends on applicationspecific requirements that dictate the desired size of our executables, how much we care about compilation costs, and how much we care about performance.
For example, if we want to maximize performance at all costs, we should make sure that we are generating and using the set of specializations that maximize the performance metrics we’re interested in, disregarding the increase in compilation costs and executable sizes.
Essentially, our goal is to find our ideal spot in the specialization spectrum.
This is our search space, with performance on one axis and code size and compilation cost on the other. The plotted points represent important applicationagnostic points in the spectrum. Those points are:
fexposeallunfoldings
and
fspecializeaggressively
. Importantly,
this is not always equivalent to max performance! If we generate useless
specializations that result in little to no performance improvements but do
grow the code size, we can end up losing performance due to more code
swapping in and out of CPU caches.The dotted line illustrates an approximate “optimal path” representing the results we might see as we generate all specializations in order of decreasing performance improvement.
This framework makes it clear that this really is just an optimization problem, with all the normal issues of traditional optimization problems at play. Unfortunately, in the absence of good tools for exploring this spectrum, it is particularly easy for programmers to get lost and go down treacherous, unoptimal paths like this:
Such cases are deceptive, making the programmer think they have landed in a good spot when they are actually in a poorperforming local optimum. Fortunately, the tools and techniques we’ll discuss in the next post of this series will greatly simplify optimal search of the specialization spectrum.
This concludes our introductory exploration of specialization. Here’s what we have learned:
In the next post of this series, we will apply all of what we have learned so far on some example applications, and demonstrate how the new tools we have developed can help us achieve optimal specialization and performance.
The effective size of an unfolding can be thought of as the number of terms in the Core representation of the unfolding, plus or minus some discounts that are applied depending on where GHC is considering inlining the unfolding.↩︎
This hints at a weak confluence of GHC Core and the reductions (i.e. optimizations) that the GHC optimizer applies to it.↩︎
Even with something like this in a cabal.project
file:
package *
ghcoptions: fexposeallunfoldings fspecializeaggressively
Some overloaded calls may still not get specialized! This can occur if a
chain of calls to overloaded functions includes a call to an overloaded
function in a GHC boot library that cannot be reinstalled by Cabal, e.g.
base
, which does not have its unfolding exposed. The only way to
specialize such calls is to build boot libraries from source with
fexposeallunfoldings
and
fspecializeaggressively
, and include
the snippet above in a cabal.project
file.
Additionally, some specific scenarios can cause overloaded calls to appear
late in the optimization pipeline. To specialize those calls,
flatespecialise
(British spelling required) is
necessary, which runs another specialization pass at the end of GHC’s Core
optimization pipeline.
Further, even after the above, some overloaded calls may still survive
without fpolymorphicspecialisation
(British spelling required), which is known to be unsound at the time of
writing. Unfortunately, in complex applications, total elimination of
overloaded calls is still quite a difficult goal to achieve.↩︎
The GHC developers are very pleased to announce the availability of the third alpha release of GHC 9.10.1. Binary distributions, source distributions, and documentation are available at downloads.haskell.org.
We hope to have this release available via ghcup shortly.
GHC 9.10 will bring a number of new features and improvements, including:
The introduction of the GHC2024
language edition, building upon
GHC2021
with the addition of a number of widelyused extensions.
Partial implementation of the GHC Proposal #281, allowing visible quantification to be used in the types of terms.
Extension of LinearTypes to allow linear let
and where
bindings
The implementation of the exception backtrace proposal, allowing the annotation of exceptions with backtraces, as well as other userdefined context
Further improvements in the info table provenance mechanism, reducing code size to allow IPE information to be enabled more widely
Javascript FFI support in the WebAssembly backend
Improvements in the fragmentation characteristics of the lowlatency nonmoving garbage collector.
… and many more
A full accounting of changes can be found in the release notes. As always, GHC’s release status, including planned future releases, can be found on the GHC Wiki status.
This alpha is the penultimate prerelease leading to 9.10.1. In two weeks we plan to publish a release candidate, followed, if all things go well, by the final release a week later.
We would like to thank GitHub, IOG, the Zw3rk stake pool, WellTyped, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell Foundation, and other anonymous contributors whose ongoing financial and inkind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of opensource contributors whose work comprise this release.
As always, do give this release a try and open a ticket if you see anything amiss.
I recently wrote about things that are backwards in Australia. I made this controversial claim:
The sun in the Southern Hemisphere moves counterclockwise across the sky over the course of the day, rather than clockwise. Instead of coming up on the left and going down on the right, as it does in the Northern Hemisphere, it comes up on the right and goes down on the left.
Many people found this confusing and I'm not sure our minds met on this. I am going to try to explain and see if I can clear up the puzzles.
“Which way are you facing?” was a frequent question. “If you're facing north, it comes up on the right, not the left.”
(To prevent endless parenthetical “(in the Northern Hemisphere)” qualifications, the rest of this article will describe how things look where I live, in the northern temperate zones. I understand that things will be reversed in the Southern Hemisphere, and quite different near the equator and the poles.)
Here's what I think the sky looks like most of the day on most of the days of the year:
The sun is in the southern sky through the entire autumn, winter, and spring. In summer it is sometimes north of the celestial equator, for up to a couple of hours after sunrise and before sunset, but it is still in the southern sky most of the time. If you are watching the sun's path through the sky, you are looking south, not north, because if you are looking north you do not see the sun, it is behind you.
Some people even tried to argue that if you face north, the sun's path is a counterclockwise circle, rather than a clockwise one. This is risible. Here's my grandfather's old grandfather clock. Notice that the hands go counterclockwise! You study the clock and disagree. They don't go counterclockwise, you say, they go clockwise, just like on every other clock. Aha, but no, I say! If you were standing behind the clock, looking into it with the back door open, then you would clearly see the hands go counterclockwise! Then you kick me in the shin, as I deserve.
Yes, if you were to face away from the sun, its path could be said to be counterclockwise, if you could see it. But that is not how we describe things. If I say that a train passed left to right, you would not normally expect me to add “but it would have been right to left, had I been facing the tracks”.
At least one person said they had imagined the sun rising directly ahead, then passing overhead, and going down in back. Okay, fair enough. You don't say that the train passed left to right if you were standing on the tracks and it ran you down.
Except that the sun does not pass directly overhead. It only does that in the tropics. If this person were really facing the sun as it rose, and stayed facing that way, the sun would go up toward their right side. If it were a train, the train tracks would go in a big curve around their right (south) side, from left to right:
Mixed gauge track (950 and 1435mm) at Sassari station, Sardinia, 1996 by user Afterbrunel, CC BYSA 3.0 DEED, via Wikimedia Commons. I added the big green arrows.
After the train passed, it would go back the other way, but they wouldn't be able see it, because it would be behind them. If they turned around to watch it go, it would still go left to right:
And if they were to turn to follow it over the course of the day, they would be turning left to right the whole time, and the sun would be moving from left to right the whole time, going up on the left and coming down on the right, like the hands of a clock — “clockwise”, as it were.
One correspondent suggested that perhaps many people in technologically advanced countries are not actually familiar with how the sun and moon move, and this was the cause of some of the confusion. Perhaps so, it's certainly tempting to dismiss my critics as not knowing how the sun behaves. The other possibility is that I am utterly confused. I took Observational Astronomy in college twice, and failed both times.
Anyway, I will maybe admit that “left to right” was unclear. But I will not recant my claim that the sun moves clockwise. E pur si muove in senso orario.
Here I was just dead wrong. I said:
In the Northern Hemisphere, the shadow of a sundial proceeds clockwise, from left to right.
Absolutely not, none of this is correct. First, “left to right”. Here's a diagram of a typical sundial:
It has a stickyup thing called a ‘gnomon’ that casts a shadow across the numbers, and the shadow moves from left to right over the course of the day. But obviously the sundial will work just as well if you walk around and look at it from the other side:
It still goes clockwise, but now clockwise is right to left instead of left to right.
It's hard to read because the numerals are upside down? Fine, whatever:
Here, unlike with the sun, “go around to the other side” is perfectly reasonable.
Talking with Joe Ardent, I realized that not even “clockwise” is required for sundials. Imagine the southfacing wall of a building, with the gnomon sticking out of it perpendicular. When the sun passes overhead, the gnomon will cast a shadow downwards on the wall, and the downwardpointing shadow will move from left to right — counterclockwise — as the sun makes its way from east to west. It's not even farfetched. Indeed, a search for “vertical sundials” produced numerous examples:
Sundial on the Moot Hall by David Dixon, CC BY 2.0 https://creativecommons.org/licenses/by/2.0, via Wikimedia Commons and Geograph.
Finally, it was reported that there were complaints on Hacker News that Australians do not celebrate July 4th. Ridiculous! All patriotic Americans celebrate July 4th.
I thought at first was going to be kind of a dumb article, because it was just going to be a list of banal stuff like:
but a couple of years back I was rather startled to realize that in the Southern Hemisphere the sun comes up on the right and goes counterclockwise through the sky instead of coming up on the left and going clockwise as I have seen it do all my life, and that was pretty interesting.
Then more recently I was thinking about it more carefully and I was stunned when I realized that the phases of the moon go the other way. So I thought I'd should actually make the list, because a good deal of it is not at all obvious. Or at least it wasn't to me!
When it's day here, it's night there, and vice versa. (This isn't a Southern Hemisphere thing, it's an Eastern Hemisphere thing.)
When it's summer here, it's winter there, and vice versa. Australians celebrate Christmas by going to the beach, and July 4th with sledding and patriotic snowball fights.
Australia's warmer zones are in the north, not the south. Their birds fly north for the winter. But winter is in July, so the reversals cancel out and birds everywhere fly south in September and October, and north in March and April, even though birds can't read.
The sun in the Southern Hemisphere moves counterclockwise across the sky over the course of the day, rather than clockwise. Instead of coming up on the left and going down on the right, as it does in the Northern Hemisphere, it comes up on the right and goes down on the left.
In the Northern Hemisphere, the shadow of a sundial proceeds clockwise, from left to right. (This is the reason clock hands also go clockwise: for backward compatibility with sundials.) But in the Southern Hemisphere, the shadow on a sundial goes counterclockwise.
In the Southern Hemisphere, the designs on the moon appear upsidedown compared with how they look in the Northern Hemisphere. Here's a picture of the full moon as seen from the Northern Hemisphere. The big crater with the bright rays that is prominent in the bottom half of the picture is Tycho.
In the Southern Hemisphere the moon looks like this, with Tycho on top:
Australians see the moon upsidedown because their heads are literally pointing in the opposite direction.
For the same reason, the Moon's phases in the Southern Hemisphere sweep from left to right instead of from right to left. In the Northern Hemisphere they go like this as the month passes from new to full:
And then in the same direction from full back to new:
But in the Southern Hemisphere the moon changes from left to right instead:
And then:
Unicode U+263D and U+263E are called FIRST QUARTER MOON
â˜½ and
LAST QUARTER MOON
â˜¾ , respectively, and are depicted Northern Hemisphere style.
(In the Southern Hemisphere, â˜½ appears during the last quarter of the month, not
the first.) Similarly the emoji U+1F311 through U+1F318,
ğŸŒ‘ğŸŒ’ğŸŒ“ğŸŒ”ğŸŒ•ğŸŒ–ğŸŒ—ğŸŒ˜ are depicted in Northern Hemisphere order, and have Northern Hemisphere
descriptions like â€œğŸŒ’ waxing crescent moonâ€�.Â In the Southern Hemisphere, ğŸŒ’ is
actually a waning crescent.
In the Northern Hemisphere a Foucault pendulum will knock down the pins in clockwise order, as shown in the picture. (This one happens to be in Barcelona.) A Southern Hemisphere Foucault pendulum will knock them down in counterclockwise order, because the Earth is turning the other way, as viewed from the fulcrum of the pendulum.
Northern Hemisphere tornadoes always rotate counterclockwise. Southern Hemisphere tornadoes always rotate clockwise.
As far as I know the thing about water going down the drain in one direction or the other is not actually true.
Several people took issue with some of the claims in this article, and the part about sundials was completely wrong. I wrote a followup.
inspectiontesting
was created over five years ago. You may want to glance over Joachim Breitner A promise checked is a promise kept: inspection testing) Haskell Symposium paper introducing it.
Already in 2018 I thought it's a fine tool, but it's more geared towards /library/ writers. They can check on (some) examples that the promises they make about the libraries they write work at least on some examples.
What we cannot do with current inspectiontesting
is check that the actual "reallife" use of the library works as intended.
Luckily, relatively recently, GHC got a feature to include all Core bindings in the interface files. While the original motivation is different (to make Template Haskell run fast), the fwriteifsimplifiedcore enables us to inspect (as in inspection testing) the "production" Core (not the test examples).
The cabalcoreinspection
is a very quick & dirty proofofconcept of this idea.
Let me illustrate this with two examples.
In neither example I need to do any test setup, other than configuring cabalcoreinspection
(though configuration is now hardcoded). Compare that to configuring e.g. HLint (HLint has user definable rules, and these are actually powerful tool). In fact, cabalcoreinspection
is nothing more than a linter for Core.
First example is countChars
as in Haskell Symposium Paper.
The promise is (actually: was) that no intermediate Text
values are created.
As far as I know, we cannot use inspectiontesting
in its current form to check anything about nonlocal bindings, so if countChars
is defined in an application, we would need to duplicate its definition in the testsuite to inspect it. That is not great.
With Core inspection, we can look at the actual Core of the module (as it is in the compiler interface file).
The prototype doesn't have any configuration, but if we imagine it has we could ask it to check that Example.countChars
should not contain type Text
. The prototype prints
Text value created with decodeUtf8With1 in countChars
So that's not the case. The intermediate Text
value is created. In fact, nowadays text
doesn't promise that toUpper
fuses with anything.
A nice thing about cabalcoreinspection
that (in theory) it could check any definition in any module as long as it's compiled with fwriteifsimplifiedcore
. So we could check things for our friends, if we care about something specific.
Second example is about GHC.Generics. I use a simple generic equality, but this could apply to any GHC.Generics
based deriving. (You should rather use deriving stock Eq
, but generic equality is a simplest example which I remembered for now).
The generic equality might be defined in a library. And library author may actually have tested it with inspectiontesting
. But does it work on our type?
If we have
data T where T1 :: Int > Char > T T2 :: Bool > Double > T deriving Generic instance Eq T where (==) = genericEq
it does. The cabalcoreinspection
doesn't complain.
But if we add a third constructor
cabalcoreinspection
barfs:
The T
becomes too large for GHC to want inline all the generics stuff.
It won't be fair to blame the library author, for example for
generic equality still optimises well, and doesn't have any traces of GHC.Generics
. We may actually need to (and may be adviced to) tune some GHC optimisation parameters. But we need a way to check whether they are enough. inspectiontesting
doesn't help, but a proper version of core inspection would be perfect for that task.
The fwriteifsimplifiedcore enables us to automate inspection of actual Core. That is a huge win. The cabalcoreinspection
is just a proofofconcept, and I might try to make it into a real thing, but right now I don't have a real use case for it.
I'm also worried about Note [Interface File with Core: Sharing RHSs]
in GHC. It says
In order to avoid duplicating definitions for bindings which already have unfoldings we do some minor headstands to avoid serialising the RHS of a definition if it has *any* unfolding.
 Only global things have unfoldings, because local things have had their unfoldings stripped.
 For any global thing which has an unstable unfolding, we just use that.
Currently this optimisation is disabled, so cabalcoreinspection
works, but if it's enabled as is; then INLINE
d bindings won't have their simplified unfoldings preserved (but rather only "inlineRHS"), and that would destroy Core inspection possibility.
But until then, cabalcoreinspection
idea works.
tl;dr If you’d like a job with us, send your application as soon as possible.
We are looking for a Haskell expert to join our team at WellTyped. We are seeking a strong allround Haskell developer who can help us with various client projects (rather than particular experience in any one specific field). This is a great opportunity for someone who is passionate about Haskell and who is keen to improve and promote Haskell in a professional context.
We are a team of top notch Haskell experts. Founded in 2008, we were the first company dedicated to promoting the mainstream commercial use of Haskell. To achieve this aim, we help companies that are using or moving to Haskell by providing a range of services including consulting, development, training, support, and improvement of the Haskell development tools.
We work with a wide range of clients, from tiny startups to wellknown multinationals. For some we do proprietary Haskell development and consulting. For others, much of the work involves opensource development and cooperating with the rest of the Haskell community. We have established a track record of technical excellence and satisfied customers.
Our company has a strong engineering culture. All our managers and decision makers are themselves Haskell developers. Most of us have an academic background and we are not afraid to apply proper computer science to customers’ problems, particularly the fruits of FP and PL research.
We are a selffunded company so we are not beholden to external investors and can concentrate on the interests of our clients, our staff and the Haskell community.
The role is not tied to a single specific project or task, is fully remote, and has flexible working hours.
In general, work for WellTyped could cover any of the projects and activities that we are involved in as a company. The work may involve:
We try wherever possible to arrange tasks within our team to suit peoples’ preferences and to rotate to provide variety and interest. At present you are more likely to be working on general Haskell development than on GHC or teaching, however.
Our ideal candidate has excellent knowledge of Haskell, whether from industry, academia or personal interest. Familiarity with other languages, lowlevel programming and good software engineering practices are also useful. Good organisation and ability to manage your own time and reliably meet deadlines is important. You should also have good communication skills.
You are likely to have a bachelor’s degree or higher in computer science or a related field, although this isn’t a requirement.
Further (optional) bonus skills:
The offer is initially for one year full time, with the intention of a long term arrangement. Living in England is not required. We may be able to offer either employment or subcontracting, depending on the jurisdiction in which you live. The salary range is 60k–100k GBP per year.
If you are interested, please apply by email to jobs@welltyped.com. Tell us why you are interested and why you would be a good fit for WellTyped, and attach your CV. Please indicate how soon you might be able to start.
The deadline for applications is Tuesday April 30th 2024.
Every year I try to solve some problems from the Advent of Code (AoC) competition in a not straightforward way. Let’s solve the part one of the day 19 problem Aplenty by compiling the problem input to an executable file.
This post was originally published on abhinavsarkar.net.
What the problem presents as input is essentially a program. Here is the example input:
Each line in the first section of the input is a code block. The bodies of the blocks have statements of these types:
A
) or Reject (R
) that terminate the program.rfg
as the last statement of the px
block in the first line.The problem calls the statements “rules”, the blocks “workflows”, and the program “system”.
All blocks of the program operates on a set of four values: x
, m
, a
, and s
. The problem calls them “ratings”, and each set of ratings is for/forms a “part”. The second section of the input specifies a bunch of these parts to run the system against.
This seems to map very well to a C program, with Accept
and Reject
returning true
and false
respectively, and jumps accomplished using goto
s. So that’s what we’ll do: we’ll compile the problem input to a C program, then compile that to an executable, and run it to get the solution to the problem.
And of course, we’ll do all this in Haskell. First some imports:
{# LANGUAGE LambdaCase #}
{# LANGUAGE StrictData #}
module Main where
import qualified Data.Array as Array
import Data.Char (digitToInt, isAlpha, isDigit)
import Data.Foldable (foldl', foldr')
import Data.Function (fix)
import Data.Functor (($>))
import qualified Data.Graph as Graph
import Data.List (intercalate, (\\))
import qualified Data.Map.Strict as Map
import System.Environment (getArgs)
import qualified Text.ParserCombinators.ReadP as P
import Prelude hiding (GT, LT)
First, we parse the input program to Haskell data types. We use the ReadP parser library built into the Haskell standard library.
data Part = Part
{ partX :: Int,
partM :: Int,
partA :: Int,
partS :: Int
} deriving (Show)
data Rating = X  M  A  S deriving (Show, Eq)
emptyPart :: Part
emptyPart = Part 0 0 0 0
addRating :: Part > (Rating, Int) > Part
addRating p (r, v) = case r of
X > p {partX = v}
M > p {partM = v}
A > p {partA = v}
S > p {partS = v}
partParser :: P.ReadP Part
partParser =
foldl' addRating emptyPart
<$> P.between (P.char '{') (P.char '}')
(partRatingParser `P.sepBy1` P.char ',')
partRatingParser :: P.ReadP (Rating, Int)
partRatingParser =
(,) <$> ratingParser <*> (P.char '=' *> intParser)
ratingParser :: P.ReadP Rating
ratingParser =
P.get >>= \case
'x' > pure X
'm' > pure M
'a' > pure A
's' > pure S
_ > P.pfail
intParser :: P.ReadP Int
intParser =
foldl' (\n d > n * 10 + d) 0 <$> P.many1 digitParser
digitParser :: P.ReadP Int
digitParser = digitToInt <$> P.satisfy isDigit
parse :: (Show a) => P.ReadP a > String > Either String a
parse parser text = case P.readP_to_S (parser <* P.eof) text of
[(res, "")] > Right res
[(_, s)] > Left $ "Leftover input: " <> s
out > Left $ "Unexpected output: " <> show out
Part
is a Haskell data type representing parts, and Rating
is an enum for, well, ratings^{1}.
Following that are parsers for parts and ratings, written in Applicative and Monadic styles using the basic parsers and combinators provided by the ReadP library.
Finally, we have the parse
function to run a parser on an input. We can try parsing parts in GHCi:
> parse partParser "{x=2127,m=1623,a=2188,s=1013}"
Right (Part {partX = 2127, partM = 1623, partA = 2188, partS = 1013})
Next, we represent and parse the program, I mean, the system:
newtype System = System (Map.Map WorkflowName Workflow) deriving (Show, Eq) data Workflow = Workflow { wName :: WorkflowName, wRules :: [Rule] } deriving (Show, Eq) type WorkflowName = String data Rule = AtomicRule AtomicRule  If Condition AtomicRule deriving (Show, Eq) data AtomicRule = Jump WorkflowName  Accept  Reject deriving (Show, Eq, Ord) data Condition = Comparison Rating CmpOp Int deriving (Show, Eq) data CmpOp = LT  GT deriving (Show, Eq)
A System
is a map of workflows by their names. A Workflow
has a name and a list of rules. A Rule
is either an AtomicRule
, or an If
rule. An AtomicRule
is either a Jump
to another workflow by name, or an Accept
or Reject
rule. The Condition
of an If
rule is a less that (LT
) or a greater than (GT
) Comparison
of some Rating
of an input part with an integer value.
Now, it’s time to parse the system:
systemParser :: P.ReadP System systemParser = System . foldl' (\m wf > Map.insert (wName wf) wf m) Map.empty <$> workflowParser `P.endBy1` P.char '\n' workflowParser :: P.ReadP Workflow workflowParser = Workflow <$> P.many1 (P.satisfy isAlpha) <*> P.between (P.char '{') (P.char '}') (ruleParser `P.sepBy1` P.char ',') ruleParser :: P.ReadP Rule ruleParser = (AtomicRule <$> atomicRuleParser) P.<++ ifRuleParser ifRuleParser :: P.ReadP Rule ifRuleParser = If <$> (Comparison <$> ratingParser <*> cmpOpParser <*> intParser) <*> (P.char ':' *> atomicRuleParser) atomicRuleParser :: P.ReadP AtomicRule atomicRuleParser = do c : _ < P.look case c of 'A' > P.char 'A' $> Accept 'R' > P.char 'R' $> Reject _ > (Jump .) . (:) <$> P.char c <*> P.many1 (P.satisfy isAlpha) cmpOpParser :: P.ReadP CmpOp cmpOpParser = P.choice [P.char '<' $> LT, P.char '>' $> GT]
Parsing is straightforward as there are no recursive data types or complicated precedence or associativity rules here. We can exercise it in GHCi (output formatted for clarity):
> parse workflowParser "px{a<2006:qkq,m>2090:A,rfg}" Right ( Workflow { wName = "px", wRules = [ If (Comparison A LT 2006) (Jump "qkq"), If (Comparison M GT 2090) Accept, AtomicRule (Jump "rfg") ] } )
Excellent! We can now combine the part parser and the system parser to parse the problem input:
data Input = Input System [Part] deriving (Show) inputParser :: P.ReadP Input inputParser = Input <$> systemParser <*> (P.char '\n' *> partParser `P.endBy1` P.char '\n')
Before moving on to translating the system to C, let’s write an interpreter so that we can compare the output of our final C program against it for validation.
Each system has a workflow named “in”, where the execution of the system starts. Running the system results in True
if the run ends with an Accept
rule, or in False
if the run ends with a Reject
rule. With this in mind, let’s cook up the interpreter:
runSystem :: System > Part > Bool runSystem (System system) part = runRule $ Jump "in" where runRule = \case Accept > True Reject > False Jump wfName > jump wfName jump wfName = case Map.lookup wfName system of Just workflow > runRules $ wRules workflow Nothing > error $ "Workflow not found in system: " <> wfName runRules = \case (rule : rest) > case rule of AtomicRule aRule > runRule aRule If cond aRule > if evalCond cond then runRule aRule else runRules rest _ > error "Workflow ended without accept/reject" evalCond = \case Comparison r LT value > rating r < value Comparison r GT value> rating r > value rating = \case X > partX part M > partM part A > partA part S > partS part
The interpreter starts by running the rule to jump to the “in” workflow. Running a rule returns True
or False
for Accept
or Reject
rules respectively, or jumps to a workflow for Jump
rules. Jumping to a workflow looks it up in the system’s map of workflows, and sequentially runs each of its rules.
An AtomicRule
is run as previously mentioned. An If
rule evaluates its condition, and either runs the consequent rule if the condition is true, or moves on to running the rest of the rules in the workflow.
That’s it for the interpreter. We can run it on the example input:
> inputText < readFile "input.txt" > Right (Input system parts) = parse inputParser inputText > runSystem system (parts !! 0) True > runSystem system (parts !! 1) False
The AoC problem requires us to return the sum total of the ratings of the parts that are accepted by the system:
solve :: Input > Int solve (Input system parts) = sum . map (\(Part x m a s) > x + m + a + s) . filter (runSystem system) $ parts
Let’s run it for the example input:
It returns the correct answer! Next up, we generate some C code.
But first, a quick digression to graphs. A Controlflow graph or CFG, is a graph of all possible paths that can be taken through a program during its execution. It has many uses in compilers, but for now, we use it to generate more readable C code.
Using the Data.Graph
module from the containers
package, we write the function to create a controlflow graph for our system/program, and use it to topologically sort the workflows:
type Graph' a =
(Graph.Graph, Graph.Vertex > (a, [a]), a > Maybe Graph.Vertex)
cfGraph :: Map.Map WorkflowName Workflow > Graph' WorkflowName
cfGraph system =
graphFromMap
. Map.toList
. flip Map.map system
$ \(Workflow _ rules) >
flip concatMap rules $ \case
AtomicRule (Jump wfName) > [wfName]
If _ (Jump wfName) > [wfName]
_ > []
where
graphFromMap :: (Ord a) => [(a, [a])] > Graph' a
graphFromMap m =
let (graph, nLookup, vLookup) =
Graph.graphFromEdges $ map (\(f, ts) > (f, f, ts)) m
in (graph, \v > let (x, _, xs) = nLookup v in (x, xs), vLookup)
toposortWorkflows :: Map.Map WorkflowName Workflow > [WorkflowName]
toposortWorkflows system =
let (cfg, nLookup, _) = cfGraph system
in map (fst . nLookup) $ Graph.topSort cfg
Graph'
is a simpler type for a graph of nodes of type a
. The cfGraph
function takes a the map from workflow names to workflows — that is, a system — and returns a controlflow graph of workflow names. It does this by finding jumps from workflows to other workflows, and connecting them.
Then, the toposortWorkflows
function uses the created CFG to topologically sort the workflows. We’ll see this in action in a bit. Moving on to …
The compiler, for now, simply generates the C code for a given system. We write a ToC
typeclass for convenience:
class ToC a where toC :: a > String instance ToC Part where toC (Part x m a s) = "{" <> intercalate ", " (map show [x, m, a, s]) <> "}" instance ToC CmpOp where toC = \case LT > "<" GT > ">" instance ToC Rating where toC = \case X > "x" M > "m" A > "a" S > "s" instance ToC AtomicRule where toC = \case Accept > "return true;" Reject > "return false;" Jump wfName > "goto " <> wfName <> ";" instance ToC Condition where toC = \case Comparison rating op val > toC rating <> " " <> toC op <> " " <> show val instance ToC Rule where toC = \case AtomicRule aRule > toC aRule If cond aRule > "if (" <> toC cond <> ") { " <> toC aRule <> " }" instance ToC Workflow where toC (Workflow wfName rules) = wfName <> ":\n" <> intercalate "\n" (map ((" " <>) . toC) rules) instance ToC System where toC (System system) = intercalate "\n" [ "bool runSystem(int x, int m, int a, int s) {", " goto in;", intercalate "\n" (map (toC . (system Map.!)) $ toposortWorkflows system), "}" ] instance ToC Input where toC (Input system parts) = intercalate "\n" [ "#include <stdbool.h>", "#include <stdio.h>\n", toC system, "int main() {", " int parts[][4] = {", intercalate ",\n" (map ((" " <>) . toC) parts), " };", " int totalRating = 0;", " for(int i = 0; i < " <> show (length parts) <> "; i++) {", " int x = parts[i][0];", " int m = parts[i][1];", " int a = parts[i][2];", " int s = parts[i][3];", " if (runSystem(x, m, a, s)) {", " totalRating += x + m + a + s;", " }", " }", " printf(\"%d\", totalRating);", " return 0;", "}" ]
As mentioned before, Accept
and Reject
rules are converted to return true
and false
respectively, and Jump
rules are converted to goto
s. If
rules become if
statements, and Workflow
s become block labels followed by block statements.
A System
is translated to a function runSystem
that takes four parameters, x
, m
, a
and s
, and runs the workflows translated to blocks by executing goto in
.
Finally, an Input
is converted to a C file with the required includes, and a main
function that solves the problem by calling the runSystem
function for all parts.
Let’s throw in a main
function to put everything together.
main :: IO () main = do file < head <$> getArgs code < readFile file case parse inputParser code of Right input > putStrLn $ toC input Left err > error err
The main
function reads the input from the file provided as the command line argument, parses it and outputs the generated C code. Let’s run it now.
We compile our compiler and run it to generate the C code for the example problem:
$ ghc make aplenty.hs
$ ./aplenty exinput.txt > aplenty.c
This is the C code it generates:
#include <stdbool.h> #include <stdio.h> bool runSystem(int x, int m, int a, int s) { goto in; in: if (s < 1351) { goto px; } goto qqz; qqz: if (s > 2770) { goto qs; } if (m < 1801) { goto hdj; } return false; qs: if (s > 3448) { return true; } goto lnx; lnx: if (m > 1548) { return true; } return true; px: if (a < 2006) { goto qkq; } if (m > 2090) { return true; } goto rfg; rfg: if (s < 537) { goto gd; } if (x > 2440) { return false; } return true; qkq: if (x < 1416) { return true; } goto crn; hdj: if (m > 838) { return true; } goto pv; pv: if (a > 1716) { return false; } return true; gd: if (a > 3333) { return false; } return false; crn: if (x > 2662) { return true; } return false; } int main() { int parts[][4] = { {787, 2655, 1222, 2876}, {1679, 44, 2067, 496}, {2036, 264, 79, 2244}, {2461, 1339, 466, 291}, {2127, 1623, 2188, 1013} }; int totalRating = 0; for(int i = 0; i < 5; i++) { int x = parts[i][0]; int m = parts[i][1]; int a = parts[i][2]; int s = parts[i][3]; if (runSystem(x, m, a, s)) { totalRating += x + m + a + s; } } printf("%d", totalRating); return 0; }
We see the toposortWorkflows
function in action, sorting the blocks in the topological order of jumps between them, as opposed to the original input. Does this work? Only one way to know:
$ gcc aplenty.c o solution
$ ./solution
19114
Perfect! The solution matches the interpreter output.
By studying the output C code, we spot some possibilities for optimizing the compiler output. Notice how the lnx
block returns same value (true
) regardless of which branch it takes:
So, we should be able to replace it with:
If we do this, the lnx
block becomes degenerate, and hence the jumps to the block can be inlined, turning the qs
block from:
to:
which makes the if
statement in the qs
block redundant as well. Hence, we can repeat the previous optimization and further reduce the generated code.
Another possible optimization is to inline the blocks to which there are only single jumps from the rest of the blocks, for example the qqz
block.
Let’s write these optimizations.
simplifyWorkflows :: System > System simplifyWorkflows (System system) = System $ Map.map simplifyWorkflow system where simplifyWorkflow (Workflow name rules) = Workflow name $ foldr' ( \r rs > case rs of [r']  ruleOutcome r == ruleOutcome r' > rs _ > r : rs ) [last rules] $ init rules ruleOutcome = \case If _ aRule > aRule AtomicRule aRule > aRule
simplifyWorkflows
goes over all workflows and repeatedly removes the statements from the end of the blocks that has same outcome as the statement previous to them.
inlineRedundantJumps :: System > System inlineRedundantJumps (System system) = System $ foldl' (flip Map.delete) (Map.map inlineJumps system) $ Map.keys redundantJumps where redundantJumps = Map.map (\wf > let ~(AtomicRule rule) = head $ wRules wf in rule) . Map.filter (\wf > length (wRules wf) == 1) $ system inlineJumps (Workflow name rules) = Workflow name $ map inlineJump rules inlineJump = \case AtomicRule (Jump wfName)  Map.member wfName redundantJumps > AtomicRule $ redundantJumps Map.! wfName If cond (Jump wfName)  Map.member wfName redundantJumps > If cond $ redundantJumps Map.! wfName rule > rule
inlineRedundantJumps
find the jumps to degenerate workflows and inlines them. It does this by first going over all workflows and creating a map of degenerate workflow names to the only rule in them, and then replacing the jumps to such workflows with the only rules.
removeJumps :: System > System removeJumps (System system) = let system' = foldl' (flip $ Map.adjust removeJumpsWithSingleJumper) system $ toposortWorkflows system in System . foldl' (flip Map.delete) system' . (\\ ["in"]) $ workflowsWithNJumpers 0 system' where removeJumpsWithSingleJumper (Workflow name rules) = Workflow name $ init rules <> case last rules of AtomicRule (Jump wfName)  wfName `elem` workflowsWithSingleJumper > let (Workflow _ rules') = system Map.! wfName in rules' rule > [rule] workflowsWithSingleJumper = workflowsWithNJumpers 1 system workflowsWithNJumpers n sys = let (cfg, nLookup, _) = cfGraph sys in map (fst . nLookup . fst) . filter (\(_, d) > d == n) . Array.assocs . Graph.indegree $ cfg
removeJumps
does two things: first, it finds blocks with only one jumper, and inlines their statements to the jump location. Then it finds blocks to which there are no jumps, and removes them entirely from the program. It uses the workflowsWithNJumpers
helper function that uses the controlflow graph of the system to find all workflows to which there are n
number of jumps, where n
is provided as an input to the function. Note the usage of the toposortWorkflows
function here, which makes sure that we remove the blocks in topological order, accumulating as many statements as possible in the final program.
With these functions in place, we write the optimize
function:
optimize :: System > System optimize = applyTillUnchanged (removeJumps . inlineRedundantJumps . simplifyWorkflows) where applyTillUnchanged :: (Eq a) => (a > a) > a > a applyTillUnchanged f = fix (\recurse x > if f x == x then x else recurse (f x))
We execute the three optimization functions repeatedly till a fixed point is reached for the resultant System
, that is, till there are no further possibilities of optimization.
Finally, we change our main
function to apply the optimizations:
main :: IO ()
main = do
file < head <$> getArgs
code < readFile file
case parse inputParser code of
Right (Input system parts) >
putStrLn . toC $ Input (optimize system) parts
Left err > error err
Compiling the optimized compiler and running it as earlier, generates this C code for the runSystem
function now:
bool runSystem(int x, int m, int a, int s) { goto in; in: if (s < 1351) { goto px; } if (s > 2770) { return true; } if (m < 1801) { goto hdj; } return false; px: if (a < 2006) { goto qkq; } if (m > 2090) { return true; } if (s < 537) { return false; } if (x > 2440) { return false; } return true; qkq: if (x < 1416) { return true; } if (x > 2662) { return true; } return false; hdj: if (m > 838) { return true; } if (a > 1716) { return false; } return true; }
It works well^{2}. We now have 1.7x fewer lines of code as compared to before^{3}.
This was another attempt to solve Advent of Code problems in somewhat unusual ways. This year we learned some basics of compilation. Swing by next year for more weird ways to solve simple problems.
The full code for this post is available here.
If you liked this post, please leave a comment.
Roman, known better online as effectfully, is interviewed by Wouter and Joachim. On his path to becoming a Plutus language developer at IOG, he learned English to read Software Foundations, Â has encountered many spaceleaks, and used Haskell to prevent robots from killing people.
Roman, known better online as effectfully, is interviewed by Wouter and Joachim. On his path to becoming a Plutus language developer at IOG, he learned English to read Software Foundations, Â has encountered many spaceleaks, and used Haskell to prevent robots from killing people.
This is the second installment of the indepth series of blogposts on developing native macOS and iOS applications using both Haskell and Swift/SwiftUI. This post covers how to call (nontrivial) Haskell functions from Swift by using a foreign function callingconvention strategy similar to that described by Calling Purgatory from Heaven: Binding to Rust in Haskell that requires argument and result marshaling. You may find the other blog posts in this series interesting.
The series of blog posts is further accompanied by a github repository where each commit matches a step of this tutorial. If in doubt regarding any step, check the matching commit to make it clearer.
This writeup has been crossposted to Rodrigo’s Blog.
We’ll pick up from where the last post ended – we have set up an XCode project
that includes our headers generated from Haskell modules with foreign export
s
and linking against the foreign library declared in the cabal file. We have
already been able to call a very simple Haskell function on integers from Swift
via Haskell’s C foreign export feature and Swift’s C interoperability.
This part concerns itself with calling idiomatic Haskell functions, which typically involve userdefined datatypes as inputs and outputs, from Swift. Moreover, these functions should be made available to Swift transparently, such that Swift calls them as it does other idiomatic functions, with user defined structs and classes.
For the running example, the following notveryinteresting function will suffice to showcase the method we will use to expose this function from Haskell to Swift, which easily scales to other complex data types and functions.
data User
= User { name :: String
, age :: Int
}
birthday :: User > User
birthday user = user{age = user.age + 1}
The Swift side should wrap Haskell’s birthday
:
struct User { let name: String let age: Int } // birthday(user: User(name: "Anton", age: 33)) = User(name: "Anton", age: 34) func birthday(user: User) > User { // Calls Haskell function... }
To support this workflow, we need a way to convert the User datatype from Haskell to Swift, and vice versa. We are going to serialize (most) inputs and outputs of a function. Even though the serialization as it will be described may seem complex, it can be automated with Template Haskell and Swift Macros and packed into a neat interface – which I’ve done in haskellswift.
As a preliminary step, we add the User
data type and birthday
function to
haskellframework/src/MyLib.hs
, and the Swift equivalents to
SwiftHaskell/ContentView.swift
from the haskellxswiftprojectsteps
example project.
Marshaling the inputs and outputs of a function, from the Swift perspective, means to serialize the input values into strings, and receive the output value as a string which is then decoded into a Swift value. The Haskell perspective is dual.
Marshaling/serializing is a very robust solution to foreign language interoperability. While there is a small overhead of encoding and decoding at a function call, it almost automatically extends to, and enables, all sorts of data to be transported across the language boundary, without it being vulnerable to compiler implementation details and memory representation incompatibilities.
We will use the same marshaling strategy that Calling Purgatory from Heaven: Binding to Rust in Haskell does. In short, the idiomatic Haskell function is wrapped by a lowlevel one which deserializes the Haskell values from the argument buffers, and serializes the function result to a buffer that the caller provides. More specifically,
For each argument of the original function, we have a Ptr CChar
and Int
– a string of characters and the size of that string (a.k.a CStringLen
)
For the result of the original function, we have two additional arguments, Ptr CChar
and Ptr Int
–
an empty buffer in memory, and a pointer to the size of that buffer, both allocated by the caller.
For each argument, we parse the C string into a Haskell value that serves as an argument to the original function.
We call the original function
We overwrite the memory location containing the original size of the buffer with the required size of the buffer to fit the result (which may be smaller or larger than the actual size). If the buffer is large enough we write the result to it.
From the Swift side, we read the amount of bytes specified in the memory location that now contains the required size. If it turns out that the required size is larger than the buffer’s size, we need to retry the function call with a larger buffer.
We will use JSON
as the serialization format: this choice is motivated
primarily by convenience because Swift can derive JSON instances for datatypes
out of the box (without incurring in extra dependencies), and in Haskell we can
use aeson
to the same effect. In practice, it could be best to use a format
such as CBOR or Borsh which are binary formats optimised for compactness and
serialization performance.
Extending the User
example requires User
to be decodable, which can be done automatically by adding to the User
declaration:
With the appropriate extensions and importing the necessary modules in MyLib
:
The MyForeignLib
module additionally must import
import Foreign.Ptr import Foreign.Storable import Foreign.Marshal import Data.Aeson import Data.ByteString import Data.ByteString.Unsafe
Now, let’s (foreign) export a function c_birthday
that wraps
birthday
above in haskellframework/flib/MyForeignLib.hs
, using the
described method.
First, the type definition of the function receives the buffer with the User
argument, and a
buffer to write the User
result to. We cannot use tuples because they are not
supported in foreign export declarations, but the intuition is that the first
two arguments represent the original User
input, and the two latter arguments
represent the returned User
.
Then, the implementation – decode the argument, encode the result, write result size to the given memory location and the result itself to the buffer, if it fits.
We transform the (Ptr CChar, Int)
pair into a ByteString
using
unsafePackCStringLen
, and decode a User
from the ByteString
using
decodeStrict
:
We apply the original birthday
function to the decoded user
. In our example,
this is a very boring function, but in reality this is likely a complex
idiomatic Haskell function that we want to expose to.
We encode the new_user :: User
as a ByteString
, and use
unsafeUseAsCStringLen
to get a pointer to the bytestring data and its length.
Finally, we get the size of the result buffer, write the actual size of the
result to the given memory location, and, if the actual size fits the buffer,
copy the bytes from the bytestring to the given buffer.
 (3) Encode result unsafeUseAsCStringLen (toStrict $ encode user_new) $ \(ptr,len) > do  (3.2) What is the size of the result buffer? size_avail < peek size_ptr  (3.3) Write actual size to the int ptr. poke size_ptr len  (3.4) If sufficient, we copy the result bytes to the given result buffer if size_avail < len then do  We need @len@ bytes available  The caller has to retry return () else do moveBytes result ptr len
If the written required size is larger than the given buffer, the caller will retry.
Of course, we must export this as a C function.
This makes the c_birthday
function wrapper available to Swift in the generated
header and at link time in the dynamic library.
In Swift, we want to be able to call the functions exposed from Haskell via
their C wrappers from a wrapper that feels idiomatic in Swift. In our example,
that means wrapping a call to c_birthday
in a new Swift birthday
function.
In ContentView.swift
, we make User
JSONencodable/decodable by conforming to
the Codable
protocol:
Then, we implement the Swift side of birthday
which simply calls
c_birthday
– the whole logic of birthday
is handled by the Haskell side
function (recall that birthday
could be incredibly complex, and other
functions exposed by Haskell will indeed be).
Note: in the implementation, a couple of blocks have to be wrapped with a do { ... } catch X { ... }
but I omit them in this text. You can see the commit
relevant to the Swift function wrapper implementation in the repo with all of
these details included.
First, we encode the Swift argument into JSON using the Data
type (plus its
length) that will serve as arguments to the foreign C function.
let enc = JSONEncoder() let dec = JSONDecoder() var data: Data = try enc.encode(user) let data_len = Int64(data.count)
However, a Swift Data
value, which represents the JSON as binary data, cannot
be passed directly to C as a pointer. For that, we must use
withUnsafeMutableBytes
to get an UnsafeMutableRawBufferPointer
out of the
Data
– that we can pass to the C foreign function. withUnsafeMutableBytes
receives a closure that uses an UnsafeMutableRawBufferPointer
in its scope and
returns the value returned by the closure. Therefore we can return the result of
calling it on the user Data
we encoded right away:
return data.withUnsafeMutableBytes { (rawPtr: UnsafeMutableRawBufferPointer) in // here goes the closure that can use the raw pointer, // the code for which we describe below }
We allocate a buffer for the C foreign function to insert the result of
calling the Haskell function, and also allocate memory to store the size of the
buffer. We use withUnsafeTemporaryAllocation
to allocate a buffer that can be
used in the C foreign function call. As for withUnsafeMutableBytes
, this
function also takes a closure and returns the value returned by the closure:
// The data buffer size let buf_size = 1024048 // 1024KB // A size=1 buffer to store the length of the result buffer return withUnsafeTemporaryAllocation(of: Int.self: 1) { size_ptr in // Store the buffer size in this memory location size_ptr.baseAddress?.pointee = buf_size // Allocate the buffer for the result (we need to wrap this in a do { ...} catch for reasons explained below) do { return withUnsafeTemporaryAllocation(byteCount: buf_size, alignment:1) { res_ptr in // Continues from here ... } } catch // We continue here in due time ... }
We are now nested deep within 3 closures: one binds the pointer to the argument’s data, the other the pointer to the buffer size, and the other the result buffer pointer. This means we can now call the C foreign function wrapping the Haskell function:
Recalling that the Haskell side will update the size pointed to by size_ptr
to
the size required to serialize the encoded result, we need to check if
this required size exceeds the buffer we allocated, or read the data otherwise:
if let required_size = size_ptr.baseAddress?.pointee { if required_size > buf_size { // Need to try again throw HsFFIError.requiredSizeIs(required_size) } } return dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none))
where HsFFIError
is a custom error defined as
We must now fill in the catch
block to retry the foreign function call with a
buffer of the right size:
} catch HsFFIError.requiredSizeIs(let required_size) { return withUnsafeTemporaryAllocation(byteCount: required_size, alignment:1) { res_ptr in size_ptr.baseAddress?.pointee = required_size c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress) return dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none)) } }
That seems like a lot of work to call a function from Haskell! However, despite this being a lot of code, not a whole lot is happening: we simply serialize the argument, allocate a buffer for the result, and deserialize the result into it. In the worst case, if the serialized result does not fit (the serialized data has over 1M characters), then we naively compute the function a second time (it should not be terribly complicated to avoid this work by caching the result and somehow resuming the serialization with the new buffer). Furthermore, there is a lot of bureocracy in getting the raw pointers to send off to Haskell land – the good news is that all of this can be automated away behind automatic code generation with Template Haskell and Swift Macros.
func birthday (user : User) > User { let enc = JSONEncoder() let dec = JSONDecoder() do { var data : Data = try enc.encode(user) let data_len = Int64(data.count) return try data.withUnsafeMutableBytes { (rawPtr:UnsafeMutableRawBufferPointer) in // Allocate buffer for result let buf_size = 1024000 return try withUnsafeTemporaryAllocation(of: Int.self, capacity: 1) { size_ptr in size_ptr.baseAddress?.pointee = buf_size do { return try withUnsafeTemporaryAllocation(byteCount: buf_size, alignment: 1) { res_ptr in c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress) if let required_size = size_ptr.baseAddress?.pointee { if required_size > buf_size { throw HsFFIError.requiredSizeIs(required_size) } } return try dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none)) } } catch HsFFIError.requiredSizeIs(let required_size) { print("Retrying with required size: \(required_size)") return try withUnsafeTemporaryAllocation(byteCount: required_size, alignment: 1) { res_ptr in size_ptr.baseAddress?.pointee = required_size c_birthday(rawPtr.baseAddress, data_len, res_ptr.baseAddress, size_ptr.baseAddress) return try dec.decode(User.self, from: Data(bytesNoCopy: res_ptr.baseAddress!, count: size_ptr.baseAddress?.pointee ?? 0, deallocator: .none)) } } } } } catch { print("Error decoding JSON probably: \(error)") return User(name: "", age: 0) } }
We can test that this is working by replacing ContentView
with:
struct ContentView: View { var body: some View { VStack { let user = birthday(user: User(name: "Ellie", age: 24)) Text("Postbirthday, \(user.name) is: \(user.age)!") } .padding() } }
And you should see:
I want to give a quick preview of what is made possible by using compiletime
code generation features (Template Haskell in Haskell, Swift Macros in Swift).
This foreign function code generation API is exposed by the
haskellswift project, namely the
swiftffi
Haskell library and haskellffi
Swift package (since it is out of
the scope of this tutorial, I will not cover how exactly the compiletime
codegeneration code works, but instead use the API provided by these
libraries).
With these toplevel foreign interaction facilities, coupled with the build tool also provided by haskellswift, one can easily bootstrap and develop programs mixing Haskell and Swift.
Let us consider the same example where we define an idiomatic birthday :: User > User
function in Haskell and want to be able to call it from Swift as
birthday(user: User) > User
To expose the birthday
function to Swift, we simply use the foreignExportSwift
Template Haskell function. The whole module could look like this:
{# LANGUAGE TemplateHaskell #}
module MyLib where
 ...
import Swift.FFI
data User
= User { name :: String
, age :: Int
}
deriving stock Generic
deriving anyclass FromJSON
deriving anyclass ToJSON
birthday :: User > User
birthday User{age=x, name=y} = User{age=x+1, name=y}
$(foreignExportSwift 'birthday)
The key bit is the last foreignExportSwift
call which will expose a C function
with the marshallingbased calling convention we outlined above.
On the Swift side, we want to use the dual @ForeignImportSwift
macro which
generates a Swift function wrapper which in turn invokes the C function exposed
by Haskell with the above marshalling strategy. The Swift file could
look like:
import HaskellFFI struct User: Codable { let name: String let age: Int } @ForeignImportHaskell func birthday(cconv: HsCallJSON, user: User) > User { stub() }
where birthday
could be called e.g. as:
The strategy of marshaling for foreign language boundary crossing is very robust and still performant, and is a great fit for the kind of mixedlanguage application we want to develop robustly.
Even though marshaling is required for robustly traversing the foreign language boundary, I will also explore, in a subsequent post, calling Haskell from Swift by instead coercing the memory representation of a Haskell value into a Swift one – this will mostly be a (very unsafe) and notatall robust curiosity, but it will give me an excuse to write a bit about lowlevel details in Haskell!
In yet another post, I also intend to introduce the hxs
tool for bootstrapping
Haskell x Swift projects and the libraries that make it so much easier to export
Haskell functions and import them from Swift.
The haskellxswiftprojectsteps git repository has a commit matching the steps of this guide, so if anything is unclear you can just let the code speak by itself in checking the commits.
Nonperiodic tilings with Penrose’s kites and darts
(An updated version, since original posting on Jan 6, 2022)
We continue our investigation of the tilings using Haskell with Haskell Diagrams. What is new is the introduction of a planar graph representation. This allows us to define more operations on finite tilings, in particular forcing and composing.
Previously in Diagrams for Penrose Tiles we implemented tools to create and draw finite patches of Penrose kites and darts (such as the samples depicted in figure 1). The code for this and for the new graph representation and tools described here can be found on GitHub https://github.com/chrisreade/PenroseKiteDart.
To describe the tiling operations it is convenient to work with the halftiles: LD
(left dart), RD
(right dart), LK
(left kite), RK
(right kite) using a polymorphic type HalfTile
(defined in a module HalfTile
)
data HalfTile rep
= LD rep  RD rep  LK rep  RK rep deriving (Show,Eq)
Here rep
is a type variable for a representation to be chosen. For drawing purposes, we chose twodimensional vectors (V2 Double
) and called these Pieces
.
type Piece = HalfTile (V2 Double)
The vector represents the join edge of the half tile (see figure 2) and thus the scale and orientation are determined (the other tile edges are derived from this when producing a diagram).
Finite tilings or patches are then lists of located pieces.
type Patch = [Located Piece]
Both Piece
and Patch
are made transformable so rotate
, and scale
can be applied to both and translate
can be applied to a Patch
. (Translate has no effect on a Piece
unless it is located.)
In Diagrams for Penrose Tiles we also discussed the rules for legal tilings and specifically the problem of incorrect tilings which are legal but get stuck so cannot continue to infinity. In order to create correct tilings we implemented the decompose
operation on patches.
The vector representation that we use for drawing is not well suited to exploring properties of a patch such as neighbours of pieces. Knowing about neighbouring tiles is important for being able to reason about composition of patches (inverting a decomposition) and to find which pieces are determined (forced) on the boundary of a patch.
However, the polymorphic type HalfTile
allows us to introduce our alternative graph representation alongside Piece
s.
In the module Tgraph.Prelude
, we have the new representation which treats half tiles as triangular faces of a planar graph – a TileFace
– by specialising HalfTile
with a triple of vertices (clockwise starting with the tile origin). For example
LD (1,3,4) RK (6,4,3)
type Vertex = Int
type TileFace = HalfTile (Vertex,Vertex,Vertex)
When we need to refer to particular vertices from a TileFace
we use originV
(the first vertex – red dot in figure 2), oppV
(the vertex at the opposite end of the join edge – dashed edge in figure 2), wingV
(the remaining vertex not on the join edge).
originV, oppV, wingV :: TileFace > Vertex
Tgraphs
The Tile Graphs implementation uses a newtype Tgraph
which is a list of tile faces.
newtype Tgraph = Tgraph [TileFace]
deriving (Show)
faces :: Tgraph > [TileFace]
faces (Tgraph fcs) = fcs
For example, fool
(short for a fool’s kite) is a Tgraph
with 6 faces (and 7 vertices), shown in figure 3.
fool = Tgraph [RD (1,2,3),LD (1,3,4),RK (6,2,5)
,LK (6,3,2),RK (6,4,3),LK (6,7,4)
]
(The fool is also called an ace in the literature)
With this representation we can investigate how composition works with whole patches. Figure 4 shows a twice decomposed sun on the left and a once decomposed sun on the right (both with vertex labels). In addition to decomposing the right Tgraph
to form the left Tgraph
, we can also compose the left Tgraph
to get the right Tgraph
.
After implementing composition, we also explore a force operation and an emplace operation to extend tilings.
There are some constraints we impose on Tgraph
s.
Tgraph
are the vertices that occur in the faces of the Tgraph
(and maxV
is the largest number occurring).Tgraph
faces and exterior region(s). This is important for adding faces.Tgraph
by starting from any single face and then add faces which share an edge with those already collected, we get all the Tgraph
faces. This is important for drawing purposes.In fact, if a Tgraph
is connected with no crossing boundaries, then it must be tile connected. (We could define tile connected to mean that the dual graph excluding exterior regions is connected.)
Figure 5 shows two excluded graphs which have crossing boundaries at 4 (left graph) and 13 (right graph). The left graph is still tile connected but the right is not tile connected (the two faces at the top right do not have an edge in common with the rest of the faces.)
Although we have allowed for Tgraphs
with holes (multiple exterior regions), we note that such holes cannot be created by adding faces one at a time without creating a crossing boundary. They can be created by removing faces from a Tgraph
without necessarily creating a crossing boundary.
Important We are using face as an abbreviation for halftile face of a Tgraph
here, and we do not count the exterior of a patch of faces to be a face. The exterior can also be disconnected when we have holes in a patch of faces and the holes are not counted as faces either. In graph theory, the term face would generally include these other regions, but we will call them exterior regions rather than faces.
In addition to the constructor Tgraph
we also use
checkedTgraph:: [TileFace] > Tgraph
which creates a Tgraph
from a list of faces, but also performs checks on the required properties of Tgraph
s. We can then remove or select faces from a Tgraph
and then use checkedTgraph
to ensure the resulting Tgraph
still satisfies the required properties.
selectFaces, removeFaces :: [TileFace] > Tgraph > Tgraph
selectFaces fcs g = checkedTgraph (faces g `intersect` fcs)
removeFaces fcs g = checkedTgraph (faces g \\ fcs)
Edges and Directed Edges
We do not explicitly record edges as part of a Tgraph, but calculate them as needed. Implicitly we are requiring
Tgraph
are the edges of the faces of the Tgraph
.To represent edges, a pair of vertices (a,b) is regarded as a directed edge from a to b. A list of such pairs will usually be regarded as a directed edge list. In the special case that the list is symmetrically closed [(b,a) is in the list whenever (a,b) is in the list] we will refer to this as an edge list rather than a directed edge list.
The following functions on TileFace
s all produce directed edges (going clockwise round a face).
type Dedge = (Vertex,Vertex)
joinE :: TileFace >Â Dedge  join edge  dashed in figure 2
shortE :: TileFace > Dedge  the short edge which is not a join edge
longE :: TileFace > Dedge  the long edge which is not a join edge
faceDedges :: TileFace > [Dedge]
 all three directed edges clockwise from origin
For the whole Tgraph
, we often want a list of all the directed edges of all the faces.
graphDedges :: Tgraph > [Dedge]
graphDedges = concatMap faceDedges . faces
Because our graphs represent tilings they are planar (can be embedded in a plane) so we know that at most two faces can share an edge and they will have opposite directions of the edge. No two faces can have the same directed edge. So from graphDedges g
we can easily calculate internal edges (edges shared by 2 faces) and boundary directed edges (directed edges round the external regions).
internalEdges, boundaryDedges :: Tgraph > [Dedge]
The internal edges of g
are those edges which occur in both directions in graphDedges g
. The boundary directed edges of g
are the missing reverse directions in graphDedges g
.
We also refer to all the long edges of a Tgraph
(including kite join edges) as phiEdges
(both directions of these edges).
phiEdges :: Tgraph > [Dedge]
This is so named because, when drawn, these long edges are phi
times the length of the short edges (phi
being the golden ratio which is approximately 1.618).
The module Tgraph.Convert
contains functions to convert a Tgraph
to our previous vector representation (Patch
) defined in TileLib
so we can use the existing tools to produce diagrams.
However, it is convenient to have an intermediate stage (a VPatch
= Vertex Patch) which contains both faces and calculated vertex locations (a finite map from vertices to locations). This allows vertex labels to be drawn and for faces to be identified and retained/excluded after the location information is calculated.
data VPatch = VPatch { vLocs :: VertexLocMap
, vpFaces::[TileFace]
} deriving Show
The conversion functions include
makeVP :: Tgraph > VPatch
For drawing purposes we introduced a class Drawable
which has a means to create a diagram when given a function to draw Pieces.
class Drawable a where
drawWith :: (Piece > Diagram B) > a > Diagram B
This allows us to make Patch
, VPatch
and Tgraph
instances of Drawable, and we can define special cases for the most frequently used drawing tools.
draw :: Drawable a => a > Diagram B
draw = drawWith drawPiece
drawj :: Drawable a => a > Diagram B
drawj = drawWith dashjPiece
We also need to be able to create diagrams with vertex labels, so we use a draw function modifier
class DrawableLabelled a where
labelSize :: Measure Double > (VPatch > Diagram B) > a > Diagram B
Both VPatch
and Tgraph
are made instances (but not Patch
as this no longer has vertex information). The type Measure
is defined in Diagrams, but we generally use a default measure for labels to define
labelled :: DrawableLabelled a => (VPatch > Diagram B) > a > Diagram B
labelled = labelSize (normalized 0.018)
This allows us to use, for example (where g
is a Tgraph
or VPatch
)
labelled draw g
labelled drawj g
One consequence of using abstract graphs is that there is no unique predefined way to orient or scale or position the VPatch
(and Patch
) arising from a Tgraph
representation. Our implementation selects a particular join edge and aligns it along the xaxis (unit length for a dart, phi
length for a kite) and tileconnectedness ensures the rest of the VPatch
(and Patch
) can be calculated from this.
We also have functions to reorient a VPatch
and lists of VPatch
s using chosen pairs of vertices. [Simply doing rotations on the final diagrams can cause problems if these include vertex labels. We do not, in general, want to rotate the labels – so we need to orient the VPatch
before converting to a diagram]
We previously implemented decomposition for patches which splits each halftile into two or three smaller scale halftiles.
decompPatch :: Patch > Patch
We now have a Tgraph
version of decomposition in the module Tgraph.Decompose
:
decompose :: Tgraph > Tgraph
Graph decomposition is particularly simple. We start by introducing one new vertex for each long edge (the phiEdges
) of the Tgraph. We then build the new faces from each old face using the new vertices.
As a running example we take fool
(mentioned above) and its decomposition foolD
*Main> foolD = decompose fool
*Main> foolD
Tgraph [LK (1,8,3),RD (2,3,8),RK (1,3,9)
,LD (4,9,3),RK (5,13,2),LK (5,10,13)
,RD (6,13,10),LK (3,2,13),RK (3,13,11)
,LD (6,11,13),RK (3,14,4),LK (3,11,14)
,RD (6,14,11),LK (7,4,14),RK (7,14,12)
,LD (6,12,14)
]
which are best seen together (fool
followed by foolD
) in figure 6.
Composing is meant to be an inverse to decomposing, and one of the main reasons for introducing our graph representation. In the literature, decomposition and composition are defined for infinite tilings and in that context they are unique inverses to each other. For finite patches, however, we will see that composition is not always uniquely determined.
In figure 7 (Two Levels) we have emphasised the larger scale faces on top of the smaller scale faces.
How do we identify the composed tiles? We start by classifying vertices which are at the wing tips of the (smaller) darts as these determine how things compose. In the interior of a graph/patch (e.g in figure 7), a dart wing tip always coincides with a second dart wing tip, and either
largeKiteCentre
and is at the centre of a larger kite. (See left vertex type in figure 8), orlargeDartBase
and is the base of a larger dart. (See right vertex type in figure 8)[We also call these (respectively) a deuce vertex type and a jack vertex type later in figure 10]
Around the boundary of a Tgraph
, the dart wing tips may not share with a second dart. Sometimes the wing tip has to be classified as unknown but often it can be decided by looking at neighbouring tiles. In this example of a four times decomposed sun (sunD4
), it is possible to classify all the dart wing tips as a largeKiteCentre
or a largeDartBase
so there are no unknowns.
If there are no unknowns, then we have a function to produce the unique composed Tgraph
.
compose:: Tgraph > Tgraph
Any correct decomposed Tgraph
without unknowns will necessarily compose back to its original. This makes compose
a left inverse to decompose
provided there are no unknowns.
For example, with an (n
times) decomposed sun we will have no unknowns, so these will all compose back up to a sun after n
applications of compose
. For n=4
(sunD4
– the smaller scale shown in figure 7) the dart wing classification returns 70 largeKiteCentre
s, 45 largeDartBase
s, and no unknown
s.
Similarly with the simpler foolD
example, if we classsify the dart wings we get
largeKiteCentres = [14,13]
largeDartBases = [3]
unknowns = []
In foolD
(the right hand Tgraph
in figure 6), nodes 14 and 13 are new kite centres and node 3 is a new dart base. There are no unknowns so we can use compose
safely
*Main> compose foolD
Tgraph [RD (1,2,3),LD (1,3,4),RK (6,2,5)
,RK (6,4,3),LK (6,3,2),LK (6,7,4)
]
which reproduces the original fool
(left hand Tgraph
in figure 6).
However, if we now check out unknowns for fool
we get
largeKiteCentres = []
largeDartBases = []
unknowns = [4,2]
So both nodes 2 and 4 are unknowns. It had looked as though fool
would simply compose into two half kites backtoback (sharing their long edge not their join), but the unknowns show there are other possible choices. Each unknown could become a largeKiteCentre
or a largeDartBase
.
The question is then what to do with unknowns.
In fact our compose
resolves two problems when dealing with finite patches. One is the unknowns and the other is critical missing faces needed to make up a new face (e.g the absence of any half dart).
It is implemented using an intermediary function for partial composition
partCompose:: Tgraph > ([TileFace],Tgraph)
partCompose
will compose everything that is uniquely determined, but will leave out faces round the boundary which cannot be determined or cannot be included in a new face. It returns the faces of the argument Tgraph
that were not used, along with the composed Tgraph
.
Figure 9 shows the result of partCompose
applied to two graphs. [These are force kiteD3
and force dartD3
on the left. Force is described later]. In each case, the excluded faces of the starting Tgraph
are shown in pale green, overlaid by the composed Tgraph
on the right.
Then compose
is simply defined to keep the composed faces and ignore the unused faces produced by partCompose
.
compose:: Tgraph > Tgraph
compose = snd . partCompose
This approach avoids making a decision about unknowns when composing, but it may lose some information by throwing away the uncomposed faces.
For correct Tgraph
s g
, if decompose g
has no unknowns, then compose
is a left inverse to decompose
. However, if we take g
to be two kite halves sharing their long edge (not their join edge), then these decompose to fool
which produces an empty Tgraph
when recomposed. Thus we do not have g = compose (decompose g)
in general. On the other hand we do have g = compose (decompose g)
for correct wholetile Tgraphs g
(wholetile means all halftiles of g
have their matching halftile on their join edge in g
)
Later (figure 21) we show another exception to g = compose (decompose g)
with an incorrect tiling.
We make use of
selectFacesVP :: [TileFace] > VPatch > VPatch
removeFacesVP :: [TileFace] > VPatch > VPatch
for creating VPatch
es from selected tile faces of a Tgraph
or VPatch
. This allows us to represent and draw a list of faces which need not be connected nor satisfy the no crossing boundaries property provided the Tgraph
it was derived from had these properties.
When building up a tiling, following the rules, there is often no choice about what tile can be added alongside certain tile edges at the boundary. Such additions are forced by the existing patch of tiles and the rules. For example, if a half tile has its join edge on the boundary, the unique mirror half tile is the only possibility for adding a face to that edge. Similarly, the short edge of a left (respectively, right) dart can only be matched with the short edge of a right (respectively, left) kite. We also make use of the fact that only 7 types of vertex can appear in (the interior of) a patch, so on a boundary vertex we sometimes have enough of the faces to determine the vertex type. These are given the following names in the literature (shown in figure 10): sun, star, jack (=largeDartBase), queen, king, ace (=fool), deuce (=largeKiteCentre).
The function
force :: Tgraph > Tgraph
will add some faces on the boundary that are forced (i.e new faces where there is exactly one possible choice). For example:
oppV
vertex must be a deuce vertex – add any missing half darts needed to complete the vertex.Figure 11 shows foolDminus
(which is foolD
with 3 faces removed) on the left and the result of forcing, ie force foolDminus
on the right which is the same Tgraph
we get from force foolD
(modulo vertex renumbering).
foolDminus =
removeFaces [RD(6,14,11), LD(6,12,14), RK(5,13,2)] foolD
Figures 12, 13 and 14 illustrate the result of forcing a 5times decomposed kite, a 5times decomposed dart, and a 5times decomposed sun (respectively). The first two figures reproduce diagrams from an article by Roger Penrose illustrating the extent of influence of tiles round a decomposed kite and dart. [Penrose R Tilings and quasicrystals; a nonlocal growth problem? in Aperiodicity and Order 2, edited by Jarich M, Academic Press, 1989. (fig 14)].
In figure 15, the bottom row shows successive decompositions of a dart (dashed blue arrows from right to left), so applying compose
to each dart will go back (green arrows from left to right). The black vertical arrows are force
. The solid blue arrows from right to left are (force . decompose)
being applied to the successive forced Tgraph
s. The green arrows in the reverse direction are compose
again and the intermediate (partCompose
) figures are shown in the top row with the remainder faces in pale green.
Figure 16 shows the forced graphs of the seven vertex types (with the starting Tgraph
s in red) along with a kite (top right).
These are related to each other as shown in the columns. Each Tgraph
composes to the one above (an empty Tgraph
for the ones in the top row) and the Tgraph
below is its forced decomposition. [The rows have been scaled differently to make the vertex types easier to see.]
This is technically tricky because we need to discover what vertices (and implicitly edges) need to be newly created and which ones already exist in the Tgraph
. This goes beyond a simple graph operation and requires use of the geometry of the faces. We have chosen not to do a full conversion to vectors to work out all the geometry, but instead we introduce a local representation of relative directions of edges at a vertex allowing a simple equality test.
Edge directions
All directions are integer multiples of 1/10th turn (mod
10) so we use these integers for face internal angles and boundary external angles. The face adding process always adds to the right of a given directed edge (a,b)
which must be a boundary directed edge. [Adding to the left of an edge (a,b)
would mean that (b,a)
will be the boundary direction and so we are really adding to the right of (b,a)
]. Face adding looks to see if either of the two other edges already exist in the Tgraph
by considering the end points a
and b
to which the new face is to be added, and checking angles.
This allows an edge in a particular sought direction to be discovered. If it is not found it is assumed not to exist. However, the search will be undermined if there are crossing boundaries. In such a case there will be more than two boundary directed edges at the vertex and there is no unique external angle.
Establishing the no crossing boundaries property ensures these failures cannot occur. We can easily check this property for newly created Tgraph
s (with checkedTgraph
) and the face adding operations cannot create crossing boundaries.
Touching Vertices and Crossing Boundaries
When a new face to be added on (a,b)
has neither of the other two edges already in the Tgraph
, the third vertex needs to be created. However it could already exist in the Tgraph
– it is not on an edge coming from a
or b
but from another nonlocal part of the Tgraph
. We call this a touching vertex. If we simply added a new vertex without checking for a clash this would create a nonsensible Tgraph
. However, if we do check and find an existing vertex, we still cannot add the face using this because it would create a crossing boundary.
Our version of forcing prevents face additions that would create a touching vertex/crossing boundary by calculating the positions of boundary vertices.
No conflicting edges
There is a final (simple) check when adding a new face, to prevent a long edge (phiEdge
) sharing with a short edge. This can arise if we force an incorrect Tgraph
(as we will see later).
Our order of forcing prioritises updates (face additions) which do not introduce a new vertex. Such safe updates are easy to recognise and they do not require a touching vertex check. Surprisingly, this pretty much removes the problem of touching vertices altogether.
As an illustration, consider foolDMinus
again on the left of figure 11. Adding the left dart onto edge (12,14)
is not a safe addition (and would create a crossing boundary at 6). However, adding the right dart RD(6,14,11)
is safe and creates the new edge (6,14) which then makes the left dart addition safe. In fact it takes some contrivance to come up with a Tgraph
with an update that could fail the check during forcing when safe cases are always done first. Figure 17 shows such a contrived Tgraph
formed by removing the faces shown in green from a twice decomposed sun on the left. The forced result is shown on the right. When there are no safe cases, we need to try an unsafe one. The four green faces at the bottom are blocked by the touching vertex check. This leaves any one of 9 halfkites at the centre which would pass the check. But after just one of these is added, the check is not needed again. There is always a safe addition to be done at each step until all the green faces are added.
Boundary information
The implementation of forcing has been made more efficient by calculating some boundary information in advance. This boundary information uses a type BoundaryState
data BoundaryState
= BoundaryState
{ boundary :: [Dedge]
, bvFacesMap :: Mapping Vertex [TileFace]
, bvLocMap :: Mapping Vertex (Point V2 Double)
, allFaces :: [TileFace]
, nextVertex :: Vertex
} deriving (Show)
This records the boundary directed edges (boundary
) plus a mapping of the boundary vertices to their incident faces (bvFacesMap
) plus a mapping of the boundary vertices to their positions (bvLocMap
). It also keeps track of all the faces and the vertex number to use when adding a vertex. The boundary information is easily incremented for each face addition without being recalculated from scratch, and a final Tgraph
with all the new faces is easily recovered from the boundary information when there are no more updates.
makeBoundaryState :: Tgraph > BoundaryState
recoverGraph :: BoundaryState > Tgraph
The saving that comes from using boundary information lies in efficient incremental changes to the boundary information and, of course, in avoiding the need to consider internal faces. As a further optimisation we keep track of updates in a mapping from boundary directed edges to updates, and supply a list of affected edges after an update so the update calculator (update generator) need only revise these. The boundary and mapping are combined in a ForceState
.
type UpdateMap = Mapping Dedge Update
type UpdateGenerator = BoundaryState > [Dedge] > UpdateMap
data ForceState = ForceState
{ boundaryState:: BoundaryState
, updateMap:: UpdateMap
}
Forcing then involves using a specific update generator (allUGenerator
) and initialising the state, then using the recursive forceAll
which keeps doing updates until there are no more, before recovering the final Tgraph
.
force:: Tgraph > Tgraph
force = forceWith allUGenerator
forceWith:: UpdateGenerator > Tgraph > Tgraph
forceWith uGen = recoverGraph . boundaryState .
forceAll uGen . initForceState uGen
forceAll :: UpdateGenerator > ForceState > ForceState
initForceState :: UpdateGenerator > Tgraph > ForceState
In addition to force
we can easily define
wholeTiles:: Tgraph > Tgraph
wholeTiles = forceWith wholeTileUpdates
which just uses the first forcing rule to make sure every halftile has a matching other half.
We also have a version of force
which counts to a specific number of face additions.
stepForce :: Int > ForceState > ForceState
This proved essential in uncovering problems of accumulated inaccuracy in calculating boundary positions (now fixed).
Below we describe results of some experiments using the tools introduced above. Specifically: emplacements, subTgraphs, incorrect tilings, and composition choices.
The finite number of rules used in forcing are based on local boundary vertex and edge information only. We thought we may be able to improve on this by considering a composition and forcing at the next level up before decomposing and forcing again. This thus considers slightly broader local information. In fact we can iterate this process to all the higher levels of composition. Some Tgraph
s produce an empty Tgraph
when composed so we can regard those as maximal compositions. For example compose fool
produces an empty Tgraph
.
The idea was to take an arbitrary Tgraph
and apply (compose . force)
repeatedly to find its maximally composed (nonempty) Tgraph
, before applying (force . decompose)
repeatedly back down to the starting level (so the same number of decompositions as compositions).
We called the function emplace
, and called the result the emplacement of the starting Tgraph
as it shows a region of influence around the starting Tgraph
.
With earlier versions of forcing when we had fewer rules, emplace g
often extended force g
for a Tgraph
g
. This allowed the identification of some new rules. However, since adding the new rules we have not found Tgraph
s where the result of force
had fewer faces than the result of emplace
.
[As an important update, we have now found examples where the result of force
strictly includes the result of emplace
(modulo vertex renumbering).
In figure 18 on the left we have a four times decomposed dart dartD4
followed by two subTgraphs brokenDart
and badlyBrokenDart
which are constructed by removing faces from dartD4
(but retaining the connectedness condition and the no crossing boundaries condition). These all produce the same forced result (depicted middle row left in figure 15).
However, if we do compositions without forcing first we find badlyBrokenDart
fails because it produces a graph with crossing boundaries after 3 compositions. So compose
on its own is not always safe, where safe means guaranteed to produce a valid Tgraph
from a valid correct Tgraph
.
In other experiments we tried force
on Tgraph
s with holes and on incomplete boundaries around a potential hole. For example, we have taken the boundary faces of a forced, 5 times decomposed dart, then removed a few more faces to make a gap (which is still a valid Tgraph
). This is shown at the top in figure 19. The result of forcing reconstructs the complete original forced graph. The bottom figure shows an intermediate stage after 2200 face additions. The gap cannot be closed off to make a hole as this would create a crossing boundary, but the channel does get filled and eventually closes the gap without creating a hole.
When we say a Tgraph
g
is correct (respectively: incorrect), we mean g
represents a correct tiling (respectively: incorrect tiling). A simple example of an incorrect Tgraph
is a kite with a dart on each side (referred to as a mistake by Penrose) shown on the left of figure 20.
*Main> mistake
Tgraph [RK (1,2,4),LK (1,3,2),RD (3,1,5)
,LD (4,6,1),LD (3,5,7),RD (4,8,6)
]
If we try to force
(or emplace
) this Tgraph
it produces an error in construction which is detected by the test for conflicting edge types (a phiEdge
sharing with a nonphiEdge
).
*Main> force mistake
... *** Exception: doUpdate:(incorrect tiling)
Conflicting new face RK (11,1,6)
with neighbouring faces
[RK (9,1,11),LK (9,5,1),RK (1,2,4),LK (1,3,2),RD (3,1,5),LD (4,6,1),RD (4,8,6)]
in boundary
BoundaryState ...
In figure 20 on the right, we see that after successfully constructing the two whole kites on the top dart short edges, there is an attempt to add an RK
on edge (1,6). The process finds an existing edge (1,11) in the correct direction for one of the new edges so tries to add the erroneous RK (11,1,6)
which fails a noConflicts
test.
So it is certainly true that incorrect Tgraph
s may fail on forcing, but forcing cannot create an incorrect Tgraph
from a correct Tgraph
.
If we apply decompose
to mistake
it produces another incorrect Tgraph
(which is similarly detected if we apply force
), but will nevertheless still compose back to mistake
if we do not try to force.
Interestingly, though, the incorrectness of a Tgraph
is not always preserved by decompose
. If we start with mistake1
which is mistake
with just two of the half darts (and also incorrect) we still get a similar failure on forcing, but decompose mistake1
is no longer incorrect. If we apply compose
to the result or force
then compose
the mistake is thrown away to leave just a kite (see figure 21). This is an example where compose
is not a left inverse to either decompose
or (force . decompose)
.
We know that unknowns indicate possible choices (although some choices may lead to incorrect Tgraph
s). As an experiment we introduce
makeChoices :: Tgraph > [Tgraph]
which produces alternatives for the 2 choices of each of unknowns (prior to composing). This uses forceLDB
which forces an unknown to be a largeDartBase
by adding an appropriate joined half dart at the node, and forceLKC
which forces an unknown to be a largeKiteCentre
by adding a half dart and a whole kite at the node (making up the 3 pieces for a larger half kite).
Figure 22 illustrates the four choices for composing fool
this way. The top row has the four choices of makeChoices fool
(with the fool shown embeded in red in each case). The bottom row shows the result of applying compose
to each choice.
In this case, all four compositions are correct tilings. The problem is that, in general, some of the choices may lead to incorrect tilings. More specifically, a choice of one unknown can determine what other unknowns have to become with constraints such as
This analysis of constraints on unknowns is not trivial. The potential exponential results from choices suggests we should compose and force as much as possible and only consider unknowns of a maximal Tgraph
.
For calculating the emplacement of a Tgraph
, we first find the forced maximal Tgraph
before decomposing. We could also consider using makeChoices
at this top step when there are unknowns, i.e a version of emplace
which produces these alternative results (emplaceChoices
)
The result of emplaceChoices
is illustrated for foolD
in figure 23. The first force and composition is unique producing the fool
level at which point we get 4 alternatives each of which compose further as previously illustrated in figure 22. Each of these are forced, then decomposed and forced, decomposed and forced again back down to the starting level. In figure 23 foolD
is overlaid on the 4 alternative results. What they have in common is (as you might expect) emplace foolD
which equals force foolD
and is the graph shown on the right of figure 11.
I am collaborating with Stephen Huggett who suggested the use of graphs for exploring properties of the tilings. We now have some tools to experiment with but we would also like to complete some formalisation and proofs.
It would also be good to establish whether it is true that g
is incorrect iff force g
fails.
We have other conjectures relating to subgraph ordering of Tgraph
s and Galois connections to explore.
We have been exploring properties of Penrose’s aperiodic tilings with kites and darts using Haskell.
Previously in Diagrams for Penrose tiles we implemented tools to draw finite tilings using Haskell diagrams. There we also noted that legal tilings are only correct tilings if they can be continued infinitely and are incorrect otherwise. In Graphs, Kites and Darts we introduced a graph representation for finite tilings (Tgraphs) which enabled us to implement operations that use neighbouring tile information. In particular we implemented a force
operation to extend a Tgraph on any boundary edge where there is a unique choice for adding a tile.
In this note we find a limitation of force
, show a way to improve on it (superForce
), and introduce boundary coverings which are used to implement superForce
and calculate empires.
A Tgraph is a collection of halftile faces representing a legal tiling and a halftile face is either an LD
(left dart) , RD
(right dart), LK
(left kite), or RK
(right kite) each with 3 vertices to form a triangle. Faces of the Tgraph which are not halftile faces are considered external regions and those edges round the external regions are the boundary edges of the Tgraph. The halftile faces in a Tgraph are required to be connected and locally tileconnected which means that there are exactly two boundary edges at any boundary vertex (no crossing boundaries).
As an example Tgraph we show kingGraph
(the three darts and two kites round a king vertex), where
kingGraph = makeTgraph
[LD (1,2,3),RD (1,11,2),LD (1,4,5),RD (1,3,4),LD (1,10,11)
,RD (1,9,10),LK (9,1,7),RK (9,7,8),RK (5,7,1),LK (5,6,7)
]
This is drawn in figure 1 using
hsep 1 [labelled drawj kingGraph, draw kingGraph]
which shows vertex labels and dashed join edges (left) and without labels and join edges (right). (hsep 1
provides a horizontal seperator of unit length.)
We know there are at most two legal possibilities for adding a halftile on a boundary edge of a Tgraph. If there are zero legal possibilities for adding a halftile to some boundary edge, we have a stuck tiling/incorrect Tgraph.
Forcing deals with all cases where there is exactly one possibility for extending on a boundary edge according to the legal tiling rules and consistent with the seven possible vertex types. That means forcing either fails at some stage with a stuck Tgraph (indicating the starting Tgraph was incorrect) or it enlarges the starting Tgraph until every boundary edge has exactly two legal possibilities (consistent with the seven vertex types) for adding a halftile so a choice would need to be made to grow the Tgraph any further.
Figure 2 shows force kingGraph
with kingGraph
shown red.
If g
is a correct Tgraph, then force g
succeeds and the resulting Tgraph will be common to all infinite tilings that extend the finite tiling represented by g
. However, we will see that force g
is not a greatest lower bound of (infinite) tilings that extend g
. Firstly, what is common to all extensions of g
may not be a connected collection of tiles. This leads to the concept of empires which we discuss later. Secondly, even if we only consider the connected common region containing g, we will see that we need to go beyond force g
to find this, leading to an operation we call superForce
.
Our empire
and superForce
operations are implemented using boundary coverings which we introduce next.
Given a successfully forced Tgraph fg
, a boundary edge covering of fg
is a list of successfully forced extensions of fg
such that
fg
remains on the boundary in each extension, andfg
.[Technically this is a covering of the choices round the boundary, but each extension is also a cover of the boundary edges.] Figure 3 shows a boundary edge covering for a forced kingGraph
(force kingGraph
is shown red in each extension).
In practice, we do not need to explore both choices for every boundary edge of fg
. When one choice is made, it may force choices for other boundary edges, reducing the number of boundary edges we need to consider further.
The main function is boundaryECovering
working on a BoundaryState
(which is a Tgraph with extra boundary information). It uses covers
which works on a list of extensions each paired with the remaining set of the original boundary edges not yet covered. (Initially covers
is given a singleton list with the starting boundary state and the full set of boundary edges to be covered.) For each extension in the list, if its uncovered set is empty, that extension is a completed cover. Otherwise covers
replaces the extension with further extensions. It picks the (lowest numbered) boundary edge in the uncovered set, tries extending with a halfdart and with a halfkite on that edge, forcing in each case, then pairs each result with its set of remaining uncovered boundary edges before adding the resulting extensions back at the front of the list to be processed again. If one of the choices for a dart/kite leads to an incorrect tiling (a stuck tiling) when forced, that choice is dropped (provided the other choice succeeds). The final list returned consists of all the completed covers.
boundaryECovering:: BoundaryState > [BoundaryState]
boundaryECovering bs = covers [(bs, Set.fromList (boundary bs))]
covers:: [(BoundaryState, Set.Set Dedge)] > [BoundaryState]
covers [] = []
covers ((bs,es):opens)
 Set.null es = bs:covers opens  bs is complete
 otherwise = covers (newcases ++ opens)
where (de,des) = Set.deleteFindMin es
newcases = fmap (\b > (b, commonBdry des b))
(atLeastOne $ tryDartAndKite bs de)
Here we have used
type Try a = Either String a
tryDartAndKite:: BoundaryState > Dedge > [Try BoundaryState]
atLeastOne :: [Try a] > [a]
We frequently use Try
as a type for results of partial functions where we need to continue computation if there is a failure. For example we have a version of force
(called tryForce
) that returns a Try Tgraph
so it does not fail by raising an error, but returns a result indicating either an explicit failure situation or a successful result with a final forced Tgraph. The function tryDartAndKite
tries adding an appropriate halfdart and halfkite on a given boundary edge, then uses tryForceBoundary
(a variant of tryForce
which works with boundary states) on each result and returns a list of Try
results. The list of Try
results is converted with atLeastOne
which collects the successful results but will raise an error when there are no successful results.
You may notice in figure 3 that the top right cover still has boundary vertices of kingGraph
on the final boundary. We use a boundary vertex covering rather than a boundary edge covering if we want to exclude these cases. This involves picking a boundary edge that includes such a vertex and continuing the process of growing possible extensions until no boundary vertices of the original remain on the boundary.
A partial example of an empire was shown in a 1977 article by Martin Gardner 1. The full empire of a finite tiling would consist of the common faces of all the infinite extensions of the tiling. This will include at least the force of the tiling but it is not obviously finite. Here we confine ourselves to the empire in finite local regions.
For example, we can calculate a local empire for a given Tgraph g
by finding the common faces of all the extensions in a boundary vertex covering of force g
(which we call empire1 g
).
This requires an efficient way to compare Tgraphs. We have implemented guided intersection and guided union operations which, when given a common edge starting point for two Tgraphs, proceed to compare the Tgraphs face by face and produce an appropriate relabelling of the second Tgraph to match the first Tgraph only in the overlap where they agree. These operations may also use geometric positioning information to deal with cases where the overlap is not just a single connected region. From these we can return a union as a single Tgraph when it exists, and an intersection as a list of common faces. Since the (guided) intersection of Tgraphs (the common faces) may not be connected, we do not have a resulting Tgraph. However we can arbitrarily pick one of the argument Tgraphs and emphasise which are the common faces in this example Tgraph.
Figure 4 (left) shows empire1 kingGraph
where the starting kingGraph
is shown in red. The greyfilled faces are the common faces from a boundary vertex covering. We can see that these are not all connected and that the force kingGraph
from figure 2 corresponds to the connected set of greyfilled faces around and including the kingGraph
in figure 4.
We call this a level 1 empire because we only explored out as far as the first boundary covering. We could instead, find further boundary coverings for each of the extensions in a boundary covering. This grows larger extensions in which to find common faces. On the right of figure 4 is a level 2 empire (empire2 kingGraph
) which finds the intersection of the combined boundary edge coverings of each extension in a boundary edge covering of force kingGraph
. Obviously this process could be continued further but, in practice, it is too inefficient to go much further.
We might hope that (when not discovering an incorrect tiling), force g
produces the maximal connected component containing g
of the common faces of all infinite extensions of g
. This is true for the kingGraph
as noted in figure 4. However, this is not the case in general.
The problem is that forcing will not discover if one of the two legal choices for extending a resulting boundary edge always leads to an incorrect Tgraph. In such a situation, the other choice would be common to all infinite extensions.
We can use a boundary edge covering to reveal such cases, leading us to a superForce
operation. For example, figure 5 shows a boundary edge covering for the forced Tgraph shown in red.
This example is particularly interesting because in every case, the leftmost end of the red forced Tgraph has a dart immediately extending it. Why is there no case extending one of the leftmost two red edges with a halfkite? The fact that such cases are missing from the boundary edge covering suggests they are not possible. Indeed we can check this by adding a halfkite to one of the edges and trying to force. This leads to a failure showing that we have an incorrect tiling. Figure 6 illustrates the Tgraph at the point that it is discovered to be stuck (at the bottom left) by forcing.
Our superForce
operation starts by forcing a Tgraph. After a successful force, it creates a boundary edge covering for the forced Tgraph and checks to see if there is any boundary edge of the forced Tgraph for which each cover has the same choice. If so, that choice is made to extend the forced Tgraph and the process is repeated by applying superForce
to the result. Otherwise, just the result of forcing is returned.
Figure 7 shows a chain of examples (rockets) where superForce
has been used. In each case, the starting Tgraph is shown red, the additional faces added by forcing are shown black, and any further extension produced by superForce
is shown in blue.
We still do not know if forcing decides that a Tgraph is correct/incorrect. Can we conclude that if force g
succeeds then g
(and force g
) are correct? We found examples (rockets in figure 7) where force succeeds but one of the 2 legal choices for extending on a boundary edge leads to an incorrect Tgraph. If we find an example g
where force g
succeeds but both legal choices on a boundary edge lead to incorrect Tgraphs we will have a counterexample. If such a g
exists then superForce g
will raise an error. [The calculation of a boundary edge covering will call atLeastOne
where both branches have led to failure for extending on an edge.]
This means that when superForce
succeeds every resulting boundary edge has two legal extensions, neither of which will get stuck when forced.
I would like to thank Stephen Huggett who suggested the idea of using graphs to represent tilings and who is working with me on proof problems relating to the kite and dart tilings.
Reference [1] Martin Gardner (1977) MATHEMATICAL GAMES. Scientific American, 236(1), (pages 110 to 121). http://www.jstor.org/stable/24953856
We continue our exploration of properties of Penrose’s aperiodic tilings with kites and darts using Haskell and Haskell Diagrams.
In this blog we discuss some interesting properties we have discovered concerning the , , and operations along with some proofs.
Index
Tgraph
s)Tgraph
s)Haskell diagrams allowed us to render finite patches of tiles easily as discussed in Diagrams for Penrose tiles. Following a suggestion of Stephen Huggett, we found that the description and manipulation of such tilings is greatly enhanced by using planar graphs. In Graphs, Kites and Darts we introduced a specialised planar graph representation for finite tilings of kites and darts which we called Tgraph
s (tile graphs). These enabled us to implement operations that use neighbouring tile information and in particular operations , , and .
For ease of reference, we reproduce the halftiles we are working with here.
Figure 1 shows the rightdart (RD), leftdart (LD), leftkite (LK) and rightkite (RK) halftiles. Each has a join edge (shown dotted) and a short edge and a long edge. The origin vertex is shown red in each case. The vertex at the opposite end of the join edge from the origin we call the opp vertex, and the remaining vertex we call the wing vertex.
If the short edges have unit length then the long edges have length (the golden ratio) and all angles are multiples of (a tenth turn) with kite halves having two 2s and a 1, and dart halves having a 3 and two 1s. This geometry of the tiles is abstracted away from at the graph representation level but used when checking validity of tile additions and by the drawing functions.
There are rules for how the tiles can be put together to make a legal tiling (see e.g. Diagrams for Penrose tiles). We defined a Tgraph
(in Graphs, Kites and Darts) as a list of such halftiles which are constrained to form a legal tiling but must also be connected with no crossing boundaries (see below).
As a simple example consider kingGraph
(2 kites and 3 darts round a king vertex). We represent each halftile as a TileFace
with three vertex numbers, then apply makeTgraph
to the list of ten Tileface
s. The function makeTgraph :: [TileFace] > Tgraph
performs the necessary checks to ensure the result is a valid Tgraph
.
kingGraph :: Tgraph
kingGraph = makeTgraph
[LD (1,2,3),RD (1,11,2),LD (1,4,5),RD (1,3,4),LD (1,10,11)
,RD (1,9,10),LK (9,1,7),RK (9,7,8),RK (5,7,1),LK (5,6,7)
]
To view the Tgraph
we simply form a diagram (in this case 2 diagrams horizontally separated by 1 unit)
hsep 1 [labelled drawj kingGraph, draw kingGraph]
and the result is shown in figure 2 with labels and dashed join edges (left) and without labels and join edges (right).
The boundary of the Tgraph
consists of the edges of halftiles which are not shared with another halftile, so they go round untiled/external regions. The no crossing boundary constraint (equivalently, locally tileconnected) means that a boundary vertex has exactly two incident boundary edges and therefore has a single external angle in the tiling. This ensures we can always locally determine the relative angles of tiles at a vertex. We say a collection of halftiles is a valid Tgraph
if it constitutes a legal tiling but also satisfies the connectedness and no crossing boundaries constraints.
Our key operations on Tgraph
s are , , and which are illustrated in figure 3.
Figure 3 shows the kingGraph
with its decomposition above it (left), the result of forcing the kingGraph
(right) and the composition of the forced kingGraph
(bottom right).
Decompose
An important property of Penrose dart and kite tilings is that it is possible to divide the halftile faces of a tiling into smaller halftile faces, to form a new (smaller scale) tiling.
Figure 4 illustrates the decomposition of a leftdart (top row) and a leftkite (bottom row). With our Tgraph
representation we simply introduce new vertices for dart and kite long edges and kite join edges and then form the new faces using these. This does not involve any geometry, because that is taken care of by drawing operations.
Force
Figure 5 illustrates the rules used by our operation (we omit a mirrorreflected version of each rule).
In each case the yellow halftile is added in the presence of the other halftiles shown. The yellow halftile is forced because, by the legal tiling rules and the seven possible vertex types, there is no choice for adding a different halftile on the edge where the yellow tile is added.
We call a Tgraph
correct if it represents a tiling which can be continued infinitely to cover the whole plane without getting stuck, and incorrect otherwise. Forcing involves adding halftiles by the illustrated rules round the boundary until either no more rules apply (in which case the result is a forced Tgraph
) or a stuck tiling is encountered (in which case an incorrect Tgraph
error is raised). Hence is a partial function but total on correct Tgraph
s.
Compose: This is discussed in the next section.
For an infinite tiling, composition is a simple inverse to decomposition. However, for a finite tiling with boundary, composition is not so straight forward. Firstly, we may need to leave halftiles out of a composition because the necessary parts of a composed halftile are missing. For example, a halfdart with a boundary short edge or a whole kite with both short edges on the boundary must necessarily be excluded from a composition. Secondly, on the boundary, there can sometimes be a problem of choosing whether a halfdart should compose to become a halfdart or a halfkite. This choice in composing only arises when there is a halfdart with its wing on the boundary but insufficient local information to determine whether it should be part of a larger halfdart or a larger halfkite.
In the literature (see for example 1 and 2) there is an often repeated method for composing (also called inflating). This method always make the kite choice when there is a choice. Whilst this is a sound method for an unbounded tiling (where there will be no choice), we show that this is an unsound method for finite tilings as follows.
Clearly composing should preserve correctness. However, figure 6 (left) shows a correct Tgraph
which is a forced queen, but the kitefavouring composition of the forced queen produces the incorrect Tgraph
shown in figure 6 (centre). Applying our function to this reveals a stuck tiling and reports an incorrect Tgraph
.
Our algorithm (discussed in Graphs, Kites and Darts) detects dart wings on the boundary where there is a choice and classifies them as unknowns. Our composition refrains from making a choice by not composing a half dart with an unknown wing vertex. The rightmost Tgraph
in figure 6 shows the result of our composition of the forced queen with the halftile faces left out of the composition (the remainder faces) shown in green. This avoidance of making a choice (when there is a choice) guarantees our composition preserves correctness.
A different composition problem can arise when we consider Tgraph
s that are not decompositions of Tgraph
s. In general, is a partial function on Tgraph
s.
Figure 7 shows a Tgraph
(left) with its sucessful composition (centre) and the halftile faces that would result from a second composition (right) which do not form a valid Tgraph
because of a crossing boundary (at vertex 6). Thus composition of a Tgraph
may fail to produce a Tgraph
when the resulting faces are disconnected or have a crossing boundary.
However, we claim that is a total function on forced Tgraph
s.
Theorem: Composition of a forced Tgraph produces a valid Tgraph.
We postpone the proof (outline) for this theorem to section 5. Meanwhile we use the result to establish relationships between , , and in the next section.
In Graphs, Kites and Darts we produced a diagram showing relationships between multiple decompositions of a dart and the forced versions of these Tgraph
s. We reproduce this here along with a similar diagram for multiple decompositions of a kite.
In figure 8 we show separate (apparently) commuting diagrams for the dart and for the kite. The bottom rows show the decompositions, the middle rows show the result of forcing the decompositions, and the top rows illustrate how the compositions of the forced Tgraph
s work by showing both the composed faces (black edges) and the remainder faces (green edges) which are removed in the composition. The diagrams are examples of some commutativity relationships concerning , and which we will prove.
It should be noted that these diagrams break down if we consider only halftiles as the starting points (bottom right of each diagram). The decomposition of a halftile does not recompose to its original, but produces an empty composition. So we do not even have in these cases. Forcing the decomposition also results in an empty composition. Clearly there is something special about the depicted cases and it is not merely that they are wholetile complete because the decompositions are not wholetile complete. [Wholetile complete means there are no join edges on the boundary, so every halftile has its other half.]
Below we have captured the properties that are sufficient for the diagrams to commute as in figure 8. In the proofs we use a partial ordering on Tgraph
s (modulo vertex relabelling) which we define next.
If and are both valid Tgraph
s and consists of a subset of the (halftile) faces of we have
which gives us a partial order on Tgraph
s. Often, though, is only isomorphic to a subset of the faces of , requiring a vertex relabelling to become a subset. In that case we write
which is also a partial ordering and induces an equivalence of Tgraph
s defined by
in which case and are isomorphic as Tgraph
s.
Both and are monotonic with respect to meaning:
We also have is monotonic, but only when restricted to correct Tgraph
s. Also, when restricted to correct Tgraph
s, we have is non decreasing because it only adds faces:
and is idempotent (forcing a forced correct Tgraph
leaves it the same):
Definition: A Tgraph
composes perfectly if all faces of are composable (i.e there are no remainder faces of when composing).
We note that the composed faces must be a valid Tgraph
(connected with no crossing boundaries) if all faces are included in the composition because has those properties. Clearly, if composes perfectly then
In general, for arbitrary where the composition is defined, we only have
Definition: A Tgraph
is a perfect composition if composes perfectly.
Clearly if is a perfect composition then
(We could use equality here because any new vertex labels introduced by will be removed by ). In general, for arbitrary ,
Lemma 1: is a perfect composition if and only if has the following 2 properties:
(Proof outline:) Firstly note that unknowns in (= ) can only come from boundary joins in . The properties 1 and 2 guarantee that has no unknowns. Since every face of has come from a decomposed face in , there can be no faces in that will not recompose, so will compose perfectly to . Conversely, if is a perfect composition, its decomposition can have no unknowns. This implies boundary joins in must satisfy properties 1 and 2.
(Note: a perfect composition may have unknowns even though its decomposition has none.)
It is easy to see two special cases:
We note that these two special cases cover all the Tgraph
s in the bottom rows of the diagrams in figure 8. So the Tgraph
s in each bottom row are perfect compositions, and furthermore, they all compose perfectly except for the rightmost Tgraph
s which have empty compositions.
In the following results we make the assumption that a Tgraph
is correct, which guarantees that when is applied, it terminates with a correct Tgraph
. We also note that preserves correctness as does (provided the composition is defined).
Lemma 2: If is a forced, correct Tgraph
then
(Proof outline:) The proof uses a case analysis of boundary and internal vertices of . For internal vertices we just check there is no change at the vertex after using figure 11 (plus an extra case for the forced star). For boundary vertices we check local contexts similar to those depicted in figure 10 (but including empty composition cases). This reveals there is no local change of the boundary at any boundary vertex, and since this is true for all boundary vertices, there can be no global change. (We omit the full details).
Lemma 3: If is a perfect composition and a correct Tgraph
, then
(Proof outline:) The proof is by analysis of each possible force rule applicable on a boundary edge of and checking local contexts to establish that (i) the result of applying to the local context must include the added halftile, and (ii) if the added half tile has a new boundary join, then the result must include both halves of the new halftile. The two properties of perfect compositions mentioned in lemma 1 are critical for the proof. However, since the result of adding a single halftile may break the condition of the Tgraph
being a pefect composition, we need to arrange that halftiles are completed first then each subsequent halftile addition is paired with its wholetile completion. This ensures the perfect composition condition holds at each step for a proof by induction. [A separate proof is needed to show that the ordering of applying force rules makes no difference to a final correct Tgraph
(apart from vertex relabelling)].
Lemma 4 If composes perfectly and is a correct Tgraph
then
Proof: Assume composes perfectly and is a correct Tgraph
. Since is nondecreasing (with respect to on correct Tgraph
s)
and since is monotonic
Since composes perfectly, the left hand side is just , so
and since is monotonic (with respect to on correct Tgraph
s)
For the opposite direction, we substitute for in lemma 3 to get
Then, since , we have
Apply to both sides (using monotonicity)
For any for which the composition is defined we have so we get
Now apply to both sides and note to get
Combining this with (*) above proves the required equivalence.
Theorem (Perfect Composition): If composes perfectly and is a correct Tgraph
then
Proof: Assume composes perfectly and is a correct Tgraph
. By lemma 4 we have
Applying to both sides, gives
Now by lemma 2, with , the right hand side is equivalent to
which establishes the result.
Corollaries (of the perfect composition theorem):
Tgraph
then
Proof: Let (so ) in the theorem.
[This result generalises lemma 2 because any correct forced Tgraph
is necessarily wholetile complete and therefore a perfect composition, and .]
Tgraph
then
Proof: Apply to both sides of the previous corollary and note that
provided the composition is defined, which it must be for a forced Tgraph
by the Compose Force theorem.
Tgraph
then
Proof: Apply to both sides of the previous corollary noting is monotonic and idempotent for correct Tgraph
s
From the fact that is non decreasing and and are monotonic, we also have
Hence combining these two subTgraph
results we have
It is important to point out that if is a correct Tgraph
and is a perfect composition then this is not the same as composes perfectly. It could be the case that has more faces than and so could have unknowns. In this case we can only prove that
As an example where this is not an equivalence, choose to be a star. Then its composition is the empty Tgraph
(which is still a pefect composition) and so the left hand side is the empty Tgraph
, but the right hand side is a sun.
The perfect composition theorem and lemmas and the three corollaries justify all the commuting implied by the diagrams in figure 8. However, one might ask more general questions like: Under what circumstances do we have (for a correct forced Tgraph
)
Definition A generator of a correct forced Tgraph
is any Tgraph
such that and .
We can now state that
Corollary If a correct forced Tgraph
has a generator which composes perfectly, then
Proof: This follows directly from lemma 4 and the perfect composition theorem.
As an example where the required generator does not exist, consider the rightmost Tgraph
of the middle row in figure 9. It is generated by the Tgraph
directly below it, but it has no generator with a perfect composition. The Tgraph
directly above it in the top row is the result of applying which has lost the leftmost dart of the Tgraph
.
We could summarise this section by saying that can lose information which cannot be recovered by a subsequent and, similarly, can lose information which cannot be recovered by a subsequent . We have defined perfect compositions which are the Tgraph
s that do not lose information when decomposed and Tgraph
s which compose perfectly which are those that do not lose information when composed. Forcing does the same thing at each level of composition (that is it commutes with composition) provided information is not lost when composing.
We know from the Compose Force
theorem that the composition of a Tgraph
that is forced is always a valid Tgraph
. In this section we use this and the results from the last section to show that composing a forced, correct Tgraph
produces a forced Tgraph
.
First we note that:
Lemma 5: The composition of a forced, correct Tgraph
is wholetile complete.
Proof: Let where is a forced, correct Tgraph
. A boundary join in implies there must be a boundary dart wing of the composable faces of . (See for example figure 4 where this would be vertex 2 for the half dart case, and vertex 5 for the halfkite face). This dart wing cannot be an unknown as the halfdart is in the composable faces. However, a known dart wing must be either a large kite centre or a large dart base and therefore internal in the composable faces of (because of the force rules) and therefore not on the boundary in . This is a contradiction showing that can have no boundary joins and is therefore wholetile complete.
Theorem: The composition of a forced, correct Tgraph
is a forced Tgraph
.
Proof: Let for some forced, correct Tgraph
, then is wholetile complete (by lemma 5) and therefore a perfect composition. Let , so composes perfectly (). By the perfect composition theorem we have
We also have
Applying to both sides, noting that is monotonic and the identity on forced Tgraph
s, we have
Applying to both sides, noting that is monotonic, we have
By (**) above, the left hand side is equivalent to so we have
but since we also have ( being nondecreasing)
we have established that
which means is a forced Tgraph
.
This result means that after forcing once we can repeatedly compose creating valid Tgraph
s until we reach the empty Tgraph
.
We can also use lemma 5 to establish the converse to a previous corollary:
Corollary If a correct forced Tgraph
satisfies:
then has a generator which composes perfectly.
Proof: By lemma 5, is wholetile complete and hence a perfect composition. This means that composes perfectly and it is also a generator for because
Theorem (Compose Force): Composition of a forced Tgraph produces a valid Tgraph.
Proof: For any forced Tgraph
we can construct the composed faces. For the result to be a valid Tgraph
we need to show no crossing boundaries and connectedness for the composed faces. These are proved separately by case analysis below.
Proof of no crossing boundaries
Assume is a forced Tgraph
and that it has a nonempty set of composed faces (we can ignore cases where the composition is empty as the empty Tgraph
is valid). Consider a vertex v
in the composed faces of and first take the case that v
is on the boundary of . We consider the possible local contexts for a vertex v
on a forced Tgraph
boundary and the nature of the composed faces at v
in each case.
Figure 10 shows local contexts for a boundary vertex v
in a forced Tgraph
where the composition is nonempty. In each case v
is shown as a red dot, and the composition is shown filled yellow. The cases for v
are shown in rows: the first row is for dart origins, the second row is for kite origins, the next two rows are for kite wings, and the last two rows are for kite opps. The dart wing cases are a subset of the kite opp cases, so not repeated, and dart opp vertices are excluded because they cannot be on the boundary of a forced Tgraph
. We only show lefthand versions, so there is a mirror symmetric set for righthand versions.
It is easy to see that there are no crossing boundaries of the composed faces at v
in each case. Since any boundary vertex of any forced Tgraph
(with a nonempty composition) must match one of these local context cases around the vertex, we can conclude that a boundary vertex of cannot become a crossing boundary in .
Next take the case where v
is an internal vertex of .
Figure 11 shows relationships between the forced Tgraph
s of the 7 (internal) vertex types (plus a kite at the top right). The red faces are those around the vertex type and the black faces are those produced by forcing (if any). Each forced Tgraph
has its composition directly above with empty compositions for the top row. We note that a (forced) star, jack, king, and queen vertex remains an internal vertex in the respective composition so cannot become a crossing boundary vertex. A deuce vertex becomes the centre of a larger kite and is no longer present in the composition (top right). That leaves cases for the sun vertex and ace vertex (=fool vertex). The sun Tgraph
(sunGraph
) and fool Tgraph
(fool
) consist of just the red faces at the respective vertex (shown top left and top centre). These both have empty compositions when there is no surrounding context. We thus need to check possible forced local contexts for sunGraph
and fool
.
The fool
case is simple and similar to a duece vertex in that it is never part of a composition. [To see this consider inverting the decomposition arrows shown in figure 4. In both cases we see the halfdart opp vertex (labelled 4 in figure 4) is removed].
For the sunGraph
there are only 7 local forced context cases to consider where the sun vertex is on the boundary of the composition.
Six of these are shown in figure 12 (the missing one is just a mirror reflection of the fourth case). Again, the relevant vertex v
is shown as a red dot and the composed faces are shown filled yellow, so it is easy to check that there is no crossing boundary of the composed faces at v
in each case. Every forced Tgraph
containing an internal sun vertex where the vertex is on the boundary of the composition must match one of the 7 cases locally round the vertex.
Thus no vertex from can become a crossing boundary vertex in the composed faces and since the vertices of the composed faces are a subset of those of , we can have no crossing boundary vertex in the composed faces.
Proof of Connectedness
Assume is a forced Tgraph
as before. We refer to the halftile faces of that get included in the composed faces as the composable faces and the rest as the remainder faces. We want to prove that the composable faces are connected as this will imply the composed faces are connected.
As before we can ignore cases where the set of composable faces is empty, and assume this is not the case. We study the nature of the remainder faces of . Firstly, we note:
Lemma (remainder faces)
The remainder faces of are made up entirely of groups of halftiles which are either:
These 3 cases of remainder face groups are shown in figure 13. In each case the border in common with composable faces is shown yellow and the red edges are necessarily on the boundary of (the black boundary could be on the boundary of or shared with another reamainder face group). [A mirror symmetric version for the first group is not shown.] Examples can be seen in e.g. figure 12 where the first Tgraph
has four examples of case 1, and two of case 2, the second has six examples of case 1 and two of case 2, and the fifth Tgraph
has an example of case 3 as well as four of case 1. [We omit the detailed proof of this lemma which reasons about what gets excluded in a composition after forcing. However, all the local context cases are included in figure 14 (lefthand versions), where we only show those contexts where there is a nonempty composition.]
We note from the (remainder faces) lemma that the common boundary of the group of remainder faces with the composable faces (shown yellow in figure 13) is just a single vertex in cases 2 and 3. In case 1, the common boundary is just a single edge of the composed faces which is made up of 2 adjacent edges of the composable faces that constitute the join of two halffools.
This means each (remainder face) group shares boundary with exactly one connected component of the composable faces.
Next we establish that if two (remainder face) groups are connected they must share boundary with the same connected component of the composable faces. We need to consider how each (remainder face) group can be connected with a neighbouring such group. It is enough to consider forced contexts of boundary dart long edges (for cases 1 and 3) and boundary kite short edges (for case 2). The cases where the composition is nonempty all appear in figure 14 (lefthand versions) along with boundary kite long edges (middle two rows) which are not relevant here.
We note that, whenever one group of the remainder faces (halffool, wholekite, wholefool) is connected to a neighbouring group of the remainder faces, the common boundary (shared edges and vertices) with the compososable faces is also connected, forming either 2 adjacent composed face boundary edges (= 4 adjacent edges of the composable faces), or a composed face boundary edge and one of its end vertices, or a single composed face boundary vertex.
It follows that any connected collection of the remainder face groups shares boundary with a unique connected component of the composable faces. Since the collection of composable and remainder faces together is connected ( is connected) the removal of the remainder faces cannot disconnect the composable faces. For this to happen, at least one connected collection of remainder face groups would have to be connected to more than one connected component of composable faces.
This establishes connectedness of any composition of a forced Tgraph
, and this completes the proof of the Compose Force theorem.
References
[1] Martin Gardner (1977) MATHEMATICAL GAMES. Scientific American, 236(1), (pages 110 to 121). http://www.jstor.org/stable/24953856
In programming languages with sophisticated type systems we easily run into inconvenience of providing many (often type) arguments explicitly. Let's take a simple map
function as an example:
If we had to always explicitly provide map
's arguments, write something like
we would immediately give up on types, and switch to use some dynamically typed programming language. It wouldn't be fun to state "the obvious" all the time.
Fortunately we know a way (unification) which can be used to infer many such argument. Therefore we can write
and the type arguments will be inferred by compiler. However we usually are able to be explicit if we want or need to be, e.g. with TypeApplications
in GHC Haskell.
Conor McBride calls a following phenomenon "Milner's Coincidence":
The HindleyMilner type system achieves the truly awesome coincidence of four distinct distinctions
 terms vs types
 explicitly written things vs implicitly written things
 presence at runtime vs erasure before runtime
 nondependent abstraction vs dependent quantification
We’re used to writing terms and leaving types to be inferred. . . and then erased. We’re used to quantifying over type variables with the corresponding type abstraction and application happening silently and statically.
GHC Haskell typesystem has been long far more expressive than vanilla HindleyMilner, and the four distrinctions are already misaligned.
GHC developers are filling the cracks: For example we'll soon ^{1} get a forall a >
(with an arrow, not a dot) quantifier, which is erased (irrelevant), explicit (visible) dependent quantification. Later we'll get foreach a.
and foreach a >
which are retained (i.e. noterased, relevant) implicit/explicit dependent quantification.
(Agda also has "different" quantifiers: explicit (x : A) > ...
and implicit {y : B} > ...
dependent quantifiers, and erased variants look like (@0 x : A) > ...
and {@0 y : B} > ...
.)
In Haskell, if we have a term with implicit quantifier (foo :: forall a. ...
), we can use TypeApplications
syntax to apply the argument explicitly:
If the quantifier is explicit, we'll (eventually) write just
or
for now.
That all is great, but consider we define a kindpolymorphic^{2} type like
then when used at type level, forall
behaves as previously, constructors
The type of constructor MkProxyE
is
So if we want to create a term of type Proxy Int
, we need to provide both k
and a
arguments:
we could also jump over k
:
The above skipping over arguments is not convenient, luckily GHC has a feature, created for other needs, which we can (ab)use here. There are inferred variables (though the better name would be "very hidden"), these are arguments for which TypeApplication
doesn't apply:
This is the way Proxy
is defined in base
(but I renamed the constructor to avoid name ambiguity)
And while GHCi prints
the @{A}
syntax is not valid Haskell, so we cannot explicitly apply inferred variables. Neither we can in types:
I think this is plainly wrong, we should be able to apply these "inferred" arguments too.
The counterargument is that, inferred variables weren't meant to be "more implicit" variables. As GHC manual explains, inferred variables are a solution to TypeApplications
with inferred types. We need to know the order of variables to be able to apply them; but especially in presence of typeclass constraints the order is arbitrary.
I'm not convinced, I think that ability to be fully explicit is way more important than a chance to write brittle code.
One solution, which I think would work, is simply to not generalise. This is controversial proposal, but as GHC Haskell is moving towards having fancier type system, something needs to be sacrificed. (MonoLocalBinds
is for local bindings, but I'd argue that should be for all bindings, not only local).
The challenge has been that library writes may not been aware of TypeApplications
, but today they have no choice. Changing from foo :: forall a b. ...
to foo :: forall b a. ...
may break some code (even though PVP doesn't explicitly write that down, that should be common sense).
So in the GHC manual example
the g
would fail to typecheck because there are unsolved typevariables. One way to think about this is that GHC would refuse to pick an order of variables. GHC could still generalise if there are no dictionary arguments, but on the other hand I don't think it would help much. It might help more if GHC wouldn't specialise as much, then
would typecheck.
This might sound like we would need to write much many type signatures. I don't think that is true: it's already a best practice to write type signatures for type level bindings, and for local bindings we would mostly need to give signatures to function bindings.
This proposal subsumes monomorphism restriction, recall that without type defaulting:
will fail to compile with
Ambiguous type variable ‘i0’ arising from a use of ‘genericLength’ prevents the constraint ‘(Num i0)’ from being solved.
error. With NoMonomophismRestriction
we have
Another, a lot simpler option, is to simply remember whether the symbols' type was inferred, and issue a warning if TypeApplications
is used with such symbol in application head. So if user writes
GHC would warn that g
has inferred type, and the TypeApplications
with g
are brittle. The solution is to give g
a type signature. This warning could be issued early in a pipeline (maybe already in renamer), so it would explain further (possibly cryptic) type errors.
Let me summarise the above: If we could apply inferred variables, i.e. use curly brace application syntax, we would have complete explicit forall a >
, implicit forall a.
and more implicit forall {a}.
dependent quantifiers. Currently the forall {a}.
quantifier is incomplete: we can abstract, but we cannot apply. We'll also need some alternative solution to TypeApplicaitons
and inferred types. We should be able to bind these variables explicitly in lambda abstractions as well: \ a >
, \ @a >
and \ @{a} >
respectively (see TypeAbstractions
).
The three level explicit/implicit/impliciter arguments may feel complicated. Doesn't other languages have similar problems, how they solve them?
As far as I'm aware Agda and Coq resolve this problem by supporting applying implicit arguments by name:
 using indices instead of parameters,
 to make constructor behave as in Haskell
data Proxy : {k : Set} (a : k) > Set1 where
MkProxy : {k : Set} {a : k} > Proxy a
t = MkProxy {a = true}
Just adding named arguments to Haskell would be a bad move. It would add another way where a subtle and wellmeaning change in the library could break downstream. For example unifying the naming scheme of typevariables in the libraries, so they are always Map k v
and not Map k a
sometimes, as it is in containers
which uses both variable namings.
We could require library authors to explicitly declare that bindings in a module can be applied by name (i.e. that they have thought about the names, and recognise that changing them will be breaking change). You would still be able to always explicitly apply implicit arguments, but sometimes you won't be able to use more convenient named syntax.
It is fair to require library authors to make adjustments so that (numerous) library users would be able to use a new language feature with that library. In a healthy ecosystem that shouldn't be a problem. Specifically it is extra fair, if the alternative is to make feature less great, as then people might not use it at all.
Another idea is to embrace implicit, more implicit and even more implicit arguments. Agda has two levels: explicit and implicit, GHC Haskell has two and a half, why stop there?
If we could start fresh, we could pick Agda's function application syntax and have
but additionally we could add
funJ {{arg}}  explicit application of implicit² argument funK {{{arg}}}  explicit application of implicit³ argument ...  and so on
With unlimited levels of implicitness we could define Proxy
as
type Proxy :: forall {k} > k > Type data Proxy a where MkProxy :: forall {{k}} > {a :: k} > Proxy a
and use it as MkProxy
, MkProxy {Int}
or MkProxy {{Type}} {Int} :: Proxy Int
. Unlimited possibilities.
For what it is worth, the implementation should be even simpler than of named arguments.
But I'd be quite happy already if GHC Haskell had a way to explicitly apply any function arguments, be it three levels (ordinary, @arg
and @{arg}
) of explicitness, many or just two; and figured another way to tackle TypeApplications
with inferred types.
GHC9.10.1 release notes (for alpha1) mention "Partial implementation of the GHC Proposal #281, allowing visible quantification to be used in the types of terms."↩︎
kind is type of types.↩︎