The project

How to verify that the compiler compiles itself, how language changes are reviewed and the twenty principles printed by heroes this.

How to verify that it compiles itself

The Heroes compiler is written in Heroes. Compiling it requires nested data, name tables, recursion, error handling and programs spread across files. It is a practical test of how the language handles a complex program.

The repository, the project’s archive, also contains the compiler as one generated C file. You can build it on a computer without Heroes installed. Then use the resulting compiler to translate its own source into C and compare the two files. Do it on a release, the version the project has published:

$ git checkout v0.5.0
$ clang -I runtime seed/heroes.c runtime/runtime.c -o heroes
$ ./heroes build selfhost/main.hero --emit-c -o out.c
$ cmp out.c seed/heroes.c
                       (cmp prints nothing: not one byte differs)

  seed/heroes.c   the C the repository ships: the way in
  out.c           what the compiler writes when it compiles itself

  over twenty megabytes of C, and more than 250,000 lines of it

If the files are identical, cmp prints nothing. The comparison verifies that the compiler reproduces the C it was built from. The third command takes under two minutes on a MacBook Air. Anyone can repeat the test: the repository is open and the four commands are the whole of it.

Between two releases the generated C file is an earlier day’s compiler. It builds the current one from the Heroes source, and that one reproduces itself one step later, which the project checks on every change. heroes --version says which one you hold: its second line is a fingerprint of the source it was built from.

• 680+modules of Heroes
• 179klines, and growing
• 0TODO, FIXME or HACK in the compiler
• 220+design sittings on the record

Heroes implements every part of the compiler: reading the words and structure of the code, connecting names to declarations, checking types, creating and checking an intermediate representation of the program, managing memory, generating C, checking external libraries, reporting errors, formatting code and handling commands.

Errors found by compiling its own source

The new compiler initially rejected its own source, incorrectly reporting a structure that contained itself. Investigating this found defects in both the new compiler and its predecessor. One involved the underline in an error message: it covered the annotation that the test used to check its position.

Compiling the compiler’s source tests many language forms in one program. Several defects had the same cause: a loop jump inside a match branch skipped the instruction that updated a manually maintained index.

The final difference between the two files was the order of the lines. Some lists were already ordered in the previous compiler; the new one needed explicit sorting, as notes in the source had warned. Maps do not guarantee an order. Adding the missing sorts made the files identical.

How language changes are reviewed · five years

Changes to syntax, language rules, error messages or compiler structure are reviewed by a panel of five seats. Each seat receives different material, and four can veto a proposal. Internal compiler or runtime changes, which affect the code that supports execution, go to two of the five if they leave syntax, error messages and the specification unchanged.

• The readerReceives the specification and sample programs, without the design document or compiler source. Tests whether the language can be understood. veto
• The compiler engineerExamines the source and assesses how much code the change requires to write and maintain. veto
• The C pragmatistCompiles the C code needed to use a library from Heroes, or documents why compilation fails. veto
• The spec wardenCompares token counts before and after the change and checks that the specification stays within its limit. Tokens are the pieces of text a model reads. veto
• The historianHas to search the web and cite dates and sources: unsourced precedent is inadmissible. It has also read the author’s history of programming languages, and uses it to know where to look rather than as the source, because a book by this project’s author cannot be evidence for this project. Advises without veto power.

The panel chooses the solution it considers strongest and most complete, then submits it to the author for approval. Work continues while confirmation is pending. If there is a more cautious alternative, it is recorded so the author can consider it. After the five verdicts a sixth reader, the completeness critic, asks what nobody listed, and more than once the solution adopted was a route no seat had proposed.

Each review is recorded in a repository file, including the material examined, the verdicts and any vetoes. There are more than a hundred and fifty reviews. Every language change requires this review process. What the panel refused, and who said no, has a page of its own: what Heroes refused →

Twenty lines

heroes this prints these principles, which are included in the compiler. They are reproduced here as the command prints them.

The Zen of Heroes

 1. This language begins with a tribute to David Bowie.
 2. It is made for language models; people can read it too.
 3. A model may have read everything; each page must still explain itself.
 4. What the eye can see is all there is: sound and vision.
 5. There is one way to say each thing; every program sings in the same voice.
 6. That way may not be obvious at first; the Starman already knows it.
 7. The semantics are familiar; the syntax is new.
 8. Turn and face the strange: every strange choice here is deliberate.
 9. Nothing changes behind your back; every change signs its name.
10. Every plausible mistake must become a compile error before the program runs.
11. Unless you mark the gap with ???: an honest hole beats a confident guess.
12. A compile-time error is an answer; a runtime error is an ambush.
13. An error should tell you how to fix the program. Anything less is a complaint.
14. Repeating a word is cheap; forgetting one can cost the whole program.
15. The specification stays small under pressure; every word must earn its place.
16. Crashes and leaks are bugs. Ashes to ashes: what Heroes takes, it returns.
17. When pretty code and safe code disagree, safe code wins.
18. There is no standard library; C is the man who sold the world.
19. Never crashing in the same car: what C might keep, Heroes assumes it keeps.
20. We can be heroes, just for one day.

Line 10 states the thesis, line 18 the initial constraint, line 19 what that constraint costs at the border with C, and line 17 the priority when two principles conflict. Why the language is built this way →

What is built, and what is next

The chain is one table in the repository, a row per milestone, closed first and scheduled after. This page reads it when the site is built: 71 milestones closed of 92, and the open one is M-panic-location.

What comes next

Next: four commands build it yourself, or take the whole language in one file.