The build log

· changes

One entry for each stage of the project up to the site’s last refresh, newest first; what closed since is in the repository’s roadmap. Full reports, including decisions, reviews and problems encountered, are in the repository, which is open to read.

The check that could not end Every issue still open, defects, tasks and features alike, was run again against the compiler as it stands and given a written answer: some had been settled along the way, some no longer applied, a few were fixed on the spot. One of them was a check that could not finish. It writes each program again with a comment in every place one can stand and a bracket broken after every word, and asks whether the formatter, the tool that lays a program out the one way the language allows, keeps it. On one test file, a single statement two thousand brackets deep, that needed more than two days of computing. The fault was the formatter’s, which at every level of a chain worked out again everything below it. Now that it works out each part once, the file was judged whole in under five hours, and the full run, more than four million variants, ended with none refused. Every program in the repository still comes out byte for byte as before. The formatter, among the thirteen verbs
The claim, put to the test For the first time the claim Heroes is built on was tested with models in the loop, the AI programs that write code: forty tasks the author wrote and sealed, every one a trap, judged with Heroes’ own rules on and off, under a prediction made public before the first answer. No model saw a task before its run: the author wrote them in a folder the assistant’s tools are barred from, and ran them from the author’s own terminal. On the way the instrument was found keeping answers from a model it had not asked, and the practice run was done again; without the specification, no model wrote a program that ran right. The result is read against the prediction alone, and the site states it at the publication gate, the moment a finished thing is announced. What is measured, and how
The seed gets a witness The seed, the one C file a machine without Heroes builds the compiler from, had only ever shown that it reproduces itself, as a lying seed could too. Now a second, independent compiler vouches for it. The project’s first compiler, written in Rust and archived, was rebuilt and wrote the first seed again, byte for byte; each of the dozens of seeds since was rebuilt from the one before. At that first step, the compiler built from the archive’s own seed was still working after seven hours; the Rust one took under a second. gcc, a second C compiler, now builds the seed beside clang, the one Heroes already used. Every new seed names the route by which the one before rebuilds it, and the CI, the checks run automatically on every change, follows it. How to check it compiles itself
Read, and never written A function that only reads what its caller holds can now say so: function peek(l: Lex lent) reads the caller’s value where it stands, never writes it, and is called as plain peek(l). Before, a function that only read a record holding a lock had two spellings and neither was true: a copy, which the language refuses for a lock, or @, which claims a change. Handed on to a C library, the value now goes with C told it may only read. Readers given only the specification needed a second try, ten times out of ten, before the word existed; with it, nine of ten were right the first time. And the specification’s new text cost exactly what the judge who weighs it had predicted, to the last token, the unit a model reads text in. The rule, in the specification
What it was built from Every program Heroes builds now carries a fingerprint of what it was made from: its sources, its library and its runtime, the support code that goes into every program. heroes --version prints the compiler’s own on its second line. The seed had been rewritten at every round of repairs, and made up more than two thirds of the repository’s history. Half its weight was lines restating a position already given; they are joined now, and the seed is rewritten only when the compiler built from it can no longer build today’s, and at each release, each version the project publishes. The day this landed, the check that judged a compiler by its file’s date refused the very compiler the landing needed; now it asks the compiler what it was built from, and the compiler answers. The seed, and the fingerprint
The value C computes When a C library’s header, the file where it describes what it offers, defines a constant, the compiler now takes the value C itself computes, by running a small piece of the program under clang’s checks. Every route that did not run the program’s own code was wrong one way or the other, over tens of thousands of real constants on three versions of clang; the one that runs it was right every time. Each hole found on the way was in the compiler’s new code, not the programs, and was found by somebody attacking the code as built, never by a test suite. A C record with long arrays is built by naming the fields that matter and ending with rest: zero; a five-field record describing the machine had needed over a thousand zeros typed out. Constants and records from C
A stopped program shows what you wrote Stop a Heroes program in a debugger, the tool that pauses a running program so you can look inside, and it now shows what you wrote: your own variables, and their values in Heroes, a list as a list and a failure as a failure. Everything the compiler had invented left the view, about ten names where there had been more than a hundred. Only the names kept the compiler’s prefix, because a bare name in the C can be taken over by any library’s header, and on one machine one was: a program with a variable called linux built on the Mac and was refused on Linux. Each repair at the C boundary that night found the next, six in a row, and each changed whom the message blames, never what it refuses. Why the compiler writes C
Every item becomes a public issue Every defect, decision and task the project has tracked is now an issue on GitHub, the public site where its code lives: more than two thousand, imported in the order they were born, each keeping its own dates. Then the folder they came from left the repository. At the author’s request, where somebody outside the project got there first, the decision names them, and so does the thanks page. The last reading found links between issues that the design panel, the five judges who review every change to the language, had ordered and nobody made. The first defect named by its GitHub number was the compiler’s own: two modules each declaring a record called Held, handed to one function written for any type, passed the check and failed to build, both copies given one name in C. Somebody got there first
A variable that changes gets a symbol of its own A variable whose value can change, a cell, is now declared v @= 0 and takes its type from its value; v @ v + 1 changes it, and a cell that nothing ever changes is refused, with a fix that writes =. The question and the spelling were the author’s: why must a cell carry its type when a fixed name does not? Because @ did two jobs, and only the type told them apart. The old line v: i64 @ 0 is refused with a fix the compiler applies by itself, and that fix rewrote 1,181 files in one commit. The rewrite went wrong in the one place the compiler cannot see, text inside a test’s string, and twelve of the compiler’s own tests failed at once. How a cell is declared
The list that would not run dry The three words written on a C library’s handles, the addresses it hands a program for things it owns, became five, and the runtime now numbers every life C hands out. An address is not an identity: C can free one and hand it out again. Before, a stale handle at the same address as a live one was taken for it, and a correct program could abort ten times out of ten on three platforms. Then every repair, tried on the cases next to it, found the next defect, more than twenty after the milestone’s journal was first drafted. The list would not run dry, and that was the method working. The author stopped it by hand, and the check that would have refused the close was taught to say so. The five words, in the specification
The word that could not reach the call A C function handed an address into a program’s memory is now taken to keep it, unless its declaration says lent: the panel had declined that default, and the author chose it on principle, provided the cost was measured, under forty billionths of a second a call. It began with how far C may read: handed an eight-byte field and a count of 64, C overwrote the field next to it in silence; now that extent is declared and checked. Where no word can reach, because C keeps the address on one call and frees it on a later one that takes none, the dying program says how many loans were still open. Its first wording blamed C; the compiler engineer and the C pragmatist vetoed it, because a library’s own failed check looks the same. Lending to C
The wall was half a mirage Text C writes into a fixed field of bytes, like the machine’s name the system fills in, can now be printed: f.validated_bytes() reads it, or says it is not text. The author asked for it when a sitting found nobody could print the machine’s name. One line was the whole difference: the function that already read C text looked for a closing zero, and a fixed field need not have one. The other wall, building a record for C to fill, was not there: you ask the library for one, and the project’s tests did it in seven places while three sittings argued how. The assistant’s instructions to them had said three times that the compiler lacked things it had; since then, a claim that something is missing is tested first, or goes out as a question. Text that comes from C
The fourth machine scored its own prediction false Heroes now builds and runs on a fourth kind of machine, Linux on ARM processors, and the first thing it did was prove its own prediction false. The prediction said C’s plain char, its one-byte number, never goes below zero on ARM. On Linux it does not; but this Mac is ARM too, and its char does. So the split belongs to the system, not the processor, and on this point three machines had all been one. The one test that could see it had carried a comment saying “char is signed here” all along. Seven routes were argued; six described the split, and the winner removes it with one flag, an option handed to clang, -fsigned-char, so the sign is the same everywhere. Linux itself had taken the mirror-image flag, for the same reason. Build it on your machine
A hole that was not where the name said The milestone went hunting for programs heroes check accepts and heroes build refuses, and found that only one of its four faces was that gap, and it was already closed, by a sitting held the same day as the one that named it. The real hole was a walk down the code that gave up after sixteen levels, defended by an argument that was true: the runtime guard exists and it fires. The argument was missing one sentence, that a guard which licenses a compile-time silence is a guard the compiler leans on to be quietly wrong. Ten defects opened by its own sittings and ten closed, one of them opened by the author asking why the CI was red; and the specification came out shorter than it went in, two rules richer. What the compiler stops
Three words for what is borrowed from C A binding now says, on every function that touches a handle, one of three things: acquires and who takes it back, consumes, borrows. None of the three is inferred by the compiler. The form that infers by itself fell when the C pragmatist opened nineteen headers of real libraries: in twelve, the same library hands out a handle to keep with one function and to look at with another, without saying so. The spelling that names who releases was proposed by the reader, who had no way to weigh it, and it turned out to be the cheaper one too. And the rule that every tool re-reading a program has to be taught a new form fired three times, once on a defect nobody had seen. Handles and the three words
The label goes on the handing over, not on the thing Writing on a handle’s type who closes it, so the compiler closes it by itself: refused, because one function hands the same connection over and another only shows it, and a label on the type cannot know which of the two gave it. The C pragmatist compiled it: a program that reads three rows read zero and said all went well. The compiler engineer cast the sitting’s only formal veto. And the count taken before arguing found, in passing, that comparing two handles always answered yes: the commonest line anybody borrowing something from C writes, the one that asks whether anything came back, had never worked. The no, in full
The library’s own name becomes the type A pointer a C library hands you can carry the name the library gives it: record Db tag sqlite3. Hand one where another belongs and the compiler stops you on that line. Three defects opened by three people on three different days were one question: what does a bare address not tell you? Not what it points at, not which one it is, not whose it is. The tool built before the fix said half the cases were already caught: it caught none, and the half were accidents of the test. In the end a sitting convened over one word kept the word, and the completeness critic found a conversion hidden at the boundary and emptied the last places in the examples where a bare address escaped into the open. Handles
The reasons had rotted under the answers Every entry on the list of postponed things received a date and a verdict: it enters, it is refused for good, or it waits with a checkable condition. Nine entries, nine sittings, and nine times the answer was fine and the reason written under it had rotted. One had a clock that had already run out; one said a thing could not be done, and eight programs did it that afternoon; one described a part of the compiler this project has never had; one was contradicted by the same document, forty pages earlier. The decision that every postponed entry gets a verdict is the author’s, and five entries became no’s in a single day. Postponed is not refused
Every ceiling says which way it may move The author said he kept hitting his head on the ceilings and asked which of them still made sense. Almost all had been set to the exact height of the thing measured, on the day somebody measured it. One was worse than the others. The rulebook the assistant reads before every piece of work was being weighed on a borrowed scale, and by that scale it sat under the limit by nine units; on the scale the reader actually uses it was eighteen hundred over, and every instrument said all was well. Now next to each number is written which direction it may move, and something notices when it should have moved and did not. The repository
The name of a thing is no longer text typed by hand A program cannot ask a value what it is made of: three no’s, and each time the written reason was false, so the reasons were replaced with measurable ones. Then came the part nobody planned. A program sending data out had to write the names of its own fields as text, and one wrong letter went out into the world with nothing complaining. Four of the five seats invented something that was not on the list, in pairs, without seeing each other: Point::x is the field’s name, and the compiler knows whether it exists. The reader and the engineer proposed it in the same hour in two spellings, and the author chose the spelling the machine had not already built, for a reason about reading. With the names as text, four typos and none caught; with the names as names, five of five. The no to reflection
The grammar went where the prose was The author asked for the language written down formally, in a file of its own. Five seats looked at it from five directions and every one broke the same half: the part that would have been copied out by hand. The reader, allowed only the specification, found it already said the thing the proposal was built to fix; the historian came back with every attempt of that kind, in every other language, abandoned. So every rule of syntax now stands in the section of the specification that explains it, in Wirth’s notation, and the part a machine can work out for itself is worked out by heroes grammar, every time it is asked. The specification
The records are no longer one file each The author asked whether two pieces of work could run at once, and the answer was a measurement: two sessions never meet in the compiler, they meet in the records, on the one line both have to write. So every entry of a record is a file, and the old file stays where it was as a map, line by line, because a hundred and sixty-one citations pointed at line numbers nobody could check. The blind spot the project had documented as unfixable became a check that runs with every verification. The repository
fold’s type says which of the two is the pile The function that combines two things receives the accumulator and the element, and if they are of one type either order compiles and the program answers something else. Now the type says which is which, and the compiler checks that the function you hand over agrees. The panel had said this was the price of a rule already written; the author said close it anyway. The gentle version caught two of the project’s seven real cases, the strict one seven, at the price that anyone writing a function for fold calls its two parameters acc and item: the author read both numbers and chose the strict one. A reader who had never seen the project wrote those two words three times in three programs without once stopping to choose them. Functions as values
A function type carries the names that can be confused The half of the hole the panel had priced and refused, the author said to build anyway: a function’s type carries the names of the arguments that can be swapped, and changing one makes it another type. Then the panel judged a thing that already existed and three of the five seats sent it back: the spec warden over a sentence with one false reading, the C pragmatist with a callback that ignores an argument, which is what callbacks do all day, and the engineer for two reasons outside the design, a tree that did not compile and a price counted with the wrong ruler. With the right one it is four hundred and thirty-two lines against a budget of about a hundred and fifty, and the number was put in front of the author. The classic swap, through a function held in a variable, went from one in six caught to six in six. Functions as values
Three routes refused by three seats, and the fourth built in twenty-five lines A label written in a call made through a function held in a variable is refused: the function has no names to compare it with. Something happened at the sitting that had never happened: three of the four ways out were refused, each by a different seat and none on cost. The spec warden refused a sentence because a running program said it was a lie; the C pragmatist refused a rule that would have deleted a working program, and brought three measurements where the same small mistake reached the memory; the engineer refused the elegant fix because it was not a fix but a rebuilding of eleven rooms of the house. Then that same engineer built the route nobody had listed, and closed half the hole. The other half is the postcard above. Functions as values
The naming rule reaches the built-in functions If two of a function’s arguments are of one type, you have to say which is which. The compiler enforced it on everything the author wrote and on nothing it provided itself. A reader who had only the specification found it, by writing the same call two ways and being told off for neither. The tool that imitates the classic mistake, swapping two arguments of one type, was producing three hundred and sixty-seven programs the compiler accepted and should not have: a hundred and fifty-nine were this one call. After the repair there were seventy-five, and none of them was it. Records and functions
Thirteen numbers over thirteen doors The author read the language’s own description and said it looked like a room nobody had ever tidied: every rule was there, each where it had been dropped. The tidying cost less than nothing. Twenty sentences that said the same thing twice became one each, and the document came out lighter, with thirteen numbered sections in the order Naur’s committee chose and Wirth kept for forty years. And because the author asked for it, six readers who had seen neither the plan nor the repository were handed the two texts unlabelled: they found three additions, all foreseen, nothing missing, and one sentence that had been ambiguous in the old text. The specification
Text with holes in it A string can now carry values: f"line {n}: {word}" writes each hole as to_str would. Any expression may stand in a hole, {{ writes one brace and a lone } is a brace, and a plain string is unchanged. The compiler refuses a hole whose value has no text, and its note says what to write instead. The example
The ruling on interpolation Three counts decided how a string may hold a value, and the counts corrected the first guess twice. Admitting any expression costs fewer words in the specification than admitting only a name, and the brace was already at work in the compiler’s own strings: a string with holes announces itself with an f, and a plain one is unchanged. The specification
Bytes a C library may keep A program can hand a C library text it will keep, and say when it is done with it: x: cstr @= s.lease(), then end_lease(@x). Every rule proposed to forbid such programs was measured and refused in turn; what closed the hole was a way to say it, not a ban. A lease nobody ends stops the program when main returns, with the count of what was left open. The C boundary
A lent string stays where it was lent A string lent to C with .cstr() may stand only as an argument of the call that reads it. Kept anywhere else, it outlived the bytes it pointed into and the program read freed memory; the compiler now refuses that shape, with its reason, before anything runs. The C boundary
A note nobody had counted Closures, functions that carry the variables around them, do not enter Heroes. A note had called them easy since the design’s first week; when the day came to build them, the first thing anybody did was count. The one-line helpers they would delete: about twenty, all in example programs, none in the compiler. Their cost: hundreds of places to change across dozens of files. The reader, who sees only the specification, said what actually stops a reader is four missing sentences about the functions the language already has, and on the author’s instruction a second sitting looked at those: a filter’s type is the same whether it keeps or discards, so the specification gained about thirty words, one clause saying which half a filter keeps. And the historian found four languages that refuse closures on purpose, one since 1988. The no, in full
The polite way to lose a failure Writing _ = in front of a call that can fail now stops the compile. Thrown away with the value went the news that it had failed, and the program finished with the number that means all is well. The author had already ruled that it must be refused; what was left was where the wall goes. The first wall had a door, and a wider one taught a way around that also defeated the rule, so the wall stands where it works and the specification says what still gets through. No fix is offered as certain: the same line repaired three ways prints 1, 9, or stops the program. More than two hundred mistakes of exactly this kind were planted in real programs, and every one was refused, more than forty only by Heroes’ own rules. Failure is a value
The repository is open The compiler’s source, the specification and every record the project keeps can now be read by anyone. The licence is Apache 2.0, so the code can be used, changed and built on. Questions and problem reports are welcome; changes sent in are not accepted, and the repository explains why in one paragraph. The repository
Freeing strings returned by C An extern declaration can name the function that frees a string allocated by a C library. The compiler calls it automatically. The change also exposed an incorrect count of the functions used in the SQLite example. The SQLite example
Checking thread stack space Threads started by Heroes now report when they run out of space for function calls. A reviewer verified that a thread created by a C library is stopped by an earlier check that names the Heroes function involved. How to use threads
Separate memory for each thread Each Heroes thread, an operation running in parallel, manages separate memory. The review found a missing check in callbacks, functions called back by C: it allowed a list to cross the C interface without being reported. Thread examples
Passing Heroes functions to C libraries You can pass a Heroes function to atexit or pthread_create. The callbacks required by qsort and signal are not supported yet. The compiler reports the limit at the declaration. A reviewer examined the full SQLite header and found different obstacles from those suggested by the initial examples. The C boundary
Larger examples to test the compiler More complex programs were added to cover the gap between small examples and the compiler. Two reviewers rejected the proposed optimisation. One then identified excessive memory use by temporary values that the proposal had not considered. The programs
An error when passing array elements A comment described a crash as impossible, but one line could reproduce it. Passing an array element to a function that modifies it gave C the address of the whole array. The examples covered record fields but missed this case. What the compiler stops
Documenting modules The documentation lacked an explanation of how to organise a program’s files. A chapter on modules was added. Read the chapter
Expanding the examples New programs were chosen to test more parts of the language. The first found a defect: it passed type checking, but compilation failed with an internal error. The chapters
Organising source files into directories The compiler’s files were organised into directories. Directory structure replaced long filename prefixes, allowing shorter names. The compiler in its own language
Using paths for modules Modules can be found in subdirectories using a path. Previously, all files had to be in the same directory. The panel examined the new rules over several reviews. Modules and files
Running commands without a shell The compiler started tools by building a string for the shell, the program that interprets commands. On Windows, the same options were interpreted differently and created a directory with the wrong name. The problem led to passing arguments directly to the process. Why the language is shaped this way
Compiling one module at a time Compilation generated one large C file even after a small change. Code generation was split by module so results for unchanged parts could be reused. One C file per module
Archiving the first compiler The first compiler was archived. The work revealed that it also contained several tools used to measure and check the project. The current compiler
Writing the test tools in Heroes The tools that check the compiler were rewritten in Heroes. They run commands, read the results and compare them with expected output, testing the compiler as a person would use it.
Distributing the compiler as a C file Building the compiler requires compiling one generated C file with clang. The repository includes this file, so a computer without Heroes can build the compiler.
The compiler written in Heroes The compiler was converted from Rust to Heroes, together with its tests. Compiling its own source exposed defects in both the new version and its predecessor. Examples and tests with expected results checked the conversion. How to verify the compiler
Converting the first module The first trial converted one file and recorded any limits encountered. The expected limit had already been removed by another change. The lexer, which divides source text into recognisable elements, worked without the predicted obstacle.
Completing support for C structures More C declarations became usable from Heroes. The work corrected compiler rejections that prevented forms already allowed by the language.
Passing structures to C functions C structures group several fields into one value, but Heroes could not yet pass them to functions. This limitation blocked many operations in a real library.
Checking C function parameters The compiler checked C function results but not their parameters. Declaration checks were extended to input types.
Finding defects with complete programs The first complete programs found new compiler defects. They included a JSON reader, Markdown converter, spreadsheet, maze solver, assembler, log analyser and game.
Integers of different widths Heroes introduced eight integer types with explicit widths. The word int is rejected because it does not specify a type. This required revisiting an initial design restriction.
Making numeric bases explicit A reviewer wrote 0700 for file permissions and asked how it was interpreted. The question exposed the need for clear rules to distinguish numeric bases.
Using constants from C headers C library options can be named in the program. Previously, programs had to copy their numeric values by hand. Reading the value from the header removes that duplication.
Checking declarations against headers An example returned 1 where 1.41 was expected: C declaration checking had a defect. Verification must compare the Heroes declaration with the actual header through clang.
Defining module names Programs can be split into files. Rules were defined for name conflicts and imported modules that are not used.
Completing generics and core operations Generic functions and core operations were completed. Open questions were checked with the compiler, including how types are represented during execution.
Maps and fallible results Implementing maps depended on values that represent a result or an error. Looking up a key can return its associated value or an error if the key is missing.
Representing recursive data Heroes can represent an expression tree without mutable references between values. Each node holds its children through an array.
Managing string memory The compiler introduced reference counting to free memory. Checks challenged the initial rule, which released each string at the end of the block that created it.
Producing the first executable A Heroes program was compiled into a native executable. The work also raised the question of which exit status to use when a program is valid but the compiler does not yet support everything it needs.
Choosing the internal representation The compiler’s intermediate representation of a program was defined. LLVM guidance led to starting with memory locations, reads and writes, leaving optimisation to a later pass.
Measuring the first checks Name, type and missing-case checks were tested by introducing mistakes into working programs. Comparing them with the Heroes-specific checks disabled measured the effect of those rules.
Connecting names to declarations Name checking rejected two declarations in a calculator program. The program defined map and fold, but both names were already reserved by the language.
Parsing and formatting code Writing the parser, which recognises code structure, revealed a contradiction in the specification. fn was both a forbidden word and the prescribed form for a function type. Parameters of that type could not be declared.
Splitting source into tokens The first compiler component divides source text into tokens, the elements the compiler recognises. Implementation exposed problems that the design alone had not revealed.
Preparing the project and tests The Rust project, test tools and first example programs were prepared. The specification became the reference. Its initial size limit was still an estimate and changed when tokens were counted.
Reviewing the plan Five reviewers critically examined the plan before implementation. They assessed language design, compiler engineering, simplicity, use by models and teaching.
The initial idea The project began with a document and a discussion about the language. The aim was a compiled language that used C libraries and was designed for models that generate code.

Full reviews, measurements and journals are in the repository, open to read. Follow the feed to hear about the next stage.