Heroes / Guide / chapter 14

The C boundary

Declaring and using functions, types and constants from C libraries.

Heroes uses C libraries for features such as sockets, databases, graphics and mathematical functions. This chapter explains how to declare them and what the compiler checks.

A group names its header

extern "math.h" link "m"
    function sqrt(x: f64) -> f64
    function pow(base: f64, exponent: f64) -> f64
    function floor(x: f64) -> f64

After these declarations, you can call the three C functions. The header is the file that describes the library’s functions and types. Clang reads it and checks the group’s declarations. This connection between Heroes and C is called a binding.

What that check catches

If you misspell a function name, the compiler reports the error at the declaration and includes clang’s response:

$ heroes build ffi.hero
error[ffi_unknown_name]: `math.h` declares no `flooor` — clang read the header and could not find it
  at ffi.hero:15:5
     |
  15 |     function flooor(x: f64) -> f64
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  note: an `extern` names what the header already has (§4.19): check the spelling, or name the header that does declare `flooor`
  fix (guess): the header declares `floor`

The suggestion on the last line comes from clang, which has read the library’s header.

Get a type wrong and the answer comes from the same place:

$ heroes build ffi.hero
error[ffi_return_type]: `puts` does not return `cstr` — that is what `stdio.h` says, and clang read it
  at ffi.hero:22:5
     |
  22 |     function puts(s: cstr) -> cstr
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  note: an `extern` is checked against the real header (§4.19): correct the result type, or name the header that declares this `puts`

Checking declarations against the header catches errors that could otherwise cause problems when the program runs.

The boundary is visible in the types

# `cstr` is a C string and `ptr` an opaque pointer. Neither is a `str`:
# the boundary is visible in the type. A second group, because this one
# comes from a different header — and on macOS neither needs `link`, since
# libm and libc are already there.
extern "stdio.h"
    function puts(s: cstr) -> i64

cstr represents a C string, and ptr an opaque pointer, an address whose contents Heroes does not read directly. Both differ from str, and conversions are explicit. s.cstr() lets C read a Heroes string for the duration of the call. c.validated() checks and copies C text, returning a str?: a value or an error if the pointer is null or the text is not valid UTF-8. Passing a null cstr to a C function stops the program with an error message.

The types distinguish data from C, and explicit conversions show where it becomes a Heroes value.

A string lent with .cstr() may stand only as an argument of the call that reads it, and only where the declaration says the call keeps nothing of it: function strlen(s: cstr lent) -> u64. Without that word a C parameter is taken to keep what it is handed, and the lend is refused, because bytes kept past the call are read after the program has changed or freed them. The word is a claim about C that nothing here can check, so the compiler never writes it for you. Kept anywhere else, the pointer would outlive the bytes it points into, and the compiler refuses that shape before anything runs. When a library keeps the text past the call, the program says so and stays in charge of the bytes. x: cstr @= s.lease() hands C a copy of the bytes on lease, a loan that ends when you say so: the copy lives until end_lease(@x) frees it. A lease nobody ends stops the program when main returns, with the count of what was left open. A program that leaves through exit() is not counted there, and on the Linux build the memory checker reports the bytes as leaked. In the example, keep_label is a C function that keeps the pointer and kept_label_length reads it later:

    while at < 3
        label: cstr @= f"row-{at}-payload".lease()
        keep_label(s: label)
        print(f"C still reads {kept_label_length()} bytes")
        end_lease(@label)
        at @ at + 1

The copy is taken when the lease begins, so later changes to the string do not reach C. And if the library still reads the text after end_lease, no compiler can see it: the header does not say how long C keeps a pointer, so that mistake stays yours, and the memory checker the Linux build runs with, the sanitizer, is what finds it.

There is one thing a declaration cannot say at all: that the C function frees what it is handed. A lease is the program’s own memory, and only end_lease may free it, so a library that frees it kills the program on the spot. No word would help, because the library may free the bytes later, from a callback, or from a call that takes no pointer at all, and a word is written on a parameter. What the program does instead is say what it saw: when it dies that way it prints how many leases were live and in which function, and leaves the reader to check whether C freed one. When a library is meant to own the memory, give it memory of its own: allocate through the library’s allocator and hand over the function that disposes of it. The whole example, with its header →

A bare ptr stays an address and nothing more: Heroes cannot know whether the one you hand back to a library is the right one, or how the library uses it. When the library has a name for its pointer, the group can use that name and the compiler checks more: those are the handles, further down this chapter. Type checks do not remove the risks of C code.

Another check runs during execution. If a C library calls a Heroes function from a thread it created itself, execution stops and the message names the function. Each Heroes thread manages its own memory; the external thread is not initialised to do so. The library must call the function from the original calling thread, or the thread must be started from Heroes.

Using them

# The hypotenuse, in the language's own arithmetic plus one C call.
#
# Both sides are `f64`, so §4.9 makes the names mandatory at the call site. C's
# numeric APIs are where same-typed parameters cluster — `pow(base:, exponent:)`
# is the same shape — so this is where the rule is felt most and where argument
# inversion would be silent without it.
function hypotenuse(a: f64, b: f64) -> f64
    return sqrt(a * a + b * b)

You call a C function like a Heroes function. The label rule from chapter 3 also applies: in pow(base:, exponent:), both arguments have the same type, so their names are required.

# No implicit conversions: an `i64` becomes an `f64` because you said so.
function diagonal_of(width: i64, height: i64) -> f64
    return hypotenuse(a: to_f64(width), b: to_f64(height))

# `pow` is C's, `floor` is C's, and the rounding is yours.
function rounded_power(base: f64, exponent: f64) -> i64
    return to_i64(floor(pow(base: base, exponent: exponent))).must()

Numeric conversions are explicit. to_f64 returns a value directly. to_i64 can fail if the value cannot be represented as an integer, so it returns an i64?. Here .must() extracts the result or stops the program on error.

Finding libraries and dependencies

link "m" names the library to link with the program. If paths or other dependencies are also needed, you can name a package whose settings are known to the system:

extern "raylib.h" package "raylib"

This declaration asks the system for header paths, libraries and other required dependencies, such as frameworks on macOS or -lGL -lX11 on Linux. The Heroes syntax stays the same across platforms. If the package requests unsupported options, the compiler lists them and stops the build.

A group can also declare a constant, whose value comes from the header rather than from your file, and a record that mirrors a C struct field for field. Both are checked the same way, by clang, against the real thing.

You can pass one of your functions to a C library for it to call later: this is a callback. Heroes allows callbacks as parameters, but not as C function results. Clang checks their types against the header. Some callbacks, such as qsort’s, need pointer checks that are not yet supported; the compiler reports the limit at the declaration.

Use owned to name the C function that frees a library-allocated string, received as a result or through an @ parameter. The compiler uses that function to free the string and returns the text as a Heroes value.

Handles, and the three words that say whose they are

Many C libraries hand you a handle: a pointer to something the library keeps for itself, a database connection, a prepared query, and that sooner or later you will have to give back. So far that would be a ptr, an address you know nothing about. Inside a group you can instead give it the name the library itself uses: a record with no fields, a tag and the name of the C type is the pointer to that type. Clang checks the name against the header, a CDb cannot be handed where a CStmt belongs, and nullptr is its empty value.

    record CDb tag sqlite3
    record CStmt tag sqlite3_stmt

Then every declaration that touches a handle says what it does, with one of three words. acquires and a name: the call hands you one, and that name is the function that takes it back. consumes: the call takes one back and ends its life. borrows: the call shows you one the library goes on keeping, and you owe nothing. None of the three is guessed by the compiler, because two functions with the same signature can do the first thing and the third: you write it.

    function sqlite3_open(filename: cstr lent, @out: CDb acquires sqlite3_close) -> i64
    function sqlite3_close(db: CDb consumes) -> i64

Two checks follow from those words. The first happens before the program runs. A parameter written without @ is borrowed: the caller keeps the value and reads it again afterwards. Handing it to a call that closes it would leave them holding an address C has already taken back, so the compiler refuses the line, and the repair is to write @ on the parameter and at the call. Here shut receives the connection on loan and tries to close it:

$ heroes check handles.hero
error[consumed_borrowed_handle]: `sqlite3_close` ends the life of what it is given, and `db` was borrowed — this function promised not to change it
  at handles.hero:8:27
    |
  8 |     _ = sqlite3_close(db: db)
    |                           ^^
  note: a parameter written without `@` is borrowed: the caller keeps the value and reads it again after the call, so a call that frees it leaves them holding an address C has taken back. Mark the parameter `@db` and the call site `@db` too, which says on both lines that the value does not survive
  note: this refuses the call, not the copy: a copy made BEFORE the call still holds the freed address, and nothing here sees that. design.md Part 8 wart 20 states that cost and names what would close it

The second check happens when the program ends. Every handle taken with a call marked acquires has to be given back with one marked consumes. When main returns, the runtime, the piece of program that sits underneath and keeps things working while it runs, counts the handles still open and, if it finds any, stops the program saying how many. This program opens a connection and never closes it:

$ heroes run forgetful.hero
opened, rc = 0
panic: 1 C handle(s) never given back — every call marked `acquires` owes one marked `consumes`, and this program is missing that many. The first is at 0x1031491f0

The same count stops a handle given back twice, which in C would free the same memory twice. And the three words check each other: once one function in the group declares consumes for a handle type, every function that hands one out has to say which of the three words applies, and one that stays silent is refused at its declaration. They make sense only on a handle, that is on a record with a tag: on a bare ptr the compiler would have nowhere to read them, and refuses them.

The program

test "the C library does the maths"
    assert hypotenuse(a: 3.0, b: 4.0) == 5.0
    assert rounded_power(base: 2.0, exponent: 10.0) == 1024

function main()
    print(
        "the 16:9 diagonal of a 1920-wide screen: ",
        diagonal_of(width: 1920, height: 1080)
    )

$ heroes run examples/gallery/08-ffi.hero
the 16:9 diagonal of a 1920-wide screen: 2202.9071700822983

Two C libraries, four declarations, no wrapper and no binding generator. The repository binds SQLite, curl, SDL and raylib the same way: a group, a header, and the declarations that program needs.

The rules covered so far

Seven words that can start a top-level line, explicit rules for data, errors and changes, C libraries for external features, and three words that say whose a handle is: these are the language elements covered so far.

That is the guide. Back to the chapter list · why the language is shaped this way. ← Previous: Tests and holes

Run a program of your own: it takes four commands. Or take the whole language in one file, and every program already written in it.