# calog -- Polyglot Script Broker: Design & Implementation Plan

A C "broker" that lets one application be written in a mix of scripting languages
(Lua and my-basic first; Squirrel and others later). Native C functions are added
once and become callable from every language. Functions and data exported from one
module are callable from modules written in another language. Threading is actor-based;
networking rides the same dispatcher. Data sharing is by-value for v1.

This document is the reconciled output of a design pass plus an adversarial verification
pass. Where the verification corrected the first-cut design, the correction is folded in
and noted as "[verified]".

---

## 1. Architecture: hub and spoke

Nothing talks to anything else directly. Every engine talks only to the broker, through
two shared contracts:

1. One universal value type, `ValueT` (a tagged union).
2. One uniform native-function signature:

```c
typedef int32_t (*NativeFnT)(ValueT *args, int32_t argCount, ValueT *result, void *userData);
```

A developer writes a native function once against that signature and registers it once.
A *script* function exported from a module is itself stored as a `NativeFnT` whose body
re-enters its owning interpreter -- so "call C from script", "call script from C", and
"call module A's function from module B" are all the same code path. Adding an engine is
O(1) adapter work, not O(N) per existing engine.

The single most important lesson from the verification pass: **there must be exactly one
`ValueT` / `AggregateT` / `ValueTypeE`, defined in `broker.h`, included verbatim by every
adapter.** The first-cut design had three divergent copies and would not have linked, let
alone round-tripped data. Section 2 is therefore the load-bearing part of this plan.

---

## 2. The canonical type system (`broker.h`) -- single source of truth

### 2.1 Value tags and the value struct

```c
typedef enum ValueTypeE {
    valueNilE       = 0,
    valueBoolE      = 1,
    valueIntE       = 2,   // int64_t
    valueRealE      = 3,   // double
    valueStringE    = 4,   // length-prefixed, binary-safe
    valueAggregateE = 5,   // hybrid array + map container
    valueFnE        = 6    // function value: refcounted handle to a CallableT
} ValueTypeE;


typedef struct StringT {
    char    *bytes;    // owned; always NUL-terminated at bytes[length] for C consumers
    int64_t  length;   // byte count excluding the convenience terminator (binary-safe)
} StringT;


typedef struct ValueT {
    ValueTypeE type;
    union {
        bool             b;
        int64_t          i;
        double           r;
        StringT          s;
        struct AggregateT *agg;       // heap-owned subtree
        struct CallableT  *callable;  // refcounted, broker-owned
    } as;
} ValueT;
```

### 2.2 The aggregate (one shape both engines map onto)

```c
typedef struct PairT {
    ValueT key;     // marshal layer constrains to int/real/string keys (see 2.5)
    ValueT value;
} PairT;


typedef enum AggregateKindE {
    aggregateListE = 0,   // empty container round-trips as a list by default
    aggregateMapE  = 1,
    aggregateBothE = 2    // array part AND pairs part populated
} AggregateKindE;


typedef struct AggregateT {
    AggregateKindE  kind;        // disambiguates empty/mixed containers across engines
    ValueT         *array;       // dense elements [0, arrayCount)
    int64_t         arrayCount;
    int64_t         arrayCap;
    PairT          *pairs;       // map part; preserves insertion order
    int64_t         pairCount;
    int64_t         pairCap;
} AggregateT;
```

A Lua table's sequence part maps to `array`, its remaining keys to `pairs`. A my-basic
`LIST` maps to `array`, a `DICT` maps to `pairs`. The explicit `kind` flag fixes two
problems the verifier flagged: an empty `{}` had no defined type on the far side, and a
mixed array+hash Lua table had no representation at all.

### 2.3 Function values: `CallableT`

`valueFnE` is the one deliberate exception to "by-value everything". A function cannot be
meaningfully copied between heaps, so it is shared by *reference* -- but safely, because it
is only ever *invoked*, never inspected, and invocation always routes to the owning
context's thread.

```c
typedef struct CallableT {
    NativeFnT  fn;            // uniform invoke entry
    void      *userData;      // luaL_ref slot, pinned mb_value_t, or C closure ctx
    uint32_t   ownerCtxId;    // context whose thread MUST run it
    uint32_t   ownerGen;      // generation of that context (UAF guard, see sec 9)
    int32_t    refCount;      // ATOMIC; shared-handle lifetime across threads
    bool       alive;         // false once the owning context is torn down
} CallableT;
```

Rules (these resolve the verifier's critical "function value breaks the no-shared-pointer
invariant" finding):

- `valueCopy` of a `valueFnE` does an **atomic refcount increment** on the same
  `CallableT` -- it does NOT clone the closure. The shared `CallableT*` across threads is
  allowed precisely because `refCount` is atomic and the closure is touched only on its
  owner thread.
- `valueFree` of a `valueFnE` does an **atomic decrement**. When it hits zero, the
  underlying closure must be released with an interpreter call (`luaL_unref`, my-basic
  unref) -- which can only run on the owner's thread. So a zero-drop on a foreign thread
  **posts a release message to the owner context** rather than calling the interpreter
  directly (sec 10).
- Invoking a dead handle (`alive == false`, owner gone) returns a clean broker error,
  never a call into a freed interpreter.

### 2.4 Value operation contract (one signature, used everywhere)

The verifier caught two designs declaring `valueCopy` with incompatible signatures. The
canonical form -- status-returning, dst-by-pointer, so OOM is checkable on the hot path:

```c
int32_t valueCopy(ValueT *dst, const ValueT *src);   // deep copy; brokerOkE / brokerErrOomE
void    valueFree(ValueT *v);                          // recursive free; leaves a safe nil
void    valueMove(ValueT *dst, ValueT *src);           // zero-alloc ownership transfer
```

`valueFree` and `valueCopy` MUST have a `default:`/explicit case for every tag including
`valueFnE` -- a missing case was how the first cut silently leaked function payloads.

### 2.5 Cross-engine fidelity table (the honest limits)

By-value marshalling across `Lua <-> broker <-> my-basic` is lossless for the common cases
and lossy at documented edges. These are inherent to my-basic's value model (verified
against `my_basic.h`/`.c`), not marshalling bugs:

| Aspect                | Lua                    | my-basic                          | Crossing Lua <-> BASIC                          |
|-----------------------|------------------------|-----------------------------------|-------------------------------------------------|
| integers              | 64-bit                 | `int_t == int` (32-bit, always)   | **truncates above 2^31** -- range-check + error |
| reals                 | double                 | `float` (double w/ `-DMB_DOUBLE_FLOAT`) | precision loss unless double build         |
| int vs real subtype   | distinct (5.4)         | integral real auto-collapses to int | **subtype not preserved** through BASIC       |
| strings               | binary-safe (length)   | bare `char*` (`strlen`)           | **embedded NUL truncates** -- detect + error    |
| array / list          | table sequence part    | `LIST`                            | OK                                              |
| map / dict            | table hash part        | `DICT` (int/real/string keys)     | OK                                              |
| mixed array+hash table| one table              | no single LIST+DICT value         | **collapses to DICT** (array part -> int keys), documented |
| empty `{}`            | table                  | LIST or DICT?                     | `kind` flag; default LIST                       |
| function value        | closure (`luaL_ref`)   | lambda/routine (`MB_DT_ROUTINE`)  | OK via `CallableT` (by-reference)               |
| nested depth          | bounded                | bounded                           | one shared cap; defined error past it           |
| cycles                | rejected on ingress    | rejected on ingress               | impossible at rest (by-value, no shared refs)   |

Policy decisions baked in to make the above deterministic:

- **One shared recursion-depth cap** applied on *every* recursive path -- both ingress and
  egress, all components -- failing with a defined status instead of overflowing the C
  stack. (The first cut bounded only Lua ingress.)
- **One strict/lenient switch**, owned by the broker and honored by both adapters: in
  strict mode an unrepresentable value/key (e.g. a function used as a table key, a 64-bit
  int into BASIC, a NUL-bearing string into BASIC) is an error; in lenient mode it is
  dropped/coerced with a documented rule. Never a silent surprise either way.
- **Keys**: broker allows int/real/string keys (my-basic dicts accept all three). Float
  keys get defined equality; non-representable keys follow the strict/lenient switch.

---

## 3. The engine vtable -- what makes the actor loop engine-agnostic

Each adapter implements one small vtable so the broker/actor core never special-cases an
engine:

```c
typedef struct EngineT {
    const char *name;
    void   *(*createInterpreter)(struct ScriptContextT *ctx);   // ON the owning thread
    void    (*destroyInterpreter)(void *interp);
    int32_t (*loadSource)(void *interp, const char *src, int64_t len);
    int32_t (*registerNative)(void *interp, const char *name, NativeFnT fn, void *userData);
    int32_t (*callExport)(void *interp, void *exportRef, ValueT *args, int32_t argCount, ValueT *result);
    void    (*releaseExport)(void *interp, void *exportRef);     // ON the owning thread
} EngineT;
```

`createInterpreter`, `destroyInterpreter`, `releaseExport`, and every `callExport` run on
the context's own thread -- that is the invariant that keeps each interpreter
single-threaded.

---

## 4. The Lua adapter

Target Lua 5.4 (note 5.1/5.3 deltas where they matter). The C API surface was verified
accurate; the fixes below are about lifecycle, not API names.

### 4.1 Native registration and the trampoline

Each registered `NativeFnT` becomes a Lua C closure: push the `NativeFnT` and its
`userData` as upvalues with `lua_pushcclosure`, then `lua_setglobal` (or into a module
table). The single trampoline recovers them from upvalues, marshals the Lua stack into a
`ValueT[]`, calls the `NativeFnT`, and pushes the result back.

- `lua_setglobal` returns `void` (the first cut documented `int` -- harmless but wrong).
- Lua allocation APIs (`lua_newuserdatauv`, `lua_createtable`, ...) **longjmp on OOM and
  never return NULL** -- so NULL-checks after them are dead code; the OOM path is a Lua
  error, not a C return. Only `luaL_newstate` can return NULL and must be checked.

### 4.2 Marshalling `ValueT <-> Lua` (by value)

Scalars are direct. Strings use `lua_tolstring` + length (binary-safe, preserves NULs).
Tables deep-copy in both directions:

- Ingress (table -> `AggregateT`): normalize the table index to absolute before `lua_next`;
  route numeric keys through `lua_tointeger`/`lua_tonumber` (do **not** let `lua_tolstring`
  mutate a numeric key in place); fill `array` for the sequence part and `pairs` for the
  rest; detect cycles via an ancestor-pointer stack (`lua_topointer`); use `lua_rawset`/
  `lua_rawget` to avoid metamethods; enforce the shared depth cap.
- Egress (`AggregateT` -> table): build with `lua_createtable`, populate `array` as the
  sequence and `pairs` as keyed entries, **with the same depth cap** (the first cut had no
  egress cap -- a deep BASIC-origin structure could overflow the stack on the way back).
  Make the builder self-balancing: record `lua_gettop` on entry and `lua_settop` back on
  any error return.

### 4.3 Exporting a Lua function (and the leak fix)

A Lua function crossing the boundary becomes a `valueFnE`: pin the closure with
`luaL_ref(L, LUA_REGISTRYINDEX)`, wrap it in a `CallableT`. The `fn` body does
`lua_rawgeti` to retrieve, marshals args onto the stack, `lua_pcall`, marshals the return;
on error it pulls the message with `luaL_tolstring` and reports it as a broker error
(sec 8).

Two bugs the verifier found, fixed here:

- **The exports array needs grow-on-demand.** The context is `calloc`'d, so the array
  starts NULL/0; the first export must `realloc` (double the cap, handle the NULL/0 seed)
  before storing, and `luaL_unref` the just-created ref if the realloc fails.
- **Transient vs persistent ownership.** Every Lua function passed as a *native argument*
  was creating a `luaL_ref` that only got released at `lua_close` -- an unbounded leak for
  any long-lived context that takes callbacks. Fix: the `CallableT` refcount owns the
  `luaL_ref`. When the last `valueFree` drops the handle to zero, the ref is released (on
  the Lua thread per sec 10). A function the broker retains as a real export holds a
  reference for as long as it is registered; a function merely borrowed for the duration
  of one call is released when that call's `ValueT` args are freed. Same mechanism, two
  lifetimes.

### 4.4 Calling a function value from Lua

A `valueFnE` marshalled *into* Lua becomes a Lua C closure over the `CallableT*`, so script
authors just write `cb(x, y)` and it transparently dispatches (to the owner's thread if the
function lives elsewhere). A universal `call(fn, ...)` native is also provided for
uniformity across engines.

### 4.5 Context lifecycle

`luaL_newstate` + `luaL_openlibs` on the owning thread; confine the `lua_State` to that
thread forever; teardown `luaL_unref`s outstanding refs then `lua_close`.

---

## 5. The my-basic adapter

Verified against `paladin-t/my_basic` (`my_basic.h` + `.c` read directly). **Zero
hallucinated calls.** The interesting work is three my-basic-specific quirks that shape the
adapter; all three are forced by the source, not stylistic.

### 5.1 Lifecycle and the inverted register result

`mb_init` (once per process) / `mb_open(&bas)` / `mb_load_string(bas, src, true)` /
`mb_run` / `mb_close` / `mb_dispose`. The broker pointer is threaded through the interp's
single userdata slot via `mb_set_userdata` / `mb_get_userdata`.

- **`mb_register_func` returns a count, not a status** -- nonzero means registered, `0`
  means duplicate/failure. That is the opposite of the `MB_FUNC_OK == 0` convention, so the
  success test must be inverted. Names are uppercased internally (`mb_strupr`), so the
  broker key is the uppercased identifier (BASIC is case-insensitive).

### 5.2 The native-function protocol and the trampoline bank

The native signature is `typedef int (*mb_func_t)(struct mb_interpreter_t*, void**);` --
**no per-callback userData parameter**, and the interpreter has only **one** userdata slot.
So a single shared C trampoline cannot tell *which* broker function it is serving.

Fix (the verifier confirmed this limitation is real): a **macro-generated bank of
trampolines** `mbTramp0 .. mbTrampN`, each hardcoding its slot index, each looking up
`ctx->nativeBank[slot]` (the `NativeFnT` + `userData`) via the interpreter's userdata
pointer. The bank size caps how many natives one my-basic context can host; size it
generously and document it.

Inside a trampoline the argument protocol is the real my-basic frame dance:
`mb_attempt_open_bracket` / loop `mb_pop_value` (honoring `mb_has_arg`) /
`mb_attempt_close_bracket` / compute / `mb_push_value` (or the typed `mb_push_*`).

### 5.3 String ownership (memdup is mandatory)

`mb_pop_string` hands back a **borrowed interior pointer** -- the broker must `strdup`/copy
it immediately. Pushed strings are taken over by the interpreter and later freed with *its*
allocator, so any string handed to `mb_push_value`/`mb_make_string` **must come from
`mb_memdup`** (not plain `malloc`). Embedded NULs cannot survive (bare `char*` + `strlen`)
-- enforce the strict/lenient policy on egress.

### 5.4 Aggregates: the collection API

There is no `mb_make_coll`. A list/dict is built by presetting `coll->type =
MB_DT_LIST`/`MB_DT_DICT` then calling `mb_init_coll`, and accessed with
`mb_get_coll` / `mb_set_coll` / `mb_remove_coll` / `mb_count_coll` / `mb_keys_of_coll`.
Collection support is on by default (`MB_ENABLE_COLLECTION_LIB`). A broker aggregate with
both `array` and `pairs` populated collapses to a DICT (array part becomes integer keys)
per the fidelity table.

### 5.5 Exporting a BASIC routine -- the parked `__BROKERSERVE` frame

This is the my-basic-specific crux. To call a BASIC routine/lambda from C you use
`mb_get_routine(s, l, name, &val)` then `mb_eval_routine(s, l, val, args, argc, &ret)` --
and **`mb_eval_routine` dereferences `*l` and hard-requires a live, non-NULL `void** l`**
(verified at `my_basic.c:14344/14358`). A valid `l` only exists *inside* a running native
call. Therefore a my-basic context cannot be driven from arbitrary C; it must be **parked
inside a native frame**.

Design: register a native `__BROKERSERVE` whose C body *is* the context's message-pump /
serve loop. A module hands control to the broker by ending with a `SERVE` call (the adapter
appends one if absent). While parked there, the loop holds a valid `l`, which it uses to
`mb_eval_routine` whenever another context calls one of this module's exported routines.
`mb_get_routine` returns `MB_FUNC_OK` with a *nil* value when a name is absent, so the
not-found test is `routine.type != MB_DT_ROUTINE`, not the status code.

For the callback direction the parked frame is not the only way in: the fork also provides
`mb_eval_routine_cold(s, routine, args, argc, ret)` (see `vendor/ourbasic/CHANGELOG`), which
invokes a routine value from an *idle* interpreter with no live `l` -- it supplies the last AST
node as a clean return landing (the head segfaults) and passes args directly. Combined with the
bare-`def`-as-value evaluation (a routine identifier not followed by `(` yields the routine
value instead of "Open bracket expected"), a my-basic script passes a top-level `def`/lambda to
`psSubscribe` / `timerAfter` / `calogExport` and it is fired later, cross-engine included. The
adapter invokes synchronously via the live-frame `mb_eval_routine` when a callback runs inside a
serving native call, and via `mb_eval_routine_cold` when it is delivered to an idle context. (A
callback must be a top-level routine; one local to a function dangles once that function returns.)

The same fork also gives my-basic bare-name resolution for exports. A third add-only patch adds
an `mb_dynamic_func_handler` hook (a field on the interpreter plus `mb_set_dynamic_func_handler`):
at the "invalid identifier usage" site in `_calc_expression` -- a bare name called like a function
that is neither variable nor routine nor collection -- the interpreter offers the name to the
handler before erroring. The adapter's handler resolves it against the export registry
(case-insensitively, since my-basic uppercases identifiers at parse time) via a fold variant of
the export resolver, and invokes it. So a BASIC script calls `exportedFn(args)` directly, without
an explicit `calogCall` -- the convenience the hook engines (Lua/JS/Squirrel/s7) get from their
unknown-name hooks. Wren and Berry, whose C APIs offer no such hook, still use `calogCall`.

### 5.6 Numeric and identity caveats

`int_t` is 32-bit unconditionally (64-bit broker ints truncate -- range-check + error or
promote to real with documented precision loss); integral reals auto-collapse to int so
real/int subtype is not preserved across a BASIC hop. Both are in the fidelity table; both
follow the strict/lenient switch.

---

## 6. Threading: the actor model

Each `ScriptContextT` owns one interpreter, one OS thread (pthreads -- chosen over C11
`<threads.h>` for portability/maturity), and one inbound MPSC message queue. Interpreters
are single-threaded; only the owning thread ever enters `callExport`. A cross-context call
is a message; the caller blocks for the reply on a per-call condvar future (lost-wakeup
safe via a predicate loop). The verifier confirmed the core is sound: no path lets two
threads touch one interpreter, the deep-copy ownership ledger is correct on the success
path, and the epoll thread enqueuing while a context is mid-dispatch is race-free.

The fixes folded in from verification:

- **One error channel.** The first cut carried a separate error `ValueT` in the reply that
  the caller never freed -- a leak on every errored call, lost error text, and a second
  source of truth contradicting the broker's "error travels in `result`" contract. Fix:
  on failure the adapter writes the error string into `result` (as `brokerSetError` does);
  the reply carries only `{status, result}`. One channel, one owner, freed once.
- **`valueCopy` checked on enqueue.** Use the canonical `int32_t valueCopy(dst, src)`,
  check each arg, and unwind partially-copied args on OOM (mirroring the broker route
  path). The actor layer should call the broker's marshalling, not reimplement a copy loop.
- **Shutdown drains everything.** On `SHUTDOWN`, error-reply every queued `CALL` *and*
  free every queued/stashed `REPLY` (result + error) before join -- the first cut leaked
  in-flight replies unwound by a nested shutdown.
- **Explicit thread stack size.** The reentrancy depth bound counts dispatch nesting, not
  C-stack bytes; set a validated stack size with `pthread_attr_setstacksize` (or lower the
  bound) so the "clean catchable depth error" promise actually holds instead of a UB
  overflow.
- **Split the ready-handshake condvar** off the queue condvar so `queueCond` has exactly
  one semantic (latent lost-wakeup footgun if a second waiter is ever added).

### 6.1 What a context does while blocked: always-live nested pump [DECIDED]

When context A makes a synchronous cross-context call and waits for the reply, A's thread
**pumps its own inbound queue** instead of sleeping idle. An incoming call to A -- including
a re-entrant B->A issued during the very call A is waiting on -- is serviced on A's own
thread, then A resumes waiting. This was chosen over strict run-to-completion because it
never deadlocks and needs no wait-for-graph deadlock detector; the rejected alternative
would have had to raise a "synchronous call cycle" error on A->B->A and would leave a busy
context unresponsive to other callers. The verifier validated the pump as sound: only A's
thread ever enters A's interpreter (the single-threaded invariant holds), reply nesting is
strict LIFO, and depth is bounded.

The contract this commits the runtime and script authors to:

- **Re-entry happens only at explicit cross-context call points** (`x = getUserInfo()`,
  `data = sockRecv(c)`), never mid-statement -- a call point is a yield point.
- **Module-global state may differ after a cross-context call returns**, because another
  call may have run on this context while it was outstanding (the same contract as any RPC).
  Local variables are unaffected.
- **Reentrancy is depth-bounded** with a catchable error (backed by an explicit pthread
  stack size, sec 6 fixes), so runaway ping-pong fails cleanly instead of overflowing.

Script code stays plain synchronous-blocking regardless -- `info = getUserInfo()` just
works; this only governs what the runtime does while a call is outstanding.

---

## 7. Networking and the dispatcher

One dedicated I/O thread runs `epoll` (Linux; `kqueue`/`poll` for portability) and owns no
interpreter. Async socket primitives (`sockConnect`, `sockListen`, `sockSend`, `sockRecv`,
`sockClose`, plus a timer) are registered once through the broker, so every language gets
them. The recommended v1 model is **synchronous-blocking at the script level**: `data =
sockRecv(conn)` parks the calling context on a reply future; when epoll reports readiness,
the I/O thread builds a `CALL`/reply and enqueues it onto the *owning* context's queue, so
the result lands on the right interpreter thread. **Callbacks are opt-in on top**: pass a
`valueFnE` (e.g. `onConnect(myFunc)`) and the completion invokes it via the same dispatch,
always back on its home thread. No separate async keyword, no per-engine coroutine support
needed.

Fixes from verification:

- The I/O command queue must be **strict tail-append FIFO** (a `sockSend` issued right
  after `sockConnect` must be processed after the connect that registered the handle);
  assert it.
- The resolver/connect path must **deep-copy `host`** before the command is freed (the
  first cut had an unconditional `free(cmd->host)` that would UAF if stored by pointer).
- Wake the epoll thread for new interest via `eventfd`/self-pipe; deregister on close;
  drain `eventfd` to `EAGAIN`.
- Portability note: `pthread_condattr_setclock(CLOCK_MONOTONIC)` is absent on Darwin --
  guard it with `#ifdef` and derive any monotonic timed wait accordingly (a monotonic
  deadline cannot be handed to a realtime-clock condvar).

---

## 8. Error model (one source of truth)

A `NativeFnT` returns a status int and, on failure, writes a human-readable message **into
`result`** (a `valueStringE` tagged as an error, or a small error-struct convention). That
single in-band channel crosses the actor boundary unchanged, is freed exactly once by the
caller, and is surfaced into the calling engine as that engine's native error
(`luaL_error`/`lua_error` for Lua, `mb_raise_error` for my-basic). There is no second error
field anywhere.

---

## 9. Context lifetime and the registry (UAF fix)

The critical use-after-free: contexts were addressed by raw `ScriptContextT*` (and exports
held raw owner pointers), while `contextShutdown` frees the context and destroys its
mutex/cond at runtime -- so a foreign thread could enqueue onto a freed `queueMutex`.

Fix:

- Address contexts by a **stable integer id** through a **locked registry**; never by raw
  pointer. `contextEnqueue`/`contextCall`/`ioDispatch` resolve id -> context under the
  registry lock and either hold the lock across enqueue or take a reference so the context
  (and its mutex) cannot be freed mid-enqueue.
- Add a **generation counter** to context ids and to `CallableT.ownerGen` so a recycled id
  cannot misroute an in-flight completion to a different context.
- `contextShutdown`: under the lock, mark dead and remove from the id map; reject new
  enqueues with a defined "dead context" error; drain and error-reply queued work; wait for
  in-flight references to drain; then free.

---

## 10. Function-value lifecycle across threads

`CallableT.refCount` is atomic. `valueCopy` bumps it; `valueFree` drops it. The subtlety:
releasing the underlying closure is an interpreter op (`luaL_unref` / my-basic unref) that
must run on the owner's thread. So when a drop reaches zero on a *foreign* thread, the
broker posts a **release message** to the owner context instead of touching the interpreter
directly; the owner releases the closure on its own thread and frees the `CallableT`. If
the owner is already gone (`alive == false`), the `CallableT` shell is freed directly (the
closure is already gone with the interpreter) and any pending invoke returns a clean error.

---

## 11. Build order

1. **Broker core**: `broker.h` (the canonical `ValueT`/`AggregateT`/`CallableT`/enums),
   `valueCopy`/`valueFree`/`valueMove` with full tag coverage and the depth cap, the
   name->entry registry, `brokerCall`, the error convention. Unit-test value round-trips
   and deep-copy/free under a leak checker before any engine exists.
2. **Lua adapter** against the core: trampoline, scalar+string marshalling, table
   deep-copy both directions with caps, native registration, export with the refcounted
   `luaL_ref` lifecycle and exports-array growth. Test C<->Lua and Lua-export-called-from-C
   single-threaded.
3. **my-basic adapter**: lifecycle, the trampoline bank, the arg-frame protocol, `mb_memdup`
   string ownership, the collection mapping, and the parked `__BROKERSERVE` export frame.
   Test C<->BASIC and the full Lua<->broker<->BASIC round-trip against the fidelity table
   (assert the lossy edges error or coerce exactly as documented).
4. **Actor layer**: `ScriptContextT`, the MPSC queue, the reply future, the id+generation
   registry, the single error channel, the chosen block-while-waiting semantics (sec 6.1),
   and shutdown drain. Stress cross-context calls and teardown under a thread sanitizer.
5. **Networking/dispatcher**: epoll I/O thread, the FIFO command queue, the socket/timer
   natives, completion dispatch onto owning queues, callbacks via `valueFnE`.
6. **Squirrel** (later): a third adapter validates that the vtable + canonical `ValueT`
   really make new engines O(1).

---

## 12. Open decisions

- **Block-while-waiting semantics: DECIDED -- always-live nested pump (sec 6.1).**
- Strict-vs-lenient default for the lossy marshal edges (recommend: strict by default so
  truncation/loss is an explicit error; lenient opt-in per call).
- my-basic native-bank size (cap on natives per BASIC context).
- Whether a foreign function injected into BASIC should be transparently callable as a
  routine value (`cb(x)`) or only via the portable `CALL(fn, ...)` primitive (Lua gets the
  transparent form for free; BASIC's transparent form needs confirming).

---

## 13. Implementation notes (as-built: broker core + both adapters)

Built and tested: `broker.h`/`value.c`/`broker.c` (core), `luaAdapter.*` (Lua 5.4),
`mybasicAdapter.*` (vendored my-basic in `vendor/`), with `testBroker`/`testLua`/
`testMyBasic`/`testPolyglot` -- 378 checks, clean under ASan+UBSan. The polyglot test
proves the thesis: one C native called from both engines, and a Lua function invoked from
a BASIC program through the broker.

**Core refinement.** The single global callable-release hook could not distinguish a Lua
closure from a my-basic routine, so release is now a per-callable `CallableReleaseFnT`
passed to `callableCreate` (design sec 10's "owner releases the closure", just synchronous
for now). Added `callableUserData` so a release fn can reach its closure handle.

**Lua adapter.** Context pointer lives in `lua_getextraspace`. Native bindings are
context-owned `{fn,userData}` structs referenced by a light-userdata upvalue on one shared
trampoline. A Lua function crossing out becomes a `CallableT` over a pinned `luaL_ref`
(released via `luaL_unref` in the per-callable release fn); transient callback args are
freed automatically because `valueFree` drops the handle. A `CallableT` crossing in becomes
a callable userdata with `__call`/`__gc`. Lua allocation APIs longjmp on OOM (no NULL
checks). Caveat: release exported callables before `luaContextDestroy` (the `luaL_ref`
lives in that state's registry).

**my-basic adapter** (the high-effort one; these rules were forced by ASan):
- Build with `-DMB_DOUBLE_FLOAT` (double reals) and link `-lm`.
- Native signature has no per-call userData and one interpreter userdata slot, so a
  macro-generated **trampoline bank** (`MB_BANK_SIZE`) supplies slot-specific entries that
  recover the binding from the context.
- `mb_register_func` returns a count: **nonzero = success, 0 = failure** (inverted vs the
  usual `MB_FUNC_OK == 0`).
- **Ownership is asymmetric and was the main source of bugs** (verified against the my-basic
  source during adversarial review):
  - A popped **collection** is owned by the consumer (`mb_dispose_value` after marshalling);
    a popped **string** is a borrowed interior pointer (copy, never free).
  - `mb_set_coll` **copies** a scalar/string key-value (dispose your copy after) but stores a
    **collection by pointer without a reference** -- so a nested collection needs an explicit
    `mb_ref_value` *before* the set, and must then NOT be disposed (the parent owns it).
  - `mb_push_value` **transfers** a collection, but **borrows** a string -- a string result
    must be pushed with `mb_push_string` (which marks it for lazy destroy), not
    `mb_push_value`.
  - `mb_eval_routine` **borrows** its arguments (it never frees them), so marshalled routine
    args are disposed by the caller after the call -- and the **return value is marshalled
    out first**, because a routine may return one of those borrowed arguments.
  - Routine values are **not** ref-counted (`mb_ref_value`/`mb_unref_value` corrupt them); a
    routine name must be **uppercased** before `mb_get_routine` (BASIC uppercases at parse).
  - int64 entering BASIC is range-checked to 32-bit `int_t` (`brokerErrRangeE` on overflow).
- **Routine export** uses `mb_get_routine`(by name) + `mb_eval_routine`, both of which need
  a live `void** l`. That cursor only exists inside a native call, so the dispatch stashes
  it in `currentL`; an exported BASIC-routine `CallableT` is therefore valid only while the
  context is *serving* (a native frame is on the stack -- what the actor layer's parked
  `__BROKERSERVE` frame will guarantee). For now, fetch and invoke within one native call.
- **One interpreter per program:** a my-basic context hosts a single program; reset+reload
  after disposing native-pushed collection intermediates is unreliable, so the tests spin a
  fresh context per run. (The actor layer will own one long-lived parked context per
  module, which sidesteps this.)

**Build/verify.** Core compiled strict (`-Wconversion -Wsign-conversion`); adapters drop
those two (engine headers use wide macros) but keep `-Wall -Wextra -Werror`. **All three
engines are vendored under `vendor/` and built from source** -- `vendor/lua` (Lua 5.4.6,
library = `src/*.c` minus the `lua.c`/`luac.c` mains), `vendor/ourbasic` (our-basic, a fork of my-basic),
`vendor/squirrel-src` (Squirrel 3.2) -- each relaxed and un-sanitized but linked into the
sanitized binaries so cross-boundary heap misuse is still caught. Nothing depends on a
system-installed engine or `pkg-config`, so the build is reproducible. The Lua platform
define is selected automatically: `$(OS)` first (Windows sets `Windows_NT` and has no
`uname` -> no define / ISO C), else from `uname -s` (LUA_USE_LINUX + `-ldl` /
LUA_USE_MACOSX / LUA_USE_POSIX). NB the project is otherwise Unix-only (pthreads,
sanitizers, `setarch`), so the Windows branch only keeps the define correct.

**Function-value lifecycle across threads (sec 10), DONE.** `callableInvoke` and the
final `callableRelease` are now thread-correct. The core exposes two installable hooks
(`callableSetInvokeHook`/`callableSetReleaseHook`, the same pattern as `brokerSetRouteHook`)
so it stays independent of the actor layer; `actorInit` installs them. An invoke from a
thread other than the callable's owner is marshalled to the owner's thread by reusing the
CALL machinery (a callable's `fn`+`userData` are exactly a native call -- `callableFn` is
the one new accessor). The final reference drop is routed too: a new `messageReleaseE`
posts the finalize (fire-and-forget) to the owner, which runs the engine release
(`luaL_unref` / `sq_release`) on its own thread. `callableFinalize` is the shared "run
release + free shell" tail; the core still runs it inline when no actor is present (so the
single-threaded `testCallableDead` semantics -- a dead callable still runs its release on
last drop -- are preserved). `testEngineLua` captures a Lua closure on its context's
thread, then invokes and releases it from the main thread; both marshal to the owner,
ASan/TSan-clean. Limit: releasing a callable whose owner *context* has been destroyed is
the deferred non-quiescent-teardown case (sec 9) -- best-effort inline finalize for now.

**JavaScript adapter (Duktape), the fourth engine.** Vendored Duktape 2.7.0 (the
single amalgamated `duktape.c`/`duktape.h`/`duk_config.h`) in `vendor/duktape`, built
relaxed/un-sanitized. `src/js/jsAdapter.*` mirrors the Lua/Squirrel adapters: one shared
trampoline recovers its binding from a hidden property on the function object
(`duk_push_current_function` + an internal `\xFF`-prefixed key) and dispatches through
`brokerCall`; marshalling covers scalars, binary-safe strings, and the hybrid aggregate
(JS array <-> list, object <-> map) with the depth cap. JS numbers are doubles, so an
integral in-range number round-trips as an int (else a real). A JS function crossing out
becomes a refcounted `CallableT` over a Duktape **heap pointer** kept alive by a per-heap
export registry object in the global stash (the slot is dropped on release) -- and it
participates in the sec-10 cross-thread routing, so a JS closure captured on its context's
thread is invoked and released from another thread correctly. `src/js/jsEngine.*` is the
EngineT binding. `testJs` (single-threaded: scalar/string/array/object marshalling, export
+ invoke-from-C, closure-as-arg callback, error paths) and `testEngineJs` (threaded:
cross-context call + the sec-10 callback) -- clean under ASan/UBSan and TSan (`make tsanjs`).
Adding the engine touched zero lines of the broker core or actor layer (one adapter TU +
one engine-binding TU + Makefile rules), re-confirming the O(1)-engine-add thesis. v1 limit
(as with Squirrel): pushing a foreign `CallableT` *into* JS is unsupported.

**Source layout.** `src/` holds the project source (core + actor directly in `src/`,
one subdir per script language: `src/lua`, `src/mybasic`, `src/squirrel`, `src/js`);
`tests/` holds the test programs; `obj/` collects every object file (ours and the vendored
engines, via `patsubst` into `obj/`); `bin/` collects the binaries. The Makefile finds our
sources by `VPATH` and groups object rules by flag set; `-MMD -MP` generate header
dependencies automatically. `make clean` removes `obj/` and `bin/`. `make test` builds and
runs all ten binaries; `make tsan`/`make tsansq`/`make tsanjs` are the ThreadSanitizer
variants.

## 16. Threading model rewrite -- host-thread natives, fire-and-forget scripts

Supersedes the earlier "natives run inline on the calling context thread" model. The
host's own thread is now an implicit **host context (id 0)**: it has a queue but no OS
thread of its own, and the host drives it by calling **`calogPump`** in its loop.

- **Scripts are fire-and-forget.** `calogContextEval(ctx, src)` enqueues the script
  onto the context's thread and returns a status (not the result); the script runs
  asynchronously, and results come back by calling natives.
- **A registered native runs on the host thread, serialized.** A script calling one
  posts a CALL onto the host queue and parks; the host runs it during `calogPump`. So
  host C code is never called concurrently and needs no locking. `actorRoute` inlines a
  call already on the host thread; otherwise it marshals to the host context (id 0).
- **`calogRegisterInline`** is the escape hatch: the registry entry's `runInline` flag
  (which replaced `ownerCtxId`) makes the native run on the calling script's thread.
- **Errors** from a fire-and-forget script are posted to the host queue and delivered
  to the `CalogErrorFnT` handler (`calogSetErrorHandler`) during `calogPump` (default:
  log to stderr).
- **Function values** (`CalogFnT`) still run on their owning engine's thread -- sec 10
  routing is unchanged, and `calogFnInvoke` from the host blocks-and-pumps the host
  queue while it waits (the same nested pump, now applied to the host context).
- **Nested eval is allowed:** a new eval that arrives while a context is mid-script
  (parked on a native call) runs nested via `pumpUntil` -- consistent with the sec-6.1
  re-entrancy contract (interpreters support nested `pcall`/`peval`).

API shape: `calogRegister(c,name,fn,ud)` / `calogRegisterInline(...)`;
`calogContextOpen(c,engine) -> CalogContextT*` (create+start merged, since nothing is
registered between them anymore) and `calogContextClose`; `calogContextEval(ctx,src)`
fire-and-forget; `calogPump`; `calogSetErrorHandler`. `CalogConfigT` and the
`createInterpreter` config parameter are gone -- a context now exposes *every*
registered native (the engine binding walks the registry via the internal
`calogForEach`). Tests rewritten to drive calog the host way (register, open, eval,
pump-until-a-native-records-the-result); `testActor` is now engine-free, exercising the
dispatch machinery with C callables (`calogFnCreate`) on synthetic contexts. Verified:
`make test` 441 checks across 11 binaries (incl. `examples/embed.c`), gcc + clang
strict, ASan/UBSan + TSan clean (`make tsan`/`tsansq`/`tsanjs`).

**`CalogT` owns its contexts; ids are unbounded.** The active-context registry moved
from `context.c` file-static globals *into* `struct CalogT` (now defined in
`calogInternal.h`): a runtime owns both its native-function registry and its
active-context registry (`ctxMutex`, `ctxSlots`, freelist). `context.c` reaches it via
one `runtime` pointer set in `calogActorInit` (which also refuses a second runtime).
So `calogDestroy` closes every still-open context automatically -- the host need not
track them (a test opens 32 and never closes them; ASan confirms no leak). Context ids
widened to **`uint64`** (32-bit slot index + 32-bit generation), so neither the live
count nor open/close churn hits a preset ceiling; `calogContextId`/`calogCurrentId`
return `uint64_t`. The now-dead `ownerGen` parameter was dropped from `calogFnCreate`
(generation lives in the packed id). Re-verified: `make test` 473 checks, ASan no
leaks, TSan clean, gcc + clang strict.

**Independent runtimes in one process.** The one-runtime limit was not fundamental --
just process-global state that hadn't moved into `CalogT`. All of it now has: the host
context, the routing hooks (`routeHook`/`invokeHook`/`releaseHook`), and the error sink
are `CalogT` fields; a `CalogFnT` carries a `runtime` pointer so `calogFnInvoke`/release
reach the right hooks (the callable path has no `CalogT` otherwise). The dispatch
reaches its runtime through the object it already holds -- the route hook is handed its
`calog`, the callable hooks read `calogFnRuntime`, context-thread code uses
`context->broker`, and the rest take an explicit `calog` argument. The **only** remaining
process global is `currentContext`, and it is thread-local (it names the calling
thread's context). So the setters (`calogSetRouteHook` etc.) are gone -- `calogActorInit`
assigns the fields directly -- and the `runtime` static that the earlier review flagged
is deleted. Runtimes are isolated: don't pass a value or callable between them (a
cross-runtime reply cannot route). A test spawns N threads, each creating, driving, and
destroying its own runtime concurrently.

**One thread may host several runtimes.** `calogPump(calog)` sets `currentContext` to
`calog`'s host context for the drain and restores it after, so a single thread can drive
many runtimes by pumping each in turn -- a native serviced during `calogPump(A)` sees
`calogCurrent() == A` even if the thread also hosts B. Two consequences fall out and are
handled: context ids number from 1 in every runtime, so the "already on the owner's
thread" (inline) and "caller can take the token/pump path" (reply) decisions match the
*runtime* too, not the id alone -- a foreign or wrong-runtime caller takes a reply box,
which cannot misroute. A test creates two runtimes on one thread, runs a script in each,
and pumps both in a loop, asserting each runtime's native resolved `calogCurrent()` to
its own runtime (it fails if the pump doesn't rebind `currentContext`). Re-verified:
`make test` 480 checks, ASan no leaks, TSan clean (both concurrent runtimes and one
thread pumping two), gcc + clang strict.

**Loading a script by filename.** Each engine carries a NULL-terminated `extensions`
list (`{"lua"}` / `{"js"}` / `{"nut"}` / `{"bas"}`), and a host makes engines available
for filename-based loading with `calogRegisterEngine(calog, &engine)`.
`calogContextLoad(calog, base)` then walks the registered engines in registration order
(each engine's extensions in order), forms `"<base>.<ext>"`, and the first one that
`fopen`s wins: it reads the file on the calling thread, opens a context on that engine,
and loads the contents fire-and-forget -- returning the context (NULL if nothing matched
or the load failed). Registration matters for more than search order: hardcoding the
built-in engine vtables in the core would force-link *all* of them (and their vendored
runtimes) into every binary, defeating the per-engine archives -- so the host opts in,
and a binary that never references an engine pulls in none (`testActor` stays
engine-free). Engine selection is fundamentally a build-time (link) choice, so
`calogRegisterBuiltinEngines` (a header-inline in `calog.h`) registers exactly the
engines whose `CALOG_WITH_<ENGINE>` macro is set -- the host defines those alongside the
archives it links, and the inline emits nothing (references no engine) unless called, so
it never force-links.

**my-basic as an actor engine.** Making my-basic loadable meant running it under the
actor model for the first time, which exposed two things. (1) Its native dispatch called
the C function *directly* instead of through `calogCall`, so natives ran on the my-basic
context thread rather than marshalling to the host -- fixed by routing `mbDispatch`
through `calogCall` (the binding now stores the registry name), matching the other
engines. (2) my-basic keeps process-global state -- lazy `mb_init` singletons and a
global `_mb_allocated` counter touched on every allocation (forced on in the vendored
header) -- so two my-basic contexts on different threads race (TSan-confirmed). The
singletons are built once by `mb_init` and read-only thereafter, so the only
execution-time shared write is that counter; a one-line vendored patch makes it
`_Atomic` (the original is preserved as `vendor/ourbasic/ourBasic.c.upstream`). With the
counter safe, the my-basic *engine* (not the adapter, which stays usable single-threaded
and lock-free) needs a lock only across *lifecycle* -- `mb_init`'s first-context build,
`mb_dispose`'s last-context teardown, and the shared context refcount -- and NOT across
`runSource`, so several my-basic scripts execute concurrently. A `tsanmb` target proves
the parallel case is race-free (verified further by a 4-context stress running
arithmetic, strings, lists, and booleans). Verified: `make test` 494 checks (13
binaries), ASan no leaks, TSan clean on all four engines
(`tsan`/`tsansq`/`tsanjs`/`tsanmb`), gcc + clang strict.

## 15. Public embedding API (`calog.h`) -- as-built (superseded by sec 16 for threading/API)

calog is packaged as an embedding library: a host links it, registers its own native
C functions, creates script contexts on an engine, and runs scripts. Every public
symbol carries a `calog` prefix (types `Calog...T`, enums `Calog...E`) so the library
is a good citizen in a host binary. The API was curated to the minimum:

- **One handle, one header.** `CalogT` is the runtime; `calogCreate()` composes the
  registry with the actor layer (installs the routing hooks) and `calogDestroy()`
  tears both down -- no separate init/shutdown for the host. The entire embedding
  surface is `src/calog.h` (~30 functions); internal machinery (the registry entry
  type, the route/invoke/release hooks, the low-level callable lifecycle, the split
  `calogBrokerCreate`/`calogActorInit`) lives in `src/calogInternal.h`, which host code
  never includes. `calog.h` leaks no internal symbol.
- **One config type.** The three per-engine configs collapsed into `CalogConfigT`
  (`exposeNames` + `exposeCount`), used by every built-in engine vtable.
- **Value model unchanged, just prefixed.** `CalogValueT`/`CalogAggT`/`CalogFnT` +
  constructors (`calogValueInt`, ...), ops (`calogValueCopy`/`Free`/`Move`/`Equals`),
  aggregates (`calogAgg*`), function values (`calogFnInvoke`/`Retain`/`Release`), and
  `calogFail`/`calogTypeName` for writing natives. Contexts: `calogContextCreate`/
  `Start`/`Eval`/`Destroy`/`Id`, plus `calogCurrentId`/`calogCurrent` for natives. The
  built-in engine vtables are `calogLuaEngine`, `calogJsEngine`, `calogSquirrelEngine`
  (a host may also supply a custom `CalogEngineT`).
- **Packaging.** `make` builds `lib/libcalog.a` (calog itself: core + actor + every
  adapter/binding) and separate vendored-engine archives (`liblua.a`, `libduktape.a`,
  `libsquirrel.a`, `libmybasic.a`). A host links `libcalog.a` plus whichever engine
  archives it uses; unused adapters (and their engine deps) stay unlinked because static
  members are pulled only when referenced -- so a JS-only host never links Lua/Squirrel.
  The tests consume the archives; the threaded/engine tests use only `calog.h`,
  validating that the public surface is complete. `examples/embed.c` is a ~30-line host
  (public header only) that registers a native and calls it from JavaScript.
- **Reconfirmed:** rename + restructure kept all 441 checks passing across 10 test
  binaries, clean under ASan/UBSan and TSan (all four engines), gcc + clang strict.

## 14. Implementation notes (as-built: actor layer, engine-on-a-thread, Squirrel)

The actor layer (`context.h`/`context.c`, build step 4) is built and tested:
`testActor` exercises cross-context routing, the always-live nested pump (the
re-entrant A->B->A deadlock test), and a concurrent fan-out stress; clean under
ASan+UBSan and ThreadSanitizer (`make tsan`, run under `setarch -R` -- some kernels
hand out more mmap randomization than TSan's shadow tolerates). One thread + one
MPSC queue per `ScriptContextT`; `brokerCall` routes through an installed hook
(`brokerSetRouteHook`) so owner-0/same-context calls run inline and others marshal
to the owning thread; an external caller blocks on a private reply box, a context
caller pumps. The reply carries only `{status, result}` -- the single error channel
(sec 8) is structural, the error string rides in `result`. `contextSendBlocking`
and `contextReply` are the shared enqueue-wait and reply tails behind both CALL and
EVAL dispatch.

**Generationed registry (sec 9), DONE.** Context ids pack a 16-bit slot index and
a 16-bit generation; the registry is a slot table plus a freelist. `contextDestroy`
unlinks a context under the registry lock (after stopping+joining its thread) and
returns the slot to the freelist; the next reuse bumps the generation. A stale id
(slot since freed/recycled) resolves to `brokerErrDeadE`, never misroutes to the
recycler -- `testActor`'s generation test proves it. The registry lock is held
across enqueue, so a foreign enqueue cannot race a destroy onto a freed queue mutex.
Still quiescence-assuming (no call to the context in flight at teardown); in-flight
reference draining is the remaining sec 9 hardening.

**Engine on a thread (the EngineT vtable).** `EngineT` gained `runSource`;
`contextEval(context, source, result)` marshals a script run onto the context's own
thread (a new `messageEvalE`) and blocks like a call. Each adapter's engine binding
lives in its own TU (`luaEngine.*`, `squirrelEngine.*`) -- the only Lua/Squirrel
files that depend on the threading layer, keeping the adapters thread-agnostic so
`testLua`/`testPolyglot` link them without `context.o`/pthread. `createInterpreter`
runs on the thread and exposes the configured natives there. Crucially, the exposed-
native trampolines now dispatch through `brokerCall` (by broker+name) instead of a
captured fn pointer, so an exposed native owned by another context is transparently
routed to its thread -- the script author still writes `doubleIt(21)`. With no route
hook installed this is identical to the old inline path (testLua still passes).
`testEngineLua` proves a real Lua interpreter on a context thread calling a thread-
agnostic native and a cross-context native, on the correct threads.

**Squirrel adapter (sec 11 step 6), the O(1)-engine-add validation.** Vendored
Squirrel 3.2 in `vendor/squirrel-src` (C++), built relaxed/un-sanitized with
`-D_SQ64 -DSQUSEDOUBLE` so `SQInteger`/`SQFloat` are 64-bit int / double matching
`ValueT` -- the adapter shares those defines so the ABI matches. `squirrelAdapter.*`
mirrors the Lua adapter: one shared trampoline recovers its binding from the
closure's single free variable (which the VM pushes onto the stack *after* the args,
so it sits at the top -- verified in `sqvm.cpp` CallNative), marshals scalars,
binary-safe strings, and the hybrid aggregate (array<->list, table<->map) with the
shared depth cap, and dispatches through `brokerCall`. `testEngineSquirrel` runs a
real VM on a thread doing the cross-context call plus string and array round-trips;
clean under ASan+UBSan and TSan (`make tsansq`). The total surface a new engine
added: one adapter TU + one engine-binding TU + Makefile rules -- no change to the
broker core or the actor layer, which is the thesis.

**Squirrel closure export, DONE.** A Squirrel closure crossing the boundary now
becomes a refcounted `CallableT` over a pinned `HSQOBJECT` (`sq_addref`/`sq_release`,
mirroring Lua's `luaL_ref` lifecycle): `squirrelExport` fetches a named global
closure, and a closure passed as a native argument is exported the same way during
ingress (the VM's foreign pointer -- finally used -- recovers the owning context).
`squirrelCallableInvoke` runs `sq_pushobject`+`sq_call` on the owner's VM and
marshals the return; `squirrelCallableRelease` `sq_release`s on the owner thread.
Single-threaded `testSquirrel` covers export+invoke-from-C, a closure passed as an
argument and called back through the broker, and the not-found/type-error paths;
ASan-clean (no addref/release leak). Caveat (same as Lua): release exported
callables before `squirrelContextDestroy`. (The reverse direction -- a foreign `CalogFnT`
pushed INTO Squirrel -- is now supported too: a native closure whose one free variable is
a release-hooked userdata holding the `CalogFnT`; see sec 18.)

`make test` runs all seven binaries (411 checks). `make tsan` covers the actor core
and the Lua engine path; `make tsansq` the Squirrel path.

**Adversarial review (3 parallel reviewers: actor concurrency, Squirrel adapter,
Lua trampoline + engine bindings).** Two real defects found and fixed:
- *NULL-interpreter crash.* `threadMain` ignores `createInterpreter`'s status, so a
  failed create (e.g. a config expose-name that was never registered) left a context
  serving with `interp == NULL`; `contextDispatchEval` only checked `runSource !=
  NULL`, so the first eval called `runSource(NULL,...)` -> NULL deref. Fixed by
  guarding `interp == NULL` (the context still serves native calls, just rejects
  evals); regression test in `testEngineLua` (testFailedInterpreter).
- *OOM lost-wakeup.* `contextReply`'s context-caller branch allocated a fresh REPLY
  and, on `calloc` failure, dropped the wakeup -- the caller hung in `pumpUntil`
  forever. Fixed by reusing the request message as the reply (it already carries the
  token and replyToId), which removes the allocation entirely, so the wakeup can no
  longer be lost to OOM.
The Squirrel adapter was traced clean against the real Squirrel source (trampoline
free-var indexing, stack balance, ValueT ownership, the throwerror/free order, binary
strings); added `sq_reservestack` guards before the recursive marshallers to match the
Lua adapter's `lua_checkstack` discipline. Documented (not changed): the registry must
be frozen before contexts start (`brokerCall` reads it locklessly from context
threads -- noted in broker.h); teardown still assumes quiescence (sec 9); and the
hybrid-aggregate-to-Squirrel-table egress flattens array indices and integer keys into
one table (same lossy edge as elsewhere in the fidelity table).

## 17. Engine expansion -- QuickJS-ng, and three new languages (Berry, s7, Wren)

calog now ships **seven** engines. Each is one adapter TU (marshalling + native
trampoline + the sec-10 callable export) plus one engine-binding TU (the four-hook
`CalogEngineT`), a vendored-from-source archive, a `testEngine*`, and a `tsan*` target --
the core, the actor layer, and `calog.h` were untouched, re-confirming the O(1)-per-engine
thesis. Which engines a binary pulls in is a link-time choice: the header-inline
`calogRegisterBuiltinEngines` references only the `CALOG_WITH_<ENGINE>`-selected vtables,
so `testActor` still links **zero** engine code.

**QuickJS-ng replaces Duktape** (same `calogJsEngine` / `.js`, same `jsAdapter.h` /
`jsEngine.c` -- only `jsAdapter.c` and the Makefile changed). The wins: a JS **BigInt
round-trips to int64 exactly** (`JS_ToBigInt64`), closing the double-only fidelity gap
Duktape had (proven by a 2^53+1 test); JS functions are **refcounted `JSValue`s**
(`JS_DupValue` / `JS_FreeValue`), replacing the Duktape heap-pointer-pinning registry
behind `CalogFnT`; and the broker/name binding rides on `JS_SetContextOpaque` +
`JS_NewCFunctionData`. One gotcha: a missing global reads back as `undefined` (not an
error), so `calogJsExport` maps `JS_IsUndefined` to not-found while a bound non-function
stays a type error. Core library = `quickjs.c` + `libregexp.c` + `libunicode.c` +
`dtoa.c`, built with `-D_GNU_SOURCE`.

**Berry** (`.be`) is a Lua-like stack VM (64-bit ints, binary-safe `be_pushnstring`).
Natives are Berry **native closures carrying two upvalues** (the context comptr and the
name), recovered with `be_getupval(vm, 0, pos)`; a Berry function crossing out is pinned
under a uniquely-named hidden global (globals are GC roots) and dropped by setting it to
nil. The sharp edge: `be_pcall(vm, argc)` leaves the **result in the function's slot
(`base+1`)**, not at `-1` (which holds the last stale argument). Vendoring needs Berry's
`coc` codegen prebuild plus its OS port (`be_port.c`) and module/class tables
(`be_modtab.c`). A subtlety for records: `be_newmap`/`be_newlist` push *raw* containers
that a script cannot subscript, so an aggregate crossing out is wrapped in its `map`/`list`
class instance (`map(raw)` via `be_getbuiltin` + `be_call`, then `be_moveto`/`be_pop` to
drop the raw and `init`'s nil return) -- then `user['name']` works. Ingress reverses it:
a list/map instance holds its raw container in the hidden `.p` member, iterated with
`be_pushiter`/`be_iter_next` (see sec 18).

**s7 Scheme** (`.scm`) uses the *current* official s7 (an older mirror lacked `s7_free`,
which would leak a heap per context). Since s7 native functions carry no user data, all
natives route through **one generic `%calog-call` dispatcher** plus a per-name Scheme
wrapper (`(define (report . a) (apply %calog-call "report" a))`); the context rides on a
`*calog-context*` c-pointer global. Callables are kept alive by `s7_gc_protect` and
invoked with `s7_call`. Because `s7_eval_c_string` evaluates a single form, `calogS7Run`
wraps the (escaped) source in `(catch #t (lambda () (eval-string ...)) handler)`, so both
read and run errors surface as a value -- a marker pair the runner detects. An aggregate
crossing out is a Scheme list, or an (applicable) hash-table when keyed, so a materialized
record reads as `(user "name")`; reading a script's keyed value back in is a v1 limit. s7's intentional
"permanent string" interning (which `s7_free` does not reclaim) is a small, bounded
allocation, suppressed with a documented, allocation-site-specific `__lsan` hook. s7 is
per-interpreter thread-safe -- no serialization needed (unlike my-basic).

**Wren** (`.wren`) is the outlier: Wren has **no bare function calls**, so every native is
reached through a single foreign method. A preamble defines `class Calog { foreign static
call(name, args) }`, and scripts call `Calog.call("report", [42])`; the C dispatcher
recovers the context from `wrenGetUserData`, marshals the argument list, and dispatches
through `calogCall`. A Wren function crossing out is a retained `WrenHandle`, invoked with
a cached per-arity `call(_)` handle. Wren numbers are IEEE **doubles**, so int64 above
2^53 loses precision (the same edge my-basic and old-JS have). An aggregate crossing out
is a Wren `List`, or a `Map` when keyed (a record reads as `user["name"]`); reading a
script's list/map back in is a v1 limit -- Wren's C API cannot enumerate a `Map`'s keys.
Wren keeps no process-global state, so contexts run in parallel.

Aggregate egress is therefore uniform across all seven engines: a host native can return a
keyed `CalogValueT` record and every engine reads its fields with native syntax
(`user.name` / `user['name']` / `user["name"]` / `(user "name")`). The reverse -- a script
handing a list/map *back* to C -- is complete on Lua/JS/Squirrel/my-basic and a v1 limit on
Berry/s7/Wren.

**Verified** across all seven engines: `make test` (531 checks, including a materialized-
record read per new engine), ASan/UBSan clean, a `tsan<engine>` target clean for each, and
gcc + clang strict on the core.

## 18. Closing the v1 marshalling limits (function-into-script, aggregate ingress)

The engines above shipped with two directional gaps in the value bridge: a foreign
function value could not be pushed *into* a script (only Lua did it), and a script could
not hand a keyed aggregate (map) *back* to C on the three newest engines. Both are now
closed everywhere they can be, with one honest exception each.

**New public API** `calogFnFromNative(out, calog, fn, userData)` -- wraps one of your
natives as a host-owned function value (owner id 0, runs on the host thread during
`calogPump`, like `calogRegister` but anonymous). Without it, function-into-script was
unusable from `calog.h` alone (`calogFnCreate` is internal), so a host could only forward
a *script*-derived callable, never one of its own natives. Return the result from a native
and the script gets a callable that routes back to the host.

**Function value -> script** (each `*FromValue` `calogFnE` case): an engine callable
object wraps the `CalogFnT*`, a trampoline marshals the script's args -> `calogFnInvoke`
-> marshals the result, and a finalizer runs `calogFnRelease`; `calogFnRetain` at push.
Per engine:
- **Lua** (pre-existing): userdata + `__call` + `__gc` metatable -- the reference for the rest.
- **JS**: a `JSClassDef` with both `.call` and `.finalizer`; the finalizer only receives
  the runtime, so `JS_SetRuntimeOpaque` carries the context to it.
- **Squirrel**: a native closure whose single free variable is a release-hooked userdata
  holding the `CalogFnT` (freeing the closure frees the userdata -> the hook releases).
- **s7**: an applicable c-object (`s7_make_c_type` + `s7_c_type_set_ref` for the call,
  `s7_c_type_set_free` for release); the `ref` fn gets `(obj . args)`, so the object is
  `s7_car`. Script calls `(f ...)`.
- **Berry**: no per-value finalizer exists, so the `CalogFnT`s pushed into a context are
  *tracked on the context* and released together in `calogBerryDestroy`; the callable is a
  native closure over `(context, CalogFnT)` comptr upvalues. (`berryFromValue` gained the
  context parameter so it could record them.)
- **Wren**: a `foreign class CalogFn { construct new() {} foreign call(args) }` whose
  `call` takes a *list* (Wren method arity is fixed, so a list absorbs any argument count);
  finalize releases. Script calls `f.call([...])`. Gotcha: Wren requires newlines between
  class members, so the preamble is multi-line.
- **MY-BASIC**: a host callable becomes a refcounted usertype-ref (`mb_make_ref_value`, whose
  dtor drops the retained `CalogFnT`); a script invokes it with `calogInvoke(fn, ...args)`,
  which marshals the args, calls the callable, and marshals the result back. (A script also
  passes its own `def`/lambda *out* to a native, invoked later via `mb_eval_routine_cold`; see
  sec 5.5.)

**Aggregate ingress** (`*ToValue` map/list): Lua/JS/Squirrel/MY-BASIC already read both.
Added:
- **s7**: read a hash-table by `s7_make_iterator` + `s7_iterate` (each yields a
  `(key . value)` cons; gotcha: at end `s7_iterate` returns a *non-pair* sentinel even
  when the at-end flag was still false, so guard `if (!s7_is_pair(pair)) break`).
- **Berry**: a list/map instance's raw container is its hidden `.p` member; iterate with
  `be_pushiter`/`be_iter_hasnext`/`be_iter_next` (which take the *container* index with the
  iterator kept on top, pushing one value for a list and key+value for a map -- restore the
  stack to `[container, iterator]` after each entry).
- **Wren**: a `List` reads back directly. A `Map` needs key enumeration, which upstream
  Wren's C API lacks (`wrenGetMapValue` is by-key only) -- so calog adds a small patch to
  the vendored `wren.c`/`wren.h` (`wrenGetMapCapacity` / `wrenGetMapEntry`, mirroring Wren's
  own internal `map_iterate`), and the adapter walks the raw table skipping empty slots.
  With the patch, Wren too reads maps back in. (Documented in LICENSE.md; re-apply if the
  amalgamation is regenerated.)

**Deliberately not "fixed" (inherent to the engine's value model):** MY-BASIC 32-bit ints
(over 2^31 range-checked to an error, not silently truncated), MY-BASIC NUL-in-string
truncation and serialize-at-load, and JS/Wren int64 above 2^53 (IEEE doubles -- JS *could*
emit a BigInt but that breaks arithmetic mixing with Number, a worse trap than the
documented precision edge). `WREN_MAX_CALL_ARITY` (16) is pinned to Wren's own engine
limit (`MAX_PARAMETERS`) and can't be raised. `MB_BANK_SIZE` (the MY-BASIC native cap) was
32 -- the one hard cap a real app could hit, since MY-BASIC natives can't carry userdata
so each needs a hand-materialized slot trampoline; it is now **256** (the trampoline bank
and its `[MB_BANK_SIZE]` table are regenerated together, so a count mismatch fails to
compile). Berry's `BE_BYTES_MAX_SIZE` was likewise raised from 32 kb to 256 MB.

**Verified**: `make test` (539 checks) with a `testForeignFunction` per engine and a
`testMapIngress` on s7 and Berry (the Berry one nests a list to exercise list ingress);
ASan-clean on every engine (retain/release balanced); gcc strict; per-engine `tsan*`.

## 19. mruby -- the eighth engine (Ruby)

**mruby** (`.rb`, `calogMrubyEngine`) is the first engine whose static library is not compiled from
vendored `.c` but **generated by the engine's own build**: mruby 4.0.0's Rake build (needs a host
Ruby + bison) emits `libmruby.a` from a trimmed embed gembox -- stdlib + math + metaprog (which
carries `mruby-compiler`, i.e. runtime `eval`) plus `mruby-bin-mrbc` as a *build-time* tool to
compile the Ruby-written stdlib into the archive; no bins, no file/socket IO. `MRB_INT64` keeps
script integers full 64-bit. The Makefile runs the Rake build as a prerequisite of the adapter
object, since mruby emits a per-build `mruby/presym.h` the adapter must include. Linked footprint is
~1.1 MB (QuickJS class -- not the CPython-scale bloat the feasibility study found real Ruby drags in).

Dispatch resolves two mruby facts. `mrb_func_t` carries **no user data**, so every native binds one
shared trampoline that recovers *which* native was called from the current method id (`mrb_get_mid`
-> `mrb_sym_name`); the context rides on **`mrb->ud`** (mrb_state's spare auxiliary pointer --
script-invisible, no global). Natives are public `Kernel` methods, so `cryptoUuid()` works bare.

Callables cross both ways with real Ruby semantics. A Ruby proc/lambda crossing *out* is GC-pinned
with `mrb_gc_register` behind `CalogFnT`, invoked with `mrb_funcall_argv(..., "call", ...)` (so a
Proc, a lambda, or a `Method` all work). A foreign `CalogFnT` crossing *in* becomes a **genuine Ruby
`Proc`** via `mrb_proc_new_cfunc_with_env` (the `CalogFnT` travels in the proc's cfunc env as a
cptr), so a script calls it `f.call(x)` / `f.(x)` and can pass it as a block -- no `.call`-method
wrapper like Wren, no `calogInvoke` helper like my-basic. As with Berry, that cfunc env has no
per-value finalizer, so foreign callables are tracked on the context and released at destroy;
marshalling is bracketed by `mrb_gc_arena_save`/`restore` so transient objects don't overflow the
arena.

Aggregates are `Array` <-> list and `Hash` <-> map (a keyed record's sequence part goes at integer
keys); strings are binary-safe (`mrb_str_new` carries a length). Bare-name **exports** resolve via
`Kernel#method_missing`: a top-level `exportedFn(args)` that is not a defined method is looked up in
the export registry and invoked -- but *only* when `self` is the main object (`mrb_obj_equal` against
`mrb_top_self`), so a missing method on any other receiver (`obj.foo`) stays a plain `NoMethodError`
and is never hijacked. This puts Ruby among the hook engines without a VM change -- its own
`method_missing` is the hook my-basic needed a fork to fake.

**IO by design.** Ruby's `File`/`IO`/`Socket` are left out (no `mruby-io`): scripts use calog's
`fs*`/`net*`/`http*` natives like every other engine, so all IO stays on one path. A partial `File`
shim would footgun (whole-file ops map to calog's `fs`, but the streaming object model does not), so
it is all-or-nothing and deferred; the adapter supplies only `puts`/`print`/`p` (to stdout).

mruby keeps no shared mutable process state -- one independent `mrb_state` per `mrb_open`, no GIL --
so contexts run in parallel, the concurrency gate mruby was chosen on (over MRI/CRuby and MicroPython,
which both fail it). Its Rake-built archive links un-sanitized, so `tsanmruby` instruments the
adapter/engine/actor code (like `tsanlibs` with Lua); each VM is single-threaded per instance.

**Verified**: `testEngineMruby` (13 checks -- natives, `Hash` egress/ingress, a foreign `Proc`, a
cross-thread lambda callback, the error handler, three concurrent `mrb_state`s), ASan-clean;
`tsanmruby` clean; cross-engine both ways (Ruby calling a Lua export, Lua invoking a Ruby lambda
export); `make test` 29/29.
