Scott Duensing 080bc95d8e Some security decisions revised. Bugs fixed.

2026-10-15 17:45:12 -05:00

20 KiB

Raw Blame History

calog

C: Any Language, One Gateway.

calog embeds scripting into a C or C++ program. You register your native C functions once, and they become callable from any embedded engine -- Lua, JavaScript, Scheme, Squirrel, Wren, Berry, Ruby, Tcl, Janet, or MY-BASIC -- with values passed through a single canonical type. Add a new scripting language to your app by linking one more archive; your native functions don't change.

#include "calog.h"

CalogT *calog = calogCreate();
calogRegister(calog, "hostLog", hostLog, NULL);       // your C function, once

CalogContextT *ctx = calogContextOpen(calog, &calogJsEngine);
calogContextEval(ctx, "hostLog('hello from JavaScript, ' + 6 * 7)");

Swap &calogJsEngine for &calogLuaEngine, &calogSquirrelEngine, &calogS7Engine, &calogWrenEngine, &calogBerryEngine, &calogMrubyEngine, &calogTclEngine, &calogJanetEngine, or &calogMyBasicEngine -- nothing else changes.

Why calog

One API, many languages. Register a native once; call it from every engine. calog.h is the entire public surface.
One value type. Everything crosses the boundary as a CalogValueT -- nil, bool, int, real, binary-safe string, a hybrid list/map aggregate, or a script function value. No per-engine glue in your code.
Natives run on your thread, serialized. A script calling a native is marshalled to the host thread and run there during calogPump(), one at a time -- so your C code is never called concurrently and needs no locking. (An opt-in escape hatch runs a native inline on the script's thread when you want it.)
Fire-and-forget scripts. Scripts run on their own context threads; the host stays responsive and drives everything from its own loop.
Callbacks both ways. A script can hand you a function value to keep and call later (routed back to the engine that owns it); and you can hand a native to a script as a callable value with calogFnFromNative.
Scripts export to each other, across languages. A script publishes a function by name with calogExport; any other script -- in any language -- calls it with calogCall (or by bare name on the hook engines). A Lua helper is callable from Ruby, a Tcl proc from JavaScript. Functions cross the language boundary, not just the host boundary, so you can write each part of a system in whatever language fits it best and still share code.
Hot reload -- patch a running system. A live context is a persistent interpreter: feed it new code at any time (taskEval, or calogContextEval from the host) to redefine a function, fix a bug, or add a feature while the process keeps running and keeps its state -- no restart. Re-export the fixed function and every caller, in every language, picks up the new version on the next call.
A polyglot web server. The httpd library serves HTTP where each route handler is a script function -- so /pay can be Ruby, /render Lua, /rules Scheme, all sharing helpers via calogExport, and you fix a bug by re-registering a handler while it keeps serving. It is the export + hot-reload + actor model in one library.
Sandbox untrusted scripts. Open a context with calogContextOpenLimited and a CalogLimitsT policy: a native allow-list (which registered natives the script may call -- enforced for every engine at the one dispatch choke point), a wall-clock budget (a runaway loop is retired -- Lua and JavaScript), and a memory cap (Lua and JavaScript, via their per-VM allocators). The sandboxed interpreter is sealed as well: its engine's own file, process, and environment access is gone, so the script reaches the host only through natives the policy can see. An unsandboxed context keeps its engine's standard library. Each context is already its own thread, so a limited plugin can misbehave without taking down the host.
Many runtimes. Independent CalogT runtimes coexist in one process; one host thread can drive several.
Load by filename. calogContextLoad(calog, "config") finds config.lua / config.js / config.nut / config.bas and runs it on the matching engine.
Reproducible & rigorous. Every engine is vendored and built from source (no system packages). The core is warning-clean on gcc and clang (-Wall -Wextra -Werror -Wconversion -Wsign-conversion) and runs clean under AddressSanitizer, UBSan, and ThreadSanitizer.
Portable, few dependencies. Linux, macOS, and Windows (mingw-w64). The OS surface is just threads + sockets + DNS, behind one abstraction (src/calogPlatform.h); everything else is portable C11. On Linux, make release yields a binary whose only shared-library dependency is the system libc, and make static on Alpine a fully static one. See PORTING.md.

Engines

Engine	Vtable	Extension	Notes
Lua 5.4	`calogLuaEngine`	`.lua`
JavaScript	`calogJsEngine`	`.js`	QuickJS-ng (ES2023+, BigInt -> int64)
Squirrel 3.2	`calogSquirrelEngine`	`.nut`	C++ VM
MY-BASIC	`calogMyBasicEngine`	`.bas`	interpreters serialize at load
Scheme	`calogS7Engine`	`.scm`	s7; 64-bit ints
Wren	`calogWrenEngine`	`.wren`	doubles only; call via `Calog.call(...)`
Berry	`calogBerryEngine`	`.be`	scalars + callbacks
Ruby	`calogMrubyEngine`	`.rb`	mruby; 64-bit ints; built by mruby's Rake
Tcl	`calogTclEngine`	`.tcl`	Tcl 9; 64-bit ints; string-typed values
Janet	`calogJanetEngine`	`.janet`	Lisp; thread-local VM; doubles (2^53)

A native can return a keyed CalogValueT record and every engine reads its fields with native syntax (user.name, user["name"], (user "name")), and a script can hand a list/map back to C on every engine. (Wren's C API can't enumerate a map's keys, so calog carries a small documented patch to the vendored Wren that adds one.) A host function value also crosses into a script on every engine, where the script gets a callable it can invoke.

You can also bring your own: CalogEngineT is a public four-function vtable.

Build

Requirements: a POSIX system (pthreads), gcc or clang, and GNU make. Developed on Linux; macOS should work, Windows via WSL.

make            # builds lib/libcalog.a, the per-engine archives, tests, and the example
make test       # runs the full suite under AddressSanitizer + UBSan

The engines are vendored under vendor/ and compiled from source, so the build is self-contained.

Linking your program

Link against lib/libcalog.a plus the archive for each engine you actually use -- unused engines (and their vendored runtimes) stay out of your binary.

# a JavaScript-only host:
cc -Isrc myhost.c lib/libcalog.a lib/libquickjs.a -pthread -lm -o myhost

Per-engine link needs:

Engine	Archive	Extra libs (Linux)
Lua	`lib/liblua.a`	`-ldl -lm`
JS	`lib/libquickjs.a`	`-lm`
Squirrel	`lib/libsquirrel.a`	`-lstdc++ -lm`
MY-BASIC	`lib/libmybasic.a`	`-lm`
Scheme	`lib/libs7.a`	`-ldl -lm`
Wren	`lib/libwren.a`	`-lm`
Berry	`lib/libberry.a`	`-lm`

Everything links with -pthread.

Running scripts with `calog`

make also builds bin/calog, a ready-to-run interpreter that links every engine and library. Point it at one or more script files:

bin/calog hello.lua                 # run on the engine its extension selects
bin/calog config                    # no extension -> search config.lua / .js / .nut / .bas / .be / .scm / .wren
bin/calog producer.js consumer.lua  # several files share one runtime (kv, pubsub, exports)

Scripts print with calogPrint(...) and stop by calling calogExit([code]), which tears everything down (calog is event-driven, so a script asks to stop). It does not return -- the statement after it never runs, so a build script's calogExit(1) is final. Ctrl-C stops a run the same way: it aborts the running scripts rather than waiting for them, so even a script stuck in a loop that calls nothing gives up promptly (on every engine but s7). A script can also end just itself with taskExit(), leaving its siblings running; once every script has ended, calog exits on its own. API.md documents every native a script can call; examples/scripts/ has runnable examples across every engine and library, plus polyglot and multi-file demos.

A complete example

#include "calog.h"

#include <stdio.h>
#include <time.h>

// A native the host provides. It runs on the host thread during calogPump, serialized
// with every other native, so it can touch host state without locking.
static int32_t hostLog(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
    (void)userData;
    calogValueNil(result);
    if (argCount != 1 || args[0].type != calogStringE) {
        return calogFail(result, calogErrArgE, "hostLog expects one string");
    }
    printf("[host] %s\n", args[0].as.s.bytes);
    return calogOkE;
}

int main(void) {
    CalogT *calog = calogCreate();
    calogRegister(calog, "hostLog", hostLog, NULL);

    CalogContextT *ctx = calogContextOpen(calog, &calogJsEngine);
    calogContextEval(ctx, "hostLog('hello from JavaScript, ' + (6 * 7))");

    // The script runs on its own thread and calls hostLog; the host services those
    // calls by pumping in its own loop. (A real app pumps inside its main loop.)
    struct timespec tick = { 0, 1000000 };  // 1 ms
    for (int i = 0; i < 50; i++) {
        calogPump(calog);
        nanosleep(&tick, NULL);
    }

    calogContextClose(ctx);
    calogDestroy(calog);
    return 0;
}

See examples/embed.c for this, ready to build (make produces bin/embed).

How it works

The canonical value

Every value crossing the C/script boundary is a CalogValueT:

typedef enum { calogNilE, calogBoolE, calogIntE, calogRealE,
               calogStringE, calogAggE, calogFnE } CalogTypeE;

struct CalogValueT {
    CalogTypeE type;
    union { bool b; int64_t i; double r;
            CalogStringT s; CalogAggT *agg; CalogFnT *fn; } as;
};

Build them with the constructors, and free owned values you're done with:

CalogValueT v;
calogValueInt(&v, 42);
calogValueString(&v, "binary\0safe", 11);   // length-prefixed, NUL-safe
// ... calogValueBool / calogValueReal / calogValueAgg / calogValueFn ...
calogValueFree(&v);

CalogAggT is a hybrid list + map (calogAggPush, calogAggSet, calogAggGet), so it round-trips onto engines that model sequences and dictionaries differently. Strings are binary-safe (length-prefixed, always NUL-terminated for convenience).

Registering natives

// Runs on the HOST thread (serialized). This is what you want almost always.
calogRegister(calog, "add", add, NULL);

// Escape hatch: runs INLINE on the calling script's thread. Faster (no host hop),
// but YOU guarantee it is thread-safe.
calogRegisterInline(calog, "clockNow", clockNow, NULL);

A native has one signature and reports failure through result:

static int32_t add(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
    (void)userData;
    if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogIntE) {
        return calogFail(result, calogErrArgE, "add expects two integers");
    }
    calogValueInt(result, args[0].as.i + args[1].as.i);
    return calogOkE;
}

Every registered native is automatically visible in every context you open.

Contexts, scripts, and the pump

A context is one engine instance on its own thread. calogContextEval is fire-and-forget: it queues the script and returns immediately.

CalogContextT *ctx = calogContextOpen(calog, &calogLuaEngine);
calogContextEval(ctx, "hostLog('running')");   // returns now; script runs on ctx's thread

Because natives are serviced on the host thread, the host must pump:

for (;;) {
    calogPump(calog);   // runs any pending native calls here, then returns
    do_other_host_work();
}

Script results come back by calling natives -- calogContextEval doesn't return a value. Script errors are delivered to an optional handler on the host thread during calogPump:

static void onError(uint64_t contextId, const char *message, void *userData) {
    fprintf(stderr, "script %llu failed: %s\n", (unsigned long long)contextId, message);
}
calogSetErrorHandler(calog, onError, NULL);   // default logs to stderr

Callbacks: script functions as values

A script can pass you a function. Keep it (retain), call it later, release it when done -- calog runs the call on the engine that owns it and returns the result to you:

static CalogFnT *savedCb = NULL;

static int32_t onEvent(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
    (void)userData;
    calogValueNil(result);
    if (argCount != 1 || args[0].type != calogFnE) {
        return calogFail(result, calogErrArgE, "onEvent expects a function");
    }
    calogFnRetain(args[0].as.fn);     // keep it beyond this call
    savedCb = args[0].as.fn;
    return calogOkE;
}

// ...later, from the host:
CalogValueT arg, out;
calogValueInt(&arg, 42);
calogFnInvoke(savedCb, &arg, 1, &out);   // routed to the script's thread, result returned here
calogValueFree(&out);
calogValueFree(&arg);
calogFnRelease(savedCb);

A reference you hold stays yours to release even after the script that made it is gone: as a context stops it releases the engine handle behind every callable it owns, so calogFnInvoke then reports calogErrDeadE instead of reaching into a torn-down interpreter.

It works the other way too: wrap one of your natives as a function value with calogFnFromNative and return it from a native, and the script gets a callable it can invoke (which routes back to your host thread). Every engine supports this; a script invokes the value directly, except in my-basic, where it is called with calogInvoke(fn, ...args).

static int32_t getAdder(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
    CalogFnT *fn;
    (void)args; (void)argCount; (void)userData;
    calogValueNil(result);
    calogFnFromNative(&fn, calog, add, NULL);   // 'add' is one of your natives
    calogValueFn(result, fn);                   // the script can now call getAdder()(2, 3)
    return calogOkE;
}

Loading a script by filename

calogRegisterEngine(calog, &calogLuaEngine);
calogRegisterEngine(calog, &calogJsEngine);

CalogContextT *ctx = calogContextLoad(calog, "config");  // tries config.lua, then config.js

Or let the build decide which engines exist. Compile the calling file with -DCALOG_WITH_LUA -DCALOG_WITH_JS ... and call the header-inline helper -- it registers exactly the engines you linked, and references no others:

calogRegisterBuiltinEngines(calog);
CalogContextT *ctx = calogContextLoad(calog, "config");

Multiple runtimes

A CalogT is a self-contained runtime; you can create several in one process. Each is driven by a host thread, and one thread may drive several by pumping each in turn. Keep values and callbacks within a single runtime -- they don't cross between runtimes.

API at a glance

// runtime
CalogT *calogCreate(void);
void    calogDestroy(CalogT *);
void    calogPump(CalogT *);                                  // service native calls here
void    calogSetErrorHandler(CalogT *, CalogErrorFnT, void *);

// natives
int32_t calogRegister(CalogT *, const char *name, CalogNativeFnT, void *userData);        // host thread
int32_t calogRegisterInline(CalogT *, const char *name, CalogNativeFnT, void *userData);  // caller thread
int32_t calogCall(CalogT *, const char *name, CalogValueT *args, int32_t n, CalogValueT *result);
int32_t calogFail(CalogValueT *result, int32_t status, const char *message);
const char *calogTypeName(CalogTypeE);

// values
void    calogValueNil/Bool/Int/Real/Agg/Fn(...);
int32_t calogValueString(CalogValueT *, const char *bytes, int64_t length);
int32_t calogValueCopy(CalogValueT *dst, const CalogValueT *src);
void    calogValueMove(CalogValueT *dst, CalogValueT *src);
void    calogValueFree(CalogValueT *);
bool    calogValueEquals(const CalogValueT *, const CalogValueT *);

// aggregates (hybrid list + map)
int32_t      calogAggCreate(CalogAggT **, CalogKindE);
void         calogAggFree(CalogAggT *);
int32_t      calogAggPush(CalogAggT *, CalogValueT *value);
int32_t      calogAggSet(CalogAggT *, CalogValueT *key, CalogValueT *value);
CalogValueT *calogAggGet(CalogAggT *, const CalogValueT *key);

// function values (script callbacks)
int32_t calogFnInvoke(CalogFnT *, CalogValueT *args, int32_t n, CalogValueT *result);
void    calogFnRetain(CalogFnT *);
void    calogFnRelease(CalogFnT *);

// contexts + engines
CalogContextT *calogContextOpen(CalogT *, const CalogEngineT *);
void           calogRegisterEngine(CalogT *, const CalogEngineT *);
CalogContextT *calogContextLoad(CalogT *, const char *baseFileName);
int32_t        calogContextEval(CalogContextT *, const char *source);   // fire-and-forget
void           calogContextClose(CalogContextT *);
uint64_t       calogContextId(const CalogContextT *);
uint64_t       calogCurrentId(void);     // 0 on the host thread
CalogT        *calogCurrent(void);

// stopping scripts (what bin/calog's calogExit is built on)
int32_t calogAbortAll(CalogT *, CalogValueT *result);   // return this from a native: every script stops
bool    calogAborting(CalogT *);                        // true once the caller has been stopped

Statuses are CalogStatusE (calogOkE, calogErrArgE, calogErrOomE, ...); calogOkE is 0.

Testing

make test       # full suite under AddressSanitizer + UBSan
make tsan       # ThreadSanitizer: actor core + Lua
make tsansq     # ThreadSanitizer: Squirrel
make tsanjs     # ThreadSanitizer: JavaScript
make tsanmb     # ThreadSanitizer: MY-BASIC

The ThreadSanitizer targets run under setarch -R (ASLR off) so TSan's shadow allocator is happy on all kernels.

make cross        # cross-build everything: musl (RUN), Windows PE, macOS Mach-O -- needs zig
make cross-smoke  # just the matrix: all ten engines per target, plus net + a real TLS run on musl
make cross-lint   # zig-free parity check; `make test` runs this one

Only cross-lint is part of make test -- the rest need a zig toolchain, which building calog natively does not. See PORTING.md.

Project layout

src/            core (broker, values, actor) + one subdir per engine (lua, js, squirrel, mybasic)
  calog.h       the entire public API
tests/          the test programs
examples/       embed.c -- a minimal host
vendor/         vendored engine sources, built from source
design.md       the full design rationale and internals

Design

design.md is the long-form record: the canonical value model, the actor/threading design, the function-value lifecycle, per-engine notes, and the build system.

License

calog is released under the MIT License. The vendored engines (Lua, Duktape, Squirrel, MY-BASIC) each retain their own MIT licenses -- see LICENSE.md for the third-party notices.

20 KiB Raw Blame History