calog/API.md

calog script API reference

Every function below is a native that calog registers into each engine, so a script in any language can call it. This reference covers the natives the calog runner and its libraries expose. (For the C embedding API -- calogCreate, calogRegister, calogFnInvoke, etc. -- see the README and src/calog.h.)

Conventions

• Calling syntax per engine. Most engines call a native by its bare name -- cryptoUuid(), kvSet("k", 1). Two differ:
  • Wren: Calog.call("name", [args]), e.g. Calog.call("kvSet", ["k", 1]).
  • Scheme (s7): s-expression form, (kvSet "k" 1).
• Output & exit (provided by the calog runner): calogPrint(...) writes to stdout; calogExit([code]) tears everything down and exits. calog is event-driven, so a script must call calogExit (or be interrupted) to end -- a finished top level does not exit.
• Values. Arguments and results marshal through one canonical type: nil, bool, int, real, string, list, and map (keyed record). Strings are binary-safe (may contain embedded NULs) everywhere the underlying library allows it.
• Signatures. Every entry is written name(param: type, ...) -> returnType, using the value types above plus handle (an opaque int64 resource handle), fn (a function value), and any (a value of any type). [x] marks an optional argument and ...x a variadic one. A signature with no -> returns nil. A return type given as T | nil may be nil (e.g. a lookup miss or end of stream).
• Handles (db connections, sockets, ssh sessions, tasks) are opaque int64 values from a process-wide table, so a handle can be passed to another context and used there -- e.g. lent to a function in another script via calogCall, which uses it and returns while the caller is blocked (a safe synchronous hand-off). What is not safe is using one handle from two contexts at the same instant, or closing it while another context is mid-operation: at most one context may touch a given handle at any moment, and exactly one closes it. (Tasks are the exception -- a task is owner-scoped: only its spawner may taskEval/taskClose it.)
• Callbacks. Natives that take a fn argument (psSubscribe, timerAfter, timerEvery, calogExport) accept a first-class function value in every engine. A function value handed to a script (say, returned by calogCall) is invoked directly -- except in my-basic, which calls it with calogInvoke(fn, ...args).
• Binary data in my-basic. my-basic strings are text (NUL-terminated), so a calog string that carries an embedded NUL arrives as a distinct byte-buffer value instead (a NUL-free string stays an ordinary string, unchanged). Byte buffers are length-carrying and binary-safe; work with them via byteLen(b) -> int, byteAt(b, i: int) -> int, byteSlice(b, start: int, count: int) -> bytes, byteConcat(...parts) -> bytes (each part a byte buffer or a string), strToByte(s: string) -> bytes, and byteToStr(b) -> string. + concatenates byte buffers (and string + bytes), and =/<> compare them by content. A byte buffer egresses back to a native as a full-length binary string. Every other engine's strings are already binary-safe, so this applies to my-basic only.
• Sandboxing. A limited context (calogContextOpenLimited) honors three per-context limits: the native allow-list (every engine), a wall-clock time budget, and a memory cap (an over-budget or runaway script is retired). The time budget and memory cap hold for all engines except s7 -- Lua, JavaScript, my-basic, Berry, Tcl, mruby, Squirrel, Wren, and Janet -- each enforced by the cleanest mechanism its VM allows (see design.md sec 24); s7 is allow-list-only, a documented limit. Bound granularity varies: exact for Lua/JS/Berry/mruby, otherwise allocation-, loop-, or statement-granular (a single operation may transiently overshoot the memory cap before the next check). For my-basic, INPUT never reads host stdin: it yields an empty line, so a script takes input through natives like every other engine.
• Availability. Every library in this reference is compiled into bin/calog: crypto, json, kv, fs, time, timer, export, pubsub, task, net (TCP / UDP / ENet), db (SQLite / PostgreSQL / MySQL), http, and ssh. ssh needs a reachable server; http needs a reachable endpoint.

---

Engine feature matrix

All ten engines share one value and callback model, so the same natives work from every language. These dimensions are identical across all ten: nil/bool/int/real scalars, binary-safe strings, list and map marshalling in both directions, function values in both directions (a script function handed to a native, and a native or host function called from the script), cross-engine calls, and the native allow-list. The table records only where engines differ -- and each difference follows from the language, not from calog.

Engine	File ext	Max integer	Bare-name exports*	Memory cap + time budget
Lua	.lua	64-bit	yes	yes
JavaScript (QuickJS)	.js	2^53**	yes	yes
my-basic	.bas	64-bit	yes	yes
Squirrel	.nut	64-bit	yes	yes
Berry	.be	64-bit	no -- use calogCall	yes
Scheme (s7)	.scm	64-bit	yes	no -- allow-list only
Wren	.wren	2^53**	no -- use Calog.call	yes
mruby	.rb	64-bit	yes	yes
Tcl	.tcl	64-bit	yes	yes
Janet	.janet	2^53**	no -- use calogCall	yes

* Registered natives are callable by bare name in every engine (report(42)). This column is only about resolving a bare name to a value exported by another context (via calogExport): seven engines resolve such a name lazily; Berry, Wren, and Janet require an explicit calogCall / Calog.call to reach an export.

** JavaScript, Wren, and Janet model numbers as IEEE doubles, so integer magnitudes above 2^53 lose precision; the other seven carry full 64-bit integers. (JavaScript can still recover a full int64 on ingress by passing a BigInt.)

The memory cap and time budget are enforced by the cleanest mechanism each VM allows (allocator refusal, a counting allocator, an instruction or heartbeat hook, the bytecode loop, a built-in limit, or -- for Janet -- a watchdog thread); design.md sec 24 has the per-engine details and the two accepted limitations (a script that deliberately catches the sandbox error and loops can pin its thread; a single allocation of script-controlled size can transiently overshoot the cap). s7 is allow-list only -- its unchecked allocator and optimizer-collapsed loops leave no safe hook point.

Value-model edges that follow from each language (not calog limits): Tcl and s7 have no true nil (it marshals out as "" / #<unspecified>) and collapse bool to 0/1; a hybrid list+map aggregate round-trips losslessly only on Lua (the others flatten it to a map); non-scalar map keys are dropped on egress (JavaScript, Berry, s7, mruby); Janet loses symbol/keyword subtype (it arrives as a string).

---

Runner (calog binary)

Function	Description
calogPrint(...values: any)	Write each argument to stdout, space-separated, with a trailing newline.
calogExit([code: int])	Tear down the runtime and exit the process with code (default 0). Does not return.

archive

Compression and archives over libarchive. All data is binary-safe. One-shot codecs work whole-buffer in memory; archive read/write use an opaque integer handle. A per-transfer 64 MiB cap guards against a decompression bomb.

Function	Description
compress(data: string, filter: string [, level: int]) -> string	Compress data. filter: "gzip", "bzip2", "xz", "zstd", "lz4", "compress", "none".
decompress(data: string [, filter: string]) -> string	Decompress; the filter is auto-detected from the magic bytes if omitted.
archiveReadOpen(bytes: string) -> handle	Open an archive held in memory.
archiveReadOpenFile(path: string) -> handle	Open an archive streamed from a file.
archiveReadNext(handle) -> map | nil	Advance one entry: { name, size, mode, mtime, type[, linkname] }; nil at the end.
archiveReadData(handle) -> string	The current entry's bytes (whole, capped).
archiveReadClose(handle)	Close and free the reader.
archiveWriteOpen(format: string, filter: string [, level: int]) -> handle	Start an in-memory archive. format is a container name (see Container formats below); filter compresses the whole archive, as for compress.
archiveWriteEntry(handle, name: string, data: string [, opts: map])	Add one entry. opts: { mode: int, mtime: int, type: "file"|"dir" }.
archiveWriteFinish(handle) -> string	Close and return the whole archive; frees the handle.

Container formats. archiveReadOpen/archiveReadOpenFile auto-detect the format from the byte stream, so extracting takes no format argument. archiveWriteOpen takes one of the names below. Compression is orthogonal to the container: a .tar.zst, for example, is format "tar" with filter "zstd".

Format	Read	Write	Notes
tar / ustar / pax / gnutar / v7tar	yes	yes	POSIX and GNU tar variants.
cpio (aliases newc, odc)	yes	yes
zip	yes	yes
7zip	yes	yes
iso9660	yes	yes	ISO CD/DVD filesystem image.
ar (alias arbsd)	yes	yes	Unix .a archive.
mtree	yes	yes	File-hierarchy listing: entry metadata, no file contents.
warc	yes	yes	Web archive.
xar	yes	yes	Extensible archive (Apple .pkg); needs vendored libxml2 + OpenSSL.
shar	no	yes	Self-extracting shell archive.
cab	yes	no	Microsoft Cabinet; extract only.
lha / lzh	yes	no	LHA; extract only.
rar / rar5	yes	no	RAR is proprietary -- libarchive can extract but not create it.

Formats marked write no are decode-only: the encoder is proprietary or absent. xar read and write are backed by the vendored libxml2 (its TOC is XML) plus OpenSSL (its entry checksums are MD5/SHA1); both are already in the build. A single unnamed stream can be written with format "raw", but for one-shot compression prefer compress/decompress.

crypto

Binary-safe cryptographic primitives over OpenSSL.

Function	Description
cryptoHashSha256(data: string) -> string	SHA-256 as a 64-char lowercase hex digest.
cryptoHashSha1(data: string) -> string	SHA-1 as a 40-char lowercase hex digest.
cryptoHmacSha256(key: string, data: string) -> string	HMAC-SHA-256 as 64-char lowercase hex.
cryptoRandomBytes(count: int) -> string	count cryptographically-random bytes.
cryptoBase64Encode(data: string) -> string	Base64-encode.
cryptoBase64Decode(text: string) -> string	Base64-decode (trailing whitespace tolerated).
cryptoHexEncode(data: string) -> string	Lowercase hex encode.
cryptoHexDecode(hexText: string) -> string	Hex decode (case-insensitive).
cryptoUuid() -> string	A random RFC 4122 version-4 UUID string.

csv

Parse and format RFC 4180 CSV. Cells are strings; quoted cells may contain the delimiter, newlines, and doubled quotes. Binary-safe.

Function	Description
csvParse(text: string [, delimiter: string]) -> list(list(string))	Parse CSV into rows of string cells. The optional delimiter is one byte (default ,).
csvFormat(rows: list(list) [, delimiter: string]) -> string	Format rows of cells (string/int/real/bool/nil) into CSV (records joined by CRLF); cells with the delimiter, a quote, or a newline are quoted.

db

SQL over SQLite, PostgreSQL, and MySQL/MariaDB. Parameters are always bound (never string-spliced), so queries are injection-safe. Values marshal as: NULL <-> nil, integers/ reals as-is, text and BLOBs as binary-safe strings.

Function	Description
dbOpen(driver: string, conn: string) -> handle	Open a connection. driver is "sqlite", "postgres", or "mysql". conn is a SQLite path or ":memory:", a libpq conninfo, or a MySQL key=value ... string.
dbExec(handle: handle, sql: string, ...params: any) -> int	Run a non-query statement with bound params; returns rows affected.
dbQuery(handle: handle, sql: string, ...params: any) -> list	Run a query; returns a list of {column: value} row maps.
dbClose(handle: handle)	Close the connection.

export

Share a function by name across contexts and engines.

Function	Description
calogExport(name: string, fn: fn)	Publish function fn under a global name.
calogUnexport(name: string)	Remove an exported name.
calogCall(name: string, ...args: any) -> any	Call an exported function by name -- works in every engine. (An export is also reachable by its bare name -- exportedFn(args) -- on Lua/JS/Squirrel/s7, on Ruby at top level (via method_missing), on Tcl (via its unknown handler), and on my-basic case-insensitively (it uppercases identifiers); Wren, Berry, and Janet always need calogCall.)

fs

POSIX filesystem access. A failed operation raises a catchable script error carrying strerror(errno).

Function	Description
fsRead(path: string) -> string	Read a whole file (binary-safe).
fsWrite(path: string, data: string)	Create/truncate and write data.
fsAppend(path: string, data: string)	Create if absent, append at the end.
fsExists(path: string) -> bool	Whether the path exists.
fsRemove(path: string)	Unlink a file.
fsMkdir(path: string)	Create one directory level (existing dir is OK).
fsList(path: string) -> list	Entry name strings, excluding . and ...
fsStat(path: string) -> map | nil	{size, isDir, isFile, mtime}, or nil if the path is absent.

http

Minimal HTTP/1.1 client over http:// and https://. Each call is its own connection (Connection: close). 3xx redirects are followed (up to 16, to break loops). https:// verifies the server certificate against the host's native trust store by default: the Windows system certificate stores, the macOS keychain (on a build made with the Apple SDK), or the well-known CA-bundle files on Linux/BSD. Overrides, highest priority first: CALOG_CA_BUNDLE (authoritative -- trust exactly that PEM bundle, so it can pin to a private CA), then the OpenSSL-standard SSL_CERT_FILE / SSL_CERT_DIR. If no trust anchors can be found, a verified request fails with a clear error rather than a misleading per-certificate failure.

Function	Description
httpGet(url: string) -> map	GET a URL, following redirects. Returns {status, body, headers} (headers keyed by lowercased name).
httpRequest(opts: map) -> map	opts is {method (default "GET"), url, headers (map), body, insecure (bool), maxRedirects (int, default 16; 0 = don't follow)}. insecure=true skips TLS verification. Returns {status, body, headers}.

httpd (a script, not a native library)

The HTTP/1.1 + WebSocket server lives entirely in examples/httpd.lua -- there is no C httpd. It is built on the generic tcp* transport above (with the tls option for HTTPS/WSS) plus the crypto natives (cryptoHashSha1 + cryptoBase64Encode for the WebSocket handshake), which is the calog thesis: systems primitives in C, protocol in a script. It runs as-is on any engine with binary-safe strings (all but my-basic, whose strings are text); a my-basic port would use the byte-buffer type and its byte* helpers for the masked/binary frames instead of string operations.

local httpd = dofile("examples/httpd.lua")   -- or paste/require the module
local s = httpd.new()
s:route("GET", "/hi", function(req) return "hello " .. req.path end)          -- string body => 200
s:route("GET", "/made", function(req) return { status = 201, body = "x" } end) -- map => custom status
s:websocket("/ws", function(msg) return "echo: " .. msg.message end)          -- reply string, or nil
s:serve(8080)                                    -- opts: { tls, cert, key, keep = fn, acceptTimeout }

A route handler receives { method, path, query, headers (lowercased), body } and returns a map { status, headers, body }, a bare string (=> 200), or nil (=> 204). The server does HTTP/1.1 keep-alive; re-registering a path hot-swaps its handler. A WebSocket handler receives { path, message } per inbound text/binary frame and returns a text reply or nil (request/reply model; no server-push). Concurrency follows the actor model: one context per accept loop (serial), or share the listener handle across several contexts for parallelism.

json

Function	Description
jsonParse(text: string) -> any	Parse JSON: object -> map, array -> list, number -> int or real, string, true/false, null -> nil.
jsonStringify(value: any) -> string	Serialize a value to compact JSON text.

kv

A process-wide, thread-safe store shared by every context and engine. Holds data only (a function value is rejected). Keys are binary-safe.

Function	Description
kvSet(key: string, value: any)	Store a deep copy of value under key (replaces any existing).
kvGet(key: string) -> any | nil	A deep copy of the stored value, or nil if absent.
kvHas(key: string) -> bool	Whether the key is present.
kvDelete(key: string)	Remove the key (no error if absent).
kvKeys() -> list	A list of the stored keys (strings).

net

Three first-class transports -- TCP, UDP, and ENet (reliable/ordered delivery over UDP) -- all always available. Payloads are binary-safe strings. Blocking calls (tcpAccept/tcpRecv/udpRecvFrom/enetService) stall only the calling context's thread.

TCP and UDP:

Function	Description
tcpConnect(host: string, port: int) -> handle	Connect to a TCP server.
tcpListen(port: int [, opts: map]) -> handle	Listen on a TCP port. opts: { tls (bool), cert (PEM path), key (PEM path) } -- with tls, every accepted connection is a TLS server session (used by the HTTPS/WSS server script).
tcpAccept(handle: handle [, timeoutMs: int]) -> handle | nil	Block for a client (completing the TLS handshake for a TLS listener); returns a connection handle. With timeoutMs, returns nil if none arrives in time, so an accept loop can re-check its own stop condition.
tcpSend(handle: handle, data: string) -> int	Send all of data; returns bytes sent.
tcpRecv(handle: handle, maxBytes: int) -> string | nil	Read up to maxBytes; nil at end of stream.
tcpClose(handle: handle)	Close a socket.
udpOpen(port: int) -> handle	Open a UDP socket (port 0 = ephemeral).
udpSendTo(handle: handle, host: string, port: int, data: string) -> int	Send a datagram; returns bytes sent.
udpRecvFrom(handle: handle, maxBytes: int) -> map	Receive one datagram: {data, host, port}.
udpClose(handle: handle)	Close a UDP socket.

ENet (reliable UDP -- ordered, reliable channels over UDP):

Function	Description
enetHost(port: int, maxPeers: int) -> handle	Create an ENet host.
enetConnect(hostHandle: handle, host: string, port: int, channels: int) -> handle	Initiate a connection to a peer; returns a peer handle.
enetService(hostHandle: handle, timeoutMs: int) -> map	Poll for one event within timeoutMs. Returns {type, ...} where type is "none", "connect", "receive" (with peer, channel, data), or "disconnect".
enetSend(peerHandle: handle, channel: int, data: string, reliable: bool)	Queue a packet on a channel.
enetDisconnect(peerHandle: handle)	Begin disconnecting a peer.
enetClose(hostHandle: handle)	Destroy an ENet host.

proc

Run a subprocess and capture its output (POSIX only; started with posix_spawn, not fork).

Function

Description

procRun(argv: list(string) [, opts: map]) -> map

Spawn argv[0], wait, and return { exit: int, stdout: string, stderr: string } (on POSIX, exit is the negative signal number if the child was killed; on Windows it is the process exit code). opts: { stdin: string, cwd: string, env: map (string->string; replaces the environment) }. Blocks the caller's context thread; stdin is fed while stdout/stderr are drained, so a large transfer cannot deadlock. POSIX spawns with posix_spawn; Windows spawns with CreateProcess.

pubsub

Deliver a message to every subscriber of a topic, across contexts and engines. Delivery is synchronous; each subscriber runs on its own context's thread and gets a deep copy of the message. Keep publish graphs acyclic.

Function	Description
psSubscribe(topic: string, fn: fn) -> int	Register fn to receive messages published on topic; returns a subscription id.
psUnsubscribe(id: int)	Drop the subscription with that id.
psPublish(topic: string, msg: any) -> int	Deliver a copy of msg to every subscriber; returns how many were invoked.

regex

Perl-compatible regular expressions over PCRE2. flags is a string of single letters: i case-insensitive, m multiline, s dot-matches-newline, x extended, g replace-all (for regexReplace). A non-participating capture group is nil. Binary-safe.

Function	Description
regexMatch(pattern, subject [, flags]) -> nil | list	First match, as [fullMatch, group1, group2, ...]; nil if no match.
regexSearch(pattern, subject [, flags]) -> nil | map	First match, as { match, start, end, groups: list } (byte offsets); nil if no match.
regexReplace(pattern, subject, replacement [, flags]) -> string	Replace matches; replacement uses $1 / ${name}. The g flag replaces all.
regexSplit(pattern, subject [, flags]) -> list(string)	Split subject on matches of pattern.

ssh

SSH/SFTP over libssh2. Requires a reachable SSH server. Payloads are binary-safe. A connection handle can be passed to and used by another context (see Handles above), but a libssh2 session is not thread-safe: never operate on one handle from two contexts at once -- calog does not serialize it for you.

Function	Description
sshConnect(host: string[, port: int]) -> handle	Connect (port defaults to 22).
sshAuthPassword(handle: handle, user: string, password: string) -> bool	Password authentication.
sshAuthKey(handle: handle, user: string, privateKeyPath: string[, publicKeyPath: string, passphrase: string]) -> bool	Public-key authentication.
sshExec(handle: handle, command: string) -> map	Run a remote command: {stdout, stderr, exitCode}.
sshClose(handle: handle)	Close the session.
sftpGet(handle: handle, remotePath: string) -> string	Read a remote file whole (binary-safe).
sftpPut(handle: handle, remotePath: string, data: string)	Create/truncate a remote file (mode 0644).
sftpList(handle: handle, path: string) -> list	[{name, size, isDir}, ...].
sftpStat(handle: handle, path: string) -> map | nil	{size, isDir}, or nil if the path is missing.
sftpRemove(handle: handle, path: string)	Remove a remote file.
sftpMkdir(handle: handle, path: string)	Create a remote directory (mode 0755).

task

Launch and manage other calog script contexts. Tasks are fire-and-forget: a spawned context runs on its own thread; results come back through host natives or the shared kv/pubsub. A task is owned by the context that spawned it, and only that owner may taskEval/taskClose. taskClose is cooperative -- a task busy in a pure loop must poll taskActive() and break out, or the close blocks until the task next returns to the runtime.

Function	Description
taskSpawn(engine: string, code: string) -> handle	Run a code string on a named engine ("lua", "js", "squirrel", "mybasic", "berry", "s7", "wren", "mruby", "tcl", "janet").
taskLoad(baseName: string) -> handle	Launch a script file (engine chosen by extension).
taskEval(handle: handle, code: string)	Feed more code into a running task (runs on its thread).
taskClose(handle: handle)	Ask a task to stop (cooperative: sets its shutdown flag, then waits for its thread to exit).
taskActive() -> bool	Called inside a task: false once its owner has asked it to stop. A long loop polls this -- while taskActive() do ... end -- so taskClose need not block.
taskActive(handle: handle) -> bool	Called in the owner: whether the task spawned under handle is still open.
taskExit()	Called inside a task: retire this task's own context. Deferred -- the current code finishes normally, then the context stops and the runtime reaps it (no owner action needed).
taskSelf() -> int	The calling script's own context id.
taskCount() -> int	Number of tasks this library currently holds open.

time

Function	Description
timeNow() -> real	Wall-clock epoch seconds, fractional (CLOCK_REALTIME).
timeMonotonic() -> real	Seconds from an unspecified origin (CLOCK_MONOTONIC); use for intervals.
timeSleep(ms: int)	Block the calling context for ms milliseconds.

timer

One background thread drives every timer; each callback runs on the context that created the timer.

Function	Description
timerAfter(ms: int, fn: fn) -> int	Fire fn once, ms milliseconds from now; returns a timer id.
timerEvery(ms: int, fn: fn) -> int	Fire fn every ms milliseconds; returns a timer id.
timerCancel(id: int)	Stop a pending or repeating timer.

xml

Parse and produce XML over vendored libxml2. An element node is a map { tag, attr, children }: tag is the element name; attr (present only when the element has attributes) maps attribute names to string values; and children is an ordered list whose items are child element maps or plain strings (text nodes). Mixed content and all character data are preserved, so xmlParse then xmlStringify round-trips the element structure; comments, processing instructions, and the XML declaration are dropped. Parsing loads no external entities and does no network I/O (no XXE), and nesting is bounded by the same maximum depth as json. Marshalling is binary-safe (UTF-8).

Function	Description
xmlParse(text: string) -> map	Parse an XML document; returns its root element node. Malformed XML raises a catchable error.
xmlStringify(node: map) -> string	Serialize an element node (the shape xmlParse returns) back to XML text.