Maybe base features complete?

This commit is contained in:
Scott Duensing 2026-07-04 18:40:32 -05:00
parent b5cf00b014
commit ac2d587aa4
49 changed files with 5132 additions and 1824 deletions

185
API.md
View file

@ -14,14 +14,25 @@ etc. -- see the README and `src/calog.h`.)
- **Output & exit** (provided by the `calog` runner): `calogPrint(...)` writes to stdout; - **Output & exit** (provided by the `calog` runner): `calogPrint(...)` writes to stdout;
`calogExit([code])` tears everything down and exits. calog is event-driven, so a script `calogExit([code])` tears everything down and exits. calog is event-driven, so a script
must call `calogExit` (or be interrupted) to end -- a finished top level does not exit. must call `calogExit` (or be interrupted) to end -- a finished top level does not exit.
- **Values.** Arguments and results marshal through one canonical type: nil, bool, integer, - **Values.** Arguments and results marshal through one canonical type: `nil`, `bool`, `int`,
real, string, list, and map (keyed record). Strings are **binary-safe** (may contain `real`, `string`, `list`, and `map` (keyed record). Strings are **binary-safe** (may contain
embedded NULs) everywhere the underlying library allows it. embedded NULs) everywhere the underlying library allows it.
- **Handles** (db connections, sockets, ssh sessions) are opaque values owned by the context - **Signatures.** Every entry is written `name(param: type, ...) -> returnType`, using the
that created them; do not share a live handle across contexts. value types above plus `handle` (an opaque int64 resource handle), `fn` (a function value),
- **Callbacks.** Natives that take a *function* argument (`psSubscribe`, `timerAfter`, and `any` (a value of any type). `[x]` marks an optional argument and `...x` a variadic one.
`timerEvery`, `calogExport`) accept a first-class function value. All engines support this, A signature with no `->` returns `nil`. A return type given as `T | nil` may be `nil` (e.g.
including my-basic (a top-level `def` or `lambda`; see `vendor/ourbasic`). a lookup miss or end of stream).
- **Handles** (db connections, sockets, ssh sessions, tasks) are opaque int64 values from a
process-wide table, so a handle can be **passed to another context and used there** -- e.g.
lent to a function in another script via `calogCall`, which uses it and returns while the
caller is blocked (a safe synchronous hand-off). What is *not* safe is using one handle from
two contexts at the same instant, or closing it while another context is mid-operation: at
most one context may touch a given handle at any moment, and exactly one closes it. (Tasks are
the exception -- a task is owner-scoped: only its spawner may `taskEval`/`taskClose` it.)
- **Callbacks.** Natives that take a `fn` argument (`psSubscribe`, `timerAfter`, `timerEvery`,
`calogExport`) accept a first-class function value in every engine. A function value handed
*to* a script (say, returned by `calogCall`) is invoked directly -- except in my-basic, which
calls it with `calogInvoke(fn, ...args)`.
- **Availability.** Every library in this reference is compiled into `bin/calog`: crypto, - **Availability.** Every library in this reference is compiled into `bin/calog`: crypto,
json, kv, fs, time, timer, export, pubsub, task, net (TCP / UDP / ENet), db (SQLite / json, kv, fs, time, timer, export, pubsub, task, net (TCP / UDP / ENet), db (SQLite /
PostgreSQL / MySQL), http, and ssh. ssh needs a reachable server; http needs a reachable PostgreSQL / MySQL), http, and ssh. ssh needs a reachable server; http needs a reachable
@ -33,8 +44,8 @@ etc. -- see the README and `src/calog.h`.)
| Function | Description | | Function | Description |
|---|---| |---|---|
| `calogPrint(...)` | Write each argument to stdout, space-separated, with a trailing newline. | | `calogPrint(...values: any)` | Write each argument to stdout, space-separated, with a trailing newline. |
| `calogExit([code])` | Tear down the runtime and exit the process with `code` (default `0`). | | `calogExit([code: int])` | Tear down the runtime and exit the process with `code` (default `0`). Does not return. |
## crypto ## crypto
@ -42,14 +53,14 @@ Binary-safe cryptographic primitives over OpenSSL.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `cryptoHashSha256(data) -> hex` | SHA-256 as a 64-char lowercase hex digest. | | `cryptoHashSha256(data: string) -> string` | SHA-256 as a 64-char lowercase hex digest. |
| `cryptoHashSha1(data) -> hex` | SHA-1 as a 40-char lowercase hex digest. | | `cryptoHashSha1(data: string) -> string` | SHA-1 as a 40-char lowercase hex digest. |
| `cryptoHmacSha256(key, data) -> hex` | HMAC-SHA-256 as 64-char lowercase hex. | | `cryptoHmacSha256(key: string, data: string) -> string` | HMAC-SHA-256 as 64-char lowercase hex. |
| `cryptoRandomBytes(count) -> bytes` | `count` cryptographically-random bytes (binary string). | | `cryptoRandomBytes(count: int) -> string` | `count` cryptographically-random bytes. |
| `cryptoBase64Encode(data) -> text` | Base64-encode. | | `cryptoBase64Encode(data: string) -> string` | Base64-encode. |
| `cryptoBase64Decode(text) -> data` | Base64-decode (trailing whitespace tolerated). | | `cryptoBase64Decode(text: string) -> string` | Base64-decode (trailing whitespace tolerated). |
| `cryptoHexEncode(data) -> text` | Lowercase hex encode. | | `cryptoHexEncode(data: string) -> string` | Lowercase hex encode. |
| `cryptoHexDecode(hexText) -> data` | Hex decode (case-insensitive). | | `cryptoHexDecode(hexText: string) -> string` | Hex decode (case-insensitive). |
| `cryptoUuid() -> string` | A random RFC 4122 version-4 UUID string. | | `cryptoUuid() -> string` | A random RFC 4122 version-4 UUID string. |
## db ## db
@ -60,10 +71,10 @@ reals as-is, text and BLOBs as binary-safe strings.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `dbOpen(driver, conn) -> handle` | Open a connection. `driver` is `"sqlite"`, `"postgres"`, or `"mysql"`. `conn` is a SQLite path or `":memory:"`, a libpq conninfo, or a MySQL `key=value ...` string. | | `dbOpen(driver: string, conn: string) -> handle` | Open a connection. `driver` is `"sqlite"`, `"postgres"`, or `"mysql"`. `conn` is a SQLite path or `":memory:"`, a libpq conninfo, or a MySQL `key=value ...` string. |
| `dbExec(handle, sql, ...params) -> rowsAffected` | Run a non-query statement with bound params; returns rows affected. | | `dbExec(handle: handle, sql: string, ...params: any) -> int` | Run a non-query statement with bound params; returns rows affected. |
| `dbQuery(handle, sql, ...params) -> rows` | Run a query; returns a list of `{column: value}` row maps. | | `dbQuery(handle: handle, sql: string, ...params: any) -> list` | Run a query; returns a list of `{column: value}` row maps. |
| `dbClose(handle)` | Close the connection. | | `dbClose(handle: handle)` | Close the connection. |
## export ## export
@ -71,9 +82,9 @@ Share a function by name across contexts and engines.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `calogExport(name, fn)` | Publish function `fn` under a global `name`. | | `calogExport(name: string, fn: fn)` | Publish function `fn` under a global `name`. |
| `calogUnexport(name)` | Remove an exported name. | | `calogUnexport(name: string)` | Remove an exported name. |
| `calogCall(name, ...args) -> result` | Call an exported function by name -- works in **every** engine. (On hook engines -- Lua/JS/Squirrel/s7 -- an export is also reachable by its bare name.) | | `calogCall(name: string, ...args: any) -> any` | Call an exported function by name -- works in **every** engine. (An export is also reachable by its bare name -- `exportedFn(args)` -- on Lua/JS/Squirrel/s7, and on my-basic case-insensitively, since it uppercases identifiers; Wren and Berry always need `calogCall`.) |
## fs ## fs
@ -82,32 +93,32 @@ POSIX filesystem access. A failed operation raises a catchable script error carr
| Function | Description | | Function | Description |
|---|---| |---|---|
| `fsRead(path) -> string` | Read a whole file (binary-safe). | | `fsRead(path: string) -> string` | Read a whole file (binary-safe). |
| `fsWrite(path, data)` | Create/truncate and write `data`. | | `fsWrite(path: string, data: string)` | Create/truncate and write `data`. |
| `fsAppend(path, data)` | Create if absent, append at the end. | | `fsAppend(path: string, data: string)` | Create if absent, append at the end. |
| `fsExists(path) -> bool` | Whether the path exists. | | `fsExists(path: string) -> bool` | Whether the path exists. |
| `fsRemove(path)` | Unlink a file. | | `fsRemove(path: string)` | Unlink a file. |
| `fsMkdir(path)` | Create one directory level (existing dir is OK). | | `fsMkdir(path: string)` | Create one directory level (existing dir is OK). |
| `fsList(path) -> list` | Entry name strings, excluding `.` and `..`. | | `fsList(path: string) -> list` | Entry name strings, excluding `.` and `..`. |
| `fsStat(path) -> map` | `{size, isDir, isFile, mtime}`, or nil if the path is absent. | | `fsStat(path: string) -> map \| nil` | `{size, isDir, isFile, mtime}`, or nil if the path is absent. |
## http ## http
Minimal HTTP/1.1 client over `http://` and `https://`. Each call is its own connection Minimal HTTP/1.1 client over `http://` and `https://`. Each call is its own connection
(`Connection: close`); redirects are not followed. `https://` verifies the server certificate (`Connection: close`). 3xx redirects are followed (up to 16, to break loops). `https://`
against the system CA store by default. verifies the server certificate against the system CA store by default.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `httpGet(url) -> map` | GET a URL. Returns `{status, body, headers}` (headers keyed by lowercased name). | | `httpGet(url: string) -> map` | GET a URL, following redirects. Returns `{status, body, headers}` (headers keyed by lowercased name). |
| `httpRequest(opts) -> map` | `opts` is `{method (default "GET"), url, headers (map), body, insecure (bool)}`. `insecure=true` skips TLS verification. Returns `{status, body, headers}`. | | `httpRequest(opts: map) -> map` | `opts` is `{method (default "GET"), url, headers (map), body, insecure (bool), maxRedirects (int, default 16; 0 = don't follow)}`. `insecure=true` skips TLS verification. Returns `{status, body, headers}`. |
## json ## json
| Function | Description | | Function | Description |
|---|---| |---|---|
| `jsonParse(text) -> value` | Parse JSON: object -> map, array -> list, number -> int or real, string, true/false, null -> nil. | | `jsonParse(text: string) -> any` | Parse JSON: object -> map, array -> list, number -> int or real, string, true/false, null -> nil. |
| `jsonStringify(value) -> text` | Serialize a value to compact JSON text. | | `jsonStringify(value: any) -> string` | Serialize a value to compact JSON text. |
## kv ## kv
@ -116,10 +127,10 @@ function value is rejected). Keys are binary-safe.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `kvSet(key, value)` | Store a deep copy of `value` under `key` (replaces any existing). | | `kvSet(key: string, value: any)` | Store a deep copy of `value` under `key` (replaces any existing). |
| `kvGet(key) -> value` | A deep copy of the stored value, or nil if absent. | | `kvGet(key: string) -> any \| nil` | A deep copy of the stored value, or nil if absent. |
| `kvHas(key) -> bool` | Whether the key is present. | | `kvHas(key: string) -> bool` | Whether the key is present. |
| `kvDelete(key)` | Remove the key (no error if absent). | | `kvDelete(key: string)` | Remove the key (no error if absent). |
| `kvKeys() -> list` | A list of the stored keys (strings). | | `kvKeys() -> list` | A list of the stored keys (strings). |
## net ## net
@ -132,27 +143,27 @@ TCP and UDP:
| Function | Description | | Function | Description |
|---|---| |---|---|
| `tcpConnect(host, port) -> handle` | Connect to a TCP server. | | `tcpConnect(host: string, port: int) -> handle` | Connect to a TCP server. |
| `tcpListen(port) -> handle` | Listen on a TCP port. | | `tcpListen(port: int) -> handle` | Listen on a TCP port. |
| `tcpAccept(handle) -> handle` | Block for a client; returns a connection handle. | | `tcpAccept(handle: handle) -> handle` | Block for a client; returns a connection handle. |
| `tcpSend(handle, data) -> bytesSent` | Send all of `data`. | | `tcpSend(handle: handle, data: string) -> int` | Send all of `data`; returns bytes sent. |
| `tcpRecv(handle, maxBytes) -> data` | Read up to `maxBytes`; nil at end of stream. | | `tcpRecv(handle: handle, maxBytes: int) -> string \| nil` | Read up to `maxBytes`; nil at end of stream. |
| `tcpClose(handle)` | Close a socket. | | `tcpClose(handle: handle)` | Close a socket. |
| `udpOpen(port) -> handle` | Open a UDP socket (`port` 0 = ephemeral). | | `udpOpen(port: int) -> handle` | Open a UDP socket (`port` 0 = ephemeral). |
| `udpSendTo(handle, host, port, data) -> bytesSent` | Send a datagram. | | `udpSendTo(handle: handle, host: string, port: int, data: string) -> int` | Send a datagram; returns bytes sent. |
| `udpRecvFrom(handle, maxBytes) -> map` | Receive one datagram: `{data, host, port}`. | | `udpRecvFrom(handle: handle, maxBytes: int) -> map` | Receive one datagram: `{data, host, port}`. |
| `udpClose(handle)` | Close a UDP socket. | | `udpClose(handle: handle)` | Close a UDP socket. |
ENet (reliable UDP -- ordered, reliable channels over UDP): ENet (reliable UDP -- ordered, reliable channels over UDP):
| Function | Description | | Function | Description |
|---|---| |---|---|
| `enetHost(port, maxPeers) -> hostHandle` | Create an ENet host. | | `enetHost(port: int, maxPeers: int) -> handle` | Create an ENet host. |
| `enetConnect(hostHandle, host, port, channels) -> peerHandle` | Initiate a connection to a peer. | | `enetConnect(hostHandle: handle, host: string, port: int, channels: int) -> handle` | Initiate a connection to a peer; returns a peer handle. |
| `enetService(hostHandle, timeoutMs) -> event` | Poll for one event within `timeoutMs`. Returns `{type, ...}` where `type` is `"none"`, `"connect"`, `"receive"` (with `peer`, `channel`, `data`), or `"disconnect"`. | | `enetService(hostHandle: handle, timeoutMs: int) -> map` | Poll for one event within `timeoutMs`. Returns `{type, ...}` where `type` is `"none"`, `"connect"`, `"receive"` (with `peer`, `channel`, `data`), or `"disconnect"`. |
| `enetSend(peerHandle, channel, data, reliable)` | Queue a packet on a channel; `reliable` is a bool. | | `enetSend(peerHandle: handle, channel: int, data: string, reliable: bool)` | Queue a packet on a channel. |
| `enetDisconnect(peerHandle)` | Begin disconnecting a peer. | | `enetDisconnect(peerHandle: handle)` | Begin disconnecting a peer. |
| `enetClose(hostHandle)` | Destroy an ENet host. | | `enetClose(hostHandle: handle)` | Destroy an ENet host. |
## pubsub ## pubsub
@ -162,42 +173,50 @@ message. Keep publish graphs acyclic.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `psSubscribe(topic, fn) -> id` | Register `fn` to receive messages published on `topic`. | | `psSubscribe(topic: string, fn: fn) -> int` | Register `fn` to receive messages published on `topic`; returns a subscription id. |
| `psUnsubscribe(id)` | Drop the subscription with that id. | | `psUnsubscribe(id: int)` | Drop the subscription with that id. |
| `psPublish(topic, msg) -> count` | Deliver a copy of `msg` to every subscriber; returns how many were invoked. | | `psPublish(topic: string, msg: any) -> int` | Deliver a copy of `msg` to every subscriber; returns how many were invoked. |
## ssh ## ssh
SSH/SFTP over libssh2. Requires a reachable SSH server. Payloads are binary-safe. SSH/SFTP over libssh2. Requires a reachable SSH server. Payloads are binary-safe. A connection
handle can be passed to and used by another context (see Handles above), but a libssh2 session
is **not thread-safe**: never operate on one handle from two contexts at once -- calog does not
serialize it for you.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `sshConnect(host[, port]) -> handle` | Connect (port defaults to 22). | | `sshConnect(host: string[, port: int]) -> handle` | Connect (port defaults to 22). |
| `sshAuthPassword(handle, user, password) -> bool` | Password authentication. | | `sshAuthPassword(handle: handle, user: string, password: string) -> bool` | Password authentication. |
| `sshAuthKey(handle, user, privateKeyPath[, publicKeyPath, passphrase]) -> bool` | Public-key authentication. | | `sshAuthKey(handle: handle, user: string, privateKeyPath: string[, publicKeyPath: string, passphrase: string]) -> bool` | Public-key authentication. |
| `sshExec(handle, command) -> map` | Run a remote command: `{stdout, stderr, exitCode}`. | | `sshExec(handle: handle, command: string) -> map` | Run a remote command: `{stdout, stderr, exitCode}`. |
| `sshClose(handle)` | Close the session. | | `sshClose(handle: handle)` | Close the session. |
| `sftpGet(handle, remotePath) -> data` | Read a remote file whole (binary-safe). | | `sftpGet(handle: handle, remotePath: string) -> string` | Read a remote file whole (binary-safe). |
| `sftpPut(handle, remotePath, data)` | Create/truncate a remote file (mode 0644). | | `sftpPut(handle: handle, remotePath: string, data: string)` | Create/truncate a remote file (mode 0644). |
| `sftpList(handle, path) -> list` | `[{name, size, isDir}, ...]`. | | `sftpList(handle: handle, path: string) -> list` | `[{name, size, isDir}, ...]`. |
| `sftpStat(handle, path) -> map` | `{size, isDir}`, or nil if the path is missing. | | `sftpStat(handle: handle, path: string) -> map \| nil` | `{size, isDir}`, or nil if the path is missing. |
| `sftpRemove(handle, path)` | Remove a remote file. | | `sftpRemove(handle: handle, path: string)` | Remove a remote file. |
| `sftpMkdir(handle, path)` | Create a remote directory (mode 0755). | | `sftpMkdir(handle: handle, path: string)` | Create a remote directory (mode 0755). |
## task ## task
Launch and manage other calog script contexts. Tasks are fire-and-forget: a spawned context Launch and manage other calog script contexts. Tasks are fire-and-forget: a spawned context
runs on its own thread; results come back through host natives or the shared kv/pubsub. A runs on its own thread; results come back through host natives or the shared kv/pubsub. A
task is owned by the context that spawned it, and only that owner may `taskEval`/`taskClose`. task is owned by the context that spawned it, and only that owner may `taskEval`/`taskClose`.
`taskClose` is cooperative -- a task busy in a pure loop must poll `taskActive()` and break
out, or the close blocks until the task next returns to the runtime.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `taskSpawn(engine, code) -> handle` | Run a code string on a named engine (`"lua"`, `"javascript"`, `"squirrel"`, `"mybasic"`, `"berry"`, `"scheme"`, `"wren"`). | | `taskSpawn(engine: string, code: string) -> handle` | Run a code string on a named engine (`"lua"`, `"js"`, `"squirrel"`, `"mybasic"`, `"berry"`, `"s7"`, `"wren"`). |
| `taskLoad(baseName) -> handle` | Launch a script *file* (engine chosen by extension). | | `taskLoad(baseName: string) -> handle` | Launch a script *file* (engine chosen by extension). |
| `taskEval(handle, code)` | Feed more code into a running task (runs on its thread). | | `taskEval(handle: handle, code: string)` | Feed more code into a running task (runs on its thread). |
| `taskClose(handle)` | Stop a task (cooperative: waits for its thread to exit). | | `taskClose(handle: handle)` | Ask a task to stop (cooperative: sets its shutdown flag, then waits for its thread to exit). |
| `taskSelf() -> id` | The calling script's own context id. | | `taskActive() -> bool` | Called *inside a task*: `false` once its owner has asked it to stop. A long loop polls this -- `while taskActive() do ... end` -- so `taskClose` need not block. |
| `taskCount() -> n` | Number of tasks this library currently holds open. | | `taskActive(handle: handle) -> bool` | Called *in the owner*: whether the task spawned under `handle` is still open. |
| `taskExit()` | Called *inside a task*: retire this task's **own** context. Deferred -- the current code finishes normally, then the context stops and the runtime reaps it (no owner action needed). |
| `taskSelf() -> int` | The calling script's own context id. |
| `taskCount() -> int` | Number of tasks this library currently holds open. |
## time ## time
@ -205,7 +224,7 @@ task is owned by the context that spawned it, and only that owner may `taskEval`
|---|---| |---|---|
| `timeNow() -> real` | Wall-clock epoch seconds, fractional (CLOCK_REALTIME). | | `timeNow() -> real` | Wall-clock epoch seconds, fractional (CLOCK_REALTIME). |
| `timeMonotonic() -> real` | Seconds from an unspecified origin (CLOCK_MONOTONIC); use for intervals. | | `timeMonotonic() -> real` | Seconds from an unspecified origin (CLOCK_MONOTONIC); use for intervals. |
| `timeSleep(ms)` | Block the calling context for `ms` milliseconds. | | `timeSleep(ms: int)` | Block the calling context for `ms` milliseconds. |
## timer ## timer
@ -214,6 +233,6 @@ timer.
| Function | Description | | Function | Description |
|---|---| |---|---|
| `timerAfter(ms, fn) -> id` | Fire `fn` once, `ms` milliseconds from now. | | `timerAfter(ms: int, fn: fn) -> int` | Fire `fn` once, `ms` milliseconds from now; returns a timer id. |
| `timerEvery(ms, fn) -> id` | Fire `fn` every `ms` milliseconds. | | `timerEvery(ms: int, fn: fn) -> int` | Fire `fn` every `ms` milliseconds; returns a timer id. |
| `timerCancel(id)` | Stop a pending or repeating timer. | | `timerCancel(id: int)` | Stop a pending or repeating timer. |

1393
AUDIT.md Normal file

File diff suppressed because it is too large Load diff

103
Makefile
View file

@ -30,8 +30,6 @@ LDFLAGS = $(SAN)
INC = -Isrc -Isrc/lua -Isrc/mybasic -Isrc/squirrel -Isrc/js -Isrc/berry -Isrc/s7 -Isrc/wren -Ilibs INC = -Isrc -Isrc/lua -Isrc/mybasic -Isrc/squirrel -Isrc/js -Isrc/berry -Isrc/s7 -Isrc/wren -Ilibs
VPATH = src:src/lua:src/mybasic:src/squirrel:src/js:src/berry:src/s7:src/wren:libs:tests VPATH = src:src/lua:src/mybasic:src/squirrel:src/js:src/berry:src/s7:src/wren:libs:tests
CORE = obj/value.o obj/broker.o
# --- vendored our-basic: calog's fork of MY-BASIC (see vendor/ourbasic/NOTICE) --- # --- vendored our-basic: calog's fork of MY-BASIC (see vendor/ourbasic/NOTICE) ---
MBDIR = vendor/ourbasic MBDIR = vendor/ourbasic
MBINC = -I$(MBDIR) MBINC = -I$(MBDIR)
@ -88,7 +86,8 @@ SQOBJ = $(patsubst $(SQDIR)/squirrel/%.cpp,obj/%.o,$(SQSRC))
QJSDIR = vendor/quickjs QJSDIR = vendor/quickjs
QJSINC = -I$(QJSDIR) QJSINC = -I$(QJSDIR)
QJSFLAGS = -std=c11 -w -g -O1 -D_GNU_SOURCE QJSFLAGS = -std=c11 -w -g -O1 -D_GNU_SOURCE
QJSOBJ = obj/quickjs.o obj/libregexp.o obj/libunicode.o obj/dtoa.o QJSNAMES = quickjs libregexp libunicode dtoa
QJSOBJ = $(patsubst %,obj/%.o,$(QJSNAMES))
# --- vendored Berry (C). The library is every src/*.c; its build needs the coc- # --- vendored Berry (C). The library is every src/*.c; its build needs the coc-
# generated headers under generate/ (regenerate with tools/coc from upstream) and # generated headers under generate/ (regenerate with tools/coc from upstream) and
@ -326,6 +325,9 @@ PGARCHIVES = vendor/postgres/src/interfaces/libpq/libpq.a vendor/postgres/src/c
MYSQLINC = -Ivendor/mariadb/include -Ivendor/mariadb/build/include MYSQLINC = -Ivendor/mariadb/include -Ivendor/mariadb/build/include
MYSQLARCH = vendor/mariadb/build/libmariadb/libmariadbclient.a MYSQLARCH = vendor/mariadb/build/libmariadb/libmariadbclient.a
SSLARCH = vendor/openssl/libssl.a vendor/openssl/libcrypto.a SSLARCH = vendor/openssl/libssl.a vendor/openssl/libcrypto.a
# All three archive groups together -- listed as real prerequisites (so a rebuilt vendored
# archive triggers a relink) but filtered out of $^ where the recipe needs a link group.
DBARCH = $(PGARCHIVES) $(MYSQLARCH) $(SSLARCH)
DBCLIENTOBJ = obj/testDbPg.o obj/testDbMysql.o DBCLIENTOBJ = obj/testDbPg.o obj/testDbMysql.o
obj/calogDbFull.o: libs/calogDb.c | obj obj/calogDbFull.o: libs/calogDb.c | obj
@ -334,11 +336,11 @@ obj/calogDbFull.o: libs/calogDb.c | obj
$(DBCLIENTOBJ): obj/%.o: %.c | obj $(DBCLIENTOBJ): obj/%.o: %.c | obj
$(CC) $(COREFLAGS) $(INC) -pthread -c -o $@ $< $(CC) $(COREFLAGS) $(INC) -pthread -c -o $@ $<
bin/testDbPg: obj/testDbPg.o obj/calogDbFull.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libsqlite3.a | bin bin/testDbPg: obj/testDbPg.o obj/calogDbFull.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libsqlite3.a $(DBARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) $(LUALIBS) $(SQLITELIBS) $(CC) $(LDFLAGS) -pthread -o $@ $(filter-out $(DBARCH),$^) -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) $(LUALIBS) $(SQLITELIBS)
bin/testDbMysql: obj/testDbMysql.o obj/calogDbFull.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libsqlite3.a | bin bin/testDbMysql: obj/testDbMysql.o obj/calogDbFull.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libsqlite3.a $(DBARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) $(LUALIBS) $(SQLITELIBS) $(CC) $(LDFLAGS) -pthread -o $@ $(filter-out $(DBARCH),$^) -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) $(LUALIBS) $(SQLITELIBS)
.PHONY: db-clients .PHONY: db-clients
db-clients: bin/testDbPg bin/testDbMysql db-clients: bin/testDbPg bin/testDbMysql
@ -389,8 +391,8 @@ bin/embed: obj/embed.o lib/libcalog.a lib/libquickjs.a | bin
bin/calog: obj/calogMain.o \ bin/calog: obj/calogMain.o \
obj/calogCrypto.o obj/calogDbFull.o obj/calogExport.o obj/calogFs.o obj/calogHttp.o obj/calogJson.o obj/calogKv.o obj/calogNet.o obj/calogPubsub.o obj/calogSsh.o obj/calogTask.o obj/calogTime.o obj/calogTimer.o obj/calogHandle.o \ obj/calogCrypto.o obj/calogDbFull.o obj/calogExport.o obj/calogFs.o obj/calogHttp.o obj/calogJson.o obj/calogKv.o obj/calogNet.o obj/calogPubsub.o obj/calogSsh.o obj/calogTask.o obj/calogTime.o obj/calogTimer.o obj/calogHandle.o \
lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a \ lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a \
lib/libsqlite3.a lib/libenet.a $(LIBSSH2LIB) | bin lib/libsqlite3.a lib/libenet.a $(LIBSSH2LIB) $(DBARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) -lstdc++ -ldl -lm -lpthread $(CC) $(LDFLAGS) -pthread -o $@ $(filter-out $(DBARCH),$^) -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) -lstdc++ -ldl -lm -lpthread
# ---- fully-static build -------------------------------------------------------------- # ---- fully-static build --------------------------------------------------------------
# A self-contained calog executable with NO shared-library dependencies at runtime. The # A self-contained calog executable with NO shared-library dependencies at runtime. The
@ -425,7 +427,7 @@ static: bin/calogStatic
# exercises calogContextLoad + calogRegisterBuiltinEngines across all four languages -- # exercises calogContextLoad + calogRegisterBuiltinEngines across all four languages --
# including my-basic under the actor model. Uses only calog.h (the extern engine vtables). # including my-basic under the actor model. Uses only calog.h (the extern engine vtables).
obj/testLoad.o: tests/testLoad.c src/calog.h | obj obj/testLoad.o: tests/testLoad.c src/calog.h | obj
$(CC) $(COREFLAGS) -Isrc -pthread -DCALOG_WITH_LUA -DCALOG_WITH_JS -DCALOG_WITH_SQUIRREL -DCALOG_WITH_MYBASIC -DCALOG_WITH_BERRY -DCALOG_WITH_S7 -DCALOG_WITH_WREN -c -o $@ $< $(CC) $(COREFLAGS) -Isrc -pthread $(ENGINEDEFS) -c -o $@ $<
bin/testLoad: obj/testLoad.o lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a | bin bin/testLoad: obj/testLoad.o lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) -lstdc++ -lm $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) -lstdc++ -lm
@ -447,8 +449,8 @@ bin/testTime: obj/testTime.o obj/calogTime.o lib/libcalog.a lib/liblua.a | bin
bin/testFs: obj/testFs.o obj/calogFs.o lib/libcalog.a lib/liblua.a | bin bin/testFs: obj/testFs.o obj/calogFs.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testCrypto: obj/testCrypto.o obj/calogCrypto.o lib/libcalog.a lib/liblua.a | bin bin/testCrypto: obj/testCrypto.o obj/calogCrypto.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(SSLARCH) $(LUALIBS) -ldl $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) -ldl
bin/testKv: obj/testKv.o obj/calogKv.o lib/libcalog.a lib/liblua.a | bin bin/testKv: obj/testKv.o obj/calogKv.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
@ -459,15 +461,15 @@ bin/testTimer: obj/testTimer.o obj/calogTimer.o lib/libcalog.a lib/liblua.a | bi
bin/testPubsub: obj/testPubsub.o obj/calogPubsub.o lib/libcalog.a lib/liblua.a | bin bin/testPubsub: obj/testPubsub.o obj/calogPubsub.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testHttp: obj/testHttp.o obj/calogHttp.o lib/libcalog.a lib/liblua.a | bin bin/testHttp: obj/testHttp.o obj/calogHttp.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(SSLARCH) $(LUALIBS) -ldl $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) -ldl
bin/testHttps: obj/testHttps.o obj/calogHttp.o lib/libcalog.a lib/liblua.a | bin bin/testHttps: obj/testHttps.o obj/calogHttp.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(SSLARCH) $(LUALIBS) -ldl $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) -ldl
# SSH/SFTP over vendored libssh2 (before OpenSSL in the link line: libssh2 depends on it). # SSH/SFTP over vendored libssh2 (before OpenSSL in the link line: libssh2 depends on it).
bin/testSsh: obj/testSsh.o obj/calogSsh.o obj/calogHandle.o lib/libcalog.a lib/liblua.a $(LIBSSH2LIB) | bin bin/testSsh: obj/testSsh.o obj/calogSsh.o obj/calogHandle.o lib/libcalog.a lib/liblua.a $(LIBSSH2LIB) $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(SSLARCH) $(LUALIBS) -ldl $(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) -ldl
# The SSH test spawns a throwaway unprivileged sshd (needs system sshd + ssh-keygen), so it is # The SSH test spawns a throwaway unprivileged sshd (needs system sshd + ssh-keygen), so it is
# NOT part of `make test`. Run it explicitly. # NOT part of `make test`. Run it explicitly.
@ -488,12 +490,15 @@ test: all
# linked un-sanitized (each VM is single-threaded, so that is sound). Run under # linked un-sanitized (each VM is single-threaded, so that is sound). Run under
# `setarch -R` (ASLR off): some kernels hand out more mmap randomization than TSan's # `setarch -R` (ASLR off): some kernels hand out more mmap randomization than TSan's
# shadow allocator tolerates, which aborts it before any test runs. # shadow allocator tolerates, which aborts it before any test runs.
TSANFLAGS = -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread
TSANCORE = src/context.c src/value.c src/broker.c
tsan: $(LUAOBJ) | bin tsan: $(LUAOBJ) | bin
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) -o bin/testActorTsan \ $(CC) $(TSANFLAGS) $(INC) -o bin/testActorTsan \
tests/testActor.c src/context.c src/value.c src/broker.c tests/testActor.c $(TSANCORE)
setarch -R ./bin/testActorTsan setarch -R ./bin/testActorTsan
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(LUAINC) -o bin/testEngineLuaTsan \ $(CC) $(TSANFLAGS) $(INC) $(LUAINC) -o bin/testEngineLuaTsan \
tests/testEngineLua.c src/lua/luaEngine.c src/lua/luaAdapter.c src/context.c src/value.c src/broker.c \ tests/testEngineLua.c src/lua/luaEngine.c src/lua/luaAdapter.c $(TSANCORE) \
$(LUAOBJ) $(LUALIBS) $(LUAOBJ) $(LUALIBS)
setarch -R ./bin/testEngineLuaTsan setarch -R ./bin/testEngineLuaTsan
@ -501,10 +506,10 @@ tsan: $(LUAOBJ) | bin
# under TSan here (throwaway obj/*.tsan.o) so races are caught across the whole # under TSan here (throwaway obj/*.tsan.o) so races are caught across the whole
# stack. Slower than `tsan`, so kept separate. # stack. Slower than `tsan`, so kept separate.
tsansq: $(SQSRC) | bin obj tsansq: $(SQSRC) | bin obj
for f in $(SQSRC); do $(CXX) $(SQXXFLAGS) -fsanitize=thread -c $$f -o obj/$$(basename $${f%.cpp}).tsan.o; done for f in $(SQSRC); do $(CXX) $(SQXXFLAGS) -fsanitize=thread -c $$f -o obj/$$(basename $${f%.cpp}).tsan.o || exit 1; done
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(SQDEF) $(SQINC) -o bin/testEngineSquirrelTsan \ $(CC) $(TSANFLAGS) $(INC) $(SQDEF) $(SQINC) -o bin/testEngineSquirrelTsan \
tests/testEngineSquirrel.c src/squirrel/squirrelEngine.c src/squirrel/squirrelAdapter.c \ tests/testEngineSquirrel.c src/squirrel/squirrelEngine.c src/squirrel/squirrelAdapter.c \
src/context.c src/value.c src/broker.c \ $(TSANCORE) \
$(patsubst $(SQDIR)/squirrel/%.cpp,obj/%.tsan.o,$(SQSRC)) -lstdc++ -lm $(patsubst $(SQDIR)/squirrel/%.cpp,obj/%.tsan.o,$(SQSRC)) -lstdc++ -lm
setarch -R ./bin/testEngineSquirrelTsan setarch -R ./bin/testEngineSquirrelTsan
rm -f $(patsubst $(SQDIR)/squirrel/%.cpp,obj/%.tsan.o,$(SQSRC)) rm -f $(patsubst $(SQDIR)/squirrel/%.cpp,obj/%.tsan.o,$(SQSRC))
@ -512,12 +517,12 @@ tsansq: $(SQSRC) | bin obj
# ThreadSanitizer build of the JS engine path: the vendored QuickJS core is # ThreadSanitizer build of the JS engine path: the vendored QuickJS core is
# recompiled under TSan (throwaway objs) so races are caught across the whole stack. # recompiled under TSan (throwaway objs) so races are caught across the whole stack.
tsanjs: | bin obj tsanjs: | bin obj
for f in quickjs libregexp libunicode dtoa; do $(CC) $(QJSFLAGS) $(QJSINC) -fsanitize=thread -c $(QJSDIR)/$$f.c -o obj/$$f.tsan.o; done for f in $(QJSNAMES); do $(CC) $(QJSFLAGS) $(QJSINC) -fsanitize=thread -c $(QJSDIR)/$$f.c -o obj/$$f.tsan.o || exit 1; done
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(QJSINC) -o bin/testEngineJsTsan \ $(CC) $(TSANFLAGS) $(INC) $(QJSINC) -o bin/testEngineJsTsan \
tests/testEngineJs.c src/js/jsEngine.c src/js/jsAdapter.c src/context.c src/value.c src/broker.c \ tests/testEngineJs.c src/js/jsEngine.c src/js/jsAdapter.c $(TSANCORE) \
obj/quickjs.tsan.o obj/libregexp.tsan.o obj/libunicode.tsan.o obj/dtoa.tsan.o -lm $(patsubst %,obj/%.tsan.o,$(QJSNAMES)) -lm
setarch -R ./bin/testEngineJsTsan setarch -R ./bin/testEngineJsTsan
rm -f obj/quickjs.tsan.o obj/libregexp.tsan.o obj/libunicode.tsan.o obj/dtoa.tsan.o rm -f $(patsubst %,obj/%.tsan.o,$(QJSNAMES))
# ThreadSanitizer build of the my-basic engine path: the vendored interpreter is # ThreadSanitizer build of the my-basic engine path: the vendored interpreter is
# recompiled under TSan (throwaway obj) so the engine's serialization of my-basic's # recompiled under TSan (throwaway obj) so the engine's serialization of my-basic's
@ -525,8 +530,8 @@ tsanjs: | bin obj
# across the whole stack -- it races without the engine lock. # across the whole stack -- it races without the engine lock.
tsanmb: | bin obj tsanmb: | bin obj
$(CC) $(MBFLAGS) -fsanitize=thread -c $(MBDIR)/ourBasic.c -o obj/ourBasic.tsan.o $(CC) $(MBFLAGS) -fsanitize=thread -c $(MBDIR)/ourBasic.c -o obj/ourBasic.tsan.o
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(MBINC) -DMB_DOUBLE_FLOAT -o bin/testEngineMyBasicTsan \ $(CC) $(TSANFLAGS) $(INC) $(MBINC) -DMB_DOUBLE_FLOAT -o bin/testEngineMyBasicTsan \
tests/testEngineMyBasic.c src/mybasic/mybasicEngine.c src/mybasic/mybasicAdapter.c src/context.c src/value.c src/broker.c \ tests/testEngineMyBasic.c src/mybasic/mybasicEngine.c src/mybasic/mybasicAdapter.c $(TSANCORE) \
obj/ourBasic.tsan.o -lm obj/ourBasic.tsan.o -lm
setarch -R ./bin/testEngineMyBasicTsan setarch -R ./bin/testEngineMyBasicTsan
rm -f obj/ourBasic.tsan.o rm -f obj/ourBasic.tsan.o
@ -534,9 +539,9 @@ tsanmb: | bin obj
# ThreadSanitizer build of the Berry engine path: the vendored VM is recompiled under # ThreadSanitizer build of the Berry engine path: the vendored VM is recompiled under
# TSan (throwaway objs) so races are caught across the whole stack. # TSan (throwaway objs) so races are caught across the whole stack.
tsanberry: | bin obj tsanberry: | bin obj
for f in $(BERRYSRC); do $(CC) $(BERRYFLAGS) $(BERRYINC) -fsanitize=thread -c $$f -o obj/$$(basename $${f%.c}).tsan.o; done for f in $(BERRYSRC); do $(CC) $(BERRYFLAGS) $(BERRYINC) -fsanitize=thread -c $$f -o obj/$$(basename $${f%.c}).tsan.o || exit 1; done
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(BERRYINC) -o bin/testEngineBerryTsan \ $(CC) $(TSANFLAGS) $(INC) $(BERRYINC) -o bin/testEngineBerryTsan \
tests/testEngineBerry.c src/berry/berryEngine.c src/berry/berryAdapter.c src/context.c src/value.c src/broker.c \ tests/testEngineBerry.c src/berry/berryEngine.c src/berry/berryAdapter.c $(TSANCORE) \
$(foreach f,$(BERRYSRC),obj/$(notdir $(f:.c=)).tsan.o) -lm $(foreach f,$(BERRYSRC),obj/$(notdir $(f:.c=)).tsan.o) -lm
setarch -R ./bin/testEngineBerryTsan setarch -R ./bin/testEngineBerryTsan
rm -f $(foreach f,$(BERRYSRC),obj/$(notdir $(f:.c=)).tsan.o) rm -f $(foreach f,$(BERRYSRC),obj/$(notdir $(f:.c=)).tsan.o)
@ -545,8 +550,8 @@ tsanberry: | bin obj
# under TSan (throwaway obj) so races are caught across the whole stack. # under TSan (throwaway obj) so races are caught across the whole stack.
tsans7: | bin obj tsans7: | bin obj
$(CC) $(S7FLAGS) $(S7INC) -fsanitize=thread -c $(S7DIR)/s7.c -o obj/s7.tsan.o $(CC) $(S7FLAGS) $(S7INC) -fsanitize=thread -c $(S7DIR)/s7.c -o obj/s7.tsan.o
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(S7INC) -o bin/testEngineS7Tsan \ $(CC) $(TSANFLAGS) $(INC) $(S7INC) -o bin/testEngineS7Tsan \
tests/testEngineS7.c src/s7/s7Engine.c src/s7/s7Adapter.c src/context.c src/value.c src/broker.c \ tests/testEngineS7.c src/s7/s7Engine.c src/s7/s7Adapter.c $(TSANCORE) \
obj/s7.tsan.o $(S7LIBS) obj/s7.tsan.o $(S7LIBS)
setarch -R ./bin/testEngineS7Tsan setarch -R ./bin/testEngineS7Tsan
rm -f obj/s7.tsan.o rm -f obj/s7.tsan.o
@ -555,8 +560,8 @@ tsans7: | bin obj
# TSan (throwaway obj) so races are caught across the whole stack. # TSan (throwaway obj) so races are caught across the whole stack.
tsanwren: | bin obj tsanwren: | bin obj
$(CC) $(WRENFLAGS) $(WRENINC) -fsanitize=thread -c $(WRENDIR)/wren.c -o obj/wren.tsan.o $(CC) $(WRENFLAGS) $(WRENINC) -fsanitize=thread -c $(WRENDIR)/wren.c -o obj/wren.tsan.o
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(WRENINC) -o bin/testEngineWrenTsan \ $(CC) $(TSANFLAGS) $(INC) $(WRENINC) -o bin/testEngineWrenTsan \
tests/testEngineWren.c src/wren/wrenEngine.c src/wren/wrenAdapter.c src/context.c src/value.c src/broker.c \ tests/testEngineWren.c src/wren/wrenEngine.c src/wren/wrenAdapter.c $(TSANCORE) \
obj/wren.tsan.o $(WRENLIBS) obj/wren.tsan.o $(WRENLIBS)
setarch -R ./bin/testEngineWrenTsan setarch -R ./bin/testEngineWrenTsan
rm -f obj/wren.tsan.o rm -f obj/wren.tsan.o
@ -565,18 +570,12 @@ tsanwren: | bin obj
# callback fan-out, kv's shared store), each over a Lua context so callbacks marshal across # callback fan-out, kv's shared store), each over a Lua context so callbacks marshal across
# threads. The unsanitized liblua.a links in fine (TSan instruments only the calog code). # threads. The unsanitized liblua.a links in fine (TSan instruments only the calog code).
tsanlibs: lib/liblua.a | bin tsanlibs: lib/liblua.a | bin
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(LUAINC) -o bin/testTimerTsan \ for n in Timer Pubsub Kv; do \
tests/testTimer.c libs/calogTimer.c src/context.c src/value.c src/broker.c src/lua/luaEngine.c src/lua/luaAdapter.c \ $(CC) $(TSANFLAGS) $(INC) $(LUAINC) -o bin/test$${n}Tsan \
lib/liblua.a $(LUALIBS) tests/test$$n.c libs/calog$$n.c $(TSANCORE) src/lua/luaEngine.c src/lua/luaAdapter.c \
setarch -R ./bin/testTimerTsan lib/liblua.a $(LUALIBS) || exit 1; \
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(LUAINC) -o bin/testPubsubTsan \ setarch -R ./bin/test$${n}Tsan || exit 1; \
tests/testPubsub.c libs/calogPubsub.c src/context.c src/value.c src/broker.c src/lua/luaEngine.c src/lua/luaAdapter.c \ done
lib/liblua.a $(LUALIBS)
setarch -R ./bin/testPubsubTsan
$(CC) -std=c11 $(WARN) -g -O1 -fsanitize=thread -pthread $(INC) $(LUAINC) -o bin/testKvTsan \
tests/testKv.c libs/calogKv.c src/context.c src/value.c src/broker.c src/lua/luaEngine.c src/lua/luaAdapter.c \
lib/liblua.a $(LUALIBS)
setarch -R ./bin/testKvTsan
clean: clean:
rm -rf obj bin lib rm -rf obj bin lib
@ -584,4 +583,4 @@ clean:
-include $(wildcard obj/*.d) -include $(wildcard obj/*.d)
-include $(wildcard obj/rel/*.d) -include $(wildcard obj/rel/*.d)
.PHONY: all test tsan tsansq tsanjs tsanmb tsanberry tsans7 tsanwren clean .PHONY: all test tsan tsansq tsanjs tsanmb tsanberry tsans7 tsanwren tsanlibs clean

View file

@ -38,7 +38,7 @@ Swap `&calogJsEngine` for `&calogLuaEngine`, `&calogSquirrelEngine`,
responsive and drives everything from its own loop. responsive and drives everything from its own loop.
- **Callbacks both ways.** A script can hand you a function value to keep and call later - **Callbacks both ways.** A script can hand you a function value to keep and call later
(routed back to the engine that owns it); and you can hand a native to a script as a (routed back to the engine that owns it); and you can hand a native to a script as a
callable value with `calogFnFromNative` (every engine but MY-BASIC). callable value with `calogFnFromNative`.
- **Many runtimes.** Independent `CalogT` runtimes coexist in one process; one host - **Many runtimes.** Independent `CalogT` runtimes coexist in one process; one host
thread can drive several. thread can drive several.
- **Load by filename.** `calogContextLoad(calog, "config")` finds `config.lua` / - **Load by filename.** `calogContextLoad(calog, "config")` finds `config.lua` /
@ -55,7 +55,7 @@ Swap `&calogJsEngine` for `&calogLuaEngine`, `&calogSquirrelEngine`,
| Lua 5.4 | `calogLuaEngine` | `.lua` | | | Lua 5.4 | `calogLuaEngine` | `.lua` | |
| JavaScript | `calogJsEngine` | `.js` | QuickJS-ng (ES2023+, BigInt → int64) | | JavaScript | `calogJsEngine` | `.js` | QuickJS-ng (ES2023+, BigInt → int64) |
| Squirrel 3.2 | `calogSquirrelEngine` | `.nut` | C++ VM | | Squirrel 3.2 | `calogSquirrelEngine` | `.nut` | C++ VM |
| MY-BASIC | `calogMyBasicEngine` | `.bas` | our-basic fork; def/lambda callbacks; loads serialize | | MY-BASIC | `calogMyBasicEngine` | `.bas` | interpreters serialize at load |
| Scheme | `calogS7Engine` | `.scm` | s7; 64-bit ints | | Scheme | `calogS7Engine` | `.scm` | s7; 64-bit ints |
| Wren | `calogWrenEngine` | `.wren` | doubles only; call via `Calog.call(…)` | | Wren | `calogWrenEngine` | `.wren` | doubles only; call via `Calog.call(…)` |
| Berry | `calogBerryEngine` | `.be` | scalars + callbacks | | Berry | `calogBerryEngine` | `.be` | scalars + callbacks |
@ -64,10 +64,8 @@ A native can return a keyed `CalogValueT` record and **every** engine reads its
native syntax (`user.name`, `user["name"]`, `(user "name")`), and a script can hand a native syntax (`user.name`, `user["name"]`, `(user "name")`), and a script can hand a
list/map *back* to C on **every** engine. (Wren's C API can't enumerate a map's keys, so list/map *back* to C on **every** engine. (Wren's C API can't enumerate a map's keys, so
calog carries a small documented patch to the vendored Wren that adds one.) A host function calog carries a small documented patch to the vendored Wren that adds one.) A host function
value also crosses *into* a script on every engine but MY-BASIC (whose value model has no value also crosses *into* a script on **every** engine, where the script gets a callable it
slot for a *foreign* callable) -- though a MY-BASIC script can now hand its own `def`/lambda can invoke.
routines *out* to a native as a callback (our vendored my-basic fork adds first-class
routine values; see below).
You can also bring your own: `CalogEngineT` is a public four-function vtable. You can also bring your own: `CalogEngineT` is a public four-function vtable.
@ -293,7 +291,8 @@ calogFnRelease(savedCb);
It works the other way too: wrap one of your natives as a function value with It works the other way too: wrap one of your natives as a function value with
`calogFnFromNative` and return it from a native, and the script gets a callable it can `calogFnFromNative` and return it from a native, and the script gets a callable it can
invoke (which routes back to your host thread). Every engine but MY-BASIC supports this. invoke (which routes back to your host thread). Every engine supports this; a script invokes
the value directly, except in my-basic, where it is called with `calogInvoke(fn, ...args)`.
```c ```c
static int32_t getAdder(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t getAdder(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {

View file

@ -343,18 +343,26 @@ appends one if absent). While parked there, the loop holds a valid `l`, which it
`mb_get_routine` returns `MB_FUNC_OK` with a *nil* value when a name is absent, so the `mb_get_routine` returns `MB_FUNC_OK` with a *nil* value when a name is absent, so the
not-found test is `routine.type != MB_DT_ROUTINE`, not the status code. not-found test is `routine.type != MB_DT_ROUTINE`, not the status code.
**Update (calog's my-basic fork -- see `vendor/ourbasic/CHANGELOG`).** The live-frame For the callback direction the parked frame is not the only way in: the fork also provides
requirement is now lifted for the callback direction. The fork adds `mb_eval_routine_cold(s, routine, args, argc, ret)` (see `vendor/ourbasic/CHANGELOG`), which
`mb_eval_routine_cold(s, routine, args, argc, ret)`, which invokes a routine value from an invokes a routine value from an *idle* interpreter with no live `l` -- it supplies the last AST
*idle* interpreter: it supplies the last AST node as a clean return landing (the head node as a clean return landing (the head segfaults) and passes args directly. Combined with the
segfaults) and passes args directly, so no live `l` is needed. Combined with the bare-`def`-as-value evaluation (a routine identifier not followed by `(` yields the routine
bare-`def`-as-value evaluator patch (a routine identifier not followed by `(` yields the value instead of "Open bracket expected"), a my-basic script passes a top-level `def`/lambda to
routine value instead of "Open bracket expected"), a my-basic script can pass a top-level `psSubscribe` / `timerAfter` / `calogExport` and it is fired later, cross-engine included. The
`def`/lambda to `psSubscribe` / `timerAfter` / `calogExport` and have it fired later -- adapter invokes synchronously via the live-frame `mb_eval_routine` when a callback runs inside a
including cross-engine -- like the other engines. The adapter invokes synchronously via the serving native call, and via `mb_eval_routine_cold` when it is delivered to an idle context. (A
live-frame `mb_eval_routine` when a callback runs inside a serving native call, and via callback must be a top-level routine; one local to a function dangles once that function returns.)
`mb_eval_routine_cold` when it is delivered to an idle context. (A callback must be a
top-level routine; one local to a function still dangles once that function returns.) The same fork also gives my-basic bare-name resolution for exports. A third add-only patch adds
an `mb_dynamic_func_handler` hook (a field on the interpreter plus `mb_set_dynamic_func_handler`):
at the "invalid identifier usage" site in `_calc_expression` -- a bare name called like a function
that is neither variable nor routine nor collection -- the interpreter offers the name to the
handler before erroring. The adapter's handler resolves it against the export registry
(case-insensitively, since my-basic uppercases identifiers at parse time) via a fold variant of
the export resolver, and invokes it. So a BASIC script calls `exportedFn(args)` directly, without
an explicit `calogCall` -- the convenience the hook engines (Lua/JS/Squirrel/s7) get from their
unknown-name hooks. Wren and Berry, whose C APIs offer no such hook, still use `calogCall`.
### 5.6 Numeric and identity caveats ### 5.6 Numeric and identity caveats
@ -922,7 +930,7 @@ natives route through **one generic `%calog-call` dispatcher** plus a per-name S
wrapper (`(define (report . a) (apply %calog-call "report" a))`); the context rides on a wrapper (`(define (report . a) (apply %calog-call "report" a))`); the context rides on a
`*calog-context*` c-pointer global. Callables are kept alive by `s7_gc_protect` and `*calog-context*` c-pointer global. Callables are kept alive by `s7_gc_protect` and
invoked with `s7_call`. Because `s7_eval_c_string` evaluates a single form, `calogS7Run` invoked with `s7_call`. Because `s7_eval_c_string` evaluates a single form, `calogS7Run`
wraps the (escaped) source in `(catch #t (lambda () (eval-string )) handler)`, so both wraps the (escaped) source in `(catch #t (lambda () (eval-string ...)) handler)`, so both
read and run errors surface as a value -- a marker pair the runner detects. An aggregate read and run errors surface as a value -- a marker pair the runner detects. An aggregate
crossing out is a Scheme list, or an (applicable) hash-table when keyed, so a materialized crossing out is a Scheme list, or an (applicable) hash-table when keyed, so a materialized
record reads as `(user "name")`; reading a script's keyed value back in is a v1 limit. s7's intentional record reads as `(user "name")`; reading a script's keyed value back in is a v1 limit. s7's intentional
@ -985,10 +993,11 @@ Per engine:
`call` takes a *list* (Wren method arity is fixed, so a list absorbs any argument count); `call` takes a *list* (Wren method arity is fixed, so a list absorbs any argument count);
finalize releases. Script calls `f.call([...])`. Gotcha: Wren requires newlines between finalize releases. Script calls `f.call([...])`. Gotcha: Wren requires newlines between
class members, so the preamble is multi-line. class members, so the preamble is multi-line.
- **MY-BASIC**: gap -- my-basic's value model has no slot for a *foreign* (host) callable, - **MY-BASIC**: a host callable becomes a refcounted usertype-ref (`mb_make_ref_value`, whose
so `calogFnFromNative` can't hand a native *into* a my-basic script. (The reverse now dtor drops the retained `CalogFnT`); a script invokes it with `calogInvoke(fn, ...args)`,
works: calog's my-basic fork makes `def`/lambda routines first-class values a script can which marshals the args, calls the callable, and marshals the result back. (A script also
pass *out* to a native and have invoked later via `mb_eval_routine_cold` -- see sec 5.5.) passes its own `def`/lambda *out* to a native, invoked later via `mb_eval_routine_cold`; see
sec 5.5.)
**Aggregate ingress** (`*ToValue` map/list): Lua/JS/Squirrel/MY-BASIC already read both. **Aggregate ingress** (`*ToValue` map/list): Lua/JS/Squirrel/MY-BASIC already read both.
Added: Added:

View file

@ -49,13 +49,16 @@ Conventions every example follows:
| `fs.lua` | fs: mkdir/write/append/read/stat/list/remove, all in a unique `/tmp` dir | | `fs.lua` | fs: mkdir/write/append/read/stat/list/remove, all in a unique `/tmp` dir |
| `time.lua` | time: `timeNow`, and measuring a `timeSleep` with `timeMonotonic` | | `time.lua` | time: `timeNow`, and measuring a `timeSleep` with `timeMonotonic` |
| `timer.lua` | timer (Lua callbacks): a `timerAfter` one-shot + a self-cancelling `timerEvery` | | `timer.lua` | timer (Lua callbacks): a `timerAfter` one-shot + a self-cancelling `timerEvery` |
| `timerInMyBasic.bas` | timer callbacks **in my-basic** -- the our-basic fork makes `def`/lambda first-class | | `timerInMyBasic.bas` | timer callbacks in my-basic: a `def` on a repeating, self-cancelling timer |
| `database.lua` | db: an in-memory SQLite table, bound-parameter inserts, a query + an aggregate | | `database.lua` | db: an in-memory SQLite table, bound-parameter inserts, a query + an aggregate |
| `export.lua` | export: publish a function, call it by name (and, on Lua, by bare name) | | `export.lua` | export: publish a function, call it by name (and, on Lua, by bare name) |
| `pubsub.lua` | pubsub: two handlers subscribe, publish delivers to both, then unsubscribe | | `pubsub.lua` | pubsub: two handlers subscribe, publish delivers to both, then unsubscribe |
| `task.lua` | task: `taskSelf`, spawn a sibling child, `taskCount` | | `task.lua` | task: `taskSelf`, spawn a sibling child, `taskCount` |
| `taskCooperative.lua` | task: a worker loop that polls `taskActive()` so `taskClose` stops it cleanly |
| `taskSelfExit.lua` | task: a worker that retires its own context with `taskExit()` (deferred; owner-free) |
| `net.lua` | net: a UDP echo over loopback between a spawned listener and a sender | | `net.lua` | net: a UDP echo over loopback between a spawned listener and a sender |
| `http.lua` | http: self-contained -- a spawned child runs a one-shot HTTP/1.1 server, then `httpGet`s it | | `http.lua` | http: self-contained -- a spawned child runs a one-shot HTTP/1.1 server, then `httpGet`s it |
| `httpRedirect.lua` | http: `httpGet` follows a 302 redirect (2-hop in-process server) |
| `ssh.lua` | ssh/sftp: a documented, safe-to-run template (prints guidance if no server is configured) | | `ssh.lua` | ssh/sftp: a documented, safe-to-run template (prints guidance if no server is configured) |
## `polyglot/` -- several languages sharing one runtime ## `polyglot/` -- several languages sharing one runtime
@ -66,6 +69,8 @@ Conventions every example follows:
| `pubsubAcrossEngines.lua` | Lua subscribes to a topic; a spawned **JavaScript** task publishes to it | | `pubsubAcrossEngines.lua` | Lua subscribes to a topic; a spawned **JavaScript** task publishes to it |
| `sharedKv.lua` | Lua seeds the shared kv store; a spawned **JavaScript** task reads it back and mutates it | | `sharedKv.lua` | Lua seeds the shared kv store; a spawned **JavaScript** task reads it back and mutates it |
| `taskFanout.lua` | one Lua script spawns a task on **five** different engines, each printing in its own language | | `taskFanout.lua` | one Lua script spawns a task on **five** different engines, each printing in its own language |
| `foreignCallable.lua` | a Lua closure is handed to a my-basic script as a value and invoked with `calogInvoke` |
| `bareExportInMyBasic.lua` | a Lua export is called from my-basic by its **bare name** (no `calogCall`), like the hook engines |
## `multifile/` -- multiple script files in one run ## `multifile/` -- multiple script files in one run

View file

@ -0,0 +1,25 @@
-- httpGet follows 3xx redirects (up to 16, to break loops). This spins up a tiny 2-hop server
-- entirely in-process: the first request gets a 302 to /final, which serves the page.
-- run: bin/calog examples/scripts/libraries/httpRedirect.lua
local server = taskSpawn("lua", [[
local ls = tcpListen(46080)
kvSet("ready", "1")
local c = tcpAccept(ls) -- hop 1: redirect
tcpRecv(c, 65536)
tcpSend(c, "HTTP/1.1 302 Found\r\nLocation: /final\r\nContent-Length: 0\r\nConnection: close\r\n\r\n")
tcpClose(c)
c = tcpAccept(ls) -- hop 2: the destination
tcpRecv(c, 65536)
local body = "arrived after a redirect"
tcpSend(c, "HTTP/1.1 200 OK\r\nContent-Length: " .. #body .. "\r\nConnection: close\r\n\r\n" .. body)
tcpClose(c)
tcpClose(ls)
]])
while not kvHas("ready") do timeSleep(10) end
timeSleep(20)
local r = httpGet("http://127.0.0.1:46080/start")
calogPrint("followed the redirect -> status", r.status .. ":", r.body)
taskClose(server)
calogExit(0)

View file

@ -0,0 +1,21 @@
-- Cooperative task shutdown with taskActive().
-- taskClose() is cooperative: it flips the task's shutdown flag and waits for the thread to
-- exit. A task busy in its own loop won't notice unless it polls taskActive(), so a
-- long-running worker checks it each iteration and breaks out when asked to stop.
-- run: bin/calog examples/scripts/libraries/taskCooperative.lua
local worker = taskSpawn("lua", [[
local ticks = 0
while taskActive() do -- false once the owner calls taskClose()
ticks = ticks + 1
timeSleep(25)
end
kvSet("workerTicks", ticks) -- runs after the loop breaks, before the thread exits
]])
calogPrint("owner: worker spawned, taskActive(handle) =", tostring(taskActive(worker)))
timeSleep(150) -- let it work for a bit
calogPrint("owner: asking the worker to stop")
taskClose(worker) -- returns as soon as the worker breaks its loop
calogPrint("owner: worker stopped after", kvGet("workerTicks"), "ticks")
calogExit(0)

View file

@ -0,0 +1,20 @@
-- Self-retiring task with taskExit().
-- A task that has finished its work can release its OWN context -- it need not wait for the
-- owner to taskClose() it. taskExit() is deferred: the rest of the current code runs, then the
-- context stops and the runtime reaps it. The owner just sees taskCount() drop.
-- run: bin/calog examples/scripts/libraries/taskSelfExit.lua
local worker = taskSpawn("lua", [[
calogPrint("worker: computing...")
local sum = 0
for i = 1, 5 do sum = sum + i end
calogPrint("worker: done (sum =", sum, "); retiring myself")
taskExit() -- retire my own context once this code finishes
]])
calogPrint("main: spawned a worker, taskCount =", taskCount())
while taskCount() > 0 do -- the worker self-exits; the runtime reaps it
timeSleep(20)
end
calogPrint("main: the worker retired itself, taskCount =", taskCount())
calogExit(0)

View file

@ -1,7 +1,5 @@
' Timer callbacks in my-basic. calog's my-basic fork (vendor/ourbasic) makes def/lambda ' Timer callbacks in my-basic: a def is armed on a repeating timer, counts ticks in a global,
' routines first-class values, so a BASIC script can hand a callback to a native just like ' and after the third tick cancels its own timer and ends the run.
' every other engine. Here a def is armed on a repeating timer; it counts ticks in a
' global, and after the third tick cancels its own timer and ends the run.
' run: bin/calog examples/scripts/libraries/timerInMyBasic.bas ' run: bin/calog examples/scripts/libraries/timerInMyBasic.bas
n = 0 n = 0

View file

@ -0,0 +1,20 @@
-- An exported function is callable by its BARE NAME in my-basic -- exportedFn(args), no explicit
-- calogCall -- the same convenience the hook engines (Lua/JS/Squirrel/s7) get. Resolution is
-- case-insensitive there (my-basic uppercases identifiers). Wren and Berry still use calogCall.
-- run: bin/calog examples/scripts/polyglot/bareExportInMyBasic.lua
local helper = taskSpawn("lua", [[
calogExport("shout", function(s) return string.upper(tostring(s)) .. "!" end)
kvSet("ready", "1")
]])
while not kvHas("ready") do timeSleep(10) end
local mb = taskSpawn("mybasic", [[
' 'shout' is a Lua export, invoked here by its bare name -- no calogCall
kvSet("out", shout("hello from basic"))
]])
while not kvHas("out") do timeSleep(20) end
calogPrint("my-basic bare-called the 'shout' export ->", kvGet("out"))
taskClose(mb); taskClose(helper)
calogExit(kvGet("out") == "HELLO FROM BASIC!" and 0 or 1)

View file

@ -0,0 +1,22 @@
-- A function value can cross INTO any engine and be invoked there. Here Lua exports a
-- higher-order function; a my-basic script receives the returned closure as a value and calls
-- it with calogInvoke(fn, ...). (Every other engine invokes such a value directly, e.g. fn(x).)
-- run: bin/calog examples/scripts/polyglot/foreignCallable.lua
local helper = taskSpawn("lua", [[
calogExport("makeGreeter", function()
return function(name) return "hello, " .. tostring(name) .. "!" end
end)
kvSet("helper_ready", "1")
]])
while not kvHas("helper_ready") do timeSleep(10) end
local mb = taskSpawn("mybasic", [[
greet = calogCall("makeGreeter") ' a Lua closure, held as a my-basic value
kvSet("greeting", calogInvoke(greet, "my-basic"))
]])
while not kvHas("greeting") do timeSleep(20) end
calogPrint("my-basic invoked the Lua closure ->", kvGet("greeting"))
taskClose(mb); taskClose(helper)
calogExit(0)

View file

@ -5,6 +5,7 @@
// of its arguments. // of its arguments.
#include "calogCrypto.h" #include "calogCrypto.h"
#include "calogInternal.h"
#include <limits.h> #include <limits.h>
#include <stdint.h> #include <stdint.h>
@ -22,11 +23,14 @@ static int32_t cryptoHashSha1Native(CalogValueT *args, int32_t argCount, CalogVa
static int32_t cryptoHashSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t cryptoHashSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t cryptoHexDecodeNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t cryptoHexDecodeNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t cryptoHexEncodeNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t cryptoHexEncodeNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t cryptoHexNibble(unsigned char c);
static int32_t cryptoHmacSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t cryptoHmacSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t cryptoRandomBytesNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t cryptoRandomBytesNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t cryptoUuidNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t cryptoUuidNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
// The single source of truth for lowercase hex digit rendering, shared by cryptoBytesToHex
// and cryptoUuidNative.
static const char cryptoHexDigits[] = "0123456789abcdef";
int32_t calogCryptoRegister(CalogT *calog) { int32_t calogCryptoRegister(CalogT *calog) {
calogRegisterInline(calog, "cryptoHashSha256", cryptoHashSha256Native, NULL); calogRegisterInline(calog, "cryptoHashSha256", cryptoHashSha256Native, NULL);
@ -76,6 +80,30 @@ static int32_t cryptoBase64DecodeNative(CalogValueT *args, int32_t argCount, Cal
if ((length % 4) != 0 || length > INT_MAX) { if ((length % 4) != 0 || length > INT_MAX) {
return calogFail(result, calogErrArgE, "cryptoBase64Decode: invalid base64 length"); return calogFail(result, calogErrArgE, "cryptoBase64Decode: invalid base64 length");
} }
// EVP_DecodeBlock treats '=' as a zero sextet no matter where it appears, so without this
// check an interior '=' (e.g. "YQ==YQ==") would decode as silent garbage instead of failing.
// Valid padding is at most the final two characters, and if present must run to the end.
{
int64_t firstPad;
int64_t padIndex;
firstPad = -1;
for (padIndex = 0; padIndex < length; padIndex++) {
if (in[padIndex] == '=') {
firstPad = padIndex;
break;
}
}
if (firstPad >= 0) {
if (firstPad < length - 2) {
return calogFail(result, calogErrArgE, "cryptoBase64Decode: invalid base64 data");
}
for (padIndex = firstPad; padIndex < length; padIndex++) {
if (in[padIndex] != '=') {
return calogFail(result, calogErrArgE, "cryptoBase64Decode: invalid base64 data");
}
}
}
}
outCap = ((size_t)length / 4) * 3; outCap = ((size_t)length / 4) * 3;
out = (unsigned char *)malloc(outCap); out = (unsigned char *)malloc(outCap);
if (out == NULL) { if (out == NULL) {
@ -116,7 +144,9 @@ static int32_t cryptoBase64EncodeNative(CalogValueT *args, int32_t argCount, Cal
if (argCount != 1 || args[0].type != calogStringE) { if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "cryptoBase64Encode expects (data)"); return calogFail(result, calogErrArgE, "cryptoBase64Encode expects (data)");
} }
if (args[0].as.s.length > INT_MAX) { // EVP_EncodeBlock returns its output length as an int, accumulated in units of 4 bytes per
// 3 input bytes; inputs beyond (INT_MAX / 4) * 3 bytes would overflow that return value.
if (args[0].as.s.length > (INT_MAX / 4) * 3) {
return calogFail(result, calogErrRangeE, "cryptoBase64Encode: input too large"); return calogFail(result, calogErrRangeE, "cryptoBase64Encode: input too large");
} }
inLen = (size_t)args[0].as.s.length; inLen = (size_t)args[0].as.s.length;
@ -139,7 +169,6 @@ static int32_t cryptoBase64EncodeNative(CalogValueT *args, int32_t argCount, Cal
static int32_t cryptoBytesToHex(CalogValueT *result, const unsigned char *bytes, size_t length) { static int32_t cryptoBytesToHex(CalogValueT *result, const unsigned char *bytes, size_t length) {
static const char digits[] = "0123456789abcdef";
char *hex; char *hex;
size_t index; size_t index;
int32_t status; int32_t status;
@ -154,8 +183,8 @@ static int32_t cryptoBytesToHex(CalogValueT *result, const unsigned char *bytes,
for (index = 0; index < length; index++) { for (index = 0; index < length; index++) {
unsigned char byte; unsigned char byte;
byte = bytes[index]; byte = bytes[index];
hex[index * 2] = digits[byte >> 4]; hex[index * 2] = cryptoHexDigits[byte >> 4];
hex[index * 2 + 1] = digits[byte & 0x0F]; hex[index * 2 + 1] = cryptoHexDigits[byte & 0x0F];
} }
status = calogValueString(result, hex, (int64_t)(length * 2)); status = calogValueString(result, hex, (int64_t)(length * 2));
free(hex); free(hex);
@ -220,8 +249,8 @@ static int32_t cryptoHexDecodeNative(CalogValueT *args, int32_t argCount, CalogV
for (index = 0; index < outLen; index++) { for (index = 0; index < outLen; index++) {
int32_t hi; int32_t hi;
int32_t lo; int32_t lo;
hi = cryptoHexNibble(in[index * 2]); hi = calogHexNibble(in[index * 2]);
lo = cryptoHexNibble(in[index * 2 + 1]); lo = calogHexNibble(in[index * 2 + 1]);
if (hi < 0 || lo < 0) { if (hi < 0 || lo < 0) {
free(out); free(out);
return calogFail(result, calogErrArgE, "cryptoHexDecode: invalid hex digit"); return calogFail(result, calogErrArgE, "cryptoHexDecode: invalid hex digit");
@ -247,20 +276,6 @@ static int32_t cryptoHexEncodeNative(CalogValueT *args, int32_t argCount, CalogV
} }
static int32_t cryptoHexNibble(unsigned char c) {
if (c >= '0' && c <= '9') {
return (int32_t)(c - '0');
}
if (c >= 'a' && c <= 'f') {
return (int32_t)(c - 'a' + 10);
}
if (c >= 'A' && c <= 'F') {
return (int32_t)(c - 'A' + 10);
}
return -1;
}
static int32_t cryptoHmacSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t cryptoHmacSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
unsigned char digest[EVP_MAX_MD_SIZE]; unsigned char digest[EVP_MAX_MD_SIZE];
unsigned int digestLen; unsigned int digestLen;
@ -316,7 +331,6 @@ static int32_t cryptoRandomBytesNative(CalogValueT *args, int32_t argCount, Calo
static int32_t cryptoUuidNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t cryptoUuidNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
static const char digits[] = "0123456789abcdef";
unsigned char bytes[16]; unsigned char bytes[16];
char text[37]; char text[37];
int32_t bi; int32_t bi;
@ -338,8 +352,8 @@ static int32_t cryptoUuidNative(CalogValueT *args, int32_t argCount, CalogValueT
if (bi == 4 || bi == 6 || bi == 8 || bi == 10) { if (bi == 4 || bi == 6 || bi == 8 || bi == 10) {
text[ti++] = '-'; text[ti++] = '-';
} }
text[ti++] = digits[bytes[bi] >> 4]; text[ti++] = cryptoHexDigits[bytes[bi] >> 4];
text[ti++] = digits[bytes[bi] & 0x0F]; text[ti++] = cryptoHexDigits[bytes[bi] & 0x0F];
} }
text[ti] = '\0'; text[ti] = '\0';
return calogValueString(result, text, 36); return calogValueString(result, text, 36);

View file

@ -8,6 +8,7 @@
#define _POSIX_C_SOURCE 200809L #define _POSIX_C_SOURCE 200809L
#include "calogDb.h" #include "calogDb.h"
#include "calogInternal.h"
#include "calogHandle.h" #include "calogHandle.h"
@ -46,6 +47,10 @@
#define DB_NUM_SCRATCH 32 #define DB_NUM_SCRATCH 32
// Cap on a mysql "key=value ..." conninfo string; longer input is rejected outright rather
// than silently truncated mid-token.
#define MYSQL_CONNINFO_CAP 512
// Process-wide DB library state: one connection registry shared by every runtime that // Process-wide DB library state: one connection registry shared by every runtime that
// registers the natives (handles are globally-unique ints, so sharing is safe). refCount is // registers the natives (handles are globally-unique ints, so sharing is safe). refCount is
// one per registered runtime, so the singleton is freed only when the last one shuts down. // one per registered runtime, so the singleton is freed only when the last one shuts down.
@ -57,26 +62,38 @@ typedef struct DbLibT {
static pthread_mutex_t gDbLibMutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t gDbLibMutex = PTHREAD_MUTEX_INITIALIZER;
static DbLibT *gDbLib = NULL; static DbLibT *gDbLib = NULL;
// A query row's columns are marshalled through this pair of callbacks, shared by every
// backend's row loop (dbMarshalRow): nameFn hands back the column name (never fails), valueFn
// marshals the column into *out (already nil'd), returning an error status on OOM.
typedef void (*DbColumnNameFnT)(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut);
typedef int32_t (*DbColumnValueFnT)(void *ctx, int32_t column, CalogValueT *out);
static int32_t dbClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t dbClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void dbCloser(uint32_t type, void *resource); static void dbCloser(uint32_t type, void *resource);
static int32_t dbExec(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t dbExec(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t dbFinishOpen(DbLibT *lib, uint32_t type, void *conn, CalogValueT *result);
static int32_t dbMarshalRow(CalogValueT *result, CalogAggT *rows, void *ctx, int32_t columnCount, DbColumnNameFnT nameFn, DbColumnValueFnT valueFn);
static int32_t dbOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t dbOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t dbQuery(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t dbQuery(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static uint32_t dbResolve(DbLibT *lib, int64_t handle, void **connOut); static uint32_t dbResolve(DbLibT *lib, int64_t handle, void **connOut);
#ifdef CALOG_WITH_MYSQL #ifdef CALOG_WITH_MYSQL
static void mysqlCleanup(MYSQL_BIND *binds, char **buffers, unsigned long *lengths, my_bool *isNull, my_bool *errors, unsigned int columnCount, MYSQL_RES *meta, MYSQL_STMT *stmt); static void mysqlCleanup(MYSQL_BIND *binds, char **buffers, unsigned long *lengths, my_bool *isNull, unsigned int columnCount, MYSQL_RES *meta, MYSQL_STMT *stmt);
static int32_t mysqlColumn(enum enum_field_types type, const char *bytes, unsigned long length, CalogValueT *out); static int32_t mysqlColumn(enum enum_field_types type, const char *bytes, unsigned long length, CalogValueT *out);
static int32_t mysqlExec(MYSQL *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result); static int32_t mysqlExec(MYSQL *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
static int32_t mysqlOpen(DbLibT *lib, const char *conninfo, CalogValueT *result); static int32_t mysqlOpen(DbLibT *lib, const char *conninfo, CalogValueT *result);
static MYSQL_STMT *mysqlPrepare(MYSQL *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result); static MYSQL_STMT *mysqlPrepare(MYSQL *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result); static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
static void mysqlRowName(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut);
static int32_t mysqlRowValue(void *ctx, int32_t column, CalogValueT *out);
#endif #endif
#ifdef CALOG_WITH_PG #ifdef CALOG_WITH_PG
static int32_t pgColumn(PGresult *res, int row, int column, CalogValueT *out); static int32_t pgColumn(PGresult *res, int row, int column, CalogValueT *out);
static int32_t pgExec(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result); static int32_t pgExec(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
static int32_t pgOpen(DbLibT *lib, const char *conninfo, CalogValueT *result); static int32_t pgOpen(DbLibT *lib, const char *conninfo, CalogValueT *result);
static int32_t pgQuery(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result); static int32_t pgQuery(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
static PGresult *pgRun(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, int resultFormat, CalogValueT *result); static void pgRowName(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut);
static int32_t pgRowValue(void *ctx, int32_t column, CalogValueT *out);
static PGresult *pgRun(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
#endif #endif
#ifdef CALOG_WITH_SQLITE #ifdef CALOG_WITH_SQLITE
static int32_t sqliteColumn(sqlite3_stmt *stmt, int column, CalogValueT *out); static int32_t sqliteColumn(sqlite3_stmt *stmt, int column, CalogValueT *out);
@ -84,10 +101,14 @@ static int32_t sqliteExec(sqlite3 *db, const CalogValueT *sqlValue, const CalogV
static int32_t sqliteOpen(DbLibT *lib, const char *path, CalogValueT *result); static int32_t sqliteOpen(DbLibT *lib, const char *path, CalogValueT *result);
static int32_t sqlitePrepare(sqlite3 *db, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, sqlite3_stmt **out, CalogValueT *result); static int32_t sqlitePrepare(sqlite3 *db, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, sqlite3_stmt **out, CalogValueT *result);
static int32_t sqliteQuery(sqlite3 *db, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result); static int32_t sqliteQuery(sqlite3 *db, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result);
static void sqliteRowName(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut);
static int32_t sqliteRowValue(void *ctx, int32_t column, CalogValueT *out);
#endif #endif
int32_t calogDbRegister(CalogT *calog) { int32_t calogDbRegister(CalogT *calog) {
int32_t status;
pthread_mutex_lock(&gDbLibMutex); pthread_mutex_lock(&gDbLibMutex);
if (gDbLib == NULL) { if (gDbLib == NULL) {
DbLibT *lib; DbLibT *lib;
@ -106,11 +127,22 @@ int32_t calogDbRegister(CalogT *calog) {
} }
gDbLib->refCount++; gDbLib->refCount++;
pthread_mutex_unlock(&gDbLibMutex); pthread_mutex_unlock(&gDbLibMutex);
calogRegisterInline(calog, "dbOpen", dbOpen, gDbLib); status = calogRegisterInline(calog, "dbOpen", dbOpen, gDbLib);
calogRegisterInline(calog, "dbExec", dbExec, gDbLib); if (status == calogOkE) {
calogRegisterInline(calog, "dbQuery", dbQuery, gDbLib); status = calogRegisterInline(calog, "dbExec", dbExec, gDbLib);
calogRegisterInline(calog, "dbClose", dbClose, gDbLib); }
return calogOkE; if (status == calogOkE) {
status = calogRegisterInline(calog, "dbQuery", dbQuery, gDbLib);
}
if (status == calogOkE) {
status = calogRegisterInline(calog, "dbClose", dbClose, gDbLib);
}
if (status != calogOkE) {
// Roll back the refCount taken above so a partially-registered runtime does not
// keep the process-wide registry alive past its own shutdown.
calogDbShutdown();
}
return status;
} }
@ -140,13 +172,9 @@ static int32_t dbClose(CalogValueT *args, int32_t argCount, CalogValueT *result,
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "dbClose expects a connection handle"); return calogFail(result, calogErrArgE, "dbClose expects a connection handle");
} }
backend = dbResolve(lib, args[0].as.i, &conn); // A single atomic remove yields the connection and its backend tag; a concurrent dbClose
if (backend == 0) { // of the same handle gets NULL here and must not double-close.
return calogFail(result, calogErrArgE, "dbClose: invalid connection handle"); conn = calogHandleRemoveAny(lib->handles, args[0].as.i, &backend);
}
// Close only the connection that this call atomically removed from the table -- a
// concurrent dbClose of the same handle gets NULL here and must not double-close.
conn = calogHandleRemove(lib->handles, args[0].as.i, backend);
if (conn == NULL) { if (conn == NULL) {
return calogFail(result, calogErrArgE, "dbClose: invalid connection handle"); return calogFail(result, calogErrArgE, "dbClose: invalid connection handle");
} }
@ -208,6 +236,69 @@ static int32_t dbExec(CalogValueT *args, int32_t argCount, CalogValueT *result,
} }
// Finish a successful backend connect: register it in the handle table (or close it back
// out on out-of-memory) and set result to the new handle. Shared tail of sqliteOpen,
// pgOpen, and mysqlOpen.
static int32_t dbFinishOpen(DbLibT *lib, uint32_t type, void *conn, CalogValueT *result) {
int64_t handle;
handle = calogHandleAdd(lib->handles, type, conn);
if (handle == 0) {
dbCloser(type, conn);
return calogFail(result, calogErrOomE, "dbOpen: out of memory");
}
calogValueInt(result, handle);
return calogOkE;
}
// Marshal one result row into a {columnName: value} map and push it onto rows. Shared row
// loop body of mysqlQuery/pgQuery/sqliteQuery: nameFn/valueFn hide the backend-specific
// column access behind ctx, so all three copies of this ~20-line block (and the error-unwind
// paths that came with them, including the uninitialized-value free bug the copies had
// drifted into) collapse to one.
static int32_t dbMarshalRow(CalogValueT *result, CalogAggT *rows, void *ctx, int32_t columnCount, DbColumnNameFnT nameFn, DbColumnValueFnT valueFn) {
CalogAggT *rowMap;
CalogValueT rowValue;
int32_t column;
int32_t status;
status = calogAggCreate(&rowMap, calogMapE);
if (status != calogOkE) {
return calogFail(result, status, "dbQuery: out of memory");
}
for (column = 0; column < columnCount; column++) {
CalogValueT key;
CalogValueT value;
const char *name;
int64_t nameLength;
calogValueNil(&value);
nameFn(ctx, column, &name, &nameLength);
status = calogValueString(&key, name, nameLength);
if (status == calogOkE) {
status = valueFn(ctx, column, &value);
}
if (status == calogOkE) {
status = calogAggSet(rowMap, &key, &value);
}
if (status != calogOkE) {
calogValueFree(&key);
calogValueFree(&value);
calogAggFree(rowMap);
return calogFail(result, status, "dbQuery: failed to build a row");
}
}
calogValueAgg(&rowValue, rowMap);
status = calogAggPush(rows, &rowValue);
if (status != calogOkE) {
calogValueFree(&rowValue);
return calogFail(result, status, "dbQuery: out of memory");
}
return calogOkE;
}
static int32_t dbOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t dbOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
DbLibT *lib; DbLibT *lib;
const char *driver; const char *driver;
@ -269,32 +360,12 @@ static int32_t dbQuery(CalogValueT *args, int32_t argCount, CalogValueT *result,
// Resolve a handle to its backend type tag and connection pointer, or 0 if not found. // Resolve a handle to its backend type tag and connection pointer, or 0 if not found.
static uint32_t dbResolve(DbLibT *lib, int64_t handle, void **connOut) { static uint32_t dbResolve(DbLibT *lib, int64_t handle, void **connOut) {
void *conn; uint32_t type;
*connOut = NULL; // One locked scan yields both the connection and its backend tag; this table holds only
#ifdef CALOG_WITH_SQLITE // this library's own connections, so the returned tag is trustworthy.
conn = calogHandleGet(lib->handles, handle, DB_TYPE_SQLITE); *connOut = calogHandleGetAny(lib->handles, handle, &type);
if (conn != NULL) { return type;
*connOut = conn;
return DB_TYPE_SQLITE;
}
#endif
#ifdef CALOG_WITH_PG
conn = calogHandleGet(lib->handles, handle, DB_TYPE_PG);
if (conn != NULL) {
*connOut = conn;
return DB_TYPE_PG;
}
#endif
#ifdef CALOG_WITH_MYSQL
conn = calogHandleGet(lib->handles, handle, DB_TYPE_MYSQL);
if (conn != NULL) {
*connOut = conn;
return DB_TYPE_MYSQL;
}
#endif
(void)conn;
return 0;
} }
@ -307,9 +378,18 @@ typedef union MysqlScalarT {
} MysqlScalarT; } MysqlScalarT;
// Free everything a query allocated: the per-column buffers, the four parallel arrays, the // Row context bound to mysqlRowName/mysqlRowValue for one mysqlQuery call.
typedef struct MysqlRowCtxT {
MYSQL_FIELD *fields;
char **buffers;
unsigned long *lengths;
my_bool *isNull;
} MysqlRowCtxT;
// Free everything a query allocated: the per-column buffers, the three parallel arrays, the
// result metadata, and the statement. Each argument may be NULL. // result metadata, and the statement. Each argument may be NULL.
static void mysqlCleanup(MYSQL_BIND *binds, char **buffers, unsigned long *lengths, my_bool *isNull, my_bool *errors, unsigned int columnCount, MYSQL_RES *meta, MYSQL_STMT *stmt) { static void mysqlCleanup(MYSQL_BIND *binds, char **buffers, unsigned long *lengths, my_bool *isNull, unsigned int columnCount, MYSQL_RES *meta, MYSQL_STMT *stmt) {
unsigned int column; unsigned int column;
if (buffers != NULL) { if (buffers != NULL) {
@ -321,7 +401,6 @@ static void mysqlCleanup(MYSQL_BIND *binds, char **buffers, unsigned long *lengt
free(buffers); free(buffers);
free(lengths); free(lengths);
free(isNull); free(isNull);
free(errors);
if (meta != NULL) { if (meta != NULL) {
mysql_free_result(meta); mysql_free_result(meta);
} }
@ -370,8 +449,7 @@ static int32_t mysqlExec(MYSQL *conn, const CalogValueT *sqlValue, const CalogVa
// uniform: host, user, password, dbname, port, socket. // uniform: host, user, password, dbname, port, socket.
static int32_t mysqlOpen(DbLibT *lib, const char *conninfo, CalogValueT *result) { static int32_t mysqlOpen(DbLibT *lib, const char *conninfo, CalogValueT *result) {
MYSQL *conn; MYSQL *conn;
int64_t handle; char buffer[MYSQL_CONNINFO_CAP];
char buffer[512];
char *host; char *host;
char *user; char *user;
char *password; char *password;
@ -392,6 +470,9 @@ static int32_t mysqlOpen(DbLibT *lib, const char *conninfo, CalogValueT *result)
sslmode = NULL; sslmode = NULL;
saveptr = NULL; saveptr = NULL;
port = 0; port = 0;
if (strlen(conninfo) >= sizeof(buffer)) {
return calogFail(result, calogErrArgE, "dbOpen: mysql connection string too long");
}
snprintf(buffer, sizeof(buffer), "%s", conninfo); snprintf(buffer, sizeof(buffer), "%s", conninfo);
token = strtok_r(buffer, " ", &saveptr); token = strtok_r(buffer, " ", &saveptr);
while (token != NULL) { while (token != NULL) {
@ -450,13 +531,7 @@ static int32_t mysqlOpen(DbLibT *lib, const char *conninfo, CalogValueT *result)
mysql_close(conn); mysql_close(conn);
return status; return status;
} }
handle = calogHandleAdd(lib->handles, DB_TYPE_MYSQL, conn); return dbFinishOpen(lib, DB_TYPE_MYSQL, conn, result);
if (handle == 0) {
mysql_close(conn);
return calogFail(result, calogErrOomE, "dbOpen: out of memory");
}
calogValueInt(result, handle);
return calogOkE;
} }
@ -561,9 +636,9 @@ static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogV
char **buffers; char **buffers;
unsigned long *lengths; unsigned long *lengths;
my_bool *isNull; my_bool *isNull;
my_bool *errors;
my_bool updateMaxLength; my_bool updateMaxLength;
CalogAggT *rows; CalogAggT *rows;
MysqlRowCtxT rowCtx;
unsigned int columnCount; unsigned int columnCount;
unsigned int column; unsigned int column;
int32_t status; int32_t status;
@ -592,7 +667,7 @@ static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogV
mysql_stmt_attr_set(stmt, STMT_ATTR_UPDATE_MAX_LENGTH, &updateMaxLength); mysql_stmt_attr_set(stmt, STMT_ATTR_UPDATE_MAX_LENGTH, &updateMaxLength);
if (mysql_stmt_store_result(stmt) != 0) { if (mysql_stmt_store_result(stmt) != 0) {
status = calogFail(result, calogErrArgE, mysql_stmt_error(stmt)); status = calogFail(result, calogErrArgE, mysql_stmt_error(stmt));
mysqlCleanup(NULL, NULL, NULL, NULL, NULL, 0, meta, stmt); mysqlCleanup(NULL, NULL, NULL, NULL, 0, meta, stmt);
return status; return status;
} }
columnCount = mysql_num_fields(meta); columnCount = mysql_num_fields(meta);
@ -601,9 +676,8 @@ static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogV
buffers = (char **)calloc(columnCount, sizeof(char *)); buffers = (char **)calloc(columnCount, sizeof(char *));
lengths = (unsigned long *)calloc(columnCount, sizeof(unsigned long)); lengths = (unsigned long *)calloc(columnCount, sizeof(unsigned long));
isNull = (my_bool *)calloc(columnCount, sizeof(my_bool)); isNull = (my_bool *)calloc(columnCount, sizeof(my_bool));
errors = (my_bool *)calloc(columnCount, sizeof(my_bool)); if (binds == NULL || buffers == NULL || lengths == NULL || isNull == NULL) {
if (binds == NULL || buffers == NULL || lengths == NULL || isNull == NULL || errors == NULL) { mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt);
return calogFail(result, calogErrOomE, "dbQuery: out of memory"); return calogFail(result, calogErrOomE, "dbQuery: out of memory");
} }
for (column = 0; column < columnCount; column++) { for (column = 0; column < columnCount; column++) {
@ -611,7 +685,7 @@ static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogV
size = (unsigned long)fields[column].max_length + 1; size = (unsigned long)fields[column].max_length + 1;
buffers[column] = (char *)malloc(size); buffers[column] = (char *)malloc(size);
if (buffers[column] == NULL) { if (buffers[column] == NULL) {
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt); mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
return calogFail(result, calogErrOomE, "dbQuery: out of memory"); return calogFail(result, calogErrOomE, "dbQuery: out of memory");
} }
binds[column].buffer_type = MYSQL_TYPE_STRING; binds[column].buffer_type = MYSQL_TYPE_STRING;
@ -619,75 +693,72 @@ static int32_t mysqlQuery(MYSQL *conn, const CalogValueT *sqlValue, const CalogV
binds[column].buffer_length = size; binds[column].buffer_length = size;
binds[column].length = &lengths[column]; binds[column].length = &lengths[column];
binds[column].is_null = &isNull[column]; binds[column].is_null = &isNull[column];
binds[column].error = &errors[column];
} }
if (mysql_stmt_bind_result(stmt, binds) != 0) { if (mysql_stmt_bind_result(stmt, binds) != 0) {
status = calogFail(result, calogErrArgE, mysql_stmt_error(stmt)); status = calogFail(result, calogErrArgE, mysql_stmt_error(stmt));
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt); mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
return status; return status;
} }
status = calogAggCreate(&rows, calogListE); status = calogAggCreate(&rows, calogListE);
if (status != calogOkE) { if (status != calogOkE) {
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt); mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
return calogFail(result, status, "dbQuery: out of memory"); return calogFail(result, status, "dbQuery: out of memory");
} }
rowCtx.fields = fields;
rowCtx.buffers = buffers;
rowCtx.lengths = lengths;
rowCtx.isNull = isNull;
while ((rc = mysql_stmt_fetch(stmt)) == 0 || rc == MYSQL_DATA_TRUNCATED) { while ((rc = mysql_stmt_fetch(stmt)) == 0 || rc == MYSQL_DATA_TRUNCATED) {
CalogAggT *rowMap; status = dbMarshalRow(result, rows, &rowCtx, (int32_t)columnCount, mysqlRowName, mysqlRowValue);
CalogValueT rowValue;
status = calogAggCreate(&rowMap, calogMapE);
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(rows); calogAggFree(rows);
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt); mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
return calogFail(result, status, "dbQuery: out of memory"); return status;
}
for (column = 0; column < columnCount; column++) {
CalogValueT key;
CalogValueT value;
calogValueNil(&value);
status = calogValueString(&key, fields[column].name, (int64_t)strlen(fields[column].name));
if (status == calogOkE) {
if (isNull[column]) {
calogValueNil(&value);
} else {
buffers[column][lengths[column]] = '\0';
status = mysqlColumn(fields[column].type, buffers[column], lengths[column], &value);
}
}
if (status == calogOkE) {
status = calogAggSet(rowMap, &key, &value);
}
if (status != calogOkE) {
calogValueFree(&key);
calogValueFree(&value);
calogAggFree(rowMap);
calogAggFree(rows);
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt);
return calogFail(result, status, "dbQuery: failed to build a row");
}
}
calogValueAgg(&rowValue, rowMap);
status = calogAggPush(rows, &rowValue);
if (status != calogOkE) {
calogValueFree(&rowValue);
calogAggFree(rows);
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt);
return calogFail(result, status, "dbQuery: out of memory");
} }
} }
if (rc != MYSQL_NO_DATA) { if (rc != MYSQL_NO_DATA) {
status = calogFail(result, calogErrArgE, mysql_stmt_error(stmt)); status = calogFail(result, calogErrArgE, mysql_stmt_error(stmt));
calogAggFree(rows); calogAggFree(rows);
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt); mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
return status; return status;
} }
mysqlCleanup(binds, buffers, lengths, isNull, errors, columnCount, meta, stmt); mysqlCleanup(binds, buffers, lengths, isNull, columnCount, meta, stmt);
calogValueAgg(result, rows); calogValueAgg(result, rows);
return calogOkE; return calogOkE;
} }
// dbMarshalRow callbacks bound to a MysqlRowCtxT for one mysqlQuery call.
static void mysqlRowName(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut) {
MysqlRowCtxT *rowCtx;
rowCtx = (MysqlRowCtxT *)ctx;
*nameOut = rowCtx->fields[column].name;
*nameLengthOut = (int64_t)strlen(rowCtx->fields[column].name);
}
static int32_t mysqlRowValue(void *ctx, int32_t column, CalogValueT *out) {
MysqlRowCtxT *rowCtx;
rowCtx = (MysqlRowCtxT *)ctx;
if (rowCtx->isNull[column]) {
return calogOkE;
}
rowCtx->buffers[column][rowCtx->lengths[column]] = '\0';
return mysqlColumn(rowCtx->fields[column].type, rowCtx->buffers[column], rowCtx->lengths[column], out);
}
#endif #endif
#ifdef CALOG_WITH_PG #ifdef CALOG_WITH_PG
// Row context bound to pgRowName/pgRowValue for one pgQuery call.
typedef struct PgRowCtxT {
PGresult *res;
int row;
} PgRowCtxT;
// Marshal a Postgres cell (text result format) to a CalogValueT: typed for the common // Marshal a Postgres cell (text result format) to a CalogValueT: typed for the common
// numeric/bool OIDs, bytea decoded from its "\x.." hex, everything else as its text bytes. // numeric/bool OIDs, bytea decoded from its "\x.." hex, everything else as its text bytes.
static int32_t pgColumn(PGresult *res, int row, int column, CalogValueT *out) { static int32_t pgColumn(PGresult *res, int row, int column, CalogValueT *out) {
@ -743,7 +814,7 @@ static int32_t pgColumn(PGresult *res, int row, int column, CalogValueT *out) {
static int32_t pgExec(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) { static int32_t pgExec(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) {
PGresult *res; PGresult *res;
res = pgRun(conn, sqlValue, params, paramCount, 0, result); res = pgRun(conn, sqlValue, params, paramCount, result);
if (res == NULL) { if (res == NULL) {
return calogErrArgE; return calogErrArgE;
} }
@ -755,7 +826,6 @@ static int32_t pgExec(PGconn *conn, const CalogValueT *sqlValue, const CalogValu
static int32_t pgOpen(DbLibT *lib, const char *conninfo, CalogValueT *result) { static int32_t pgOpen(DbLibT *lib, const char *conninfo, CalogValueT *result) {
PGconn *conn; PGconn *conn;
int64_t handle;
conn = PQconnectdb(conninfo); conn = PQconnectdb(conninfo);
if (PQstatus(conn) != CONNECTION_OK) { if (PQstatus(conn) != CONNECTION_OK) {
@ -764,25 +834,20 @@ static int32_t pgOpen(DbLibT *lib, const char *conninfo, CalogValueT *result) {
PQfinish(conn); PQfinish(conn);
return status; return status;
} }
handle = calogHandleAdd(lib->handles, DB_TYPE_PG, conn); return dbFinishOpen(lib, DB_TYPE_PG, conn, result);
if (handle == 0) {
PQfinish(conn);
return calogFail(result, calogErrOomE, "dbOpen: out of memory");
}
calogValueInt(result, handle);
return calogOkE;
} }
static int32_t pgQuery(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) { static int32_t pgQuery(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) {
PGresult *res; PGresult *res;
CalogAggT *rows; CalogAggT *rows;
PgRowCtxT rowCtx;
int rowCount; int rowCount;
int columnCount; int columnCount;
int row; int row;
int32_t status; int32_t status;
res = pgRun(conn, sqlValue, params, paramCount, 0, result); res = pgRun(conn, sqlValue, params, paramCount, result);
if (res == NULL) { if (res == NULL) {
return calogErrArgE; return calogErrArgE;
} }
@ -793,44 +858,14 @@ static int32_t pgQuery(PGconn *conn, const CalogValueT *sqlValue, const CalogVal
} }
rowCount = PQntuples(res); rowCount = PQntuples(res);
columnCount = PQnfields(res); columnCount = PQnfields(res);
rowCtx.res = res;
for (row = 0; row < rowCount; row++) { for (row = 0; row < rowCount; row++) {
CalogAggT *rowMap; rowCtx.row = row;
CalogValueT rowValue; status = dbMarshalRow(result, rows, &rowCtx, columnCount, pgRowName, pgRowValue);
int column;
status = calogAggCreate(&rowMap, calogMapE);
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(rows); calogAggFree(rows);
PQclear(res); PQclear(res);
return calogFail(result, status, "dbQuery: out of memory"); return status;
}
for (column = 0; column < columnCount; column++) {
CalogValueT key;
CalogValueT value;
const char *name;
name = PQfname(res, column);
status = calogValueString(&key, name, (int64_t)strlen(name));
if (status == calogOkE) {
status = pgColumn(res, row, column, &value);
}
if (status == calogOkE) {
status = calogAggSet(rowMap, &key, &value);
}
if (status != calogOkE) {
calogValueFree(&key);
calogValueFree(&value);
calogAggFree(rowMap);
calogAggFree(rows);
PQclear(res);
return calogFail(result, status, "dbQuery: failed to build a row");
}
}
calogValueAgg(&rowValue, rowMap);
status = calogAggPush(rows, &rowValue);
if (status != calogOkE) {
calogValueFree(&rowValue);
calogAggFree(rows);
PQclear(res);
return calogFail(result, status, "dbQuery: out of memory");
} }
} }
calogValueAgg(result, rows); calogValueAgg(result, rows);
@ -839,11 +874,32 @@ static int32_t pgQuery(PGconn *conn, const CalogValueT *sqlValue, const CalogVal
} }
// dbMarshalRow callbacks bound to a PgRowCtxT for one pgQuery call.
static void pgRowName(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut) {
PgRowCtxT *rowCtx;
const char *name;
rowCtx = (PgRowCtxT *)ctx;
name = PQfname(rowCtx->res, column);
*nameOut = name;
*nameLengthOut = (int64_t)strlen(name);
}
static int32_t pgRowValue(void *ctx, int32_t column, CalogValueT *out) {
PgRowCtxT *rowCtx;
rowCtx = (PgRowCtxT *)ctx;
return pgColumn(rowCtx->res, rowCtx->row, column, out);
}
// Run a parameterized statement ($1..$N): nil -> SQL NULL; bool as 't'/'f'; ints/reals // Run a parameterized statement ($1..$N): nil -> SQL NULL; bool as 't'/'f'; ints/reals
// formatted as text; strings bound in BINARY format with an explicit length, so they are // formatted as text; strings bound in BINARY format with an explicit length, so they are
// binary-safe (embedded NULs survive, unlike NUL-terminated text params). On any error the // binary-safe (embedded NULs survive, unlike NUL-terminated text params). Always requests a
// message is copied into result via calogFail and NULL is returned. // text-format result (pgColumn only ever decodes text). On any error the message is copied
static PGresult *pgRun(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, int resultFormat, CalogValueT *result) { // into result via calogFail and NULL is returned.
static PGresult *pgRun(PGconn *conn, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) {
const char **values; const char **values;
int *lengths; int *lengths;
int *formats; int *formats;
@ -890,6 +946,14 @@ static PGresult *pgRun(PGconn *conn, const CalogValueT *sqlValue, const CalogVal
values[index] = buffer; values[index] = buffer;
break; break;
case calogStringE: case calogStringE:
if (param->as.s.length > INT32_MAX) {
free((void *)values);
free(lengths);
free(formats);
free(scratch);
calogFail(result, calogErrRangeE, "dbExec/dbQuery: a bound string is too long");
return NULL;
}
values[index] = param->as.s.bytes; values[index] = param->as.s.bytes;
lengths[index] = (int)param->as.s.length; lengths[index] = (int)param->as.s.length;
formats[index] = 1; formats[index] = 1;
@ -903,7 +967,8 @@ static PGresult *pgRun(PGconn *conn, const CalogValueT *sqlValue, const CalogVal
return NULL; return NULL;
} }
} }
res = PQexecParams(conn, sqlValue->as.s.bytes, paramCount, NULL, values, lengths, formats, resultFormat); // Always text-format result (0): pgColumn only decodes text.
res = PQexecParams(conn, sqlValue->as.s.bytes, paramCount, NULL, values, lengths, formats, 0);
free((void *)values); free((void *)values);
free(lengths); free(lengths);
free(formats); free(formats);
@ -964,7 +1029,6 @@ static int32_t sqliteExec(sqlite3 *db, const CalogValueT *sqlValue, const CalogV
static int32_t sqliteOpen(DbLibT *lib, const char *path, CalogValueT *result) { static int32_t sqliteOpen(DbLibT *lib, const char *path, CalogValueT *result) {
sqlite3 *db; sqlite3 *db;
int64_t handle;
db = NULL; db = NULL;
if (sqlite3_open(path, &db) != SQLITE_OK) { if (sqlite3_open(path, &db) != SQLITE_OK) {
@ -973,13 +1037,7 @@ static int32_t sqliteOpen(DbLibT *lib, const char *path, CalogValueT *result) {
sqlite3_close(db); sqlite3_close(db);
return status; return status;
} }
handle = calogHandleAdd(lib->handles, DB_TYPE_SQLITE, db); return dbFinishOpen(lib, DB_TYPE_SQLITE, db, result);
if (handle == 0) {
sqlite3_close(db);
return calogFail(result, calogErrOomE, "dbOpen: out of memory");
}
calogValueInt(result, handle);
return calogOkE;
} }
@ -990,6 +1048,9 @@ static int32_t sqlitePrepare(sqlite3 *db, const CalogValueT *sqlValue, const Cal
int32_t index; int32_t index;
*out = NULL; *out = NULL;
if (sqlValue->as.s.length > INT32_MAX) {
return calogFail(result, calogErrRangeE, "dbExec/dbQuery: sql text is too long");
}
if (sqlite3_prepare_v2(db, sqlValue->as.s.bytes, (int)sqlValue->as.s.length, &stmt, NULL) != SQLITE_OK) { if (sqlite3_prepare_v2(db, sqlValue->as.s.bytes, (int)sqlValue->as.s.length, &stmt, NULL) != SQLITE_OK) {
return calogFail(result, calogErrArgE, sqlite3_errmsg(db)); return calogFail(result, calogErrArgE, sqlite3_errmsg(db));
} }
@ -1011,6 +1072,10 @@ static int32_t sqlitePrepare(sqlite3 *db, const CalogValueT *sqlValue, const Cal
rc = sqlite3_bind_double(stmt, index + 1, param->as.r); rc = sqlite3_bind_double(stmt, index + 1, param->as.r);
break; break;
case calogStringE: case calogStringE:
if (param->as.s.length > INT32_MAX) {
sqlite3_finalize(stmt);
return calogFail(result, calogErrRangeE, "dbExec/dbQuery: a bound string is too long");
}
rc = sqlite3_bind_text(stmt, index + 1, param->as.s.bytes, (int)param->as.s.length, SQLITE_TRANSIENT); rc = sqlite3_bind_text(stmt, index + 1, param->as.s.bytes, (int)param->as.s.length, SQLITE_TRANSIENT);
break; break;
default: default:
@ -1032,6 +1097,7 @@ static int32_t sqlitePrepare(sqlite3 *db, const CalogValueT *sqlValue, const Cal
static int32_t sqliteQuery(sqlite3 *db, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) { static int32_t sqliteQuery(sqlite3 *db, const CalogValueT *sqlValue, const CalogValueT *params, int32_t paramCount, CalogValueT *result) {
sqlite3_stmt *stmt; sqlite3_stmt *stmt;
CalogAggT *rows; CalogAggT *rows;
int32_t columns;
int32_t status; int32_t status;
int rc; int rc;
@ -1044,46 +1110,13 @@ static int32_t sqliteQuery(sqlite3 *db, const CalogValueT *sqlValue, const Calog
sqlite3_finalize(stmt); sqlite3_finalize(stmt);
return calogFail(result, status, "dbQuery: out of memory"); return calogFail(result, status, "dbQuery: out of memory");
} }
while ((rc = sqlite3_step(stmt)) == SQLITE_ROW) {
CalogAggT *rowMap;
CalogValueT rowValue;
int columns;
int column;
status = calogAggCreate(&rowMap, calogMapE);
if (status != calogOkE) {
calogAggFree(rows);
sqlite3_finalize(stmt);
return calogFail(result, status, "dbQuery: out of memory");
}
columns = sqlite3_column_count(stmt); columns = sqlite3_column_count(stmt);
for (column = 0; column < columns; column++) { while ((rc = sqlite3_step(stmt)) == SQLITE_ROW) {
CalogValueT key; status = dbMarshalRow(result, rows, stmt, columns, sqliteRowName, sqliteRowValue);
CalogValueT value;
const char *name;
name = sqlite3_column_name(stmt, column);
status = calogValueString(&key, name, (int64_t)strlen(name));
if (status == calogOkE) {
status = sqliteColumn(stmt, column, &value);
}
if (status == calogOkE) {
status = calogAggSet(rowMap, &key, &value);
}
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(&key);
calogValueFree(&value);
calogAggFree(rowMap);
calogAggFree(rows); calogAggFree(rows);
sqlite3_finalize(stmt); sqlite3_finalize(stmt);
return calogFail(result, status, "dbQuery: failed to build a row"); return status;
}
}
calogValueAgg(&rowValue, rowMap);
status = calogAggPush(rows, &rowValue);
if (status != calogOkE) {
calogValueFree(&rowValue);
calogAggFree(rows);
sqlite3_finalize(stmt);
return calogFail(result, status, "dbQuery: out of memory");
} }
} }
if (rc != SQLITE_DONE) { if (rc != SQLITE_DONE) {
@ -1096,4 +1129,19 @@ static int32_t sqliteQuery(sqlite3 *db, const CalogValueT *sqlValue, const Calog
sqlite3_finalize(stmt); sqlite3_finalize(stmt);
return calogOkE; return calogOkE;
} }
// dbMarshalRow callbacks bound directly to the sqlite3_stmt for one sqliteQuery call.
static void sqliteRowName(void *ctx, int32_t column, const char **nameOut, int64_t *nameLengthOut) {
const char *name;
name = sqlite3_column_name((sqlite3_stmt *)ctx, column);
*nameOut = name;
*nameLengthOut = (int64_t)strlen(name);
}
static int32_t sqliteRowValue(void *ctx, int32_t column, CalogValueT *out) {
return sqliteColumn((sqlite3_stmt *)ctx, column, out);
}
#endif #endif

View file

@ -18,11 +18,11 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define EXPORT_INITIAL_CAP 8 #include "calogInternal.h"
#define EXPORT_GROWTH 2
typedef struct ExportEntryT { typedef struct ExportEntryT {
char *name; char *name;
int64_t nameLen;
CalogFnT *fn; CalogFnT *fn;
} ExportEntryT; } ExportEntryT;
@ -36,43 +36,32 @@ static pthread_mutex_t gInitMutex = PTHREAD_MUTEX_INITIALIZER;
static int32_t gRefCount = 0; static int32_t gRefCount = 0;
static int32_t exportCall(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t exportCall(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t exportFindLocked(const char *name); static int32_t exportFindFoldLocked(const char *bytes, int64_t length);
static int32_t exportFindLocked(const char *bytes, int64_t length);
static void exportFreeAll(void);
static int32_t exportPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t exportPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t exportRemove(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t exportRemove(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t exportResolve(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t exportResolve(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t exportResolveByFind(CalogValueT *args, int32_t argCount, CalogValueT *result, int32_t (*find)(const char *, int64_t));
static int32_t exportResolveFold(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
int32_t calogExportRegister(CalogT *calog) { int32_t calogExportRegister(CalogT *calog) {
pthread_mutex_lock(&gInitMutex); calogRegistryRetain(&gInitMutex, &gRefCount);
gRefCount++;
pthread_mutex_unlock(&gInitMutex);
calogRegisterInline(calog, "calogExport", exportPublish, NULL); calogRegisterInline(calog, "calogExport", exportPublish, NULL);
calogRegisterInline(calog, "calogUnexport", exportRemove, NULL); calogRegisterInline(calog, "calogUnexport", exportRemove, NULL);
calogRegisterInline(calog, "calogCall", exportCall, NULL); calogRegisterInline(calog, "calogCall", exportCall, NULL);
calogRegisterInline(calog, "__calogExportResolve", exportResolve, NULL); calogRegisterInline(calog, "__calogExportResolve", exportResolve, NULL);
// Case-insensitive variant: my-basic uppercases every identifier at parse time, so its
// bare-name resolver matches an export regardless of the case it was registered with.
calogRegisterInline(calog, "__calogExportResolveFold", exportResolveFold, NULL);
return calogOkE; return calogOkE;
} }
void calogExportShutdown(void) { void calogExportShutdown(void) {
pthread_mutex_lock(&gInitMutex); // calogRegistryRelease holds gInitMutex for the whole call, including exportFreeAll below.
if (gRefCount > 0) { calogRegistryRelease(&gInitMutex, &gRefCount, exportFreeAll);
gRefCount--;
}
if (gRefCount <= 0) {
int32_t index;
pthread_mutex_lock(&gMapMutex);
for (index = 0; index < gCount; index++) {
free(gEntries[index].name);
calogFnRelease(gEntries[index].fn);
}
free(gEntries);
gEntries = NULL;
gCount = 0;
gCap = 0;
pthread_mutex_unlock(&gMapMutex);
}
pthread_mutex_unlock(&gInitMutex);
} }
@ -87,7 +76,7 @@ static int32_t exportCall(CalogValueT *args, int32_t argCount, CalogValueT *resu
return calogFail(result, calogErrArgE, "calogCall expects (name, ...args)"); return calogFail(result, calogErrArgE, "calogCall expects (name, ...args)");
} }
pthread_mutex_lock(&gMapMutex); pthread_mutex_lock(&gMapMutex);
index = exportFindLocked(args[0].as.s.bytes); index = exportFindLocked(args[0].as.s.bytes, args[0].as.s.length);
if (index < 0) { if (index < 0) {
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
return calogFail(result, calogErrNotFoundE, "calogCall: no such exported function"); return calogFail(result, calogErrNotFoundE, "calogCall: no such exported function");
@ -102,11 +91,30 @@ static int32_t exportCall(CalogValueT *args, int32_t argCount, CalogValueT *resu
} }
static int32_t exportFindLocked(const char *name) { static int32_t exportFindFoldLocked(const char *bytes, int64_t length) {
int32_t index; int32_t index;
int64_t at;
for (index = 0; index < gCount; index++) { for (index = 0; index < gCount; index++) {
if (strcmp(gEntries[index].name, name) == 0) { if (gEntries[index].nameLen != length) {
continue;
}
for (at = 0; at < length; at++) {
char lhs;
char rhs;
lhs = gEntries[index].name[at];
rhs = bytes[at];
if (lhs >= 'A' && lhs <= 'Z') {
lhs = (char)(lhs - 'A' + 'a');
}
if (rhs >= 'A' && rhs <= 'Z') {
rhs = (char)(rhs - 'A' + 'a');
}
if (lhs != rhs) {
break;
}
}
if (at == length) {
return index; return index;
} }
} }
@ -114,10 +122,44 @@ static int32_t exportFindLocked(const char *name) {
} }
static int32_t exportFindLocked(const char *bytes, int64_t length) {
int32_t index;
for (index = 0; index < gCount; index++) {
if (gEntries[index].nameLen == length && memcmp(gEntries[index].name, bytes, (size_t)length) == 0) {
return index;
}
}
return -1;
}
// Invoked by calogRegistryRelease while gInitMutex is held, exactly once, when the last
// reference drops.
static void exportFreeAll(void) {
int32_t index;
pthread_mutex_lock(&gMapMutex);
for (index = 0; index < gCount; index++) {
free(gEntries[index].name);
calogFnRelease(gEntries[index].fn);
}
free(gEntries);
gEntries = NULL;
gCount = 0;
gCap = 0;
pthread_mutex_unlock(&gMapMutex);
}
static int32_t exportPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t exportPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
void *buffer;
int64_t capacity;
const char *name; const char *name;
char *nameCopy; char *nameCopy;
int64_t nameLen;
int32_t index; int32_t index;
int32_t status;
(void)userData; (void)userData;
calogValueNil(result); calogValueNil(result);
@ -125,8 +167,9 @@ static int32_t exportPublish(CalogValueT *args, int32_t argCount, CalogValueT *r
return calogFail(result, calogErrArgE, "calogExport expects (name, function)"); return calogFail(result, calogErrArgE, "calogExport expects (name, function)");
} }
name = args[0].as.s.bytes; name = args[0].as.s.bytes;
nameLen = args[0].as.s.length;
pthread_mutex_lock(&gMapMutex); pthread_mutex_lock(&gMapMutex);
index = exportFindLocked(name); index = exportFindLocked(name, nameLen);
if (index >= 0) { if (index >= 0) {
// Replace: drop the old function, retain the new one. // Replace: drop the old function, retain the new one.
calogFnRelease(gEntries[index].fn); calogFnRelease(gEntries[index].fn);
@ -135,25 +178,27 @@ static int32_t exportPublish(CalogValueT *args, int32_t argCount, CalogValueT *r
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
return calogOkE; return calogOkE;
} }
if (gCount == gCap) { buffer = gEntries;
int32_t newCap; capacity = gCap;
ExportEntryT *grown; status = calogGrow(&buffer, &capacity, (int64_t)gCount + 1, sizeof(ExportEntryT));
newCap = (gCap == 0) ? EXPORT_INITIAL_CAP : gCap * EXPORT_GROWTH; if (status != calogOkE) {
grown = (ExportEntryT *)realloc(gEntries, (size_t)newCap * sizeof(ExportEntryT));
if (grown == NULL) {
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
return calogFail(result, calogErrOomE, "calogExport: out of memory"); return calogFail(result, status, "calogExport: out of memory");
} }
gEntries = grown; gEntries = (ExportEntryT *)buffer;
gCap = newCap; gCap = (int32_t)capacity;
} nameCopy = (char *)malloc((size_t)nameLen + 1);
nameCopy = strdup(name);
if (nameCopy == NULL) { if (nameCopy == NULL) {
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
return calogFail(result, calogErrOomE, "calogExport: out of memory"); return calogFail(result, calogErrOomE, "calogExport: out of memory");
} }
if (nameLen > 0) {
memcpy(nameCopy, name, (size_t)nameLen);
}
nameCopy[nameLen] = '\0';
calogFnRetain(args[1].as.fn); calogFnRetain(args[1].as.fn);
gEntries[gCount].name = nameCopy; gEntries[gCount].name = nameCopy;
gEntries[gCount].nameLen = nameLen;
gEntries[gCount].fn = args[1].as.fn; gEntries[gCount].fn = args[1].as.fn;
gCount++; gCount++;
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
@ -170,7 +215,7 @@ static int32_t exportRemove(CalogValueT *args, int32_t argCount, CalogValueT *re
return calogFail(result, calogErrArgE, "calogUnexport expects (name)"); return calogFail(result, calogErrArgE, "calogUnexport expects (name)");
} }
pthread_mutex_lock(&gMapMutex); pthread_mutex_lock(&gMapMutex);
index = exportFindLocked(args[0].as.s.bytes); index = exportFindLocked(args[0].as.s.bytes, args[0].as.s.length);
if (index >= 0) { if (index >= 0) {
free(gEntries[index].name); free(gEntries[index].name);
calogFnRelease(gEntries[index].fn); calogFnRelease(gEntries[index].fn);
@ -183,15 +228,20 @@ static int32_t exportRemove(CalogValueT *args, int32_t argCount, CalogValueT *re
static int32_t exportResolve(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t exportResolve(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
return exportResolveByFind(args, argCount, result, exportFindLocked);
}
static int32_t exportResolveByFind(CalogValueT *args, int32_t argCount, CalogValueT *result, int32_t (*find)(const char *, int64_t)) {
int32_t index; int32_t index;
(void)userData;
calogValueNil(result); calogValueNil(result);
if (argCount != 1 || args[0].type != calogStringE) { if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "__calogExportResolve expects (name)"); return calogFail(result, calogErrArgE, "export resolve expects (name)");
} }
pthread_mutex_lock(&gMapMutex); pthread_mutex_lock(&gMapMutex);
index = exportFindLocked(args[0].as.s.bytes); index = find(args[0].as.s.bytes, args[0].as.s.length);
if (index >= 0) { if (index >= 0) {
CalogFnT *fn; CalogFnT *fn;
fn = gEntries[index].fn; fn = gEntries[index].fn;
@ -202,3 +252,9 @@ static int32_t exportResolve(CalogValueT *args, int32_t argCount, CalogValueT *r
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
return calogOkE; return calogOkE;
} }
static int32_t exportResolveFold(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
return exportResolveByFind(args, argCount, result, exportFindFoldLocked);
}

View file

@ -9,19 +9,20 @@
// (plain `export` is a JavaScript keyword; plain `call` is a my-basic keyword). // (plain `export` is a JavaScript keyword; plain `call` is a my-basic keyword).
// //
// On engines with a runtime unknown-name hook (Lua, Squirrel, JavaScript, s7), an exported // On engines with a runtime unknown-name hook (Lua, Squirrel, JavaScript, s7), an exported
// name is ALSO reachable by its BARE name, e.g. `luaExample(1, 2)`; engines that resolve // name is ALSO reachable by its BARE name, e.g. `luaExample(1, 2)`. my-basic likewise bare-
// names statically (Wren, Berry, my-basic) use calogCall('luaExample', ...). Resolution is // resolves a name called like a function, but CASE-INSENSITIVELY (it uppercases identifiers at
// dynamic: a name exported at any time becomes callable immediately, and calogUnexport takes // parse time), so `luaExample(1, 2)` there matches the export ignoring case. Wren and Berry
// effect at once. // resolve names statically and use calogCall('luaExample', ...). Resolution is dynamic: a name
// exported at any time becomes callable immediately, and calogUnexport takes effect at once.
// //
// An exported function is an ordinary calog function value, so a call runs in the exporter's // An exported function is an ordinary calog function value, so a call runs in the exporter's
// own context/thread (marshalled like any callable), and a call after the exporter is gone // own context/thread (marshalled like any callable), and a call after the exporter is gone
// fails cleanly. The registry is process-wide and reference-counted across runtimes. // fails cleanly. The registry is process-wide and reference-counted across runtimes.
// //
// NOTE (bare-name shadowing): on the hook engines, reading an otherwise-undefined global // NOTE (bare-name shadowing): on the hook engines (and, for a name called like a function, on
// whose name matches an export resolves to that export -- an exported name is visible to // my-basic), an otherwise-undefined name that matches an export resolves to that export -- an
// every context, so pick export names that will not collide with scripts' feature-detection // exported name is visible to every context, so pick export names that will not collide with
// of undefined globals. // scripts' feature-detection of undefined globals.
#ifndef CALOG_EXPORT_H #ifndef CALOG_EXPORT_H
#define CALOG_EXPORT_H #define CALOG_EXPORT_H

View file

@ -7,6 +7,7 @@
#define _POSIX_C_SOURCE 200809L #define _POSIX_C_SOURCE 200809L
#include "calogFs.h" #include "calogFs.h"
#include "calogInternal.h"
#include <dirent.h> #include <dirent.h>
#include <errno.h> #include <errno.h>
@ -18,6 +19,10 @@
#include <sys/types.h> #include <sys/types.h>
#include <unistd.h> #include <unistd.h>
// fsRead's starting buffer capacity when fstat's st_size reports 0 (procfs/sysfs/FIFO-style
// files, whose content is read regardless by growing this buffer to EOF).
#define FS_READ_INITIAL_CAP 4096
static int32_t fsAppendNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t fsAppendNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t fsExistsNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t fsExistsNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t fsFail(CalogValueT *result, int32_t err); static int32_t fsFail(CalogValueT *result, int32_t err);
@ -167,12 +172,10 @@ static int32_t fsMkdirNative(CalogValueT *args, int32_t argCount, CalogValueT *r
// An already-existing directory is success; anything else (including a non-directory // An already-existing directory is success; anything else (including a non-directory
// squatting on the name) is the failure the caller sees. // squatting on the name) is the failure the caller sees.
if (errno == EEXIST && stat(path, &st) == 0 && S_ISDIR(st.st_mode)) { if (errno == EEXIST && stat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
calogValueNil(result);
return calogOkE; return calogOkE;
} }
return fsFail(result, errno); return fsFail(result, errno);
} }
calogValueNil(result);
return calogOkE; return calogOkE;
} }
@ -218,9 +221,11 @@ static int32_t fsReadNative(CalogValueT *args, int32_t argCount, CalogValueT *re
struct stat st; struct stat st;
const char *path; const char *path;
char *data; char *data;
void *buffer;
ssize_t got; ssize_t got;
int64_t cap;
int64_t initialCap;
size_t offset; size_t offset;
size_t total;
int fd; int fd;
int saved; int saved;
int32_t status; int32_t status;
@ -240,15 +245,31 @@ static int32_t fsReadNative(CalogValueT *args, int32_t argCount, CalogValueT *re
close(fd); close(fd);
return fsFail(result, saved); return fsFail(result, saved);
} }
total = (st.st_size > 0) ? (size_t)st.st_size : (size_t)0; // st_size is only a hint: procfs/sysfs/FIFO-style files can report 0 (or an
data = (char *)malloc(total + 1); // understated size) yet still have real content, so keep reading past it until
if (data == NULL) { // read() hits EOF instead of trusting st_size as the exact length.
initialCap = (st.st_size > 0) ? (int64_t)st.st_size + 1 : FS_READ_INITIAL_CAP;
buffer = NULL;
cap = 0;
status = calogGrow(&buffer, &cap, initialCap, sizeof(char));
if (status != calogOkE) {
close(fd); close(fd);
return calogFail(result, calogErrOomE, "fsRead: out of memory"); return calogFail(result, status, "fsRead: out of memory");
} }
data = (char *)buffer;
offset = 0; offset = 0;
while (offset < total) { for (;;) {
got = read(fd, data + offset, total - offset); if (offset == (size_t)cap) {
buffer = data;
status = calogGrow(&buffer, &cap, cap + 1, sizeof(char));
if (status != calogOkE) {
free(data);
close(fd);
return calogFail(result, status, "fsRead: out of memory");
}
data = (char *)buffer;
}
got = read(fd, data + offset, (size_t)cap - offset);
if (got < 0) { if (got < 0) {
if (errno == EINTR) { if (errno == EINTR) {
continue; continue;
@ -304,7 +325,6 @@ static int32_t fsStatNative(CalogValueT *args, int32_t argCount, CalogValueT *re
if (stat(path, &st) != 0) { if (stat(path, &st) != 0) {
// A missing path is not an error: report nil so callers can probe with fsStat. // A missing path is not an error: report nil so callers can probe with fsStat.
if (errno == ENOENT) { if (errno == ENOENT) {
calogValueNil(result);
return calogOkE; return calogOkE;
} }
return fsFail(result, errno); return fsFail(result, errno);

View file

@ -102,6 +102,28 @@ void *calogHandleGet(CalogHandleTableT *table, int64_t handle, uint32_t type) {
} }
void *calogHandleGetAny(CalogHandleTableT *table, int64_t handle, uint32_t *typeOut) {
void *resource;
int32_t index;
*typeOut = 0;
if (handle <= 0) {
return NULL;
}
resource = NULL;
pthread_mutex_lock(&table->mutex);
for (index = 0; index < table->count; index++) {
if (table->entries[index].handle == handle) {
resource = table->entries[index].resource;
*typeOut = table->entries[index].type;
break;
}
}
pthread_mutex_unlock(&table->mutex);
return resource;
}
void *calogHandleRemove(CalogHandleTableT *table, int64_t handle, uint32_t type) { void *calogHandleRemove(CalogHandleTableT *table, int64_t handle, uint32_t type) {
void *resource; void *resource;
int32_t index; int32_t index;
@ -125,6 +147,31 @@ void *calogHandleRemove(CalogHandleTableT *table, int64_t handle, uint32_t type)
} }
void *calogHandleRemoveAny(CalogHandleTableT *table, int64_t handle, uint32_t *typeOut) {
void *resource;
int32_t index;
*typeOut = 0;
if (handle <= 0) {
return NULL;
}
resource = NULL;
pthread_mutex_lock(&table->mutex);
for (index = 0; index < table->count; index++) {
if (table->entries[index].handle == handle) {
resource = table->entries[index].resource;
*typeOut = table->entries[index].type;
table->entries[index].handle = 0;
table->entries[index].type = 0;
table->entries[index].resource = NULL;
break;
}
}
pthread_mutex_unlock(&table->mutex);
return resource;
}
CalogHandleTableT *calogHandleTableCreate(void) { CalogHandleTableT *calogHandleTableCreate(void) {
CalogHandleTableT *table; CalogHandleTableT *table;
@ -153,3 +200,23 @@ void calogHandleTableDestroy(CalogHandleTableT *table, CalogHandleCloserT closer
pthread_mutex_destroy(&table->mutex); pthread_mutex_destroy(&table->mutex);
free(table); free(table);
} }
void *calogHandleTakeIf(CalogHandleTableT *table, uint32_t type, bool (*pred)(void *resource, void *userData), void *userData) {
void *resource;
int32_t index;
resource = NULL;
pthread_mutex_lock(&table->mutex);
for (index = 0; index < table->count; index++) {
if (table->entries[index].handle != 0 && table->entries[index].type == type && pred(table->entries[index].resource, userData)) {
resource = table->entries[index].resource;
table->entries[index].handle = 0;
table->entries[index].type = 0;
table->entries[index].resource = NULL;
break;
}
}
pthread_mutex_unlock(&table->mutex);
return resource;
}

View file

@ -10,6 +10,7 @@
#ifndef CALOG_HANDLE_H #ifndef CALOG_HANDLE_H
#define CALOG_HANDLE_H #define CALOG_HANDLE_H
#include <stdbool.h>
#include <stdint.h> #include <stdint.h>
typedef struct CalogHandleTableT CalogHandleTableT; typedef struct CalogHandleTableT CalogHandleTableT;
@ -27,7 +28,18 @@ int64_t calogHandleAdd(CalogHandleTableT *table, uint32_t type, void
void *calogHandleGet(CalogHandleTableT *table, int64_t handle, uint32_t type); void *calogHandleGet(CalogHandleTableT *table, int64_t handle, uint32_t type);
void *calogHandleRemove(CalogHandleTableT *table, int64_t handle, uint32_t type); void *calogHandleRemove(CalogHandleTableT *table, int64_t handle, uint32_t type);
// GetAny/RemoveAny resolve a handle of UNKNOWN type in a single locked scan, returning the
// resource and writing its type to *typeOut (0 when the handle is absent). For a table whose
// entries are all one library's own resources, this replaces a fan of type-specific probes.
void *calogHandleGetAny(CalogHandleTableT *table, int64_t handle, uint32_t *typeOut);
void *calogHandleRemoveAny(CalogHandleTableT *table, int64_t handle, uint32_t *typeOut);
// Number of live entries of the given type currently in the table. // Number of live entries of the given type currently in the table.
int32_t calogHandleCount(CalogHandleTableT *table, uint32_t type); int32_t calogHandleCount(CalogHandleTableT *table, uint32_t type);
// Atomically remove and return the first live entry of `type` whose resource satisfies pred
// (called under the table lock), or NULL if none. Lets an owner sweep entries by a runtime
// property (e.g. a task whose context thread has finished) in one locked pass.
void *calogHandleTakeIf(CalogHandleTableT *table, uint32_t type, bool (*pred)(void *resource, void *userData), void *userData);
#endif #endif

View file

@ -10,15 +10,21 @@
#define _GNU_SOURCE #define _GNU_SOURCE
#include "calogHttp.h" #include "calogHttp.h"
#include "calogInternal.h"
#include <errno.h> #include <errno.h>
#include <fcntl.h>
#include <limits.h> #include <limits.h>
#include <netdb.h> #include <netdb.h>
#include <netinet/in.h> #include <netinet/in.h>
#include <poll.h>
#include <signal.h>
#include <stdint.h> #include <stdint.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include <strings.h>
#include <sys/socket.h> #include <sys/socket.h>
#include <sys/time.h>
#include <unistd.h> #include <unistd.h>
#include <openssl/ssl.h> #include <openssl/ssl.h>
@ -30,6 +36,15 @@
#define HTTP_HOST_MAX 256 #define HTTP_HOST_MAX 256
#define HTTP_PORT_MAX 16 #define HTTP_PORT_MAX 16
#define HTTP_PATH_MAX 4096 #define HTTP_PATH_MAX 4096
// Connect, read, and write are all bounded by this single deadline (seconds): a stalled or
// blackholed peer must not hang the calling script thread (or a shutdown joining it) forever.
#define HTTP_TIMEOUT_SEC 30
#define HTTP_PORT_HTTP "80"
#define HTTP_PORT_HTTPS "443"
// Follow 3xx redirects, but stop after this many hops so a redirect loop cannot spin forever.
#define HTTP_MAX_REDIRECTS 16
#define HTTP_METHOD_MAX 16
#define HTTP_URL_MAX (HTTP_HOST_MAX + HTTP_PORT_MAX + HTTP_PATH_MAX + 32)
// A growable output byte buffer for the request and for decoded bodies. // A growable output byte buffer for the request and for decoded bodies.
typedef struct HttpBufT { typedef struct HttpBufT {
@ -38,9 +53,13 @@ typedef struct HttpBufT {
size_t cap; size_t cap;
} HttpBufT; } HttpBufT;
// A parsed absolute URL. host/port/path are NUL-terminated so they double as C strings. // A parsed absolute URL. host/port/path are NUL-terminated so they double as C strings. ipv6 is
// true when the URL wrote the host as a bracketed literal ("[::1]"); host itself is stored
// WITHOUT the brackets (getaddrinfo/SNI/certificate-hostname all want the bare address), and
// the Host header re-adds them.
typedef struct HttpUrlT { typedef struct HttpUrlT {
bool https; bool https;
bool ipv6;
char host[HTTP_HOST_MAX]; char host[HTTP_HOST_MAX];
char port[HTTP_PORT_MAX]; char port[HTTP_PORT_MAX];
char path[HTTP_PATH_MAX]; char path[HTTP_PATH_MAX];
@ -56,6 +75,7 @@ typedef struct HttpConnT {
bool verify; bool verify;
} HttpConnT; } HttpConnT;
static CalogValueT *httpAggField(CalogAggT *agg, const char *name);
static int32_t httpAppendHeaders(HttpBufT *buf, const CalogAggT *headers); static int32_t httpAppendHeaders(HttpBufT *buf, const CalogAggT *headers);
static int32_t httpBufEnsure(HttpBufT *buf, size_t extra); static int32_t httpBufEnsure(HttpBufT *buf, size_t extra);
static void httpBufFree(HttpBufT *buf); static void httpBufFree(HttpBufT *buf);
@ -63,6 +83,7 @@ static int32_t httpBufPutBytes(HttpBufT *buf, const char *bytes, size_t length);
static int32_t httpBufPutStr(HttpBufT *buf, const char *s); static int32_t httpBufPutStr(HttpBufT *buf, const char *s);
static int32_t httpBuildResult(const char *raw, size_t rawLen, CalogValueT *result); static int32_t httpBuildResult(const char *raw, size_t rawLen, CalogValueT *result);
static void httpConnClose(HttpConnT *conn); static void httpConnClose(HttpConnT *conn);
static bool httpConnectTimeout(int fd, const struct sockaddr *addr, socklen_t addrLen, int timeoutSec);
static int32_t httpConnOpen(HttpConnT *conn, const HttpUrlT *url, CalogValueT *result); static int32_t httpConnOpen(HttpConnT *conn, const HttpUrlT *url, CalogValueT *result);
static ssize_t httpConnRead(HttpConnT *conn, char *buf, size_t length); static ssize_t httpConnRead(HttpConnT *conn, char *buf, size_t length);
static int32_t httpConnWriteAll(HttpConnT *conn, const char *bytes, size_t length); static int32_t httpConnWriteAll(HttpConnT *conn, const char *bytes, size_t length);
@ -70,28 +91,57 @@ static bool httpContainsChunked(const char *bytes, size_t length);
static bool httpCopyField(char *dst, size_t cap, const char *src, size_t length); static bool httpCopyField(char *dst, size_t cap, const char *src, size_t length);
static int32_t httpDechunk(const char *bytes, size_t length, HttpBufT *out); static int32_t httpDechunk(const char *bytes, size_t length, HttpBufT *out);
static bool httpDefaultPort(const HttpUrlT *url); static bool httpDefaultPort(const HttpUrlT *url);
static CalogAggT *httpFilterHeaders(const CalogAggT *headers, bool dropCreds, bool dropContent);
static int32_t httpFollow(const char *method, const char *urlBytes, int64_t urlLen, const CalogAggT *headers, const char *body, int64_t bodyLen, bool verify, int32_t maxRedirects, CalogValueT *result);
static int32_t httpGetNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t httpGetNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static bool httpHasBadByte(const char *bytes, size_t length, bool rejectSpace);
static bool httpHeaderIn(const char *name, int64_t len, const char *const *list);
static int32_t httpHexVal(char c); static int32_t httpHexVal(char c);
static bool httpIsRedirect(int32_t code);
static char httpLower(char c); static char httpLower(char c);
static int32_t httpMapSetAgg(CalogAggT *map, const char *key, CalogAggT *inner); static int32_t httpMapSetAgg(CalogAggT *map, const char *key, CalogAggT *inner);
static int32_t httpMapSetInt(CalogAggT *map, const char *key, int64_t value); static bool httpMethodHasBody(const char *method);
static int32_t httpMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length); static CalogValueT *httpOptGet(CalogAggT *opts, const char *name, int32_t *status);
static int64_t httpParseInt(const char *bytes, size_t length); static int64_t httpParseInt(const char *bytes, size_t length);
static int32_t httpParseStatus(const char *line, size_t length, int32_t *codeOut); static int32_t httpParseStatus(const char *line, size_t length, int32_t *codeOut);
static int32_t httpParseUrl(const char *url, int64_t urlLen, HttpUrlT *out, CalogValueT *result); static int32_t httpParseUrl(const char *url, int64_t urlLen, HttpUrlT *out, CalogValueT *result);
static int32_t httpPerform(const char *method, const char *urlBytes, int64_t urlLen, const CalogAggT *headers, const char *body, int64_t bodyLen, bool verify, CalogValueT *result); static int32_t httpPerform(const char *method, const char *urlBytes, int64_t urlLen, const CalogAggT *headers, const char *body, int64_t bodyLen, bool verify, CalogValueT *result);
static int32_t httpReadResponse(HttpConnT *conn, HttpBufT *raw); static int32_t httpReadResponse(HttpConnT *conn, HttpBufT *raw);
static int32_t httpRequestNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t httpRequestNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static bool httpResolveLocation(const HttpUrlT *base, const char *loc, char *out, size_t cap);
static const CalogValueT *httpResultHeader(const CalogValueT *result, const char *name);
static int32_t httpResultStatus(const CalogValueT *result);
static bool httpSameOrigin(const HttpUrlT *a, const HttpUrlT *b);
static void httpSetTimeouts(int fd, int timeoutSec);
static int32_t httpTlsHandshake(HttpConnT *conn, const HttpUrlT *url, CalogValueT *result); static int32_t httpTlsHandshake(HttpConnT *conn, const HttpUrlT *url, CalogValueT *result);
static int httpWriteAuthority(char *out, size_t cap, const char *scheme, const HttpUrlT *base);
int32_t calogHttpRegister(CalogT *calog) { int32_t calogHttpRegister(CalogT *calog) {
// SSL_write/SSL_shutdown (and plain send()) can hit a broken pipe if the peer resets or
// closes mid-write; with the default disposition that raises SIGPIPE and kills the whole
// broker process. The plain path already passes MSG_NOSIGNAL, but OpenSSL's socket BIO does
// not, so ignore SIGPIPE process-wide (idempotent; safe to call from every ...Register).
signal(SIGPIPE, SIG_IGN);
calogRegisterInline(calog, "httpGet", httpGetNative, NULL); calogRegisterInline(calog, "httpGet", httpGetNative, NULL);
calogRegisterInline(calog, "httpRequest", httpRequestNative, NULL); calogRegisterInline(calog, "httpRequest", httpRequestNative, NULL);
return calogOkE; return calogOkE;
} }
// Look up a string-keyed field in an aggregate with a stack key (no allocation). The returned
// pointer aliases the aggregate and is valid only until it is freed or mutated.
static CalogValueT *httpAggField(CalogAggT *agg, const char *name) {
CalogValueT key;
calogValueNil(&key);
key.type = calogStringE;
key.as.s.bytes = (char *)name;
key.as.s.length = (int64_t)strlen(name);
return calogAggGet(agg, &key);
}
static int32_t httpAppendHeaders(HttpBufT *buf, const CalogAggT *headers) { static int32_t httpAppendHeaders(HttpBufT *buf, const CalogAggT *headers) {
int64_t index; int64_t index;
int32_t status; int32_t status;
@ -272,7 +322,7 @@ static int32_t httpBuildResult(const char *raw, size_t rawLen, CalogValueT *resu
lower[i] = httpLower(nameStart[i]); lower[i] = httpLower(nameStart[i]);
} }
lower[nameLen] = '\0'; lower[nameLen] = '\0';
status = httpMapSetStr(headersMap, lower, valStart, (int64_t)valLen); status = calogMapSetStr(headersMap, lower, valStart, (int64_t)valLen);
if (status != calogOkE) { if (status != calogOkE) {
free(lower); free(lower);
calogAggFree(headersMap); calogAggFree(headersMap);
@ -308,8 +358,16 @@ static int32_t httpBuildResult(const char *raw, size_t rawLen, CalogValueT *resu
finalBody = dechunked.bytes; finalBody = dechunked.bytes;
finalLen = dechunked.length; finalLen = dechunked.length;
} else if (haveContentLength) { } else if (haveContentLength) {
// A body shorter than the promised Content-Length means a read error (connection reset,
// unclean TLS close) was silently mapped to EOF upstream -- surface it as a failure
// rather than delivering a truncated body as a status-200 success.
if (bodyRawLen < (size_t)contentLength) {
httpBufFree(&dechunked);
calogAggFree(headersMap);
return calogFail(result, calogErrArgE, "http: response body shorter than Content-Length");
}
finalBody = bodyRaw; finalBody = bodyRaw;
finalLen = ((size_t)contentLength < bodyRawLen) ? (size_t)contentLength : bodyRawLen; finalLen = (size_t)contentLength;
} else { } else {
finalBody = bodyRaw; finalBody = bodyRaw;
finalLen = bodyRawLen; finalLen = bodyRawLen;
@ -322,14 +380,14 @@ static int32_t httpBuildResult(const char *raw, size_t rawLen, CalogValueT *resu
calogAggFree(headersMap); calogAggFree(headersMap);
return calogFail(result, status, "http: out of memory"); return calogFail(result, status, "http: out of memory");
} }
status = httpMapSetInt(map, "status", (int64_t)code); status = calogMapSetInt(map, "status", (int64_t)code);
if (status != calogOkE) { if (status != calogOkE) {
httpBufFree(&dechunked); httpBufFree(&dechunked);
calogAggFree(map); calogAggFree(map);
calogAggFree(headersMap); calogAggFree(headersMap);
return calogFail(result, status, "http: out of memory"); return calogFail(result, status, "http: out of memory");
} }
status = httpMapSetStr(map, "body", (finalBody != NULL) ? finalBody : "", (int64_t)finalLen); status = calogMapSetStr(map, "body", (finalBody != NULL) ? finalBody : "", (int64_t)finalLen);
if (status != calogOkE) { if (status != calogOkE) {
httpBufFree(&dechunked); httpBufFree(&dechunked);
calogAggFree(map); calogAggFree(map);
@ -368,6 +426,43 @@ static void httpConnClose(HttpConnT *conn) {
} }
static bool httpConnectTimeout(int fd, const struct sockaddr *addr, socklen_t addrLen, int timeoutSec) {
int flags;
int rc;
int err;
socklen_t errLen;
struct pollfd pfd;
flags = fcntl(fd, F_GETFL, 0);
if (flags < 0 || fcntl(fd, F_SETFL, flags | O_NONBLOCK) < 0) {
return connect(fd, addr, addrLen) == 0; // best effort: no bounded-connect fallback
}
rc = connect(fd, addr, addrLen);
if (rc == 0) {
fcntl(fd, F_SETFL, flags);
return true;
}
if (errno != EINPROGRESS) {
fcntl(fd, F_SETFL, flags);
return false;
}
pfd.fd = fd;
pfd.events = POLLOUT;
rc = poll(&pfd, 1, timeoutSec * 1000);
if (rc <= 0) {
fcntl(fd, F_SETFL, flags);
return false; // timeout or poll error
}
errLen = sizeof(err);
if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &err, &errLen) < 0 || err != 0) {
fcntl(fd, F_SETFL, flags);
return false;
}
fcntl(fd, F_SETFL, flags);
return true;
}
static int32_t httpConnOpen(HttpConnT *conn, const HttpUrlT *url, CalogValueT *result) { static int32_t httpConnOpen(HttpConnT *conn, const HttpUrlT *url, CalogValueT *result) {
struct addrinfo hints; struct addrinfo hints;
struct addrinfo *res; struct addrinfo *res;
@ -391,7 +486,8 @@ static int32_t httpConnOpen(HttpConnT *conn, const HttpUrlT *url, CalogValueT *r
if (fd < 0) { if (fd < 0) {
continue; continue;
} }
if (connect(fd, rp->ai_addr, rp->ai_addrlen) == 0) { httpSetTimeouts(fd, HTTP_TIMEOUT_SEC);
if (httpConnectTimeout(fd, rp->ai_addr, rp->ai_addrlen, HTTP_TIMEOUT_SEC)) {
break; break;
} }
close(fd); close(fd);
@ -418,13 +514,20 @@ static ssize_t httpConnRead(HttpConnT *conn, char *buf, size_t length) {
if (conn->ssl != NULL) { if (conn->ssl != NULL) {
int chunk; int chunk;
int r; int r;
int err;
chunk = (length > (size_t)INT_MAX) ? INT_MAX : (int)length; chunk = (length > (size_t)INT_MAX) ? INT_MAX : (int)length;
r = SSL_read(conn->ssl, buf, chunk); r = SSL_read(conn->ssl, buf, chunk);
// A non-positive return (clean shutdown or error) ends the read-to-EOF loop. if (r > 0) {
if (r <= 0) { return (ssize_t)r;
}
// Distinguish a clean TLS close_notify (end of response, like a plain-socket EOF) from
// any other error (reset, protocol violation, truncated shutdown) -- the latter must NOT
// be treated as a successful end of response.
err = SSL_get_error(conn->ssl, r);
if (err == SSL_ERROR_ZERO_RETURN) {
return 0; return 0;
} }
return (ssize_t)r; return -1;
} }
for (;;) { for (;;) {
ssize_t r; ssize_t r;
@ -564,9 +667,173 @@ static int32_t httpDechunk(const char *bytes, size_t length, HttpBufT *out) {
static bool httpDefaultPort(const HttpUrlT *url) { static bool httpDefaultPort(const HttpUrlT *url) {
if (url->https) { if (url->https) {
return strcmp(url->port, "443") == 0; return strcmp(url->port, HTTP_PORT_HTTPS) == 0;
} }
return strcmp(url->port, "80") == 0; return strcmp(url->port, HTTP_PORT_HTTP) == 0;
}
// Copy headers, dropping credential headers when dropCreds (a redirect crossed to another origin)
// and/or body-describing headers when dropContent (a redirect dropped the request body). Returns
// an owned aggregate, or NULL on out of memory.
static CalogAggT *httpFilterHeaders(const CalogAggT *headers, bool dropCreds, bool dropContent) {
static const char *const creds[] = { "authorization", "cookie", "proxy-authorization", NULL };
static const char *const content[] = { "content-type", "content-length", "content-encoding", "transfer-encoding", NULL };
CalogAggT *out;
int64_t index;
if (calogAggCreate(&out, calogMapE) != calogOkE) {
return NULL;
}
for (index = 0; index < headers->pairCount; index++) {
const CalogValueT *key;
CalogValueT keyCopy;
CalogValueT valCopy;
key = &headers->pairs[index].key;
if (key->type == calogStringE && ((dropCreds && httpHeaderIn(key->as.s.bytes, key->as.s.length, creds)) || (dropContent && httpHeaderIn(key->as.s.bytes, key->as.s.length, content)))) {
continue;
}
if (calogValueCopy(&keyCopy, &headers->pairs[index].key) != calogOkE) {
calogAggFree(out);
return NULL;
}
if (calogValueCopy(&valCopy, &headers->pairs[index].value) != calogOkE) {
calogValueFree(&keyCopy);
calogAggFree(out);
return NULL;
}
if (calogAggSet(out, &keyCopy, &valCopy) != calogOkE) {
calogValueFree(&keyCopy);
calogValueFree(&valCopy);
calogAggFree(out);
return NULL;
}
}
return out;
}
// Perform a request and follow up to maxRedirects 3xx redirects (0 disables following, handing
// back the raw 3xx). Each hop reissues httpPerform against the resolved Location; the response
// map of every non-final hop is freed before the next. Method/body follow the usual rules:
// 303 (and 301/302 for a body-bearing method) drop to GET with no body, 307/308 are preserved.
static int32_t httpFollow(const char *method, const char *urlBytes, int64_t urlLen, const CalogAggT *headers, const char *body, int64_t bodyLen, bool verify, int32_t maxRedirects, CalogValueT *result) {
char curUrl[HTTP_URL_MAX];
char curMethod[HTTP_METHOD_MAX];
const char *curBody;
const CalogAggT *curHeaders;
CalogAggT *filtered; // owned; headers with credential/content headers stripped, or NULL
HttpUrlT origin; // scheme+host+port of the ORIGINAL request, for cross-origin checks
int64_t curBodyLen;
int32_t redirects;
int32_t status;
bool bodyDropped;
calogValueNil(result);
filtered = NULL;
curHeaders = headers;
bodyDropped = false;
if (urlLen < 0 || (size_t)urlLen >= sizeof(curUrl)) {
return calogFail(result, calogErrArgE, "http: URL too long");
}
if (strlen(method) >= sizeof(curMethod)) {
return calogFail(result, calogErrArgE, "http: method too long");
}
memcpy(curUrl, urlBytes, (size_t)urlLen);
curUrl[urlLen] = '\0';
strcpy(curMethod, method);
curBody = body;
curBodyLen = bodyLen;
status = httpParseUrl(curUrl, (int64_t)strlen(curUrl), &origin, result);
if (status != calogOkE) {
return status;
}
for (redirects = 0; ; redirects++) {
int32_t code;
const CalogValueT *location;
HttpUrlT base;
HttpUrlT target;
char locBuf[HTTP_URL_MAX];
char next[HTTP_URL_MAX];
size_t locLen;
status = httpPerform(curMethod, curUrl, (int64_t)strlen(curUrl), curHeaders, curBody, curBodyLen, verify, result);
if (status != calogOkE) {
goto done;
}
code = httpResultStatus(result);
if (maxRedirects <= 0 || !httpIsRedirect(code)) {
goto done; // redirects disabled, or a final (non-3xx) response
}
location = httpResultHeader(result, "location");
if (location == NULL || location->type != calogStringE || location->as.s.length == 0 || memchr(location->as.s.bytes, '\0', (size_t)location->as.s.length) != NULL) {
goto done; // a 3xx with no usable Location (missing, empty, or containing a NUL)
}
if (redirects >= maxRedirects) {
calogValueFree(result);
status = calogFail(result, calogErrArgE, "http: too many redirects");
goto done;
}
// Copy the Location out of the response map, then free the map (which the pointer aliases).
locLen = (size_t)location->as.s.length;
if (locLen >= sizeof(locBuf)) {
calogValueFree(result);
status = calogFail(result, calogErrArgE, "http: redirect Location too long");
goto done;
}
memcpy(locBuf, location->as.s.bytes, locLen);
locBuf[locLen] = '\0';
calogValueFree(result);
// Resolve the (possibly relative) Location against the URL just fetched.
status = httpParseUrl(curUrl, (int64_t)strlen(curUrl), &base, result);
if (status != calogOkE) {
goto done;
}
if (!httpResolveLocation(&base, locBuf, next, sizeof(next))) {
status = calogFail(result, calogErrArgE, "http: could not resolve redirect Location");
goto done;
}
status = httpParseUrl(next, (int64_t)strlen(next), &target, result);
if (status != calogOkE) {
goto done;
}
// 303 (and 301/302 for a body-bearing method) -> GET with no body; 307/308 keep both.
if (code == 303 || ((code == 301 || code == 302) && httpMethodHasBody(curMethod))) {
strcpy(curMethod, "GET");
curBody = NULL;
curBodyLen = 0;
bodyDropped = true;
}
// Choose the headers for the next hop: drop credential headers (Authorization/Cookie/...)
// when the target is a different origin -- so a secret is never resent to another host,
// including over an https->http downgrade -- and drop Content-* once the body is gone.
if (headers != NULL) {
bool dropCreds;
dropCreds = !httpSameOrigin(&origin, &target);
if (dropCreds || bodyDropped) {
if (filtered != NULL) {
calogAggFree(filtered);
}
filtered = httpFilterHeaders(headers, dropCreds, bodyDropped);
if (filtered == NULL) {
status = calogFail(result, calogErrOomE, "http: out of memory");
goto done;
}
curHeaders = filtered;
} else {
curHeaders = headers;
}
}
strcpy(curUrl, next);
}
done:
if (filtered != NULL) {
calogAggFree(filtered);
}
return status;
} }
@ -576,7 +843,53 @@ static int32_t httpGetNative(CalogValueT *args, int32_t argCount, CalogValueT *r
if (argCount != 1 || args[0].type != calogStringE) { if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "httpGet expects (url)"); return calogFail(result, calogErrArgE, "httpGet expects (url)");
} }
return httpPerform("GET", args[0].as.s.bytes, args[0].as.s.length, NULL, NULL, 0, true, result); return httpFollow("GET", args[0].as.s.bytes, args[0].as.s.length, NULL, NULL, 0, true, HTTP_MAX_REDIRECTS, result);
}
static bool httpHasBadByte(const char *bytes, size_t length, bool rejectSpace) {
size_t i;
// CR/LF would let a raw byte string smuggle extra header lines (or a second request) into
// the request line; a method additionally cannot carry a bare space.
for (i = 0; i < length; i++) {
if (bytes[i] == '\r' || bytes[i] == '\n') {
return true;
}
if (rejectSpace && bytes[i] == ' ') {
return true;
}
}
return false;
}
// Case-insensitive membership test of a (length-carried) header name in a NULL-terminated list
// of lowercase names.
static bool httpHeaderIn(const char *name, int64_t len, const char *const *list) {
size_t which;
for (which = 0; list[which] != NULL; which++) {
size_t nameLen;
size_t index;
bool match;
nameLen = strlen(list[which]);
if ((size_t)len != nameLen) {
continue;
}
match = true;
for (index = 0; index < nameLen; index++) {
if (httpLower(name[index]) != list[which][index]) {
match = false;
break;
}
}
if (match) {
return true;
}
}
return false;
} }
@ -594,6 +907,12 @@ static int32_t httpHexVal(char c) {
} }
// The 3xx codes that carry a Location this client follows (301/302/303 and 307/308).
static bool httpIsRedirect(int32_t code) {
return code == 301 || code == 302 || code == 303 || code == 307 || code == 308;
}
static char httpLower(char c) { static char httpLower(char c) {
if (c >= 'A' && c <= 'Z') { if (c >= 'A' && c <= 'Z') {
return (char)(c - 'A' + 'a'); return (char)(c - 'A' + 'a');
@ -622,44 +941,25 @@ static int32_t httpMapSetAgg(CalogAggT *map, const char *key, CalogAggT *inner)
} }
static int32_t httpMapSetInt(CalogAggT *map, const char *key, int64_t value) { static bool httpMethodHasBody(const char *method) {
CalogValueT keyValue; // RFC 9110 8.6: a request with a method that commonly carries content SHOULD send
CalogValueT intValue; // Content-Length even when the body is empty; strict servers/proxies otherwise answer
int32_t status; // 411 Length Required. GET/HEAD never carry content, so they alone are excluded.
return strcasecmp(method, "GET") != 0 && strcasecmp(method, "HEAD") != 0;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
calogValueInt(&intValue, value);
status = calogAggSet(map, &keyValue, &intValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
}
return status;
} }
static int32_t httpMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length) { static CalogValueT *httpOptGet(CalogAggT *opts, const char *name, int32_t *status) {
CalogValueT keyValue; CalogValueT keyValue;
CalogValueT stringValue; CalogValueT *value;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key)); *status = calogValueString(&keyValue, name, (int64_t)strlen(name));
if (status != calogOkE) { if (*status != calogOkE) {
return status; return NULL;
} }
status = calogValueString(&stringValue, bytes, length); value = calogAggGet(opts, &keyValue);
if (status != calogOkE) {
calogValueFree(&keyValue); calogValueFree(&keyValue);
return status; return value;
}
status = calogAggSet(map, &keyValue, &stringValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
calogValueFree(&stringValue);
}
return status;
} }
@ -690,6 +990,7 @@ static int64_t httpParseInt(const char *bytes, size_t length) {
static int32_t httpParseStatus(const char *line, size_t length, int32_t *codeOut) { static int32_t httpParseStatus(const char *line, size_t length, int32_t *codeOut) {
int32_t code; int32_t code;
size_t i; size_t i;
size_t digits;
bool any; bool any;
i = 0; i = 0;
@ -702,8 +1003,14 @@ static int32_t httpParseStatus(const char *line, size_t length, int32_t *codeOut
} }
code = 0; code = 0;
any = false; any = false;
digits = 0;
// RFC 7230 status codes are exactly 3 digits; cap accumulation there so a hostile/garbage
// status line (e.g. 11+ digits) cannot signed-overflow code (matches httpParseInt's guard).
while (i < length && line[i] >= '0' && line[i] <= '9') { while (i < length && line[i] >= '0' && line[i] <= '9') {
if (digits < 3) {
code = code * 10 + (int32_t)(line[i] - '0'); code = code * 10 + (int32_t)(line[i] - '0');
digits++;
}
any = true; any = true;
i++; i++;
} }
@ -732,13 +1039,33 @@ static int32_t httpParseUrl(const char *url, int64_t urlLen, HttpUrlT *out, Calo
return calogFail(result, calogErrArgE, "http: URL must start with http:// or https://"); return calogFail(result, calogErrArgE, "http: URL must start with http:// or https://");
} }
hostStart = pos; hostStart = pos;
if (pos < len && url[pos] == '[') {
// IPv6 literal host, e.g. "[::1]:8080" -- scan to the matching ']' instead of stopping
// at the first ':' (an IPv6 address is full of colons).
pos++;
hostStart = pos;
while (pos < len && url[pos] != ']') {
pos++;
}
if (pos >= len) {
return calogFail(result, calogErrArgE, "http: URL has an unterminated IPv6 host literal");
}
hostEnd = pos;
pos++; // consume ']'
out->ipv6 = true;
} else {
while (pos < len && url[pos] != ':' && url[pos] != '/') { while (pos < len && url[pos] != ':' && url[pos] != '/') {
pos++; pos++;
} }
hostEnd = pos; hostEnd = pos;
out->ipv6 = false;
}
if (hostEnd == hostStart) { if (hostEnd == hostStart) {
return calogFail(result, calogErrArgE, "http: URL has an empty host"); return calogFail(result, calogErrArgE, "http: URL has an empty host");
} }
if (httpHasBadByte(url + hostStart, hostEnd - hostStart, false)) {
return calogFail(result, calogErrArgE, "http: URL host contains invalid characters");
}
if (!httpCopyField(out->host, sizeof(out->host), url + hostStart, hostEnd - hostStart)) { if (!httpCopyField(out->host, sizeof(out->host), url + hostStart, hostEnd - hostStart)) {
return calogFail(result, calogErrArgE, "http: URL host too long"); return calogFail(result, calogErrArgE, "http: URL host too long");
} }
@ -756,9 +1083,12 @@ static int32_t httpParseUrl(const char *url, int64_t urlLen, HttpUrlT *out, Calo
return calogFail(result, calogErrArgE, "http: URL has an empty port"); return calogFail(result, calogErrArgE, "http: URL has an empty port");
} }
} else { } else {
strcpy(out->port, out->https ? "443" : "80"); strcpy(out->port, out->https ? HTTP_PORT_HTTPS : HTTP_PORT_HTTP);
} }
if (pos < len) { if (pos < len) {
if (httpHasBadByte(url + pos, len - pos, false)) {
return calogFail(result, calogErrArgE, "http: URL path contains invalid characters");
}
if (!httpCopyField(out->path, sizeof(out->path), url + pos, len - pos)) { if (!httpCopyField(out->path, sizeof(out->path), url + pos, len - pos)) {
return calogFail(result, calogErrArgE, "http: URL path too long"); return calogFail(result, calogErrArgE, "http: URL path too long");
} }
@ -800,9 +1130,15 @@ static int32_t httpPerform(const char *method, const char *urlBytes, int64_t url
if (status == calogOkE) { if (status == calogOkE) {
status = httpBufPutStr(&request, " HTTP/1.1\r\nHost: "); status = httpBufPutStr(&request, " HTTP/1.1\r\nHost: ");
} }
if (status == calogOkE && url.ipv6) {
status = httpBufPutStr(&request, "[");
}
if (status == calogOkE) { if (status == calogOkE) {
status = httpBufPutStr(&request, url.host); status = httpBufPutStr(&request, url.host);
} }
if (status == calogOkE && url.ipv6) {
status = httpBufPutStr(&request, "]");
}
if (status == calogOkE && !httpDefaultPort(&url)) { if (status == calogOkE && !httpDefaultPort(&url)) {
status = httpBufPutStr(&request, ":"); status = httpBufPutStr(&request, ":");
if (status == calogOkE) { if (status == calogOkE) {
@ -826,8 +1162,10 @@ static int32_t httpPerform(const char *method, const char *urlBytes, int64_t url
} }
} }
// Content-Length + the blank line + body. // Content-Length + the blank line + body. A method that commonly carries content still gets
if (bodyLen > 0) { // an explicit Content-Length: 0 even with an empty/absent body (see httpMethodHasBody);
// otherwise strict servers/proxies may answer 411 Length Required.
if (bodyLen > 0 || httpMethodHasBody(method)) {
char lengthLine[64]; char lengthLine[64];
int n; int n;
n = snprintf(lengthLine, sizeof(lengthLine), "Content-Length: %lld\r\n", (long long)bodyLen); n = snprintf(lengthLine, sizeof(lengthLine), "Content-Length: %lld\r\n", (long long)bodyLen);
@ -880,7 +1218,12 @@ static int32_t httpReadResponse(HttpConnT *conn, HttpBufT *raw) {
ssize_t n; ssize_t n;
int32_t status; int32_t status;
n = httpConnRead(conn, chunk, sizeof(chunk)); n = httpConnRead(conn, chunk, sizeof(chunk));
if (n <= 0) { if (n < 0) {
// A genuine read error (reset, unclean TLS shutdown) is not the same as a clean EOF
// -- do not hand a truncated response to the script as a success.
return calogErrArgE;
}
if (n == 0) {
break; break;
} }
if (raw->length + (size_t)n > (size_t)HTTP_MAX_RESPONSE) { if (raw->length + (size_t)n > (size_t)HTTP_MAX_RESPONSE) {
@ -902,12 +1245,13 @@ static int32_t httpRequestNative(CalogValueT *args, int32_t argCount, CalogValue
CalogValueT *headersValue; CalogValueT *headersValue;
CalogValueT *bodyValue; CalogValueT *bodyValue;
CalogValueT *insecureValue; CalogValueT *insecureValue;
CalogValueT keyValue; CalogValueT *redirectValue;
const CalogAggT *headers; const CalogAggT *headers;
const char *method; const char *method;
const char *body; const char *body;
int64_t bodyLen; int64_t bodyLen;
int32_t status; int32_t status;
int32_t maxRedirects;
bool verify; bool verify;
(void)userData; (void)userData;
@ -918,36 +1262,36 @@ static int32_t httpRequestNative(CalogValueT *args, int32_t argCount, CalogValue
opts = args[0].as.agg; opts = args[0].as.agg;
// url (required). // url (required).
status = calogValueString(&keyValue, "url", 3); status = calogOkE;
urlValue = httpOptGet(opts, "url", &status);
if (status != calogOkE) { if (status != calogOkE) {
return calogFail(result, status, "httpRequest: out of memory"); return calogFail(result, status, "httpRequest: out of memory");
} }
urlValue = calogAggGet(opts, &keyValue);
calogValueFree(&keyValue);
if (urlValue == NULL || urlValue->type != calogStringE) { if (urlValue == NULL || urlValue->type != calogStringE) {
return calogFail(result, calogErrArgE, "httpRequest: options.url must be a string"); return calogFail(result, calogErrArgE, "httpRequest: options.url must be a string");
} }
// method (optional; default GET). // method (optional; default GET).
method = "GET"; method = "GET";
status = calogValueString(&keyValue, "method", 6); status = calogOkE;
methodValue = httpOptGet(opts, "method", &status);
if (status != calogOkE) { if (status != calogOkE) {
return calogFail(result, status, "httpRequest: out of memory"); return calogFail(result, status, "httpRequest: out of memory");
} }
methodValue = calogAggGet(opts, &keyValue);
calogValueFree(&keyValue);
if (methodValue != NULL && methodValue->type == calogStringE) { if (methodValue != NULL && methodValue->type == calogStringE) {
method = methodValue->as.s.bytes; method = methodValue->as.s.bytes;
} }
if (httpHasBadByte(method, strlen(method), true)) {
return calogFail(result, calogErrArgE, "httpRequest: options.method contains invalid characters");
}
// headers (optional). // headers (optional).
headers = NULL; headers = NULL;
status = calogValueString(&keyValue, "headers", 7); status = calogOkE;
headersValue = httpOptGet(opts, "headers", &status);
if (status != calogOkE) { if (status != calogOkE) {
return calogFail(result, status, "httpRequest: out of memory"); return calogFail(result, status, "httpRequest: out of memory");
} }
headersValue = calogAggGet(opts, &keyValue);
calogValueFree(&keyValue);
if (headersValue != NULL && headersValue->type == calogAggE) { if (headersValue != NULL && headersValue->type == calogAggE) {
headers = headersValue->as.agg; headers = headersValue->as.agg;
} }
@ -955,12 +1299,11 @@ static int32_t httpRequestNative(CalogValueT *args, int32_t argCount, CalogValue
// body (optional). // body (optional).
body = NULL; body = NULL;
bodyLen = 0; bodyLen = 0;
status = calogValueString(&keyValue, "body", 4); status = calogOkE;
bodyValue = httpOptGet(opts, "body", &status);
if (status != calogOkE) { if (status != calogOkE) {
return calogFail(result, status, "httpRequest: out of memory"); return calogFail(result, status, "httpRequest: out of memory");
} }
bodyValue = calogAggGet(opts, &keyValue);
calogValueFree(&keyValue);
if (bodyValue != NULL && bodyValue->type == calogStringE) { if (bodyValue != NULL && bodyValue->type == calogStringE) {
body = bodyValue->as.s.bytes; body = bodyValue->as.s.bytes;
bodyLen = bodyValue->as.s.length; bodyLen = bodyValue->as.s.length;
@ -968,17 +1311,155 @@ static int32_t httpRequestNative(CalogValueT *args, int32_t argCount, CalogValue
// insecure (optional): true skips certificate + hostname verification for an https request. // insecure (optional): true skips certificate + hostname verification for an https request.
verify = true; verify = true;
status = calogValueString(&keyValue, "insecure", 8); status = calogOkE;
insecureValue = httpOptGet(opts, "insecure", &status);
if (status != calogOkE) { if (status != calogOkE) {
return calogFail(result, status, "httpRequest: out of memory"); return calogFail(result, status, "httpRequest: out of memory");
} }
insecureValue = calogAggGet(opts, &keyValue);
calogValueFree(&keyValue);
if (insecureValue != NULL && insecureValue->type == calogBoolE && insecureValue->as.b) { if (insecureValue != NULL && insecureValue->type == calogBoolE && insecureValue->as.b) {
verify = false; verify = false;
} }
return httpPerform(method, urlValue->as.s.bytes, urlValue->as.s.length, headers, body, bodyLen, verify, result); // maxRedirects (optional): how many 3xx redirects to follow. Default 16; 0 (or negative)
// disables following and returns the raw 3xx. Values above 16 are capped, so a caller can
// never widen the loop guard.
maxRedirects = HTTP_MAX_REDIRECTS;
status = calogOkE;
redirectValue = httpOptGet(opts, "maxRedirects", &status);
if (status != calogOkE) {
return calogFail(result, status, "httpRequest: out of memory");
}
if (redirectValue != NULL && redirectValue->type == calogIntE) {
if (redirectValue->as.i < 0) {
maxRedirects = 0;
} else if (redirectValue->as.i < HTTP_MAX_REDIRECTS) {
maxRedirects = (int32_t)redirectValue->as.i;
}
}
return httpFollow(method, urlValue->as.s.bytes, urlValue->as.s.length, headers, body, bodyLen, verify, maxRedirects, result);
}
// Resolve a redirect Location against the URL just fetched, writing an absolute URL into out.
// Handles an absolute URL, a protocol-relative "//host/path", an absolute path "/path", and a
// relative path (merged against the base path's directory). Leading whitespace is skipped;
// dot-segments (../) are NOT normalized. Returns false on overflow.
static bool httpResolveLocation(const HttpUrlT *base, const char *loc, char *out, size_t cap) {
const char *scheme;
int authority;
while (*loc == ' ' || *loc == '\t') {
loc++;
}
if (strncmp(loc, "http://", 7) == 0 || strncmp(loc, "https://", 8) == 0) {
if (strlen(loc) >= cap) {
return false;
}
strcpy(out, loc);
return true;
}
scheme = base->https ? "https" : "http";
if (loc[0] == '/' && loc[1] == '/') {
int n;
n = snprintf(out, cap, "%s:%s", scheme, loc);
return n > 0 && (size_t)n < cap;
}
authority = httpWriteAuthority(out, cap, scheme, base);
if (authority < 0) {
return false;
}
if (loc[0] == '/') {
if ((size_t)authority + strlen(loc) >= cap) {
return false;
}
strcpy(out + authority, loc);
return true;
}
if (loc[0] == '?' || loc[0] == '#') {
// Query- or fragment-only reference (RFC 3986 5.3): keep the whole base path, replace the
// rest -- NOT the directory-merge below, which would drop the base path's last segment.
size_t basePathLen;
basePathLen = strcspn(base->path, "?#");
if ((size_t)authority + basePathLen + strlen(loc) >= cap) {
return false;
}
memcpy(out + authority, base->path, basePathLen);
strcpy(out + (size_t)authority + basePathLen, loc);
return true;
}
// A relative path: append the base path's directory (up to and including its last '/', within
// the path part before any query/fragment), then loc.
{
size_t pathLen;
size_t dirLen;
size_t index;
pathLen = strcspn(base->path, "?#");
dirLen = 1; // base->path always begins with '/'
for (index = 0; index < pathLen; index++) {
if (base->path[index] == '/') {
dirLen = index + 1;
}
}
if ((size_t)authority + dirLen + strlen(loc) >= cap) {
return false;
}
memcpy(out + authority, base->path, dirLen);
strcpy(out + (size_t)authority + dirLen, loc);
return true;
}
}
// A response header value (name lowercase, as httpBuildResult keys them) from a built result
// map, or NULL. Aliases the map; valid until it is freed.
static const CalogValueT *httpResultHeader(const CalogValueT *result, const char *name) {
CalogValueT *headers;
if (result->type != calogAggE) {
return NULL;
}
headers = httpAggField(result->as.agg, "headers");
if (headers == NULL || headers->type != calogAggE) {
return NULL;
}
return httpAggField(headers->as.agg, name);
}
// The integer status code from a built result map, or -1 if absent/malformed.
static int32_t httpResultStatus(const CalogValueT *result) {
CalogValueT *value;
if (result->type != calogAggE) {
return -1;
}
value = httpAggField(result->as.agg, "status");
if (value == NULL || value->type != calogIntE) {
return -1;
}
return (int32_t)value->as.i;
}
// Same origin = same scheme, host, and port -- the boundary at which credential headers are
// dropped from a followed redirect.
static bool httpSameOrigin(const HttpUrlT *a, const HttpUrlT *b) {
return a->https == b->https && strcmp(a->host, b->host) == 0 && strcmp(a->port, b->port) == 0;
}
static void httpSetTimeouts(int fd, int timeoutSec) {
struct timeval tv;
tv.tv_sec = timeoutSec;
tv.tv_usec = 0;
// Best effort: if the kernel rejects these (unusual), the connection just falls back to no
// read/write deadline rather than failing the request outright.
(void)setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
(void)setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &tv, sizeof(tv));
} }
@ -1017,3 +1498,21 @@ static int32_t httpTlsHandshake(HttpConnT *conn, const HttpUrlT *url, CalogValue
} }
return calogOkE; return calogOkE;
} }
// Write "scheme://host" (bracketing an IPv6 host, and adding ":port" when non-default) into out.
// Returns the number of bytes written (excluding the NUL), or -1 if it does not fit.
static int httpWriteAuthority(char *out, size_t cap, const char *scheme, const HttpUrlT *base) {
int n;
if (base->ipv6 && !httpDefaultPort(base)) {
n = snprintf(out, cap, "%s://[%s]:%s", scheme, base->host, base->port);
} else if (base->ipv6) {
n = snprintf(out, cap, "%s://[%s]", scheme, base->host);
} else if (!httpDefaultPort(base)) {
n = snprintf(out, cap, "%s://%s:%s", scheme, base->host, base->port);
} else {
n = snprintf(out, cap, "%s://%s", scheme, base->host);
}
return (n > 0 && (size_t)n < cap) ? n : -1;
}

View file

@ -5,11 +5,13 @@
// httpGet(url) -> { status, body, headers } // httpGet(url) -> { status, body, headers }
// httpRequest(opts) -> { status, body, headers } // httpRequest(opts) -> { status, body, headers }
// where opts is a map { method (default "GET"), url, headers (map of name->value), body, // where opts is a map { method (default "GET"), url, headers (map of name->value), body,
// insecure (bool) }. The result map carries the integer status code, the binary-safe response // insecure (bool), maxRedirects (int) }. The result map carries the integer status code, the
// body, and a headers map keyed by the lowercased header name. Each call opens its own // binary-safe response body, and a headers map keyed by the lowercased header name. Each call
// connection ("Connection: close"), reads the whole response to EOF, and decodes the body // opens its own connection ("Connection: close"), reads the whole response to EOF, and decodes
// honoring Transfer-Encoding: chunked / Content-Length / read-to-close. Redirects are NOT // the body honoring Transfer-Encoding: chunked / Content-Length / read-to-close. 3xx redirects
// followed (a 3xx is returned as-is). For https:// the TLS handshake uses SNI and, by DEFAULT, // ARE followed (up to maxRedirects, default 16, capped at 16 to break loops; 0 disables and
// returns the raw 3xx). 303 -- and 301/302 for a body-bearing method -- become a bodyless GET;
// 307/308 keep the method and body. For https:// the TLS handshake uses SNI and, by DEFAULT,
// verifies the server certificate chain against the system CA store and checks it matches the // verifies the server certificate chain against the system CA store and checks it matches the
// requested hostname (an untrusted or mismatched certificate fails the request). Pass // requested hostname (an untrusted or mismatched certificate fails the request). Pass
// insecure=true to httpRequest to skip verification (self-signed / development endpoints). // insecure=true to httpRequest to skip verification (self-signed / development endpoints).

View file

@ -5,6 +5,7 @@
#define _POSIX_C_SOURCE 200809L #define _POSIX_C_SOURCE 200809L
#include "calogJson.h" #include "calogJson.h"
#include "calogInternal.h"
#include <errno.h> #include <errno.h>
#include <math.h> #include <math.h>
@ -145,10 +146,16 @@ static int32_t jsonEncodeAgg(JsonBufT *buf, const CalogAggT *agg, int32_t depth)
// A map (or a hybrid that carries keyed pairs) serializes as a JSON object; a pure // A map (or a hybrid that carries keyed pairs) serializes as a JSON object; a pure
// sequence serializes as a JSON array. In the hybrid case the array elements are emitted // sequence serializes as a JSON array. In the hybrid case the array elements are emitted
// under their integer indices so nothing is silently dropped. // under their integer indices, unless a pair key would render to the identical JSON key
// (e.g. an explicit key 0 alongside array index 0) -- serializing that would produce a
// duplicate object key and silently drop the array element on re-parse, so we refuse.
if (agg->pairCount > 0 || agg->kind == calogMapE) { if (agg->pairCount > 0 || agg->kind == calogMapE) {
bool first; bool first;
JsonBufT keyScratch;
first = true; first = true;
keyScratch.bytes = NULL;
keyScratch.length = 0;
keyScratch.cap = 0;
status = jsonBufPutChar(buf, '{'); status = jsonBufPutChar(buf, '{');
if (status != calogOkE) { if (status != calogOkE) {
return status; return status;
@ -156,23 +163,46 @@ static int32_t jsonEncodeAgg(JsonBufT *buf, const CalogAggT *agg, int32_t depth)
for (index = 0; index < agg->arrayCount; index++) { for (index = 0; index < agg->arrayCount; index++) {
char tmp[32]; char tmp[32];
int n; int n;
int64_t pairIndex;
bool collides;
n = snprintf(tmp, sizeof(tmp), "\"%lld\":", (long long)index);
collides = false;
for (pairIndex = 0; pairIndex < agg->pairCount; pairIndex++) {
keyScratch.length = 0;
status = jsonEncodeKey(&keyScratch, &agg->pairs[pairIndex].key);
if (status != calogOkE) {
jsonBufFree(&keyScratch);
return status;
}
if ((size_t)(n - 1) == keyScratch.length && memcmp(tmp, keyScratch.bytes, keyScratch.length) == 0) {
collides = true;
break;
}
}
if (collides) {
jsonBufFree(&keyScratch);
return calogErrTypeE;
}
if (!first) { if (!first) {
status = jsonBufPutChar(buf, ','); status = jsonBufPutChar(buf, ',');
if (status != calogOkE) { if (status != calogOkE) {
jsonBufFree(&keyScratch);
return status; return status;
} }
} }
first = false; first = false;
n = snprintf(tmp, sizeof(tmp), "\"%lld\":", (long long)index);
status = jsonBufPutBytes(buf, tmp, (size_t)n); status = jsonBufPutBytes(buf, tmp, (size_t)n);
if (status != calogOkE) { if (status != calogOkE) {
jsonBufFree(&keyScratch);
return status; return status;
} }
status = jsonEncode(buf, &agg->array[index], depth + 1); status = jsonEncode(buf, &agg->array[index], depth + 1);
if (status != calogOkE) { if (status != calogOkE) {
jsonBufFree(&keyScratch);
return status; return status;
} }
} }
jsonBufFree(&keyScratch);
for (index = 0; index < agg->pairCount; index++) { for (index = 0; index < agg->pairCount; index++) {
if (!first) { if (!first) {
status = jsonBufPutChar(buf, ','); status = jsonBufPutChar(buf, ',');
@ -315,13 +345,8 @@ static int32_t jsonHexQuad(JsonParseT *p, uint32_t *out) {
if (p->cur >= p->end) { if (p->cur >= p->end) {
return calogErrArgE; return calogErrArgE;
} }
if (*p->cur >= '0' && *p->cur <= '9') { digit = calogHexNibble((unsigned char)*p->cur);
digit = *p->cur - '0'; if (digit < 0) {
} else if (*p->cur >= 'a' && *p->cur <= 'f') {
digit = *p->cur - 'a' + 10;
} else if (*p->cur >= 'A' && *p->cur <= 'F') {
digit = *p->cur - 'A' + 10;
} else {
return calogErrArgE; return calogErrArgE;
} }
value = (value << 4) | (uint32_t)digit; value = (value << 4) | (uint32_t)digit;
@ -419,12 +444,10 @@ static int32_t jsonParseNative(CalogValueT *args, int32_t argCount, CalogValueT
parse.end = args[0].as.s.bytes + args[0].as.s.length; parse.end = args[0].as.s.bytes + args[0].as.s.length;
status = jsonParseValue(&parse, result, 0); status = jsonParseValue(&parse, result, 0);
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(result);
return calogFail(result, status, "jsonParse: invalid JSON"); return calogFail(result, status, "jsonParse: invalid JSON");
} }
jsonSkipWs(&parse); jsonSkipWs(&parse);
if (parse.cur != parse.end) { if (parse.cur != parse.end) {
calogValueFree(result);
return calogFail(result, calogErrArgE, "jsonParse: trailing characters after JSON value"); return calogFail(result, calogErrArgE, "jsonParse: trailing characters after JSON value");
} }
return calogOkE; return calogOkE;
@ -436,27 +459,54 @@ static int32_t jsonParseNumber(JsonParseT *p, CalogValueT *out) {
char *stop; char *stop;
bool isReal; bool isReal;
// Strict RFC 8259 grammar: number = [ "-" ] int [ frac ] [ exp ], where int is either a
// single "0" or a non-zero digit followed by more digits (no other leading zeros), frac
// requires at least one digit after ".", and exp requires at least one digit after "e"/"E"
// and an optional sign. This rejects forms strtod/strtoll alone would accept ("+1", ".5",
// "5.", "01"); strtod/strtoll below only compute the value of text already known-valid.
calogValueNil(out); calogValueNil(out);
start = p->cur; start = p->cur;
isReal = false; isReal = false;
if (p->cur < p->end && *p->cur == '-') { if (p->cur < p->end && *p->cur == '-') {
p->cur++; p->cur++;
} }
while (p->cur < p->end) { if (p->cur >= p->end || *p->cur < '0' || *p->cur > '9') {
char c; return calogErrArgE;
c = *p->cur; }
if (c >= '0' && c <= '9') { if (*p->cur == '0') {
p->cur++;
} else if (c == '.' || c == 'e' || c == 'E' || c == '+' || c == '-') {
isReal = true;
p->cur++; p->cur++;
} else { } else {
break; while (p->cur < p->end && *p->cur >= '0' && *p->cur <= '9') {
p->cur++;
} }
} }
if (p->cur == start) { if (p->cur < p->end && *p->cur == '.') {
const char *fracStart;
isReal = true;
p->cur++;
fracStart = p->cur;
while (p->cur < p->end && *p->cur >= '0' && *p->cur <= '9') {
p->cur++;
}
if (p->cur == fracStart) {
return calogErrArgE; return calogErrArgE;
} }
}
if (p->cur < p->end && (*p->cur == 'e' || *p->cur == 'E')) {
const char *expStart;
isReal = true;
p->cur++;
if (p->cur < p->end && (*p->cur == '+' || *p->cur == '-')) {
p->cur++;
}
expStart = p->cur;
while (p->cur < p->end && *p->cur >= '0' && *p->cur <= '9') {
p->cur++;
}
if (p->cur == expStart) {
return calogErrArgE;
}
}
if (isReal) { if (isReal) {
double r; double r;
r = strtod(start, &stop); r = strtod(start, &stop);

View file

@ -22,8 +22,7 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define KV_INITIAL_CAP 8 #include "calogInternal.h"
#define KV_GROWTH 2
typedef struct KvEntryT { typedef struct KvEntryT {
char *keyBytes; char *keyBytes;
@ -43,6 +42,7 @@ static int32_t gRefCount = 0;
static bool kvContainsFn(const CalogValueT *value, int32_t depth); static bool kvContainsFn(const CalogValueT *value, int32_t depth);
static int32_t kvDelete(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t kvDelete(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t kvFindLocked(const char *bytes, int64_t length); static int32_t kvFindLocked(const char *bytes, int64_t length);
static void kvFreeAll(void);
static int32_t kvGet(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t kvGet(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t kvHas(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t kvHas(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t kvKeys(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t kvKeys(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
@ -50,9 +50,7 @@ static int32_t kvSet(CalogValueT *args, int32_t argCount, CalogValueT *result, v
int32_t calogKvRegister(CalogT *calog) { int32_t calogKvRegister(CalogT *calog) {
pthread_mutex_lock(&gInitMutex); calogRegistryRetain(&gInitMutex, &gRefCount);
gRefCount++;
pthread_mutex_unlock(&gInitMutex);
calogRegisterInline(calog, "kvSet", kvSet, NULL); calogRegisterInline(calog, "kvSet", kvSet, NULL);
calogRegisterInline(calog, "kvGet", kvGet, NULL); calogRegisterInline(calog, "kvGet", kvGet, NULL);
calogRegisterInline(calog, "kvHas", kvHas, NULL); calogRegisterInline(calog, "kvHas", kvHas, NULL);
@ -63,24 +61,8 @@ int32_t calogKvRegister(CalogT *calog) {
void calogKvShutdown(void) { void calogKvShutdown(void) {
pthread_mutex_lock(&gInitMutex); // calogRegistryRelease holds gInitMutex for the whole call, including kvFreeAll below.
if (gRefCount > 0) { calogRegistryRelease(&gInitMutex, &gRefCount, kvFreeAll);
gRefCount--;
}
if (gRefCount <= 0) {
int32_t index;
pthread_mutex_lock(&gMapMutex);
for (index = 0; index < gCount; index++) {
free(gEntries[index].keyBytes);
calogValueFree(&gEntries[index].value);
}
free(gEntries);
gEntries = NULL;
gCount = 0;
gCap = 0;
pthread_mutex_unlock(&gMapMutex);
}
pthread_mutex_unlock(&gInitMutex);
} }
@ -148,6 +130,24 @@ static int32_t kvFindLocked(const char *bytes, int64_t length) {
} }
// Invoked by calogRegistryRelease while gInitMutex is held, exactly once, when the last
// reference drops.
static void kvFreeAll(void) {
int32_t index;
pthread_mutex_lock(&gMapMutex);
for (index = 0; index < gCount; index++) {
free(gEntries[index].keyBytes);
calogValueFree(&gEntries[index].value);
}
free(gEntries);
gEntries = NULL;
gCount = 0;
gCap = 0;
pthread_mutex_unlock(&gMapMutex);
}
static int32_t kvGet(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t kvGet(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
int32_t index; int32_t index;
int32_t status; int32_t status;
@ -165,7 +165,6 @@ static int32_t kvGet(CalogValueT *args, int32_t argCount, CalogValueT *result, v
} }
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(result);
return calogFail(result, status, "kvGet: out of memory"); return calogFail(result, status, "kvGet: out of memory");
} }
return calogOkE; return calogOkE;
@ -225,6 +224,8 @@ static int32_t kvKeys(CalogValueT *args, int32_t argCount, CalogValueT *result,
static int32_t kvSet(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t kvSet(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
void *buffer;
int64_t capacity;
CalogValueT copy; CalogValueT copy;
char *keyBytes; char *keyBytes;
int64_t keyLen; int64_t keyLen;
@ -241,7 +242,6 @@ static int32_t kvSet(CalogValueT *args, int32_t argCount, CalogValueT *result, v
} }
status = calogValueCopy(&copy, &args[1]); status = calogValueCopy(&copy, &args[1]);
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(&copy);
return calogFail(result, status, "kvSet: out of memory"); return calogFail(result, status, "kvSet: out of memory");
} }
keyLen = args[0].as.s.length; keyLen = args[0].as.s.length;
@ -254,19 +254,16 @@ static int32_t kvSet(CalogValueT *args, int32_t argCount, CalogValueT *result, v
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
return calogOkE; return calogOkE;
} }
if (gCount == gCap) { buffer = gEntries;
int32_t newCap; capacity = gCap;
KvEntryT *grown; status = calogGrow(&buffer, &capacity, (int64_t)gCount + 1, sizeof(KvEntryT));
newCap = (gCap == 0) ? KV_INITIAL_CAP : gCap * KV_GROWTH; if (status != calogOkE) {
grown = (KvEntryT *)realloc(gEntries, (size_t)newCap * sizeof(KvEntryT));
if (grown == NULL) {
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);
calogValueFree(&copy); calogValueFree(&copy);
return calogFail(result, calogErrOomE, "kvSet: out of memory"); return calogFail(result, status, "kvSet: out of memory");
}
gEntries = grown;
gCap = newCap;
} }
gEntries = (KvEntryT *)buffer;
gCap = (int32_t)capacity;
keyBytes = (char *)malloc((size_t)keyLen + 1); keyBytes = (char *)malloc((size_t)keyLen + 1);
if (keyBytes == NULL) { if (keyBytes == NULL) {
pthread_mutex_unlock(&gMapMutex); pthread_mutex_unlock(&gMapMutex);

View file

@ -7,6 +7,7 @@
#include "calogNet.h" #include "calogNet.h"
#include "calogHandle.h" #include "calogHandle.h"
#include "calogInternal.h"
#include <arpa/inet.h> #include <arpa/inet.h>
#include <errno.h> #include <errno.h>
@ -46,6 +47,12 @@ typedef struct NetLibT {
int32_t refCount; int32_t refCount;
} NetLibT; } NetLibT;
// One row of the registration table below: native name paired with its implementation.
typedef struct NetNativeT {
const char *name;
CalogNativeFnT fn;
} NetNativeT;
static pthread_mutex_t gNetLibMutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t gNetLibMutex = PTHREAD_MUTEX_INITIALIZER;
static NetLibT *gNetLib = NULL; static NetLibT *gNetLib = NULL;
@ -56,10 +63,11 @@ static int32_t enetHost(CalogValueT *args, int32_t argCount, CalogValueT *result
static int32_t enetSend(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t enetSend(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t enetService(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t enetService(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void netCloser(uint32_t type, void *resource); static void netCloser(uint32_t type, void *resource);
static int32_t netMapSetInt(CalogAggT *map, const char *key, int64_t value);
static int32_t netMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length);
static int32_t netOpenBound(uint16_t port, int socktype, bool doListen, CalogValueT *result, int *fdOut); static int32_t netOpenBound(uint16_t port, int socktype, bool doListen, CalogValueT *result, int *fdOut);
static bool netPortOk(int64_t port);
static int netResolve(const char *host, uint16_t port, int socktype, bool passive, struct addrinfo **out); static int netResolve(const char *host, uint16_t port, int socktype, bool passive, struct addrinfo **out);
static int32_t netSocketClose(NetLibT *lib, int64_t handleId, uint32_t type1, uint32_t type2, CalogValueT *result, const char *message);
static void netSocketFree(NetSocketT *sock);
static int32_t netStore(NetLibT *lib, int fd, uint32_t type, CalogValueT *result); static int32_t netStore(NetLibT *lib, int fd, uint32_t type, CalogValueT *result);
static int32_t tcpAccept(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t tcpAccept(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t tcpClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t tcpClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
@ -72,48 +80,71 @@ static int32_t udpOpen(CalogValueT *args, int32_t argCount, CalogValueT *result,
static int32_t udpRecvFrom(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t udpRecvFrom(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t udpSendTo(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t udpSendTo(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
// Table-driven registration so a failed calogRegisterInline call can be detected (and the
// whole batch unwound) instead of the return value being silently discarded per call.
static const NetNativeT gNetNatives[] = {
{ "tcpConnect", tcpConnect },
{ "tcpListen", tcpListen },
{ "tcpAccept", tcpAccept },
{ "tcpSend", tcpSend },
{ "tcpRecv", tcpRecv },
{ "tcpClose", tcpClose },
{ "udpOpen", udpOpen },
{ "udpSendTo", udpSendTo },
{ "udpRecvFrom", udpRecvFrom },
{ "udpClose", udpClose },
{ "enetHost", enetHost },
{ "enetConnect", enetConnect },
{ "enetService", enetService },
{ "enetSend", enetSend },
{ "enetDisconnect", enetDisconnect },
{ "enetClose", enetClose },
};
int32_t calogNetRegister(CalogT *calog) { int32_t calogNetRegister(CalogT *calog) {
NetLibT *lib;
int32_t status;
size_t nativeIndex;
pthread_mutex_lock(&gNetLibMutex); pthread_mutex_lock(&gNetLibMutex);
if (gNetLib == NULL) { if (gNetLib == NULL) {
NetLibT *lib; NetLibT *newLib;
lib = (NetLibT *)calloc(1, sizeof(*lib)); newLib = (NetLibT *)calloc(1, sizeof(*newLib));
if (lib == NULL) { if (newLib == NULL) {
pthread_mutex_unlock(&gNetLibMutex); pthread_mutex_unlock(&gNetLibMutex);
return calogErrOomE; return calogErrOomE;
} }
lib->handles = calogHandleTableCreate(); newLib->handles = calogHandleTableCreate();
if (lib->handles == NULL) { if (newLib->handles == NULL) {
free(lib); free(newLib);
pthread_mutex_unlock(&gNetLibMutex); pthread_mutex_unlock(&gNetLibMutex);
return calogErrOomE; return calogErrOomE;
} }
if (enet_initialize() != 0) { if (enet_initialize() != 0) {
calogHandleTableDestroy(lib->handles, NULL); calogHandleTableDestroy(newLib->handles, NULL);
free(lib); free(newLib);
pthread_mutex_unlock(&gNetLibMutex); pthread_mutex_unlock(&gNetLibMutex);
return calogErrOomE; return calogErrOomE;
} }
gNetLib = lib; gNetLib = newLib;
} }
gNetLib->refCount++; gNetLib->refCount++;
lib = gNetLib;
pthread_mutex_unlock(&gNetLibMutex); pthread_mutex_unlock(&gNetLibMutex);
calogRegisterInline(calog, "tcpConnect", tcpConnect, gNetLib); status = calogOkE;
calogRegisterInline(calog, "tcpListen", tcpListen, gNetLib); for (nativeIndex = 0; nativeIndex < sizeof(gNetNatives) / sizeof(gNetNatives[0]); nativeIndex++) {
calogRegisterInline(calog, "tcpAccept", tcpAccept, gNetLib); status = calogRegisterInline(calog, gNetNatives[nativeIndex].name, gNetNatives[nativeIndex].fn, lib);
calogRegisterInline(calog, "tcpSend", tcpSend, gNetLib); if (status != calogOkE) {
calogRegisterInline(calog, "tcpRecv", tcpRecv, gNetLib); break;
calogRegisterInline(calog, "tcpClose", tcpClose, gNetLib); }
calogRegisterInline(calog, "udpOpen", udpOpen, gNetLib); }
calogRegisterInline(calog, "udpSendTo", udpSendTo, gNetLib); if (status != calogOkE) {
calogRegisterInline(calog, "udpRecvFrom", udpRecvFrom, gNetLib); // Roll back the refcount bump (and, if we were the sole holder, the whole registry)
calogRegisterInline(calog, "udpClose", udpClose, gNetLib); // so a partially-registered runtime does not leave a phantom reference behind.
calogRegisterInline(calog, "enetHost", enetHost, gNetLib); calogNetShutdown();
calogRegisterInline(calog, "enetConnect", enetConnect, gNetLib); return status;
calogRegisterInline(calog, "enetService", enetService, gNetLib); }
calogRegisterInline(calog, "enetSend", enetSend, gNetLib);
calogRegisterInline(calog, "enetDisconnect", enetDisconnect, gNetLib);
calogRegisterInline(calog, "enetClose", enetClose, gNetLib);
return calogOkE; return calogOkE;
} }
@ -139,13 +170,9 @@ static void netCloser(uint32_t type, void *resource) {
switch (type) { switch (type) {
case NET_TYPE_TCP: case NET_TYPE_TCP:
case NET_TYPE_TCP_LISTEN: case NET_TYPE_TCP_LISTEN:
case NET_TYPE_UDP: { case NET_TYPE_UDP:
NetSocketT *sock; netSocketFree((NetSocketT *)resource);
sock = (NetSocketT *)resource;
close(sock->fd);
free(sock);
break; break;
}
case NET_TYPE_ENET_HOST: case NET_TYPE_ENET_HOST:
enet_host_destroy((ENetHost *)resource); enet_host_destroy((ENetHost *)resource);
break; break;
@ -158,50 +185,6 @@ static void netCloser(uint32_t type, void *resource) {
} }
// Set map[key] = integer. On failure the (freshly built) key is released; the scalar value
// needs none.
static int32_t netMapSetInt(CalogAggT *map, const char *key, int64_t value) {
CalogValueT keyValue;
CalogValueT intValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
calogValueInt(&intValue, value);
status = calogAggSet(map, &keyValue, &intValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
}
return status;
}
// Set map[key] = binary-safe string. On failure any built values are released.
static int32_t netMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length) {
CalogValueT keyValue;
CalogValueT stringValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
status = calogValueString(&stringValue, bytes, length);
if (status != calogOkE) {
calogValueFree(&keyValue);
return status;
}
status = calogAggSet(map, &keyValue, &stringValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
calogValueFree(&stringValue);
}
return status;
}
// Create a socket bound to the given local port (0 = ephemeral), optionally listening. // Create a socket bound to the given local port (0 = ephemeral), optionally listening.
// Returns calogOkE with *fdOut set, or an error with result populated. // Returns calogOkE with *fdOut set, or an error with result populated.
static int32_t netOpenBound(uint16_t port, int socktype, bool doListen, CalogValueT *result, int *fdOut) { static int32_t netOpenBound(uint16_t port, int socktype, bool doListen, CalogValueT *result, int *fdOut) {
@ -245,6 +228,13 @@ static int32_t netOpenBound(uint16_t port, int socktype, bool doListen, CalogVal
} }
// True if port is a valid IPv4/IPv6 port number (0 = ephemeral is allowed by callers that
// permit it; this only checks the range).
static bool netPortOk(int64_t port) {
return port >= 0 && port <= NET_PORT_MAX;
}
static int netResolve(const char *host, uint16_t port, int socktype, bool passive, struct addrinfo **out) { static int netResolve(const char *host, uint16_t port, int socktype, bool passive, struct addrinfo **out) {
struct addrinfo hints; struct addrinfo hints;
char portBuffer[8]; char portBuffer[8];
@ -260,6 +250,31 @@ static int netResolve(const char *host, uint16_t port, int socktype, bool passiv
} }
// Remove and close a socket handle, trying type1 then (if non-zero) type2. Used by tcpClose
// (TCP + TCP_LISTEN) and udpClose (UDP alone).
static int32_t netSocketClose(NetLibT *lib, int64_t handleId, uint32_t type1, uint32_t type2, CalogValueT *result, const char *message) {
NetSocketT *sock;
sock = (NetSocketT *)calogHandleRemove(lib->handles, handleId, type1);
if (sock == NULL && type2 != 0) {
sock = (NetSocketT *)calogHandleRemove(lib->handles, handleId, type2);
}
if (sock == NULL) {
return calogFail(result, calogErrArgE, message);
}
netSocketFree(sock);
return calogOkE;
}
// Close the fd and release the NetSocketT. Shared by netSocketClose and the handle-table
// teardown path (netCloser).
static void netSocketFree(NetSocketT *sock) {
close(sock->fd);
free(sock);
}
// Wrap an open fd in a handle-table entry, transferring ownership. On failure the fd is // Wrap an open fd in a handle-table entry, transferring ownership. On failure the fd is
// closed. Sets result to the new integer handle on success. // closed. Sets result to the new integer handle on success.
static int32_t netStore(NetLibT *lib, int fd, uint32_t type, CalogValueT *result) { static int32_t netStore(NetLibT *lib, int fd, uint32_t type, CalogValueT *result) {
@ -307,23 +322,13 @@ static int32_t tcpAccept(CalogValueT *args, int32_t argCount, CalogValueT *resul
static int32_t tcpClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t tcpClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
NetLibT *lib; NetLibT *lib;
NetSocketT *sock;
lib = (NetLibT *)userData; lib = (NetLibT *)userData;
calogValueNil(result); calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "tcpClose expects (handle)"); return calogFail(result, calogErrArgE, "tcpClose expects (handle)");
} }
sock = (NetSocketT *)calogHandleRemove(lib->handles, args[0].as.i, NET_TYPE_TCP); return netSocketClose(lib, args[0].as.i, NET_TYPE_TCP, NET_TYPE_TCP_LISTEN, result, "tcpClose: invalid handle");
if (sock == NULL) {
sock = (NetSocketT *)calogHandleRemove(lib->handles, args[0].as.i, NET_TYPE_TCP_LISTEN);
}
if (sock == NULL) {
return calogFail(result, calogErrArgE, "tcpClose: invalid handle");
}
close(sock->fd);
free(sock);
return calogOkE;
} }
@ -339,7 +344,7 @@ static int32_t tcpConnect(CalogValueT *args, int32_t argCount, CalogValueT *resu
if (argCount != 2 || args[0].type != calogStringE || args[1].type != calogIntE) { if (argCount != 2 || args[0].type != calogStringE || args[1].type != calogIntE) {
return calogFail(result, calogErrArgE, "tcpConnect expects (host, port)"); return calogFail(result, calogErrArgE, "tcpConnect expects (host, port)");
} }
if (args[1].as.i < 0 || args[1].as.i > NET_PORT_MAX) { if (!netPortOk(args[1].as.i)) {
return calogFail(result, calogErrArgE, "tcpConnect: port out of range"); return calogFail(result, calogErrArgE, "tcpConnect: port out of range");
} }
rc = netResolve(args[0].as.s.bytes, (uint16_t)args[1].as.i, SOCK_STREAM, false, &res); rc = netResolve(args[0].as.s.bytes, (uint16_t)args[1].as.i, SOCK_STREAM, false, &res);
@ -376,7 +381,7 @@ static int32_t tcpListen(CalogValueT *args, int32_t argCount, CalogValueT *resul
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "tcpListen expects (port)"); return calogFail(result, calogErrArgE, "tcpListen expects (port)");
} }
if (args[0].as.i < 0 || args[0].as.i > NET_PORT_MAX) { if (!netPortOk(args[0].as.i)) {
return calogFail(result, calogErrArgE, "tcpListen: port out of range"); return calogFail(result, calogErrArgE, "tcpListen: port out of range");
} }
status = netOpenBound((uint16_t)args[0].as.i, SOCK_STREAM, true, result, &fd); status = netOpenBound((uint16_t)args[0].as.i, SOCK_STREAM, true, result, &fd);
@ -457,20 +462,13 @@ static int32_t tcpSend(CalogValueT *args, int32_t argCount, CalogValueT *result,
static int32_t udpClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t udpClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
NetLibT *lib; NetLibT *lib;
NetSocketT *sock;
lib = (NetLibT *)userData; lib = (NetLibT *)userData;
calogValueNil(result); calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "udpClose expects (handle)"); return calogFail(result, calogErrArgE, "udpClose expects (handle)");
} }
sock = (NetSocketT *)calogHandleRemove(lib->handles, args[0].as.i, NET_TYPE_UDP); return netSocketClose(lib, args[0].as.i, NET_TYPE_UDP, 0, result, "udpClose: invalid handle");
if (sock == NULL) {
return calogFail(result, calogErrArgE, "udpClose: invalid handle");
}
close(sock->fd);
free(sock);
return calogOkE;
} }
@ -484,7 +482,7 @@ static int32_t udpOpen(CalogValueT *args, int32_t argCount, CalogValueT *result,
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "udpOpen expects (port)"); return calogFail(result, calogErrArgE, "udpOpen expects (port)");
} }
if (args[0].as.i < 0 || args[0].as.i > NET_PORT_MAX) { if (!netPortOk(args[0].as.i)) {
return calogFail(result, calogErrArgE, "udpOpen: port out of range"); return calogFail(result, calogErrArgE, "udpOpen: port out of range");
} }
status = netOpenBound((uint16_t)args[0].as.i, SOCK_DGRAM, false, result, &fd); status = netOpenBound((uint16_t)args[0].as.i, SOCK_DGRAM, false, result, &fd);
@ -537,13 +535,13 @@ static int32_t udpRecvFrom(CalogValueT *args, int32_t argCount, CalogValueT *res
free(buffer); free(buffer);
return calogFail(result, status, "udpRecvFrom: out of memory"); return calogFail(result, status, "udpRecvFrom: out of memory");
} }
status = netMapSetStr(map, "data", buffer, (int64_t)received); status = calogMapSetStr(map, "data", buffer, (int64_t)received);
free(buffer); free(buffer);
if (status == calogOkE) { if (status == calogOkE) {
status = netMapSetStr(map, "host", hostBuffer, (int64_t)strlen(hostBuffer)); status = calogMapSetStr(map, "host", hostBuffer, (int64_t)strlen(hostBuffer));
} }
if (status == calogOkE) { if (status == calogOkE) {
status = netMapSetInt(map, "port", (int64_t)ntohs(from.sin_port)); status = calogMapSetInt(map, "port", (int64_t)ntohs(from.sin_port));
} }
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(map); calogAggFree(map);
@ -566,7 +564,7 @@ static int32_t udpSendTo(CalogValueT *args, int32_t argCount, CalogValueT *resul
if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogIntE || args[3].type != calogStringE) { if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogIntE || args[3].type != calogStringE) {
return calogFail(result, calogErrArgE, "udpSendTo expects (handle, host, port, data)"); return calogFail(result, calogErrArgE, "udpSendTo expects (handle, host, port, data)");
} }
if (args[2].as.i < 0 || args[2].as.i > NET_PORT_MAX) { if (!netPortOk(args[2].as.i)) {
return calogFail(result, calogErrArgE, "udpSendTo: port out of range"); return calogFail(result, calogErrArgE, "udpSendTo: port out of range");
} }
sock = (NetSocketT *)calogHandleGet(lib->handles, args[0].as.i, NET_TYPE_UDP); sock = (NetSocketT *)calogHandleGet(lib->handles, args[0].as.i, NET_TYPE_UDP);
@ -632,7 +630,7 @@ static int32_t enetConnect(CalogValueT *args, int32_t argCount, CalogValueT *res
if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogIntE || args[3].type != calogIntE) { if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogIntE || args[3].type != calogIntE) {
return calogFail(result, calogErrArgE, "enetConnect expects (hostHandle, host, port, channels)"); return calogFail(result, calogErrArgE, "enetConnect expects (hostHandle, host, port, channels)");
} }
if (args[2].as.i < 0 || args[2].as.i > NET_PORT_MAX) { if (!netPortOk(args[2].as.i)) {
return calogFail(result, calogErrArgE, "enetConnect: port out of range"); return calogFail(result, calogErrArgE, "enetConnect: port out of range");
} }
if (args[3].as.i < 1) { if (args[3].as.i < 1) {
@ -692,7 +690,7 @@ static int32_t enetHost(CalogValueT *args, int32_t argCount, CalogValueT *result
if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogIntE) { if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogIntE) {
return calogFail(result, calogErrArgE, "enetHost expects (port, maxPeers)"); return calogFail(result, calogErrArgE, "enetHost expects (port, maxPeers)");
} }
if (args[0].as.i < 0 || args[0].as.i > NET_PORT_MAX) { if (!netPortOk(args[0].as.i)) {
return calogFail(result, calogErrArgE, "enetHost: port out of range"); return calogFail(result, calogErrArgE, "enetHost: port out of range");
} }
if (args[1].as.i < 1) { if (args[1].as.i < 1) {
@ -730,7 +728,7 @@ static int32_t enetSend(CalogValueT *args, int32_t argCount, CalogValueT *result
if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogIntE || args[2].type != calogStringE || args[3].type != calogBoolE) { if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogIntE || args[2].type != calogStringE || args[3].type != calogBoolE) {
return calogFail(result, calogErrArgE, "enetSend expects (peerHandle, channel, data, reliable)"); return calogFail(result, calogErrArgE, "enetSend expects (peerHandle, channel, data, reliable)");
} }
if (args[1].as.i < 0 || args[1].as.i > 255) { if (args[1].as.i < 0 || args[1].as.i > ENET_PROTOCOL_MAXIMUM_CHANNEL_COUNT) {
return calogFail(result, calogErrArgE, "enetSend: channel out of range"); return calogFail(result, calogErrArgE, "enetSend: channel out of range");
} }
peer = (ENetPeer *)calogHandleGet(lib->handles, args[0].as.i, NET_TYPE_ENET_PEER); peer = (ENetPeer *)calogHandleGet(lib->handles, args[0].as.i, NET_TYPE_ENET_PEER);
@ -767,23 +765,29 @@ static int32_t enetService(CalogValueT *args, int32_t argCount, CalogValueT *res
if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogIntE) { if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogIntE) {
return calogFail(result, calogErrArgE, "enetService expects (hostHandle, timeoutMs)"); return calogFail(result, calogErrArgE, "enetService expects (hostHandle, timeoutMs)");
} }
if (args[1].as.i < 0) { // enet_host_service takes the timeout as enet_uint32 milliseconds; reject anything that
return calogFail(result, calogErrArgE, "enetService: timeout must be non-negative"); // would silently wrap (a value >= 2^32 would otherwise become a near-zero busy-poll).
if (args[1].as.i < 0 || args[1].as.i > (int64_t)UINT32_MAX) {
return calogFail(result, calogErrArgE, "enetService: timeout out of range");
} }
host = (ENetHost *)calogHandleGet(lib->handles, args[0].as.i, NET_TYPE_ENET_HOST); host = (ENetHost *)calogHandleGet(lib->handles, args[0].as.i, NET_TYPE_ENET_HOST);
if (host == NULL) { if (host == NULL) {
return calogFail(result, calogErrArgE, "enetService: invalid host handle"); return calogFail(result, calogErrArgE, "enetService: invalid host handle");
} }
serviced = enet_host_service(host, &event, (enet_uint32)args[1].as.i); // Build the result map before consuming the event: on OOM here there is nothing yet to
if (serviced < 0) { // leak or leave in a stale handle-table state (see the failure paths below, which used to
return calogFail(result, calogErrArgE, "enetService: service failed"); // run after the event was already dequeued).
}
status = calogAggCreate(&map, calogMapE); status = calogAggCreate(&map, calogMapE);
if (status != calogOkE) { if (status != calogOkE) {
return calogFail(result, status, "enetService: out of memory"); return calogFail(result, status, "enetService: out of memory");
} }
serviced = enet_host_service(host, &event, (enet_uint32)args[1].as.i);
if (serviced < 0) {
calogAggFree(map);
return calogFail(result, calogErrArgE, "enetService: service failed");
}
if (serviced == 0 || event.type == ENET_EVENT_TYPE_NONE) { if (serviced == 0 || event.type == ENET_EVENT_TYPE_NONE) {
status = netMapSetStr(map, "type", "none", 4); status = calogMapSetStr(map, "type", "none", (int64_t)strlen("none"));
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(map); calogAggFree(map);
return calogFail(result, status, "enetService: out of memory"); return calogFail(result, status, "enetService: out of memory");
@ -807,30 +811,33 @@ static int32_t enetService(CalogValueT *args, int32_t argCount, CalogValueT *res
} }
switch (event.type) { switch (event.type) {
case ENET_EVENT_TYPE_CONNECT: case ENET_EVENT_TYPE_CONNECT:
status = netMapSetStr(map, "type", "connect", 7); status = calogMapSetStr(map, "type", "connect", (int64_t)strlen("connect"));
break; break;
case ENET_EVENT_TYPE_RECEIVE: case ENET_EVENT_TYPE_RECEIVE:
status = netMapSetStr(map, "type", "receive", 7); status = calogMapSetStr(map, "type", "receive", (int64_t)strlen("receive"));
if (status == calogOkE) { if (status == calogOkE) {
status = netMapSetInt(map, "channel", (int64_t)event.channelID); status = calogMapSetInt(map, "channel", (int64_t)event.channelID);
} }
if (status == calogOkE) { if (status == calogOkE) {
status = netMapSetStr(map, "data", (const char *)event.packet->data, (int64_t)event.packet->dataLength); status = calogMapSetStr(map, "data", (const char *)event.packet->data, (int64_t)event.packet->dataLength);
} }
enet_packet_destroy(event.packet); enet_packet_destroy(event.packet);
break; break;
case ENET_EVENT_TYPE_DISCONNECT: case ENET_EVENT_TYPE_DISCONNECT:
status = netMapSetStr(map, "type", "disconnect", 10); status = calogMapSetStr(map, "type", "disconnect", (int64_t)strlen("disconnect"));
// The peer is now invalid; drop its handle but still report it in this event. // The peer is now invalid; drop its handle but still report it in this event.
calogHandleRemove(lib->handles, peerHandle, NET_TYPE_ENET_PEER); calogHandleRemove(lib->handles, peerHandle, NET_TYPE_ENET_PEER);
event.peer->data = NULL; event.peer->data = NULL;
break; break;
default: default:
status = netMapSetStr(map, "type", "none", 4); // Unreachable: ENET_EVENT_TYPE_NONE is handled above and ENetEventType has no
break; // other values. Kept only to satisfy -Wswitch; a peer handle was possibly just
// allocated above for this event, so this is not a safe fallback to "none".
calogAggFree(map);
return calogFail(result, calogErrArgE, "enetService: unexpected event type");
} }
if (status == calogOkE) { if (status == calogOkE) {
status = netMapSetInt(map, "peer", peerHandle); status = calogMapSetInt(map, "peer", peerHandle);
} }
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(map); calogAggFree(map);

View file

@ -27,8 +27,7 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define PUBSUB_INITIAL_CAP 8 #include "calogInternal.h"
#define PUBSUB_GROWTH 2
typedef struct SubEntryT { typedef struct SubEntryT {
int64_t id; int64_t id;
@ -49,15 +48,14 @@ static pthread_mutex_t gInitMutex = PTHREAD_MUTEX_INITIALIZER;
static int32_t gRefCount = 0; static int32_t gRefCount = 0;
static int32_t pubsubFindByIdLocked(int64_t id); static int32_t pubsubFindByIdLocked(int64_t id);
static void pubsubFreeAll(void);
static int32_t pubsubPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t pubsubPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t pubsubSubscribe(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t pubsubSubscribe(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t pubsubUnsubscribe(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t pubsubUnsubscribe(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
int32_t calogPubsubRegister(CalogT *calog) { int32_t calogPubsubRegister(CalogT *calog) {
pthread_mutex_lock(&gInitMutex); calogRegistryRetain(&gInitMutex, &gRefCount);
gRefCount++;
pthread_mutex_unlock(&gInitMutex);
calogRegisterInline(calog, "psSubscribe", pubsubSubscribe, NULL); calogRegisterInline(calog, "psSubscribe", pubsubSubscribe, NULL);
calogRegisterInline(calog, "psUnsubscribe", pubsubUnsubscribe, NULL); calogRegisterInline(calog, "psUnsubscribe", pubsubUnsubscribe, NULL);
calogRegisterInline(calog, "psPublish", pubsubPublish, NULL); calogRegisterInline(calog, "psPublish", pubsubPublish, NULL);
@ -66,24 +64,8 @@ int32_t calogPubsubRegister(CalogT *calog) {
void calogPubsubShutdown(void) { void calogPubsubShutdown(void) {
pthread_mutex_lock(&gInitMutex); // calogRegistryRelease holds gInitMutex for the whole call, including pubsubFreeAll below.
if (gRefCount > 0) { calogRegistryRelease(&gInitMutex, &gRefCount, pubsubFreeAll);
gRefCount--;
}
if (gRefCount <= 0) {
int32_t index;
pthread_mutex_lock(&gListMutex);
for (index = 0; index < gCount; index++) {
free(gEntries[index].topic);
calogFnRelease(gEntries[index].callback);
}
free(gEntries);
gEntries = NULL;
gCount = 0;
gCap = 0;
pthread_mutex_unlock(&gListMutex);
}
pthread_mutex_unlock(&gInitMutex);
} }
@ -99,6 +81,24 @@ static int32_t pubsubFindByIdLocked(int64_t id) {
} }
// Invoked by calogRegistryRelease while gInitMutex is held, exactly once, when the last
// reference drops.
static void pubsubFreeAll(void) {
int32_t index;
pthread_mutex_lock(&gListMutex);
for (index = 0; index < gCount; index++) {
free(gEntries[index].topic);
calogFnRelease(gEntries[index].callback);
}
free(gEntries);
gEntries = NULL;
gCount = 0;
gCap = 0;
pthread_mutex_unlock(&gListMutex);
}
static int32_t pubsubPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t pubsubPublish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
CalogFnT **collected; CalogFnT **collected;
int64_t topicLen; int64_t topicLen;
@ -140,11 +140,9 @@ static int32_t pubsubPublish(CalogValueT *args, int32_t argCount, CalogValueT *r
int32_t status; int32_t status;
status = calogValueCopy(&messageCopy, &args[1]); status = calogValueCopy(&messageCopy, &args[1]);
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(&messageCopy);
calogFnRelease(collected[index]); calogFnRelease(collected[index]);
continue; continue;
} }
calogValueNil(&callResult);
status = calogFnInvoke(collected[index], &messageCopy, 1, &callResult); status = calogFnInvoke(collected[index], &messageCopy, 1, &callResult);
calogValueFree(&callResult); calogValueFree(&callResult);
calogValueFree(&messageCopy); calogValueFree(&messageCopy);
@ -162,9 +160,12 @@ static int32_t pubsubPublish(CalogValueT *args, int32_t argCount, CalogValueT *r
static int32_t pubsubSubscribe(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t pubsubSubscribe(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
void *buffer;
int64_t capacity;
char *topicCopy; char *topicCopy;
int64_t id; int64_t id;
int64_t topicLen; int64_t topicLen;
int32_t status;
(void)userData; (void)userData;
calogValueNil(result); calogValueNil(result);
@ -179,19 +180,16 @@ static int32_t pubsubSubscribe(CalogValueT *args, int32_t argCount, CalogValueT
memcpy(topicCopy, args[0].as.s.bytes, (size_t)topicLen); memcpy(topicCopy, args[0].as.s.bytes, (size_t)topicLen);
topicCopy[topicLen] = '\0'; topicCopy[topicLen] = '\0';
pthread_mutex_lock(&gListMutex); pthread_mutex_lock(&gListMutex);
if (gCount == gCap) { buffer = gEntries;
int32_t newCap; capacity = gCap;
SubEntryT *grown; status = calogGrow(&buffer, &capacity, (int64_t)gCount + 1, sizeof(SubEntryT));
newCap = (gCap == 0) ? PUBSUB_INITIAL_CAP : gCap * PUBSUB_GROWTH; if (status != calogOkE) {
grown = (SubEntryT *)realloc(gEntries, (size_t)newCap * sizeof(SubEntryT));
if (grown == NULL) {
pthread_mutex_unlock(&gListMutex); pthread_mutex_unlock(&gListMutex);
free(topicCopy); free(topicCopy);
return calogFail(result, calogErrOomE, "psSubscribe: out of memory"); return calogFail(result, status, "psSubscribe: out of memory");
}
gEntries = grown;
gCap = newCap;
} }
gEntries = (SubEntryT *)buffer;
gCap = (int32_t)capacity;
id = gNextId; id = gNextId;
gNextId++; gNextId++;
calogFnRetain(args[1].as.fn); calogFnRetain(args[1].as.fn);

View file

@ -1,14 +1,15 @@
// calogSsh.c -- calog SSH library (see calogSsh.h). SSH command execution and SFTP file // calogSsh.c -- calog SSH library (see calogSsh.h). SSH command execution and SFTP file
// transfer over the shared typed handle table, bound to libssh2. Every blocking call is an // transfer over the shared typed handle table, bound to libssh2. Every blocking call is an
// inline native, so it stalls only the calling script's context thread. Sessions run in // inline native, so it stalls only the calling script's context thread. Sessions run in
// libssh2 blocking mode; each connection handle is owner-scoped to the context that created // libssh2 blocking mode. A connection handle has no owning context -- any context it is passed
// it (see SshConnT.ownerId). // to may use it -- so the caller must serialize access (see the threading notes in calogSsh.h).
#define _GNU_SOURCE #define _GNU_SOURCE
#include "calogSsh.h" #include "calogSsh.h"
#include "calogHandle.h" #include "calogHandle.h"
#include "calogInternal.h"
#include <netdb.h> #include <netdb.h>
#include <pthread.h> #include <pthread.h>
@ -17,6 +18,7 @@
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include <sys/select.h>
#include <sys/socket.h> #include <sys/socket.h>
#include <unistd.h> #include <unistd.h>
@ -31,16 +33,22 @@
// Upper bound on a single read-to-EOF transfer (sshExec output, sftpGet file), matching // Upper bound on a single read-to-EOF transfer (sshExec output, sftpGet file), matching
// calogNet/calogHttp: without it a huge/endless remote source would grow an unbounded buffer // calogNet/calogHttp: without it a huge/endless remote source would grow an unbounded buffer
// and OOM-kill the shared host process. Exceeding it fails with calogErrRangeE. // and OOM-kill the shared host process. Exceeding it fails with calogErrRangeE; a stream of
// exactly this many bytes succeeds (see sshDrainStep's boundary probe).
#define SSH_MAX_TRANSFER (64 * 1024 * 1024) #define SSH_MAX_TRANSFER (64 * 1024 * 1024)
// What a connection handle owns: the socket, the libssh2 session, a lazily-opened+cached SFTP // libssh2_sftp_readdir fails the whole listing (LIBSSH2_ERROR_BUFFER_TOO_SMALL) on a remote
// subsystem, and the id of the context that created it. ONLY that owner may operate on it. // entry name at or beyond this size; real filesystems cap names at 255 bytes (NAME_MAX), so
// this leaves comfortable margin.
#define SSH_NAME_MAX 512
// What a connection handle owns: the socket, the libssh2 session, and a lazily-opened+cached
// SFTP subsystem. The handle is a plain process-wide entry with no owning context: any context
// it is passed to may operate on it (see the threading notes in calogSsh.h).
typedef struct SshConnT { typedef struct SshConnT {
int sock; int sock;
LIBSSH2_SESSION *session; LIBSSH2_SESSION *session;
LIBSSH2_SFTP *sftp; LIBSSH2_SFTP *sftp;
uint64_t ownerId;
} SshConnT; } SshConnT;
// Process-wide SSH library state shared by every runtime that registers the natives; refCount // Process-wide SSH library state shared by every runtime that registers the natives; refCount
@ -51,6 +59,17 @@ typedef struct SshLibT {
int32_t refCount; int32_t refCount;
} SshLibT; } SshLibT;
// Read adapter so sshDrainStep works uniformly over a channel stream (libssh2_channel_read_ex)
// and an SFTP file (libssh2_sftp_read); the SFTP adapter ignores streamId.
typedef ssize_t (*SshReadFnT)(void *handle, int streamId, char *buffer, size_t length);
// Outcome of one sshDrainStep call.
typedef enum SshStepE {
sshStepDataE,
sshStepEofE,
sshStepAgainE
} SshStepE;
static pthread_mutex_t gSshLibMutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t gSshLibMutex = PTHREAD_MUTEX_INITIALIZER;
static SshLibT *gSshLib = NULL; static SshLibT *gSshLib = NULL;
@ -64,14 +83,17 @@ static int32_t sshAuthKey(CalogValueT *args, int32_t argCount, CalogValueT *resu
static int32_t sshAuthPassword(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t sshAuthPassword(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t sshClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t sshClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void sshCloser(uint32_t type, void *resource); static void sshCloser(uint32_t type, void *resource);
static void sshConnDestroy(SshConnT *conn, const char *reason);
static int32_t sshConnect(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t sshConnect(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t sshDrain(LIBSSH2_CHANNEL *channel, int streamId, char **bytesOut, int64_t *lengthOut); static int32_t sshDrainStep(SshReadFnT readFn, void *handle, int streamId, char **bytes, int64_t *cap, int64_t *length, SshStepE *step);
static int32_t sshEnsureSftp(SshConnT *conn, CalogValueT *result); static int32_t sshEnsureSftp(SshConnT *conn, CalogValueT *result);
static int32_t sshExec(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t sshExec(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t sshMapSetBool(CalogAggT *map, const char *key, bool value); static int32_t sshExecDrain(int sock, LIBSSH2_SESSION *session, LIBSSH2_CHANNEL *channel, char **outBytes, int64_t *outLength, char **errBytes, int64_t *errLength);
static int32_t sshMapSetInt(CalogAggT *map, const char *key, int64_t value); static ssize_t sshReadChannel(void *handle, int streamId, char *buffer, size_t length);
static int32_t sshMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length); static ssize_t sshReadSftpFile(void *handle, int streamId, char *buffer, size_t length);
static int32_t sshResolve(SshLibT *lib, int64_t handle, CalogValueT *result, SshConnT **out); static int32_t sshResolve(SshLibT *lib, int64_t handle, CalogValueT *result, SshConnT **out);
static void sshSessionDestroy(LIBSSH2_SESSION *session, int sock, const char *reason);
static void sshWaitSocket(int sock, LIBSSH2_SESSION *session);
int32_t calogSshRegister(CalogT *calog) { int32_t calogSshRegister(CalogT *calog) {
@ -137,8 +159,9 @@ static int32_t sftpGet(CalogValueT *args, int32_t argCount, CalogValueT *result,
SshConnT *conn; SshConnT *conn;
LIBSSH2_SFTP_HANDLE *file; LIBSSH2_SFTP_HANDLE *file;
char *buffer; char *buffer;
size_t cap; int64_t cap;
size_t length; int64_t length;
SshStepE step;
int32_t status; int32_t status;
lib = (SshLibT *)userData; lib = (SshLibT *)userData;
@ -162,43 +185,24 @@ static int32_t sftpGet(CalogValueT *args, int32_t argCount, CalogValueT *result,
cap = 0; cap = 0;
length = 0; length = 0;
for (;;) { for (;;) {
ssize_t n; status = sshDrainStep(sshReadSftpFile, file, 0, &buffer, &cap, &length, &step);
if (length == cap) { if (status != calogOkE) {
size_t want;
char *grown;
if (cap >= SSH_MAX_TRANSFER) {
free(buffer); free(buffer);
libssh2_sftp_close(file); libssh2_sftp_close(file);
return calogFail(result, calogErrRangeE, "sftpGet: file exceeds the maximum transfer size"); if (status == calogErrRangeE) {
return calogFail(result, status, "sftpGet: file exceeds the maximum transfer size");
} }
want = (cap == 0) ? 65536 : cap * CALOG_GROWTH_FACTOR; if (status == calogErrOomE) {
if (want > SSH_MAX_TRANSFER) { return calogFail(result, status, "sftpGet: out of memory");
want = SSH_MAX_TRANSFER;
} }
grown = (char *)realloc(buffer, want); return calogFail(result, status, "sftpGet: read failed");
if (grown == NULL) {
free(buffer);
libssh2_sftp_close(file);
return calogFail(result, calogErrOomE, "sftpGet: out of memory");
} }
buffer = grown; if (step == sshStepEofE) {
cap = want;
}
n = libssh2_sftp_read(file, buffer + length, cap - length);
if (n > 0) {
length += (size_t)n;
} else if (n == 0) {
break; break;
} else if (n == LIBSSH2_ERROR_EAGAIN) {
continue;
} else {
free(buffer);
libssh2_sftp_close(file);
return calogFail(result, calogErrArgE, "sftpGet: read failed");
} }
} }
libssh2_sftp_close(file); libssh2_sftp_close(file);
status = calogValueString(result, (buffer != NULL) ? buffer : "", (int64_t)length); status = calogValueString(result, buffer, (int64_t)length);
free(buffer); free(buffer);
return status; return status;
} }
@ -237,16 +241,13 @@ static int32_t sftpList(CalogValueT *args, int32_t argCount, CalogValueT *result
LIBSSH2_SFTP_ATTRIBUTES attrs; LIBSSH2_SFTP_ATTRIBUTES attrs;
CalogAggT *entry; CalogAggT *entry;
CalogValueT entryValue; CalogValueT entryValue;
char name[512]; char name[SSH_NAME_MAX];
int rc; int rc;
rc = libssh2_sftp_readdir(dir, name, sizeof(name), &attrs); rc = libssh2_sftp_readdir(dir, name, sizeof(name), &attrs);
if (rc == 0) { if (rc == 0) {
break; break;
} }
if (rc == LIBSSH2_ERROR_EAGAIN) {
continue;
}
if (rc < 0) { if (rc < 0) {
calogAggFree(list); calogAggFree(list);
libssh2_sftp_closedir(dir); libssh2_sftp_closedir(dir);
@ -262,16 +263,16 @@ static int32_t sftpList(CalogValueT *args, int32_t argCount, CalogValueT *result
libssh2_sftp_closedir(dir); libssh2_sftp_closedir(dir);
return calogFail(result, status, "sftpList: out of memory"); return calogFail(result, status, "sftpList: out of memory");
} }
status = sshMapSetStr(entry, "name", name, (int64_t)rc); status = calogMapSetStr(entry, "name", name, (int64_t)rc);
if (status == calogOkE) { if (status == calogOkE) {
int64_t size; int64_t size;
size = (attrs.flags & LIBSSH2_SFTP_ATTR_SIZE) ? (int64_t)attrs.filesize : 0; size = (attrs.flags & LIBSSH2_SFTP_ATTR_SIZE) ? (int64_t)attrs.filesize : 0;
status = sshMapSetInt(entry, "size", size); status = calogMapSetInt(entry, "size", size);
} }
if (status == calogOkE) { if (status == calogOkE) {
bool isDir; bool isDir;
isDir = (attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) && LIBSSH2_SFTP_S_ISDIR(attrs.permissions); isDir = (attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) && LIBSSH2_SFTP_S_ISDIR(attrs.permissions);
status = sshMapSetBool(entry, "isDir", isDir); status = calogMapSetBool(entry, "isDir", isDir);
} }
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(entry); calogAggFree(entry);
@ -349,9 +350,6 @@ static int32_t sftpPut(CalogValueT *args, int32_t argCount, CalogValueT *result,
ssize_t n; ssize_t n;
n = libssh2_sftp_write(file, args[2].as.s.bytes + total, (size_t)(args[2].as.s.length - total)); n = libssh2_sftp_write(file, args[2].as.s.bytes + total, (size_t)(args[2].as.s.length - total));
if (n < 0) { if (n < 0) {
if (n == LIBSSH2_ERROR_EAGAIN) {
continue;
}
libssh2_sftp_close(file); libssh2_sftp_close(file);
return calogFail(result, calogErrArgE, "sftpPut: write failed"); return calogFail(result, calogErrArgE, "sftpPut: write failed");
} }
@ -421,9 +419,9 @@ static int32_t sftpStat(CalogValueT *args, int32_t argCount, CalogValueT *result
} }
size = (attrs.flags & LIBSSH2_SFTP_ATTR_SIZE) ? (int64_t)attrs.filesize : 0; size = (attrs.flags & LIBSSH2_SFTP_ATTR_SIZE) ? (int64_t)attrs.filesize : 0;
isDir = (attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) && LIBSSH2_SFTP_S_ISDIR(attrs.permissions); isDir = (attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) && LIBSSH2_SFTP_S_ISDIR(attrs.permissions);
status = sshMapSetInt(map, "size", size); status = calogMapSetInt(map, "size", size);
if (status == calogOkE) { if (status == calogOkE) {
status = sshMapSetBool(map, "isDir", isDir); status = calogMapSetBool(map, "isDir", isDir);
} }
if (status != calogOkE) { if (status != calogOkE) {
calogAggFree(map); calogAggFree(map);
@ -507,41 +505,33 @@ static int32_t sshClose(CalogValueT *args, int32_t argCount, CalogValueT *result
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "sshClose expects (handle)"); return calogFail(result, calogErrArgE, "sshClose expects (handle)");
} }
// Peek first to enforce owner-scoping; because the owner is a single thread, the peek and // Atomically claim the handle -- a concurrent sshClose of the same handle gets NULL and must
// the remove below cannot race another context claiming this handle. // not double-destroy. ssh handles are shareable across contexts (no owner check); the caller
conn = (SshConnT *)calogHandleGet(lib->handles, args[0].as.i, SSH_TYPE_CONN); // is responsible for ensuring nobody is mid-operation on the connection when it is closed
if (conn == NULL) { // (see the threading notes in calogSsh.h).
return calogFail(result, calogErrNotFoundE, "sshClose: invalid ssh handle");
}
if (conn->ownerId != calogCurrentId()) {
return calogFail(result, calogErrArgE, "sshClose: ssh handle belongs to another context");
}
conn = (SshConnT *)calogHandleRemove(lib->handles, args[0].as.i, SSH_TYPE_CONN); conn = (SshConnT *)calogHandleRemove(lib->handles, args[0].as.i, SSH_TYPE_CONN);
if (conn == NULL) { if (conn == NULL) {
return calogFail(result, calogErrNotFoundE, "sshClose: invalid ssh handle"); return calogFail(result, calogErrNotFoundE, "sshClose: invalid ssh handle");
} }
if (conn->sftp != NULL) { sshConnDestroy(conn, "calog: closing");
libssh2_sftp_shutdown(conn->sftp);
}
libssh2_session_disconnect(conn->session, "calog: closing");
libssh2_session_free(conn->session);
close(conn->sock);
free(conn);
return calogOkE; return calogOkE;
} }
static void sshCloser(uint32_t type, void *resource) { static void sshCloser(uint32_t type, void *resource) {
SshConnT *conn;
(void)type; (void)type;
conn = (SshConnT *)resource; sshConnDestroy((SshConnT *)resource, "calog: shutdown");
}
// Tear down a connection handle: shut down any cached SFTP subsystem, then the session and
// socket, then free the handle itself. Shared by sshClose and the handle table's closer so
// the release order lives in exactly one place.
static void sshConnDestroy(SshConnT *conn, const char *reason) {
if (conn->sftp != NULL) { if (conn->sftp != NULL) {
libssh2_sftp_shutdown(conn->sftp); libssh2_sftp_shutdown(conn->sftp);
} }
libssh2_session_disconnect(conn->session, "calog: shutdown"); sshSessionDestroy(conn->session, conn->sock, reason);
libssh2_session_free(conn->session);
close(conn->sock);
free(conn); free(conn);
} }
@ -611,21 +601,15 @@ static int32_t sshConnect(CalogValueT *args, int32_t argCount, CalogValueT *resu
} }
conn = (SshConnT *)calloc(1, sizeof(*conn)); conn = (SshConnT *)calloc(1, sizeof(*conn));
if (conn == NULL) { if (conn == NULL) {
libssh2_session_disconnect(session, "out of memory"); sshSessionDestroy(session, fd, "out of memory");
libssh2_session_free(session);
close(fd);
return calogFail(result, calogErrOomE, "sshConnect: out of memory"); return calogFail(result, calogErrOomE, "sshConnect: out of memory");
} }
conn->sock = fd; conn->sock = fd;
conn->session = session; conn->session = session;
conn->sftp = NULL; conn->sftp = NULL;
conn->ownerId = calogCurrentId();
handle = calogHandleAdd(lib->handles, SSH_TYPE_CONN, conn); handle = calogHandleAdd(lib->handles, SSH_TYPE_CONN, conn);
if (handle == 0) { if (handle == 0) {
libssh2_session_disconnect(session, "out of memory"); sshConnDestroy(conn, "out of memory");
libssh2_session_free(session);
close(fd);
free(conn);
return calogFail(result, calogErrOomE, "sshConnect: out of memory"); return calogFail(result, calogErrOomE, "sshConnect: out of memory");
} }
calogValueInt(result, handle); calogValueInt(result, handle);
@ -633,52 +617,55 @@ static int32_t sshConnect(CalogValueT *args, int32_t argCount, CalogValueT *resu
} }
// Drain one channel stream (0 = stdout, SSH_EXTENDED_DATA_STDERR = stderr) into a fresh // Attempt one read of *bytes/*cap/*length via readFn, growing the buffer first if it is
// growable buffer, binary-safe. Returns calogOkE with *bytesOut owned by the caller (freed on // full. At the SSH_MAX_TRANSFER cap, instead of failing outright, probes with a single-byte
// every path), or an error (the buffer is released here on failure). // read so a stream of exactly SSH_MAX_TRANSFER bytes still succeeds -- only a stream with
static int32_t sshDrain(LIBSSH2_CHANNEL *channel, int streamId, char **bytesOut, int64_t *lengthOut) { // MORE than the cap fails (calogErrRangeE). Shared by sftpGet (a single SFTP file stream)
char *buffer; // and sshExecDrain (two interleaved channel streams). Never frees *bytes on failure -- that
size_t cap; // stays the caller's job, since callers may be juggling more than one buffer.
size_t length; static int32_t sshDrainStep(SshReadFnT readFn, void *handle, int streamId, char **bytes, int64_t *cap, int64_t *length, SshStepE *step) {
void *grown;
buffer = NULL; char probe[1];
cap = 0;
length = 0;
for (;;) {
ssize_t n; ssize_t n;
if (length == cap) { int32_t status;
size_t want;
char *grown; if (*length == *cap) {
if (cap >= SSH_MAX_TRANSFER) { if (*cap >= SSH_MAX_TRANSFER) {
free(buffer); n = readFn(handle, streamId, probe, sizeof(probe));
return calogErrRangeE; if (n == 0) {
*step = sshStepEofE;
return calogOkE;
} }
want = (cap == 0) ? 4096 : cap * CALOG_GROWTH_FACTOR; if (n == LIBSSH2_ERROR_EAGAIN) {
if (want > SSH_MAX_TRANSFER) { *step = sshStepAgainE;
want = SSH_MAX_TRANSFER; return calogOkE;
} }
grown = (char *)realloc(buffer, want); if (n < 0) {
if (grown == NULL) {
free(buffer);
return calogErrOomE;
}
buffer = grown;
cap = want;
}
n = libssh2_channel_read_ex(channel, streamId, buffer + length, cap - length);
if (n > 0) {
length += (size_t)n;
} else if (n == 0) {
break;
} else if (n == LIBSSH2_ERROR_EAGAIN) {
continue;
} else {
free(buffer);
return calogErrArgE; return calogErrArgE;
} }
return calogErrRangeE;
}
grown = *bytes;
status = calogGrow(&grown, cap, *length + 1, 1);
if (status != calogOkE) {
return status;
}
*bytes = (char *)grown;
if (*cap > SSH_MAX_TRANSFER) {
*cap = SSH_MAX_TRANSFER;
}
}
n = readFn(handle, streamId, *bytes + *length, (size_t)(*cap - *length));
if (n > 0) {
*length += n;
*step = sshStepDataE;
} else if (n == 0) {
*step = sshStepEofE;
} else if (n == LIBSSH2_ERROR_EAGAIN) {
*step = sshStepAgainE;
} else {
return calogErrArgE;
} }
*bytesOut = buffer;
*lengthOut = (int64_t)length;
return calogOkE; return calogOkE;
} }
@ -716,6 +703,9 @@ static int32_t sshExec(CalogValueT *args, int32_t argCount, CalogValueT *result,
if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogStringE) { if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "sshExec expects (handle, command)"); return calogFail(result, calogErrArgE, "sshExec expects (handle, command)");
} }
if (args[1].as.s.length > SSH_MAX_TRANSFER) {
return calogFail(result, calogErrRangeE, "sshExec: command too long");
}
status = sshResolve(lib, args[0].as.i, result, &conn); status = sshResolve(lib, args[0].as.i, result, &conn);
if (status != calogOkE) { if (status != calogOkE) {
return status; return status;
@ -724,27 +714,22 @@ static int32_t sshExec(CalogValueT *args, int32_t argCount, CalogValueT *result,
if (channel == NULL) { if (channel == NULL) {
return calogFail(result, calogErrArgE, "sshExec: could not open a channel"); return calogFail(result, calogErrArgE, "sshExec: could not open a channel");
} }
if (args[1].as.s.length > SSH_MAX_TRANSFER) {
libssh2_channel_free(channel);
return calogFail(result, calogErrRangeE, "sshExec: command too long");
}
// Not libssh2_channel_exec (it derives the length with strlen, truncating a command with an // Not libssh2_channel_exec (it derives the length with strlen, truncating a command with an
// embedded NUL); pass the binary-safe length explicitly. // embedded NUL); pass the binary-safe length explicitly.
if (libssh2_channel_process_startup(channel, "exec", 4u, args[1].as.s.bytes, (unsigned int)args[1].as.s.length) != 0) { if (libssh2_channel_process_startup(channel, "exec", 4u, args[1].as.s.bytes, (unsigned int)args[1].as.s.length) != 0) {
libssh2_channel_free(channel); libssh2_channel_free(channel);
return calogFail(result, calogErrArgE, "sshExec: could not start the command"); return calogFail(result, calogErrArgE, "sshExec: could not start the command");
} }
// Blocking reads: drain stdout to EOF, then stderr (libssh2 buffers the other stream). status = sshExecDrain(conn->sock, conn->session, channel, &outBytes, &outLength, &errBytes, &errLength);
status = sshDrain(channel, 0, &outBytes, &outLength);
if (status != calogOkE) { if (status != calogOkE) {
libssh2_channel_free(channel); libssh2_channel_free(channel);
if (status == calogErrRangeE) {
return calogFail(result, status, "sshExec: command output exceeds the maximum transfer size");
}
if (status == calogErrOomE) {
return calogFail(result, status, "sshExec: out of memory"); return calogFail(result, status, "sshExec: out of memory");
} }
status = sshDrain(channel, SSH_EXTENDED_DATA_STDERR, &errBytes, &errLength); return calogFail(result, status, "sshExec: channel read failed");
if (status != calogOkE) {
free(outBytes);
libssh2_channel_free(channel);
return calogFail(result, status, "sshExec: out of memory");
} }
libssh2_channel_close(channel); libssh2_channel_close(channel);
exitCode = libssh2_channel_get_exit_status(channel); exitCode = libssh2_channel_get_exit_status(channel);
@ -755,12 +740,12 @@ static int32_t sshExec(CalogValueT *args, int32_t argCount, CalogValueT *result,
free(errBytes); free(errBytes);
return calogFail(result, status, "sshExec: out of memory"); return calogFail(result, status, "sshExec: out of memory");
} }
status = sshMapSetStr(map, "stdout", outBytes, outLength); status = calogMapSetStr(map, "stdout", outBytes, outLength);
if (status == calogOkE) { if (status == calogOkE) {
status = sshMapSetStr(map, "stderr", errBytes, errLength); status = calogMapSetStr(map, "stderr", errBytes, errLength);
} }
if (status == calogOkE) { if (status == calogOkE) {
status = sshMapSetInt(map, "exitCode", (int64_t)exitCode); status = calogMapSetInt(map, "exitCode", (int64_t)exitCode);
} }
free(outBytes); free(outBytes);
free(errBytes); free(errBytes);
@ -773,72 +758,95 @@ static int32_t sshExec(CalogValueT *args, int32_t argCount, CalogValueT *result,
} }
// Set map[key] = boolean. On failure the (freshly built) key is released; the scalar needs // Run the remote command's stdout and stderr to completion, interleaving reads of both
// none. // streams in one loop (never draining one to EOF before starting the other). The two streams
static int32_t sshMapSetBool(CalogAggT *map, const char *key, bool value) { // share one flow-control window, so fully draining stdout first while the remote blocks
CalogValueT keyValue; // writing a flood of stderr can hang forever waiting on stdout, which never arrives because
CalogValueT boolValue; // the remote process never gets to exit. Drops the session into non-blocking mode for the
// duration (restored before every return) so a stream with nothing ready right now cannot
// block progress on the other; sshWaitSocket parks the calling thread only when neither
// stream can make progress right now. On failure both output buffers are freed here.
static int32_t sshExecDrain(int sock, LIBSSH2_SESSION *session, LIBSSH2_CHANNEL *channel, char **outBytes, int64_t *outLength, char **errBytes, int64_t *errLength) {
char *outBuf;
char *errBuf;
int64_t outCap;
int64_t errCap;
int64_t outLen;
int64_t errLen;
bool outDone;
bool errDone;
bool progressed;
SshStepE step;
int32_t status; int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key)); outBuf = NULL;
errBuf = NULL;
outCap = 0;
errCap = 0;
outLen = 0;
errLen = 0;
outDone = false;
errDone = false;
libssh2_session_set_blocking(session, 0);
while (!outDone || !errDone) {
progressed = false;
if (!outDone) {
status = sshDrainStep(sshReadChannel, channel, 0, &outBuf, &outCap, &outLen, &step);
if (status != calogOkE) { if (status != calogOkE) {
libssh2_session_set_blocking(session, 1);
free(outBuf);
free(errBuf);
return status; return status;
} }
calogValueBool(&boolValue, value); if (step == sshStepEofE) {
status = calogAggSet(map, &keyValue, &boolValue); outDone = true;
if (status != calogOkE) { progressed = true;
calogValueFree(&keyValue); } else if (step == sshStepDataE) {
progressed = true;
} }
}
if (!errDone) {
status = sshDrainStep(sshReadChannel, channel, SSH_EXTENDED_DATA_STDERR, &errBuf, &errCap, &errLen, &step);
if (status != calogOkE) {
libssh2_session_set_blocking(session, 1);
free(outBuf);
free(errBuf);
return status; return status;
}
if (step == sshStepEofE) {
errDone = true;
progressed = true;
} else if (step == sshStepDataE) {
progressed = true;
}
}
if (!progressed && (!outDone || !errDone)) {
sshWaitSocket(sock, session);
}
}
libssh2_session_set_blocking(session, 1);
*outBytes = outBuf;
*outLength = outLen;
*errBytes = errBuf;
*errLength = errLen;
return calogOkE;
} }
// Set map[key] = integer. On failure the (freshly built) key is released; the scalar needs static ssize_t sshReadChannel(void *handle, int streamId, char *buffer, size_t length) {
// none. return libssh2_channel_read_ex((LIBSSH2_CHANNEL *)handle, streamId, buffer, length);
static int32_t sshMapSetInt(CalogAggT *map, const char *key, int64_t value) {
CalogValueT keyValue;
CalogValueT intValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
calogValueInt(&intValue, value);
status = calogAggSet(map, &keyValue, &intValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
}
return status;
} }
// Set map[key] = binary-safe string. On failure any built values are released. static ssize_t sshReadSftpFile(void *handle, int streamId, char *buffer, size_t length) {
static int32_t sshMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length) { (void)streamId;
CalogValueT keyValue; return libssh2_sftp_read((LIBSSH2_SFTP_HANDLE *)handle, buffer, length);
CalogValueT stringValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
status = calogValueString(&stringValue, bytes, length);
if (status != calogOkE) {
calogValueFree(&keyValue);
return status;
}
status = calogAggSet(map, &keyValue, &stringValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
calogValueFree(&stringValue);
}
return status;
} }
// Look up a connection handle and enforce owner-scoping. Not-found resolves to // Look up a connection handle. Not-found resolves to calogErrNotFoundE. There is NO owner
// calogErrNotFoundE; a handle owned by another context resolves to calogErrArgE. // check: an ssh handle may be used by any context it is passed to (see the threading notes in
// calogSsh.h) -- the caller must serialize access, since a libssh2 session is not thread-safe.
static int32_t sshResolve(SshLibT *lib, int64_t handle, CalogValueT *result, SshConnT **out) { static int32_t sshResolve(SshLibT *lib, int64_t handle, CalogValueT *result, SshConnT **out) {
SshConnT *conn; SshConnT *conn;
@ -846,9 +854,40 @@ static int32_t sshResolve(SshLibT *lib, int64_t handle, CalogValueT *result, Ssh
if (conn == NULL) { if (conn == NULL) {
return calogFail(result, calogErrNotFoundE, "invalid ssh handle"); return calogFail(result, calogErrNotFoundE, "invalid ssh handle");
} }
if (conn->ownerId != calogCurrentId()) {
return calogFail(result, calogErrArgE, "ssh handle belongs to another context");
}
*out = conn; *out = conn;
return calogOkE; return calogOkE;
} }
// Tear down a raw session and its socket (disconnect, free the session, close the fd). Used
// by sshConnDestroy and by sshConnect's post-handshake failure paths, before a SshConnT
// exists to own the session.
static void sshSessionDestroy(LIBSSH2_SESSION *session, int sock, const char *reason) {
libssh2_session_disconnect(session, reason);
libssh2_session_free(session);
close(sock);
}
// Block the calling thread until the socket is ready in whatever direction libssh2 is
// currently waiting on. Used only by sshExecDrain's non-blocking interleave; every other
// native in this file runs its session in blocking mode and never calls this.
static void sshWaitSocket(int sock, LIBSSH2_SESSION *session) {
fd_set fdSet;
fd_set *readSet;
fd_set *writeSet;
int direction;
FD_ZERO(&fdSet);
FD_SET(sock, &fdSet);
readSet = NULL;
writeSet = NULL;
direction = libssh2_session_block_directions(session);
if (direction & LIBSSH2_SESSION_BLOCK_INBOUND) {
readSet = &fdSet;
}
if (direction & LIBSSH2_SESSION_BLOCK_OUTBOUND) {
writeSet = &fdSet;
}
select(sock + 1, readSet, writeSet, NULL, NULL);
}

View file

@ -14,12 +14,21 @@
// sftpMkdir(handle, path) (mode 0755) // sftpMkdir(handle, path) (mode 0755)
// Payloads are binary-safe strings. The blocking natives are INLINE, so they stall only the // Payloads are binary-safe strings. The blocking natives are INLINE, so they stall only the
// calling script's context thread, not the host. Sessions are put in libssh2 blocking mode. // calling script's context thread, not the host. Sessions are put in libssh2 blocking mode.
// A connection handle belongs to the context that created it: using one from another context //
// fails (calogErrArgE) rather than racing. The SFTP subsystem is opened lazily on the first // THREADING / cross-context use. A connection handle is a plain process-wide value (an opaque
// sftp* call and cached on the connection. Handles are NOT reference-counted; closing one // int64) with no owning context, so it can be PASSED to another script and used there -- e.g.
// while another context is mid-operation is undefined (use-after-free). Hostname resolution // lent to a function in another context via calogCall, which uses it and returns while the
// uses getaddrinfo (works in dynamic builds; a fully-static glibc build cannot resolve names // caller is blocked. That synchronous lend is SAFE: only one context ever touches the session
// -- connect by IP, or link musl). // at a time, and the handle stays with the lender. What is NOT safe is CONCURRENT use of one
// handle: two contexts operating on it at the same instant (a libssh2 session is NOT thread-safe
// -- concurrent use corrupts session state), a borrowed handle stashed and later used from a
// timer/pubsub callback while the owner also uses it, or closing a handle while another context
// is mid-operation on it (use-after-free). The rule is: at most one context uses a given handle
// at any instant, and exactly one context closes it. calog does NOT serialize this for you.
//
// The SFTP subsystem is opened lazily on the first sftp* call and cached on the connection.
// Handles are NOT reference-counted. Hostname resolution uses getaddrinfo (works in dynamic
// builds; a fully-static glibc build cannot resolve names -- connect by IP, or link musl).
#ifndef CALOG_SSH_H #ifndef CALOG_SSH_H
#define CALOG_SSH_H #define CALOG_SSH_H

View file

@ -5,9 +5,11 @@
#include "calogTask.h" #include "calogTask.h"
#include "calogHandle.h" #include "calogHandle.h"
#include "calogInternal.h"
#include <pthread.h> #include <pthread.h>
#include <stdint.h> #include <stdint.h>
#include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
@ -25,14 +27,22 @@ typedef struct TaskLibT {
// ONLY the owner may eval or close a task. Because a context is single-threaded, that rules // ONLY the owner may eval or close a task. Because a context is single-threaded, that rules
// out the two ways taskClose's pthread_join could misbehave: a task can never own its own // out the two ways taskClose's pthread_join could misbehave: a task can never own its own
// handle (so it can't self-join and free itself mid-run), and two tasks can never each own // handle (so it can't self-join and free itself mid-run), and two tasks can never each own
// the other's handle (so they can't dead-lock closing each other). // the other's handle (so they can't dead-lock closing each other). The get-then-check-
// ownerId sequence in taskEval/taskClose still races a concurrent taskClose's get-then-
// remove-then-free of the SAME handle from another context, so both natives serialize their
// whole get/check/(remove) critical section under gTaskLibMutex -- see taskEval/taskClose.
typedef struct TaskT { typedef struct TaskT {
CalogContextT *context; CalogContextT *context;
uint64_t ownerId; uint64_t ownerId;
} TaskT; } TaskT;
// Maps an engine name to its vtable, for the engines compiled in. References the extern // Maps an engine name to its vtable, for the engines compiled in. References the extern
// vtables only under their CALOG_WITH_* guard, so unselected engines stay unlinked. // vtables only under their CALOG_WITH_* guard, so unselected engines stay unlinked. The
// names below are taskSpawn's own short spellings (e.g. "js", "s7") and are intentionally
// NOT the same string as the vtable's own CalogEngineT.name (e.g. "javascript", "scheme"
// -- used for engine self-identification elsewhere, not for lookup here); collapsing this
// table to key off ->name would change taskSpawn's public argument spelling and break the
// tested API (tests/testTask.c uses taskSpawn('js', ...)).
typedef struct TaskEngineT { typedef struct TaskEngineT {
const char *name; const char *name;
const CalogEngineT *engine; const CalogEngineT *engine;
@ -66,14 +76,18 @@ static const TaskEngineT gTaskEngines[] = {
static pthread_mutex_t gTaskLibMutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t gTaskLibMutex = PTHREAD_MUTEX_INITIALIZER;
static TaskLibT *gTaskLib = NULL; static TaskLibT *gTaskLib = NULL;
static int32_t taskActive(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t taskClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t taskClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void taskCloser(uint32_t type, void *resource); static void taskCloser(uint32_t type, void *resource);
static bool taskContextFinished(void *resource, void *userData);
static int32_t taskCount(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t taskCount(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static const CalogEngineT *taskEngineByName(const char *name); static const CalogEngineT *taskEngineByName(const char *name);
static int32_t taskEval(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t taskEval(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t taskExit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t taskLoad(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t taskLoad(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t taskSelf(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t taskSelf(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t taskSpawn(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t taskSpawn(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int64_t taskWrapContext(TaskLibT *lib, CalogContextT *context, CalogValueT *result, const char *label);
int32_t calogTaskRegister(CalogT *calog) { int32_t calogTaskRegister(CalogT *calog) {
@ -99,6 +113,8 @@ int32_t calogTaskRegister(CalogT *calog) {
calogRegisterInline(calog, "taskLoad", taskLoad, gTaskLib); calogRegisterInline(calog, "taskLoad", taskLoad, gTaskLib);
calogRegisterInline(calog, "taskEval", taskEval, gTaskLib); calogRegisterInline(calog, "taskEval", taskEval, gTaskLib);
calogRegisterInline(calog, "taskClose", taskClose, gTaskLib); calogRegisterInline(calog, "taskClose", taskClose, gTaskLib);
calogRegisterInline(calog, "taskActive", taskActive, gTaskLib);
calogRegisterInline(calog, "taskExit", taskExit, gTaskLib);
calogRegisterInline(calog, "taskSelf", taskSelf, gTaskLib); calogRegisterInline(calog, "taskSelf", taskSelf, gTaskLib);
calogRegisterInline(calog, "taskCount", taskCount, gTaskLib); calogRegisterInline(calog, "taskCount", taskCount, gTaskLib);
return calogOkE; return calogOkE;
@ -123,6 +139,64 @@ void calogTaskShutdown(void) {
} }
// Reap every task whose context THREAD has already exited -- a task that called taskExit(), or
// one that simply ran off the end while its owner never closed it. Each wrapper is removed
// under gTaskLibMutex (so it cannot race taskClose on the same handle), then the finished
// context is joined + freed outside the lock. Safe from ANY thread, including a non-owner host:
// a finished thread joins instantly, so this sidesteps the owner-only rule that guards LIVE
// joins. The host runner should call it periodically (e.g. once per calogPump). Returns the
// number reaped.
int32_t calogTaskReap(void) {
int32_t reaped;
reaped = 0;
for (;;) {
TaskT *task;
pthread_mutex_lock(&gTaskLibMutex);
task = (gTaskLib != NULL) ? (TaskT *)calogHandleTakeIf(gTaskLib->handles, TASK_TYPE_CONTEXT, taskContextFinished, NULL) : NULL;
pthread_mutex_unlock(&gTaskLibMutex);
if (task == NULL) {
break;
}
calogContextClose(task->context);
free(task);
reaped++;
}
return reaped;
}
// taskActive() -> bool: the SELF form a long-running task polls to cooperate with taskClose.
// taskClose is cooperative -- it flips this context's shutdown flag then blocks joining the
// thread -- so a task busy in a pure loop never notices unless it checks. It returns false
// once the owner has asked this task to stop; a loop does `while taskActive() do ... end`.
// A task has no handle to itself (it can never own its own handle -- that invariant is what
// keeps taskClose's join from self-joining), so the self form takes no argument.
//
// taskActive(handle) -> bool: the OWNER form -- true while the task the caller spawned under
// handle is still open (not yet taskClose'd). Only the owner gets a meaningful answer.
static int32_t taskActive(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
TaskLibT *lib;
TaskT *task;
lib = (TaskLibT *)userData;
calogValueNil(result);
if (argCount == 0) {
calogValueBool(result, !calogCurrentShuttingDown());
return calogOkE;
}
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "taskActive expects () or (handle)");
}
pthread_mutex_lock(&gTaskLibMutex);
task = (TaskT *)calogHandleGet(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT);
calogValueBool(result, task != NULL && task->ownerId == calogCurrentId());
pthread_mutex_unlock(&gTaskLibMutex);
return calogOkE;
}
static int32_t taskClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t taskClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
TaskLibT *lib; TaskLibT *lib;
TaskT *task; TaskT *task;
@ -132,17 +206,21 @@ static int32_t taskClose(CalogValueT *args, int32_t argCount, CalogValueT *resul
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "taskClose expects (handle)"); return calogFail(result, calogErrArgE, "taskClose expects (handle)");
} }
// Only the owner may close (see TaskT). This check makes a self-join or a mutual close // Only the owner may close (see TaskT). Get, the ownerId check, and Remove all run
// unreachable, and because the owner is a single thread it also serialises the peek // under gTaskLibMutex so a concurrent taskEval/taskClose on the SAME handle from
// below with the remove -- no other thread can claim this handle. // another context can never observe this task between its removal and its free below.
pthread_mutex_lock(&gTaskLibMutex);
task = (TaskT *)calogHandleGet(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT); task = (TaskT *)calogHandleGet(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT);
if (task == NULL) { if (task == NULL) {
pthread_mutex_unlock(&gTaskLibMutex);
return calogFail(result, calogErrArgE, "taskClose: invalid task handle"); return calogFail(result, calogErrArgE, "taskClose: invalid task handle");
} }
if (task->ownerId != calogCurrentId()) { if (task->ownerId != calogCurrentId()) {
pthread_mutex_unlock(&gTaskLibMutex);
return calogFail(result, calogErrArgE, "taskClose: only the task's owner may close it"); return calogFail(result, calogErrArgE, "taskClose: only the task's owner may close it");
} }
task = (TaskT *)calogHandleRemove(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT); task = (TaskT *)calogHandleRemove(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT);
pthread_mutex_unlock(&gTaskLibMutex);
if (task == NULL) { if (task == NULL) {
return calogFail(result, calogErrArgE, "taskClose: invalid task handle"); return calogFail(result, calogErrArgE, "taskClose: invalid task handle");
} }
@ -159,6 +237,12 @@ static void taskCloser(uint32_t type, void *resource) {
} }
static bool taskContextFinished(void *resource, void *userData) {
(void)userData;
return calogContextFinished(((TaskT *)resource)->context);
}
static int32_t taskCount(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t taskCount(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
TaskLibT *lib; TaskLibT *lib;
@ -185,31 +269,55 @@ static const CalogEngineT *taskEngineByName(const char *name) {
static int32_t taskEval(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t taskEval(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
TaskLibT *lib; TaskLibT *lib;
TaskT *task; TaskT *task;
CalogContextT *context;
lib = (TaskLibT *)userData; lib = (TaskLibT *)userData;
calogValueNil(result); calogValueNil(result);
if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogStringE) { if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "taskEval expects (handle, code)"); return calogFail(result, calogErrArgE, "taskEval expects (handle, code)");
} }
// See taskClose: Get and the ownerId check run under gTaskLibMutex so a concurrent
// taskClose on the SAME handle from the owner can never free this task while we are
// still reading it. Once we know we are the owner, the context pointer is ours to use
// outside the lock -- the owner is single-threaded, so it cannot be closing the same
// task from another thread while this call is in flight.
pthread_mutex_lock(&gTaskLibMutex);
task = (TaskT *)calogHandleGet(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT); task = (TaskT *)calogHandleGet(lib->handles, args[0].as.i, TASK_TYPE_CONTEXT);
if (task == NULL) { if (task == NULL) {
pthread_mutex_unlock(&gTaskLibMutex);
return calogFail(result, calogErrArgE, "taskEval: invalid task handle"); return calogFail(result, calogErrArgE, "taskEval: invalid task handle");
} }
if (task->ownerId != calogCurrentId()) { if (task->ownerId != calogCurrentId()) {
pthread_mutex_unlock(&gTaskLibMutex);
return calogFail(result, calogErrArgE, "taskEval: only the task's owner may control it"); return calogFail(result, calogErrArgE, "taskEval: only the task's owner may control it");
} }
if (calogContextEval(task->context, args[1].as.s.bytes) != calogOkE) { context = task->context;
pthread_mutex_unlock(&gTaskLibMutex);
if (calogContextEval(context, args[1].as.s.bytes) != calogOkE) {
return calogFail(result, calogErrArgE, "taskEval: could not queue the code"); return calogFail(result, calogErrArgE, "taskEval: could not queue the code");
} }
return calogOkE; return calogOkE;
} }
static int32_t taskExit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
calogValueNil(result);
// A task retires ITSELF. It cannot free its own context synchronously (it runs on the very
// thread a close would join), so it flags the context to retire once the CURRENT code
// finishes (the rest of this eval runs normally); the finished context is then reaped by
// calogTaskReap, which the host calls while pumping. No handle -- a task has none to itself.
calogCurrentRetire();
return calogOkE;
}
static int32_t taskLoad(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t taskLoad(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
TaskLibT *lib; TaskLibT *lib;
CalogT *calog; CalogT *calog;
CalogContextT *context; CalogContextT *context;
TaskT *task;
int64_t handle; int64_t handle;
lib = (TaskLibT *)userData; lib = (TaskLibT *)userData;
@ -225,18 +333,9 @@ static int32_t taskLoad(CalogValueT *args, int32_t argCount, CalogValueT *result
if (context == NULL) { if (context == NULL) {
return calogFail(result, calogErrArgE, "taskLoad: no matching script file, or the load failed"); return calogFail(result, calogErrArgE, "taskLoad: no matching script file, or the load failed");
} }
task = (TaskT *)malloc(sizeof(*task)); handle = taskWrapContext(lib, context, result, "taskLoad");
if (task == NULL) {
calogContextClose(context);
return calogFail(result, calogErrOomE, "taskLoad: out of memory");
}
task->context = context;
task->ownerId = calogCurrentId();
handle = calogHandleAdd(lib->handles, TASK_TYPE_CONTEXT, task);
if (handle == 0) { if (handle == 0) {
free(task); return calogErrOomE;
calogContextClose(context);
return calogFail(result, calogErrOomE, "taskLoad: out of memory");
} }
calogValueInt(result, handle); calogValueInt(result, handle);
return calogOkE; return calogOkE;
@ -277,21 +376,12 @@ static int32_t taskSpawn(CalogValueT *args, int32_t argCount, CalogValueT *resul
if (context == NULL) { if (context == NULL) {
return calogFail(result, calogErrArgE, "taskSpawn: could not open a context"); return calogFail(result, calogErrArgE, "taskSpawn: could not open a context");
} }
task = (TaskT *)malloc(sizeof(*task)); handle = taskWrapContext(lib, context, result, "taskSpawn");
if (task == NULL) {
calogContextClose(context);
return calogFail(result, calogErrOomE, "taskSpawn: out of memory");
}
task->context = context;
task->ownerId = calogCurrentId();
handle = calogHandleAdd(lib->handles, TASK_TYPE_CONTEXT, task);
if (handle == 0) { if (handle == 0) {
free(task); return calogErrOomE;
calogContextClose(context);
return calogFail(result, calogErrOomE, "taskSpawn: out of memory");
} }
if (calogContextEval(context, args[1].as.s.bytes) != calogOkE) { if (calogContextEval(context, args[1].as.s.bytes) != calogOkE) {
calogHandleRemove(lib->handles, handle, TASK_TYPE_CONTEXT); task = (TaskT *)calogHandleRemove(lib->handles, handle, TASK_TYPE_CONTEXT);
free(task); free(task);
calogContextClose(context); calogContextClose(context);
return calogFail(result, calogErrArgE, "taskSpawn: could not queue the code"); return calogFail(result, calogErrArgE, "taskSpawn: could not queue the code");
@ -299,3 +389,32 @@ static int32_t taskSpawn(CalogValueT *args, int32_t argCount, CalogValueT *resul
calogValueInt(result, handle); calogValueInt(result, handle);
return calogOkE; return calogOkE;
} }
// Shared tail of taskLoad/taskSpawn: wrap an already-open context in a TaskT and file it in
// the handle table. Returns the new handle, or 0 after calling calogFail (with calogErrOomE)
// and closing the context. label is the caller's name, used only in the failure message.
static int64_t taskWrapContext(TaskLibT *lib, CalogContextT *context, CalogValueT *result, const char *label) {
TaskT *task;
int64_t handle;
char message[CALOG_ERR_MSG_CAP];
task = (TaskT *)malloc(sizeof(*task));
if (task == NULL) {
calogContextClose(context);
snprintf(message, sizeof(message), "%s: out of memory", label);
calogFail(result, calogErrOomE, message);
return 0;
}
task->context = context;
task->ownerId = calogCurrentId();
handle = calogHandleAdd(lib->handles, TASK_TYPE_CONTEXT, task);
if (handle == 0) {
free(task);
calogContextClose(context);
snprintf(message, sizeof(message), "%s: out of memory", label);
calogFail(result, calogErrOomE, message);
return 0;
}
return handle;
}

View file

@ -5,6 +5,9 @@
// taskLoad(baseName) -> handle launch a script FILE (engine chosen by extension) // taskLoad(baseName) -> handle launch a script FILE (engine chosen by extension)
// taskEval(handle, code) feed more code into a running task (runs on its thread) // taskEval(handle, code) feed more code into a running task (runs on its thread)
// taskClose(handle) stop a task (cooperative shutdown; see below) // taskClose(handle) stop a task (cooperative shutdown; see below)
// taskActive() -> bool (in a task) false once its owner has asked it to stop
// taskActive(handle) -> bool (in the owner) is that spawned task still open?
// taskExit() (in a task) retire this task's own context (deferred)
// taskSelf() -> id the calling script's own context id // taskSelf() -> id the calling script's own context id
// taskCount() -> n number of tasks this library currently holds open // taskCount() -> n number of tasks this library currently holds open
// //
@ -21,7 +24,9 @@
// taskClose is COOPERATIVE (it is the "kill"): it queues a shutdown and waits for the task's // taskClose is COOPERATIVE (it is the "kill"): it queues a shutdown and waits for the task's
// thread to exit. A task blocked calling back into the host (a host-thread native) has that // thread to exit. A task blocked calling back into the host (a host-thread native) has that
// call interrupted; a task busy in a pure computation is not force-cancelled (that would // call interrupted; a task busy in a pure computation is not force-cancelled (that would
// corrupt interpreter state), so a task spinning in an infinite loop cannot be force-killed. // corrupt interpreter state), so a task spinning in an infinite loop cannot be force-killed --
// it must poll taskActive() and break its own loop, or taskClose will block until it happens
// to return.
// (A task that, during its shutdown, synchronously invokes a callable owned by the closing // (A task that, during its shutdown, synchronously invokes a callable owned by the closing
// context can still dead-lock, since the closer is blocked in the join -- avoid that.) // context can still dead-lock, since the closer is blocked in the join -- avoid that.)
// Engines are addressed by name for // Engines are addressed by name for
@ -38,6 +43,12 @@
// calogOkE or an error. // calogOkE or an error.
int32_t calogTaskRegister(CalogT *calog); int32_t calogTaskRegister(CalogT *calog);
// Reap tasks whose context thread has already finished -- a task that called taskExit(), or
// one that ran to the end while its owner never closed it. The host should call this
// periodically (once per calogPump is ideal) so self-retired tasks are joined and freed
// promptly instead of lingering until calogDestroy. Returns how many were reaped.
int32_t calogTaskReap(void);
// Free the process-wide task registry. Call it AFTER the runtime is torn down // Free the process-wide task registry. Call it AFTER the runtime is torn down
// (calogDestroy), which is what actually closes any still-open task contexts. // (calogDestroy), which is what actually closes any still-open task contexts.
void calogTaskShutdown(void); void calogTaskShutdown(void);

View file

@ -10,6 +10,10 @@
#include <errno.h> #include <errno.h>
#include <time.h> #include <time.h>
// Conversion factors shared by timeSleepNative and timeToSeconds.
#define TIME_NS_PER_MS 1000000.0
#define TIME_NS_PER_SEC 1000000000
static int32_t timeMonotonicNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t timeMonotonicNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t timeNowNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t timeNowNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t timeSleepNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t timeSleepNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
@ -73,9 +77,14 @@ static int32_t timeSleepNative(CalogValueT *args, int32_t argCount, CalogValueT
if (!(ms >= 0.0)) { if (!(ms >= 0.0)) {
return calogFail(result, calogErrRangeE, "timeSleep: milliseconds must be non-negative"); return calogFail(result, calogErrRangeE, "timeSleep: milliseconds must be non-negative");
} }
nanos = (int64_t)(ms * 1000000.0); // Reject ms large enough that ms * TIME_NS_PER_MS would overflow int64_t --
request.tv_sec = (time_t)(nanos / 1000000000); // the cast below is UB otherwise (C11 6.3.1.4p1).
request.tv_nsec = (long)(nanos % 1000000000); if (ms > (double)INT64_MAX / TIME_NS_PER_MS) {
return calogFail(result, calogErrRangeE, "timeSleep: milliseconds out of range");
}
nanos = (int64_t)(ms * TIME_NS_PER_MS);
request.tv_sec = (time_t)(nanos / TIME_NS_PER_SEC);
request.tv_nsec = (long)(nanos % TIME_NS_PER_SEC);
while (nanosleep(&request, &remaining) != 0) { while (nanosleep(&request, &remaining) != 0) {
if (errno != EINTR) { if (errno != EINTR) {
return calogFail(result, calogErrUnsupportedE, "timeSleep: nanosleep failed"); return calogFail(result, calogErrUnsupportedE, "timeSleep: nanosleep failed");
@ -87,5 +96,5 @@ static int32_t timeSleepNative(CalogValueT *args, int32_t argCount, CalogValueT
static double timeToSeconds(const struct timespec *ts) { static double timeToSeconds(const struct timespec *ts) {
return (double)ts->tv_sec + (double)ts->tv_nsec / 1000000000.0; return (double)ts->tv_sec + (double)ts->tv_nsec / TIME_NS_PER_SEC;
} }

View file

@ -26,8 +26,10 @@
#include <stdlib.h> #include <stdlib.h>
#include <time.h> #include <time.h>
#define TIMER_INITIAL_CAP 8 #include "calogInternal.h"
#define TIMER_GROWTH 2
#define NS_PER_SEC 1000000000LL
#define NS_PER_MS 1000000.0
// One scheduled timer. intervalNs == 0 marks a one-shot; a periodic timer keeps a non-zero // One scheduled timer. intervalNs == 0 marks a one-shot; a periodic timer keeps a non-zero
// interval. A cancelled timer is tombstoned here and pruned by the thread at the top of its // interval. A cancelled timer is tombstoned here and pruned by the thread at the top of its
@ -62,6 +64,7 @@ static int32_t timerCancelNative(CalogValueT *args, int32_t argCount, CalogValue
static int32_t timerEnsureThreadLocked(void); static int32_t timerEnsureThreadLocked(void);
static int32_t timerEveryNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t timerEveryNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t timerFindByIdLocked(int64_t id); static int32_t timerFindByIdLocked(int64_t id);
static void timerFreeAll(void);
static int64_t timerNowNs(void); static int64_t timerNowNs(void);
static void timerPruneLocked(void); static void timerPruneLocked(void);
static int32_t timerSchedule(CalogValueT *args, int32_t argCount, CalogValueT *result, bool periodic); static int32_t timerSchedule(CalogValueT *args, int32_t argCount, CalogValueT *result, bool periodic);
@ -69,9 +72,7 @@ static void *timerThreadMain(void *arg);
int32_t calogTimerRegister(CalogT *calog) { int32_t calogTimerRegister(CalogT *calog) {
pthread_mutex_lock(&gInitMutex); calogRegistryRetain(&gInitMutex, &gRefCount);
gRefCount++;
pthread_mutex_unlock(&gInitMutex);
calogRegisterInline(calog, "timerAfter", timerAfterNative, NULL); calogRegisterInline(calog, "timerAfter", timerAfterNative, NULL);
calogRegisterInline(calog, "timerEvery", timerEveryNative, NULL); calogRegisterInline(calog, "timerEvery", timerEveryNative, NULL);
calogRegisterInline(calog, "timerCancel", timerCancelNative, NULL); calogRegisterInline(calog, "timerCancel", timerCancelNative, NULL);
@ -80,48 +81,10 @@ int32_t calogTimerRegister(CalogT *calog) {
void calogTimerShutdown(void) { void calogTimerShutdown(void) {
pthread_t threadHandle; // calogRegistryRelease holds gInitMutex for the whole call, including timerFreeAll below,
bool joinThread; // so a timerSchedule that checks gRefCount under gInitMutex (see timerSchedule) can never
int32_t index; // observe a mid-teardown state: it either runs entirely before this or entirely after.
calogRegistryRelease(&gInitMutex, &gRefCount, timerFreeAll);
joinThread = false;
pthread_mutex_lock(&gInitMutex);
if (gRefCount > 0) {
gRefCount--;
}
if (gRefCount > 0) {
pthread_mutex_unlock(&gInitMutex);
return;
}
// Last runtime is gone: stop the thread (if it ever started), then release every remaining
// callback and free the list. Contexts are still alive here, so a callback release routes a
// finalize to its owner thread just like calogExport.
pthread_mutex_lock(&gTimerMutex);
gShutdown = true;
threadHandle = gThread;
if (gThreadStarted) {
joinThread = true;
pthread_cond_signal(&gTimerCond);
}
pthread_mutex_unlock(&gTimerMutex);
if (joinThread) {
pthread_join(threadHandle, NULL);
}
pthread_mutex_lock(&gTimerMutex);
for (index = 0; index < gCount; index++) {
calogFnRelease(gTimers[index].callback);
}
free(gTimers);
gTimers = NULL;
gCount = 0;
gCap = 0;
if (joinThread) {
pthread_cond_destroy(&gTimerCond);
}
gThreadStarted = false;
gShutdown = false;
pthread_mutex_unlock(&gTimerMutex);
pthread_mutex_unlock(&gInitMutex);
} }
@ -136,6 +99,9 @@ static int32_t timerCancelNative(CalogValueT *args, int32_t argCount, CalogValue
(void)userData; (void)userData;
calogValueNil(result); calogValueNil(result);
// Ids are always minted as calogIntE (timerSchedule -> calogValueInt), so require exactly
// that here rather than also coercing a calogRealE -- keeps the id-validation policy for
// library-issued ids strict and uniform instead of silently accepting drifted types.
if (argCount != 1 || args[0].type != calogIntE) { if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "timerCancel expects (id)"); return calogFail(result, calogErrArgE, "timerCancel expects (id)");
} }
@ -195,11 +161,49 @@ static int32_t timerFindByIdLocked(int64_t id) {
} }
// Invoked by calogRegistryRelease while gInitMutex is held, exactly once, when the last
// registered runtime releases the timer library. Stops the thread (if it ever started), then
// releases every remaining callback and frees the list. Contexts are still alive here, so a
// callback release routes a finalize to its owner thread just like calogExport.
static void timerFreeAll(void) {
pthread_t threadHandle;
bool joinThread;
int32_t index;
joinThread = false;
pthread_mutex_lock(&gTimerMutex);
gShutdown = true;
threadHandle = gThread;
if (gThreadStarted) {
joinThread = true;
pthread_cond_signal(&gTimerCond);
}
pthread_mutex_unlock(&gTimerMutex);
if (joinThread) {
pthread_join(threadHandle, NULL);
}
pthread_mutex_lock(&gTimerMutex);
for (index = 0; index < gCount; index++) {
calogFnRelease(gTimers[index].callback);
}
free(gTimers);
gTimers = NULL;
gCount = 0;
gCap = 0;
if (joinThread) {
pthread_cond_destroy(&gTimerCond);
}
gThreadStarted = false;
gShutdown = false;
pthread_mutex_unlock(&gTimerMutex);
}
static int64_t timerNowNs(void) { static int64_t timerNowNs(void) {
struct timespec ts; struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts); clock_gettime(CLOCK_MONOTONIC, &ts);
return (int64_t)ts.tv_sec * 1000000000 + (int64_t)ts.tv_nsec; return (int64_t)ts.tv_sec * NS_PER_SEC + (int64_t)ts.tv_nsec;
} }
@ -221,6 +225,8 @@ static void timerPruneLocked(void) {
static int32_t timerSchedule(CalogValueT *args, int32_t argCount, CalogValueT *result, bool periodic) { static int32_t timerSchedule(CalogValueT *args, int32_t argCount, CalogValueT *result, bool periodic) {
const char *label; const char *label;
void *buffer;
int64_t capacity;
double ms; double ms;
int64_t delayNs; int64_t delayNs;
int64_t id; int64_t id;
@ -238,23 +244,31 @@ static int32_t timerSchedule(CalogValueT *args, int32_t argCount, CalogValueT *r
if (!(ms >= 0.0 && ms <= 9.0e12)) { if (!(ms >= 0.0 && ms <= 9.0e12)) {
return calogFail(result, calogErrRangeE, "timer: milliseconds out of range"); return calogFail(result, calogErrRangeE, "timer: milliseconds out of range");
} }
delayNs = (int64_t)(ms * 1000000.0); delayNs = (int64_t)(ms * NS_PER_MS);
// Hold gInitMutex for the whole schedule so it can never straddle a calogTimerShutdown:
// either gRefCount is still positive and the thread this call may (re)start is one that
// shutdown has not yet torn down, or shutdown already ran and this call fails outright
// instead of silently resurrecting an unjoined background thread.
pthread_mutex_lock(&gInitMutex);
if (gRefCount <= 0) {
pthread_mutex_unlock(&gInitMutex);
return calogFail(result, calogErrDeadE, "timer: library has been shut down");
}
pthread_mutex_lock(&gTimerMutex); pthread_mutex_lock(&gTimerMutex);
if (gCount == gCap) { buffer = gTimers;
int32_t newCap; capacity = gCap;
TimerEntryT *grown; status = calogGrow(&buffer, &capacity, (int64_t)gCount + 1, sizeof(TimerEntryT));
newCap = (gCap == 0) ? TIMER_INITIAL_CAP : gCap * TIMER_GROWTH; if (status != calogOkE) {
grown = (TimerEntryT *)realloc(gTimers, (size_t)newCap * sizeof(TimerEntryT));
if (grown == NULL) {
pthread_mutex_unlock(&gTimerMutex); pthread_mutex_unlock(&gTimerMutex);
return calogFail(result, calogErrOomE, "timer: out of memory"); pthread_mutex_unlock(&gInitMutex);
} return calogFail(result, status, "timer: out of memory");
gTimers = grown;
gCap = newCap;
} }
gTimers = (TimerEntryT *)buffer;
gCap = (int32_t)capacity;
status = timerEnsureThreadLocked(); status = timerEnsureThreadLocked();
if (status != calogOkE) { if (status != calogOkE) {
pthread_mutex_unlock(&gTimerMutex); pthread_mutex_unlock(&gTimerMutex);
pthread_mutex_unlock(&gInitMutex);
return calogFail(result, status, "timer: could not start the timer thread"); return calogFail(result, status, "timer: could not start the timer thread");
} }
id = gNextId++; id = gNextId++;
@ -268,6 +282,7 @@ static int32_t timerSchedule(CalogValueT *args, int32_t argCount, CalogValueT *r
gCount++; gCount++;
pthread_cond_signal(&gTimerCond); pthread_cond_signal(&gTimerCond);
pthread_mutex_unlock(&gTimerMutex); pthread_mutex_unlock(&gTimerMutex);
pthread_mutex_unlock(&gInitMutex);
calogValueInt(result, id); calogValueInt(result, id);
return calogOkE; return calogOkE;
} }
@ -301,8 +316,8 @@ static void *timerThreadMain(void *arg) {
struct timespec target; struct timespec target;
int64_t fireAt; int64_t fireAt;
fireAt = gTimers[minIndex].nextFireMonoNs; fireAt = gTimers[minIndex].nextFireMonoNs;
target.tv_sec = (time_t)(fireAt / 1000000000); target.tv_sec = (time_t)(fireAt / NS_PER_SEC);
target.tv_nsec = (long)(fireAt % 1000000000); target.tv_nsec = (long)(fireAt % NS_PER_SEC);
pthread_cond_timedwait(&gTimerCond, &gTimerMutex, &target); pthread_cond_timedwait(&gTimerCond, &gTimerMutex, &target);
continue; continue;
} }

85
src/adapterBinding.h Normal file
View file

@ -0,0 +1,85 @@
// adapterBinding.h -- shared broker+name binding registry for native-trampoline
// engine adapters (Lua, Squirrel, ...).
//
// An exposed native is bound by broker + name, not a captured fn pointer, so a
// trampoline dispatches through calogCall and honors the actor route hook: a native
// owned by another context is marshalled to its thread instead of being run (wrongly)
// on this state's thread. With no hook installed, calogCall runs it inline --
// identical to the single-threaded path.
//
// Every adapter that installs BindingT trampolines shares the same expose-time
// bookkeeping (malloc + strdup + growable pointer array) and the same teardown loop;
// this header holds that one copy so each adapter file supplies only its VM-specific
// install step (pushing the binding as an upvalue/free-variable) and its own
// bindings/bindingCount/bindingCap fields.
#ifndef CALOG_ADAPTER_BINDING_H
#define CALOG_ADAPTER_BINDING_H
#include "calogInternal.h"
#include <stdlib.h>
#include <string.h>
typedef struct BindingT {
CalogT *broker;
char *name;
} BindingT;
// Look up `name` in `broker`, allocate a BindingT for it, and append it to
// *bindings/*count/*cap (grown with calogGrow). On success *out is the new binding
// (still owned by the list; the caller does not free it). On failure *out is NULL and
// nothing is appended or leaked.
static inline int32_t adapterBindingExpose(CalogT *broker, const char *name, BindingT ***bindings, int32_t *count, int32_t *cap, BindingT **out) {
CalogEntryT *entry;
BindingT *binding;
void *buffer;
int64_t cap64;
int32_t status;
*out = NULL;
entry = calogLookup(broker, name);
if (entry == NULL) {
return calogErrNotFoundE;
}
binding = (BindingT *)malloc(sizeof(*binding));
if (binding == NULL) {
return calogErrOomE;
}
binding->broker = broker;
binding->name = strdup(name);
if (binding->name == NULL) {
free(binding);
return calogErrOomE;
}
buffer = *bindings;
cap64 = *cap;
status = calogGrow(&buffer, &cap64, (int64_t)*count + 1, sizeof(BindingT *));
if (status != calogOkE) {
free(binding->name);
free(binding);
return status;
}
*bindings = (BindingT **)buffer;
*cap = (int32_t)cap64;
(*bindings)[*count] = binding;
(*count)++;
*out = binding;
return calogOkE;
}
// Free every binding in the list, then the list array itself. Does not touch the
// caller's count/cap fields; the caller is tearing the whole context down.
static inline void adapterBindingDestroyAll(BindingT **bindings, int32_t count) {
int32_t index;
for (index = 0; index < count; index++) {
free(bindings[index]->name);
free(bindings[index]);
}
free(bindings);
}
#endif

View file

@ -25,7 +25,6 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define BERRY_ERR_CAP 256
#define BERRY_REF_CAP 32 #define BERRY_REF_CAP 32
#define BERRY_FOREIGN_INITIAL 8 #define BERRY_FOREIGN_INITIAL 8
@ -37,12 +36,16 @@ struct CalogBerryT {
CalogFnT **foreignFns; // foreign function values pushed into this VM CalogFnT **foreignFns; // foreign function values pushed into this VM
int32_t foreignCount; int32_t foreignCount;
int32_t foreignCap; int32_t foreignCap;
int32_t *freeRefs; // reclaimed _calog_fn_N slot numbers, reused before minting new ones
int32_t freeCount;
int32_t freeCap;
}; };
// Backs a CalogFnT exported from this VM: the owning context and the name of the hidden // Backs a CalogFnT exported from this VM: the owning context, the reclaimable slot
// global that keeps the Berry function GC-reachable. // number, and the name of the hidden global that keeps the Berry function GC-reachable.
typedef struct BerryExportT { typedef struct BerryExportT {
CalogBerryT *context; CalogBerryT *context;
int32_t refNum;
char refName[BERRY_REF_CAP]; char refName[BERRY_REF_CAP];
} BerryExportT; } BerryExportT;
@ -90,7 +93,9 @@ static int32_t berryCallableInvoke(CalogValueT *args, int32_t argCount, CalogVal
vm = context->vm; vm = context->vm;
calogValueNil(result); calogValueNil(result);
base = be_top(vm); base = be_top(vm);
if (!be_getglobal(vm, export->refName)) { be_getglobal(vm, export->refName);
if (be_isnil(vm, -1)) {
be_pop(vm, be_top(vm) - base);
return calogFail(result, calogErrDeadE, "berry callable no longer exists"); return calogFail(result, calogErrDeadE, "berry callable no longer exists");
} }
for (index = 0; index < argCount; index++) { for (index = 0; index < argCount; index++) {
@ -116,14 +121,29 @@ static int32_t berryCallableInvoke(CalogValueT *args, int32_t argCount, CalogVal
static void berryCallableRelease(CalogFnT *callable) { static void berryCallableRelease(CalogFnT *callable) {
BerryExportT *export; BerryExportT *export;
CalogBerryT *context;
bvm *vm; bvm *vm;
void *buffer;
int64_t cap64;
export = (BerryExportT *)calogFnUserData(callable); export = (BerryExportT *)calogFnUserData(callable);
vm = export->context->vm; context = export->context;
vm = context->vm;
// Drop the hidden global so the pinned function becomes collectable. // Drop the hidden global so the pinned function becomes collectable.
be_pushnil(vm); be_pushnil(vm);
be_setglobal(vm, export->refName); be_setglobal(vm, export->refName);
be_pop(vm, 1); be_pop(vm, 1);
// Reclaim the slot number so the next export reuses it instead of minting a fresh
// hidden global. If the free list cannot grow, the number is simply not reused --
// no leak, just a missed optimization on this release.
buffer = context->freeRefs;
cap64 = context->freeCap;
if (calogGrow(&buffer, &cap64, (int64_t)context->freeCount + 1, sizeof(int32_t)) == calogOkE) {
context->freeRefs = (int32_t *)buffer;
context->freeCap = (int32_t)cap64;
context->freeRefs[context->freeCount] = export->refNum;
context->freeCount++;
}
free(export); free(export);
} }
@ -139,6 +159,7 @@ void calogBerryDestroy(CalogBerryT *context) {
calogFnRelease(context->foreignFns[index]); calogFnRelease(context->foreignFns[index]);
} }
free(context->foreignFns); free(context->foreignFns);
free(context->freeRefs);
if (context->vm != NULL) { if (context->vm != NULL) {
be_vm_delete(context->vm); be_vm_delete(context->vm);
} }
@ -180,8 +201,17 @@ static int32_t berryExportValue(CalogBerryT *context, int index, CalogFnT **out)
return calogErrOomE; return calogErrOomE;
} }
export->context = context; export->context = context;
snprintf(export->refName, sizeof(export->refName), "_calog_fn_%d", context->nextRef); if (context->freeCount > 0) {
// Reuse a slot released by a prior berryCallableRelease instead of minting a
// new global name, so the hidden global table stays bounded by the peak number
// of simultaneously live exports.
context->freeCount--;
export->refNum = context->freeRefs[context->freeCount];
} else {
export->refNum = context->nextRef;
context->nextRef++; context->nextRef++;
}
snprintf(export->refName, sizeof(export->refName), "_calog_fn_%d", export->refNum);
// Pin the function under a hidden global name (globals are GC roots). // Pin the function under a hidden global name (globals are GC roots).
be_pushvalue(vm, index); be_pushvalue(vm, index);
be_setglobal(vm, export->refName); be_setglobal(vm, export->refName);
@ -232,7 +262,7 @@ static int berryForeignCall(bvm *vm) {
int argc; int argc;
int index; int index;
int32_t status; int32_t status;
char message[BERRY_ERR_CAP]; char message[CALOG_ERR_MSG_CAP];
argc = be_top(vm); // arguments occupy stack slots 1..argc argc = be_top(vm); // arguments occupy stack slots 1..argc
be_getupval(vm, 0, 0); be_getupval(vm, 0, 0);
@ -271,8 +301,13 @@ static int berryForeignCall(bvm *vm) {
be_raise(vm, "calog_error", message); be_raise(vm, "calog_error", message);
return 0; return 0;
} }
berryFromValue(context, &result, 0); status = berryFromValue(context, &result, 0);
calogValueFree(&result); calogValueFree(&result);
if (status != calogOkE) {
be_pop(vm, 1);
be_raise(vm, "type_error", "failed to marshal the function-value result");
return 0;
}
be_return(vm); be_return(vm);
} }
@ -281,7 +316,7 @@ static int32_t berryFromValue(CalogBerryT *context, const CalogValueT *value, in
bvm *vm; bvm *vm;
vm = context->vm; vm = context->vm;
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
be_pushnil(vm); be_pushnil(vm);
return calogErrDepthE; return calogErrDepthE;
} }
@ -305,16 +340,27 @@ static int32_t berryFromValue(CalogBerryT *context, const CalogValueT *value, in
CalogAggT *aggregate; CalogAggT *aggregate;
const char *className; const char *className;
int64_t index; int64_t index;
int containerBase;
int32_t childStatus;
aggregate = value->as.agg; aggregate = value->as.agg;
containerBase = be_top(vm);
// Populate a raw container: anything keyed (or an explicit map) is a map with // Populate a raw container: anything keyed (or an explicit map) is a map with
// the sequence part at integer keys, else a list. be_setindex and be_data_push // the sequence part at integer keys, else a list. be_setindex and be_data_push
// take the value on top and do not pop, so drop it after each. // take the value on top and do not pop, so drop it after each. A failed
if (aggregate->pairCount > 0 || aggregate->kind == calogMapE) { // recursive marshal leaves a nil on top of the failing key (or bare), so
// unwind the whole container (Berry GC owns it) and propagate the error --
// a single nil is left on top to keep this frame's own stack contract.
if (calogAggIsKeyed(aggregate)) {
className = "map"; className = "map";
be_newmap(vm); be_newmap(vm);
for (index = 0; index < aggregate->arrayCount; index++) { for (index = 0; index < aggregate->arrayCount; index++) {
be_pushint(vm, (bint)index); be_pushint(vm, (bint)index);
berryFromValue(context, &aggregate->array[index], depth + 1); childStatus = berryFromValue(context, &aggregate->array[index], depth + 1);
if (childStatus != calogOkE) {
be_pop(vm, be_top(vm) - containerBase);
be_pushnil(vm);
return childStatus;
}
be_setindex(vm, -3); be_setindex(vm, -3);
be_pop(vm, 2); be_pop(vm, 2);
} }
@ -328,7 +374,12 @@ static int32_t berryFromValue(CalogBerryT *context, const CalogValueT *value, in
} else { } else {
continue; // non-scalar key: no Berry equivalent, drop the pair continue; // non-scalar key: no Berry equivalent, drop the pair
} }
berryFromValue(context, &aggregate->pairs[index].value, depth + 1); childStatus = berryFromValue(context, &aggregate->pairs[index].value, depth + 1);
if (childStatus != calogOkE) {
be_pop(vm, be_top(vm) - containerBase);
be_pushnil(vm);
return childStatus;
}
be_setindex(vm, -3); be_setindex(vm, -3);
be_pop(vm, 2); be_pop(vm, 2);
} }
@ -336,7 +387,12 @@ static int32_t berryFromValue(CalogBerryT *context, const CalogValueT *value, in
className = "list"; className = "list";
be_newlist(vm); be_newlist(vm);
for (index = 0; index < aggregate->arrayCount; index++) { for (index = 0; index < aggregate->arrayCount; index++) {
berryFromValue(context, &aggregate->array[index], depth + 1); childStatus = berryFromValue(context, &aggregate->array[index], depth + 1);
if (childStatus != calogOkE) {
be_pop(vm, be_top(vm) - containerBase);
be_pushnil(vm);
return childStatus;
}
be_data_push(vm, -2); be_data_push(vm, -2);
be_pop(vm, 1); be_pop(vm, 1);
} }
@ -404,7 +460,7 @@ static int32_t berryToValue(CalogBerryT *context, int index, CalogValueT *out, i
vm = context->vm; vm = context->vm;
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
if (be_isnil(vm, index)) { if (be_isnil(vm, index)) {
@ -551,7 +607,7 @@ static int berryTrampoline(bvm *vm) {
int argc; int argc;
int index; int index;
int32_t status; int32_t status;
char message[BERRY_ERR_CAP]; char message[CALOG_ERR_MSG_CAP];
argc = be_top(vm); // arguments occupy stack slots 1..argc argc = be_top(vm); // arguments occupy stack slots 1..argc
// Recover the binding from the closure's upvalues (pushed above the args). // Recover the binding from the closure's upvalues (pushed above the args).

View file

@ -190,8 +190,11 @@ CalogContextT *calogContextLoad(CalogT *calog, const char *baseFileName);
int32_t calogContextEval(CalogContextT *context, const char *source); // fire-and-forget int32_t calogContextEval(CalogContextT *context, const char *source); // fire-and-forget
void calogContextClose(CalogContextT *context); void calogContextClose(CalogContextT *context);
uint64_t calogContextId(const CalogContextT *context); uint64_t calogContextId(const CalogContextT *context);
bool calogContextFinished(const CalogContextT *context); // true once its thread has exited (safe to reap)
uint64_t calogCurrentId(void); // context id of the calling thread (0 = host thread) uint64_t calogCurrentId(void); // context id of the calling thread (0 = host thread)
CalogT *calogCurrent(void); // runtime of the calling context (NULL if none) CalogT *calogCurrent(void); // runtime of the calling context (NULL if none)
void calogCurrentRetire(void); // ask the calling context to retire itself (deferred; see calogTask)
bool calogCurrentShuttingDown(void); // true once the calling context has been asked to close
extern const CalogEngineT calogLuaEngine; extern const CalogEngineT calogLuaEngine;
extern const CalogEngineT calogJsEngine; extern const CalogEngineT calogJsEngine;

View file

@ -73,6 +73,7 @@ struct CalogT {
int64_t ctxFreeCap; int64_t ctxFreeCap;
// actor layer (context.c): installed by calogActorInit, all NULL on a bare broker // actor layer (context.c): installed by calogActorInit, all NULL on a bare broker
CalogContextT *hostContext; // id 0, no thread; driven by calogPump CalogContextT *hostContext; // id 0, no thread; driven by calogPump
pthread_t hostThread; // thread that ran calogActorInit (this runtime's host)
CalogRouteFnT routeHook; CalogRouteFnT routeHook;
CalogInvokeHookT invokeHook; CalogInvokeHookT invokeHook;
CalogReleaseHookT releaseHook; CalogReleaseHookT releaseHook;
@ -109,4 +110,56 @@ CalogT *calogContextBroker(const CalogContextT *context);
void *calogContextInterp(CalogContextT *context); void *calogContextInterp(CalogContextT *context);
bool calogContextRegistered(CalogT *runtime, uint64_t ctxId); bool calogContextRegistered(CalogT *runtime, uint64_t ctxId);
// ---- shared helpers (one source of truth; value.c) ----
// A formatted engine error message is copied into a stack buffer of this size before the
// interpreter's throw/longjmp releases the underlying result.
#define CALOG_ERR_MSG_CAP 256
// Exact-representable int64 range as doubles: a finite double N is an exact integer in
// int64 range iff floor(N) == N and CALOG_INT64_MIN_DOUBLE <= N < CALOG_INT64_MAX_DOUBLE.
// The upper bound 2^63 is itself not representable as an int64, hence the strict <.
#define CALOG_INT64_MIN_DOUBLE (-9223372036854775808.0)
#define CALOG_INT64_MAX_DOUBLE (9223372036854775808.0)
// Decode one ASCII hex digit to 0..15, or -1 if the byte is not a hex digit. The single
// source for hex-nibble decoding (crypto hex strings and JSON \\uXXXX escapes).
static inline int32_t calogHexNibble(unsigned char c) {
if (c >= '0' && c <= '9') {
return (int32_t)(c - '0');
}
if (c >= 'a' && c <= 'f') {
return (int32_t)(c - 'a' + 10);
}
if (c >= 'A' && c <= 'F') {
return (int32_t)(c - 'A' + 10);
}
return -1;
}
// Grow *buffer to hold at least `needed` elements of `elemSize` bytes, doubling from a
// minimum and guarding both the capacity multiply and the size_t byte multiply against
// overflow. The single growth policy for every dynamic array in the project.
int32_t calogGrow(void **buffer, int64_t *cap, int64_t needed, size_t elemSize);
// Canonical "an engine's number is a double" ingress: emit an int64 when the double is an
// exact integer in int64 range, otherwise a real.
void calogValueFromDouble(CalogValueT *out, double number);
// Canonical egress classification: true if a hybrid aggregate materializes as a keyed
// map/object/table, false if a sequence/array/list.
bool calogAggIsKeyed(const CalogAggT *aggregate);
// Build map[key] = value with correct ownership on failure (key is a C string; its byte
// length is strlen(key)). Returns calogOkE or an error, releasing anything it built.
int32_t calogMapSetInt(CalogAggT *map, const char *key, int64_t value);
int32_t calogMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length);
int32_t calogMapSetBool(CalogAggT *map, const char *key, bool flag);
// Refcounted process-wide library registry: retain once per calog...Register, release once
// per ...Shutdown. release invokes freeAll (while holding initMutex) exactly when the last
// reference drops.
void calogRegistryRetain(pthread_mutex_t *initMutex, int32_t *refCount);
void calogRegistryRelease(pthread_mutex_t *initMutex, int32_t *refCount, void (*freeAll)(void));
#endif #endif

View file

@ -57,7 +57,6 @@
static const CalogEngineT *engineForExtension(const char *ext); static const CalogEngineT *engineForExtension(const char *ext);
static const char *extensionOf(const char *arg); static const char *extensionOf(const char *arg);
static bool fileReadable(const char *path);
static int32_t nativeCalogExit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t nativeCalogExit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeCalogPrint(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t nativeCalogPrint(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData); static void onError(uint64_t contextId, const char *message, void *userData);
@ -97,6 +96,28 @@ static const CalogEngineT *const gEngines[] = {
&calogWrenEngine &calogWrenEngine
}; };
// The built-in libraries, in registration order. One source of truth: main() registers every
// entry from this table and printUsage names them from it, so the help text can never drift
// from what is actually registered.
static const struct {
const char *name;
int32_t (*reg)(CalogT *calog);
} gLibraries[] = {
{ "crypto", calogCryptoRegister },
{ "db", calogDbRegister },
{ "export", calogExportRegister },
{ "fs", calogFsRegister },
{ "http", calogHttpRegister },
{ "json", calogJsonRegister },
{ "kv", calogKvRegister },
{ "net", calogNetRegister },
{ "pubsub", calogPubsubRegister },
{ "ssh", calogSshRegister },
{ "task", calogTaskRegister },
{ "time", calogTimeRegister },
{ "timer", calogTimerRegister }
};
// s7 interns a small, bounded set of "permanent" strings it never reclaims (an s7 trait, not a // s7 interns a small, bounded set of "permanent" strings it never reclaims (an s7 trait, not a
// leak); suppress exactly that site so an ASan/LSan build of the runner stays quiet. LSan calls // leak); suppress exactly that site so an ASan/LSan build of the runner stays quiet. LSan calls
// this weak hook automatically. // this weak hook automatically.
@ -151,18 +172,6 @@ static const char *extensionOf(const char *arg) {
} }
static bool fileReadable(const char *path) {
FILE *file;
file = fopen(path, "rb");
if (file == NULL) {
return false;
}
fclose(file);
return true;
}
// calogExit([code]) -- a script asks the runner to tear everything down and exit. Inline: it // calogExit([code]) -- a script asks the runner to tear everything down and exit. Inline: it
// only sets two atomics, which is thread-safe, so it needs no host hop. // only sets two atomics, which is thread-safe, so it needs no host hop.
static int32_t nativeCalogExit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t nativeCalogExit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
@ -227,17 +236,38 @@ static void onSignal(int sig) {
static void printUsage(FILE *stream, const char *program) { static void printUsage(FILE *stream, const char *program) {
size_t index;
fprintf(stream, "usage: %s <script> [script ...]\n", program); fprintf(stream, "usage: %s <script> [script ...]\n", program);
fprintf(stream, "\n"); fprintf(stream, "\n");
fprintf(stream, "Run each script through the calog multi-language runtime.\n"); fprintf(stream, "Run each script through the calog multi-language runtime.\n");
fprintf(stream, "A name with a known extension runs on that engine:\n"); fprintf(stream, "A name with a known extension runs on that engine:\n");
fprintf(stream, " .lua .js .nut .bas .be .scm .wren\n"); fprintf(stream, " ");
// Derived from gEngines so the help text can never drift from the resolver's own truth.
for (index = 0; index < sizeof(gEngines) / sizeof(gEngines[0]); index++) {
const CalogEngineT *engine;
int32_t which;
engine = gEngines[index];
if (engine->extensions == NULL) {
continue;
}
for (which = 0; engine->extensions[which] != NULL; which++) {
fprintf(stream, " .%s", engine->extensions[which]);
}
}
fprintf(stream, "\n");
fprintf(stream, "A name with no recognized extension is searched for as <name>.<ext>\n"); fprintf(stream, "A name with no recognized extension is searched for as <name>.<ext>\n");
fprintf(stream, "in that priority order; the first existing file wins.\n"); fprintf(stream, "in that priority order; the first existing file wins.\n");
fprintf(stream, "\n"); fprintf(stream, "\n");
fprintf(stream, "Scripts share the crypto, db, export, fs, http, json, kv, net, pubsub,\n"); fprintf(stream, "Scripts share these libraries (plus calogPrint):");
fprintf(stream, "ssh, task, time, and timer libraries plus calogPrint(). Call calogExit([code])\n"); // Named from the same gLibraries table main() registers, so the help cannot drift.
fprintf(stream, "to tear everything down and exit; otherwise calog runs until interrupted.\n"); for (index = 0; index < sizeof(gLibraries) / sizeof(gLibraries[0]); index++) {
fprintf(stream, " %s", gLibraries[index].name);
}
fprintf(stream, "\n");
fprintf(stream, "Call calogExit([code]) to tear everything down and exit; otherwise calog\n");
fprintf(stream, "runs until interrupted.\n");
} }
@ -345,12 +375,14 @@ static bool resolveArg(const char *arg, const CalogEngineT **outEngine, char **o
return false; return false;
} }
snprintf(path, pathSize, "%s.%s", arg, engine->extensions[which]); snprintf(path, pathSize, "%s.%s", arg, engine->extensions[which]);
if (!fileReadable(path)) { source = readFile(path);
if (source == NULL) {
// Not found here is expected -- keep searching. Any other failure (permission,
// out of memory) is a hard stop, same as the exact-extension case above.
if (errno == ENOENT) {
free(path); free(path);
continue; continue;
} }
source = readFile(path);
if (source == NULL) {
fprintf(stderr, "calog: %s: %s\n", path, strerror(errno)); fprintf(stderr, "calog: %s: %s\n", path, strerror(errno));
free(path); free(path);
return false; return false;
@ -368,14 +400,16 @@ static bool resolveArg(const char *arg, const CalogEngineT **outEngine, char **o
int main(int argc, char **argv) { int main(int argc, char **argv) {
const CalogEngineT **engines; const CalogEngineT **engines = NULL;
char **sources; char **sources = NULL;
const char *program; const char *program;
CalogT *calog; CalogT *calog = NULL;
struct timespec tick = { 0, PUMP_INTERVAL_NS }; struct timespec tick = { 0, PUMP_INTERVAL_NS };
int32_t scriptCount; int32_t scriptCount = 0;
int32_t resolvedCount = 0;
int32_t liveCount; int32_t liveCount;
int32_t index; int32_t index;
int32_t status = 0;
program = strrchr(argv[0], '/'); program = strrchr(argv[0], '/');
program = (program != NULL) ? program + 1 : argv[0]; program = (program != NULL) ? program + 1 : argv[0];
@ -396,59 +430,39 @@ int main(int argc, char **argv) {
sources = (char **)malloc((size_t)scriptCount * sizeof(*sources)); sources = (char **)malloc((size_t)scriptCount * sizeof(*sources));
if (engines == NULL || sources == NULL) { if (engines == NULL || sources == NULL) {
fprintf(stderr, "calog: out of memory\n"); fprintf(stderr, "calog: out of memory\n");
free(engines); status = 1;
free(sources); goto cleanup;
return 1;
} }
// Resolve every argument before running any (fail fast): one bad name runs nothing. // Resolve every argument before running any (fail fast): one bad name runs nothing.
for (index = 0; index < scriptCount; index++) { for (index = 0; index < scriptCount; index++) {
if (!resolveArg(argv[index + 1], &engines[index], &sources[index])) { if (!resolveArg(argv[index + 1], &engines[index], &sources[index])) {
int32_t undo; status = 1;
for (undo = 0; undo < index; undo++) { goto cleanup;
free(sources[undo]);
}
free(engines);
free(sources);
return 1;
} }
resolvedCount++;
} }
calog = calogCreate(); calog = calogCreate();
if (calog == NULL) { if (calog == NULL) {
fprintf(stderr, "calog: failed to create runtime\n"); fprintf(stderr, "calog: failed to create runtime\n");
for (index = 0; index < scriptCount; index++) { status = 1;
free(sources[index]); goto cleanup;
}
free(engines);
free(sources);
return 1;
} }
calogSetErrorHandler(calog, onError, NULL); calogSetErrorHandler(calog, onError, NULL);
if (calogRegister(calog, "calogPrint", nativeCalogPrint, NULL) != calogOkE || if (calogRegister(calog, "calogPrint", nativeCalogPrint, NULL) != calogOkE ||
calogRegisterInline(calog, "calogExit", nativeCalogExit, NULL) != calogOkE || calogRegisterInline(calog, "calogExit", nativeCalogExit, NULL) != calogOkE) {
calogCryptoRegister(calog) != calogOkE || fprintf(stderr, "calog: failed to register the runner natives\n");
calogDbRegister(calog) != calogOkE || status = 1;
calogExportRegister(calog) != calogOkE || goto cleanup;
calogFsRegister(calog) != calogOkE || }
calogHttpRegister(calog) != calogOkE || for (index = 0; index < (int32_t)(sizeof(gLibraries) / sizeof(gLibraries[0])); index++) {
calogJsonRegister(calog) != calogOkE || if (gLibraries[index].reg(calog) != calogOkE) {
calogKvRegister(calog) != calogOkE || fprintf(stderr, "calog: failed to register the %s library\n", gLibraries[index].name);
calogNetRegister(calog) != calogOkE || status = 1;
calogPubsubRegister(calog) != calogOkE || goto cleanup;
calogSshRegister(calog) != calogOkE ||
calogTaskRegister(calog) != calogOkE ||
calogTimeRegister(calog) != calogOkE ||
calogTimerRegister(calog) != calogOkE) {
fprintf(stderr, "calog: failed to register libraries\n");
calogDestroy(calog);
for (index = 0; index < scriptCount; index++) {
free(sources[index]);
} }
free(engines);
free(sources);
return 1;
} }
calogRegisterBuiltinEngines(calog); calogRegisterBuiltinEngines(calog);
@ -461,13 +475,8 @@ int main(int argc, char **argv) {
gLaunched = (LaunchedT *)calloc((size_t)scriptCount, sizeof(*gLaunched)); gLaunched = (LaunchedT *)calloc((size_t)scriptCount, sizeof(*gLaunched));
if (gLaunched == NULL) { if (gLaunched == NULL) {
fprintf(stderr, "calog: out of memory\n"); fprintf(stderr, "calog: out of memory\n");
calogDestroy(calog); status = 1;
for (index = 0; index < scriptCount; index++) { goto cleanup;
free(sources[index]);
}
free(engines);
free(sources);
return 1;
} }
for (index = 0; index < scriptCount; index++) { for (index = 0; index < scriptCount; index++) {
CalogContextT *context; CalogContextT *context;
@ -475,9 +484,11 @@ int main(int argc, char **argv) {
context = calogContextOpen(calog, engines[index]); context = calogContextOpen(calog, engines[index]);
if (context == NULL) { if (context == NULL) {
fprintf(stderr, "calog: %s: failed to open a context\n", argv[index + 1]); fprintf(stderr, "calog: %s: failed to open a context\n", argv[index + 1]);
atomic_store(&gExitCode, 1); // failed launch: never report success on exit
} else if (calogContextEval(context, sources[index]) != calogOkE) { } else if (calogContextEval(context, sources[index]) != calogOkE) {
fprintf(stderr, "calog: %s: failed to start script\n", argv[index + 1]); fprintf(stderr, "calog: %s: failed to start script\n", argv[index + 1]);
calogContextClose(context); calogContextClose(context);
atomic_store(&gExitCode, 1); // failed launch: never report success on exit
} else { } else {
gLaunched[gLaunchedCount].id = calogContextId(context); gLaunched[gLaunchedCount].id = calogContextId(context);
gLaunched[gLaunchedCount].context = context; gLaunched[gLaunchedCount].context = context;
@ -488,6 +499,8 @@ int main(int argc, char **argv) {
} }
free(engines); free(engines);
free(sources); free(sources);
engines = NULL;
sources = NULL;
liveCount = gLaunchedCount; liveCount = gLaunchedCount;
if (liveCount == 0) { if (liveCount == 0) {
@ -499,6 +512,7 @@ int main(int argc, char **argv) {
// the last live context is gone, the runner exits. // the last live context is gone, the runner exits.
while (!atomic_load(&gShutdown)) { while (!atomic_load(&gShutdown)) {
calogPump(calog); calogPump(calog);
calogTaskReap(); // join + free any task that retired itself via taskExit()
for (index = 0; index < gLaunchedCount; index++) { for (index = 0; index < gLaunchedCount; index++) {
if (gLaunched[index].context != NULL && atomic_load(&gLaunched[index].failed)) { if (gLaunched[index].context != NULL && atomic_load(&gLaunched[index].failed)) {
int32_t expected; int32_t expected;
@ -516,6 +530,7 @@ int main(int argc, char **argv) {
nanosleep(&tick, NULL); nanosleep(&tick, NULL);
} }
calogPump(calog); calogPump(calog);
calogTaskReap();
// Release cross-context references while the contexts are still alive, tear the runtime // Release cross-context references while the contexts are still alive, tear the runtime
// down (joining every context thread), then free the registries that outlive it. // down (joining every context thread), then free the registries that outlive it.
@ -531,4 +546,17 @@ int main(int argc, char **argv) {
free(gLaunched); free(gLaunched);
return (int)atomic_load(&gExitCode); return (int)atomic_load(&gExitCode);
cleanup:
// Shared teardown for every pre-run failure path: free what resolveArg produced, the two
// argv-length arrays, and destroy the runtime if calogCreate got far enough to build one.
for (index = 0; index < resolvedCount; index++) {
free(sources[index]);
}
free(engines);
free(sources);
if (calog != NULL) {
calogDestroy(calog);
}
return status;
} }

View file

@ -32,6 +32,7 @@
#include "calogInternal.h" #include "calogInternal.h"
#include <pthread.h> #include <pthread.h>
#include <stdatomic.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
@ -54,10 +55,9 @@
typedef enum MessageKindE { typedef enum MessageKindE {
messageCallE = 0, messageCallE = 0,
messageReplyE = 1, messageReplyE = 1,
messageShutdownE = 2, messageEvalE = 2,
messageEvalE = 3, messageReleaseE = 3,
messageReleaseE = 4, messageErrorE = 4
messageErrorE = 5
} MessageKindE; } MessageKindE;
typedef struct ReplyBoxT { typedef struct ReplyBoxT {
@ -96,7 +96,9 @@ struct CalogContextT {
MessageT *tail; MessageT *tail;
MessageT *stash; MessageT *stash;
uint64_t nextToken; uint64_t nextToken;
bool shuttingDown; _Atomic bool shuttingDown; // set by contextRequestShutdown; polled lock-free by calogCurrentShuttingDown
_Atomic bool retireRequested; // set by calogCurrentRetire (taskExit); serveLoop retires AFTER the current eval
_Atomic bool finished; // set by threadMain just before the thread returns; lets any thread safely reap it
bool started; bool started;
bool interpDead; // set (under ctxMutex) once the interpreter is torn down bool interpDead; // set (under ctxMutex) once the interpreter is torn down
}; };
@ -119,7 +121,9 @@ static void contextDrainQueue(CalogContextT *context);
static int32_t contextEnqueue(CalogT *calog, uint64_t targetId, MessageT *message); static int32_t contextEnqueue(CalogT *calog, uint64_t targetId, MessageT *message);
static int32_t contextPostRelease(CalogT *calog, uint64_t targetId, CalogFnT *callable); static int32_t contextPostRelease(CalogT *calog, uint64_t targetId, CalogFnT *callable);
static void contextReply(CalogT *calog, MessageT *message, int32_t status, CalogValueT *result); static void contextReply(CalogT *calog, MessageT *message, int32_t status, CalogValueT *result);
static void contextRequestShutdown(CalogContextT *context);
static int32_t contextSendBlocking(CalogT *calog, uint64_t targetId, MessageT *message, CalogValueT *result); static int32_t contextSendBlocking(CalogT *calog, uint64_t targetId, MessageT *message, CalogValueT *result);
static bool dispatchCommon(CalogContextT *context, MessageT *message);
static void enqueueRaw(CalogContextT *context, MessageT *message); static void enqueueRaw(CalogContextT *context, MessageT *message);
static void hostDispatch(CalogT *calog, MessageT *message); static void hostDispatch(CalogT *calog, MessageT *message);
static uint64_t idCompose(int64_t index, uint32_t generation); static uint64_t idCompose(int64_t index, uint32_t generation);
@ -130,7 +134,7 @@ static void messageFree(MessageT *message);
static bool onOwnerThread(CalogT *runtime, uint64_t owner); static bool onOwnerThread(CalogT *runtime, uint64_t owner);
static void postError(CalogT *calog, uint64_t contextId, const CalogValueT *result); static void postError(CalogT *calog, uint64_t contextId, const CalogValueT *result);
static int32_t pumpUntil(CalogContextT *context, uint64_t token, int32_t *outStatus, CalogValueT *result); static int32_t pumpUntil(CalogContextT *context, uint64_t token, int32_t *outStatus, CalogValueT *result);
static int32_t registryFreePush(CalogT *calog, int64_t index); static void registryFreePush(CalogT *calog, int64_t index);
static CalogContextT *registryResolveLocked(CalogT *calog, uint64_t id); static CalogContextT *registryResolveLocked(CalogT *calog, uint64_t id);
static void serveLoop(CalogContextT *context); static void serveLoop(CalogContextT *context);
static MessageT *tryDequeue(CalogContextT *context); static MessageT *tryDequeue(CalogContextT *context);
@ -148,6 +152,7 @@ int32_t calogActorInit(CalogT *calog) {
calog->hostContext->broker = calog; calog->hostContext->broker = calog;
pthread_mutex_init(&calog->hostContext->queueMutex, NULL); pthread_mutex_init(&calog->hostContext->queueMutex, NULL);
pthread_cond_init(&calog->hostContext->queueCond, NULL); pthread_cond_init(&calog->hostContext->queueCond, NULL);
calog->hostThread = pthread_self();
currentContext = calog->hostContext; currentContext = calog->hostContext;
pthread_mutex_init(&calog->ctxMutex, NULL); pthread_mutex_init(&calog->ctxMutex, NULL);
@ -207,7 +212,12 @@ static void actorReleaseCallable(CalogFnT *callable) {
static int32_t actorRoute(CalogT *calog, CalogEntryT *entry, CalogValueT *args, int32_t argCount, CalogValueT *result) { static int32_t actorRoute(CalogT *calog, CalogEntryT *entry, CalogValueT *args, int32_t argCount, CalogValueT *result) {
// An inline native, or any call already on this runtime's host thread, runs inline; // An inline native, or any call already on this runtime's host thread, runs inline;
// otherwise the native is marshalled to the host thread (serviced by calogPump). // otherwise the native is marshalled to the host thread (serviced by calogPump).
if (entry->runInline || currentContext == calog->hostContext) { // currentContext alone is not enough: it is a single per-thread slot, so a thread
// hosting several runtimes in turn (calogCreate, calogCreate) only matches the most
// recently created one. pthread_equal against the physical host thread recorded at
// calogActorInit catches the older runtime too -- nobody else will ever pump its
// host queue, so marshalling there would deadlock forever.
if (entry->runInline || currentContext == calog->hostContext || pthread_equal(pthread_self(), calog->hostThread)) {
return entry->fn(args, argCount, result, entry->userData); return entry->fn(args, argCount, result, entry->userData);
} }
return contextDispatch(calog, CALOG_HOST_ID, entry->fn, entry->userData, args, argCount, result); return contextDispatch(calog, CALOG_HOST_ID, entry->fn, entry->userData, args, argCount, result);
@ -216,7 +226,6 @@ static int32_t actorRoute(CalogT *calog, CalogEntryT *entry, CalogValueT *args,
void calogActorShutdown(CalogT *calog) { void calogActorShutdown(CalogT *calog) {
int64_t index; int64_t index;
MessageT *message;
for (index = 0; index < calog->ctxCount; index++) { for (index = 0; index < calog->ctxCount; index++) {
CalogContextT *context; CalogContextT *context;
@ -224,11 +233,7 @@ void calogActorShutdown(CalogT *calog) {
if (context == NULL || !context->started) { if (context == NULL || !context->started) {
continue; continue;
} }
message = (MessageT *)calloc(1, sizeof(*message)); contextRequestShutdown(context);
if (message != NULL) {
message->kind = messageShutdownE;
enqueueRaw(context, message);
}
} }
// Join EVERY thread before freeing ANY context: a context tearing down may release a // Join EVERY thread before freeing ANY context: a context tearing down may release a
// callable owned by a sibling (marshalled across contexts), which resolves/posts to that // callable owned by a sibling (marshalled across contexts), which resolves/posts to that
@ -272,9 +277,9 @@ void calogActorShutdown(CalogT *calog) {
if (calog->hostContext != NULL) { if (calog->hostContext != NULL) {
bool wasHost; bool wasHost;
wasHost = (currentContext == calog->hostContext); wasHost = (currentContext == calog->hostContext);
while ((message = tryDequeue(calog->hostContext)) != NULL) { // contextDrainQueue (not a bare messageFree loop) finalizes any orphaned
messageFree(message); // messageReleaseE so a pending callable is not leaked.
} contextDrainQueue(calog->hostContext);
pthread_mutex_destroy(&calog->hostContext->queueMutex); pthread_mutex_destroy(&calog->hostContext->queueMutex);
pthread_cond_destroy(&calog->hostContext->queueCond); pthread_cond_destroy(&calog->hostContext->queueCond);
free(calog->hostContext); free(calog->hostContext);
@ -347,10 +352,7 @@ CalogContextT *calogContextOpen(CalogT *broker, const CalogEngineT *engine) {
} else { } else {
if (broker->ctxCount + 1 > (int64_t)CONTEXT_INDEX_MASK) { if (broker->ctxCount + 1 > (int64_t)CONTEXT_INDEX_MASK) {
pthread_mutex_unlock(&broker->ctxMutex); pthread_mutex_unlock(&broker->ctxMutex);
pthread_mutex_destroy(&context->queueMutex); goto fail;
pthread_cond_destroy(&context->queueCond);
free(context);
return NULL;
} }
if (broker->ctxCount == broker->ctxCap) { if (broker->ctxCount == broker->ctxCap) {
int64_t newCap; int64_t newCap;
@ -359,10 +361,7 @@ CalogContextT *calogContextOpen(CalogT *broker, const CalogEngineT *engine) {
grown = (CalogRegistrySlotT *)realloc(broker->ctxSlots, (size_t)newCap * sizeof(CalogRegistrySlotT)); grown = (CalogRegistrySlotT *)realloc(broker->ctxSlots, (size_t)newCap * sizeof(CalogRegistrySlotT));
if (grown == NULL) { if (grown == NULL) {
pthread_mutex_unlock(&broker->ctxMutex); pthread_mutex_unlock(&broker->ctxMutex);
pthread_mutex_destroy(&context->queueMutex); goto fail;
pthread_cond_destroy(&context->queueCond);
free(context);
return NULL;
} }
broker->ctxSlots = grown; broker->ctxSlots = grown;
broker->ctxCap = newCap; broker->ctxCap = newCap;
@ -381,13 +380,18 @@ CalogContextT *calogContextOpen(CalogT *broker, const CalogEngineT *engine) {
broker->ctxSlots[index].context = NULL; broker->ctxSlots[index].context = NULL;
registryFreePush(broker, index); registryFreePush(broker, index);
pthread_mutex_unlock(&broker->ctxMutex); pthread_mutex_unlock(&broker->ctxMutex);
goto fail;
}
context->started = true;
return context;
fail:
// Shared teardown for every failure above: nothing beyond the queue's own
// mutex/cond and the context shell has been allocated at any of these points.
pthread_mutex_destroy(&context->queueMutex); pthread_mutex_destroy(&context->queueMutex);
pthread_cond_destroy(&context->queueCond); pthread_cond_destroy(&context->queueCond);
free(context); free(context);
return NULL; return NULL;
}
context->started = true;
return context;
} }
@ -470,6 +474,29 @@ uint64_t calogCurrentId(void) {
} }
// Ask THIS context to retire itself, requested from inside the task (which has no handle to
// itself). Unlike an owner's taskClose, this does NOT shut the context down mid-eval: it sets
// a deferred flag so the CURRENT native/eval finishes normally (host calls and all), and
// serveLoop stops the context only once that eval returns. The thread then exits and a reaper
// (calogTaskReap) joins and frees it. No-op on the host context, which has no thread to end.
void calogCurrentRetire(void) {
if (currentContext != NULL && currentContext->id != CALOG_HOST_ID) {
atomic_store(&currentContext->retireRequested, true);
}
}
// True once this context's owner has requested it close (calogContextClose set the flag and
// is blocked joining this thread). A long-running script polls it to cooperate; the host
// thread, which has no context to close, always reads false. Lock-free (the flag is atomic).
bool calogCurrentShuttingDown(void) {
if (currentContext == NULL) {
return false;
}
return atomic_load(&currentContext->shuttingDown);
}
// Tear down a single context (quiescence assumed). The thread is stopped and joined, // Tear down a single context (quiescence assumed). The thread is stopped and joined,
// then the slot is unlinked under the registry lock so no foreign enqueue can reach // then the slot is unlinked under the registry lock so no foreign enqueue can reach
// the freed queue mutex; the freed index returns to the freelist. // the freed queue mutex; the freed index returns to the freelist.
@ -482,12 +509,7 @@ void calogContextClose(CalogContextT *context) {
} }
broker = context->broker; broker = context->broker;
if (context->started) { if (context->started) {
MessageT *message; contextRequestShutdown(context);
message = (MessageT *)calloc(1, sizeof(*message));
if (message != NULL) {
message->kind = messageShutdownE;
enqueueRaw(context, message);
}
pthread_join(context->thread, NULL); pthread_join(context->thread, NULL);
} }
pthread_mutex_lock(&broker->ctxMutex); pthread_mutex_lock(&broker->ctxMutex);
@ -620,6 +642,15 @@ static void contextDrainQueue(CalogContextT *context) {
messageFree(message); messageFree(message);
} }
} }
// A nested pumpUntil frame can abandon its token during shutdown (queue-empty exit)
// while a still-active outer frame stashes its late reply (token mismatch) and then
// never claims it once the thread ends. tryDequeue never visits context->stash, so
// free those replies here too.
while (context->stash != NULL) {
message = context->stash;
context->stash = message->next;
messageFree(message);
}
} }
@ -693,6 +724,13 @@ int32_t calogContextEval(CalogContextT *context, const char *source) {
} }
// True once this context's thread has fully exited (serveLoop returned, interpreter destroyed).
// A reaper polls it so it only pthread_joins a context that will not block, then frees it.
bool calogContextFinished(const CalogContextT *context) {
return atomic_load(&context->finished);
}
uint64_t calogContextId(const CalogContextT *context) { uint64_t calogContextId(const CalogContextT *context) {
return context->id; return context->id;
} }
@ -790,6 +828,18 @@ static void contextReply(CalogT *calog, MessageT *message, int32_t status, Calog
} }
// Ask a context's own thread to stop once its queue drains. Allocation-free (unlike an
// enqueued SHUTDOWN message would be) so it cannot fail under OOM and leave the caller's
// pthread_join waiting forever: it just flips the flag messageDequeue already waits on,
// under the same mutex, and signals the cond so a thread parked in cond_wait wakes.
static void contextRequestShutdown(CalogContextT *context) {
pthread_mutex_lock(&context->queueMutex);
context->shuttingDown = true;
pthread_cond_signal(&context->queueCond);
pthread_mutex_unlock(&context->queueMutex);
}
// Enqueue an already-built CALL to targetId and wait for its reply. A context caller // Enqueue an already-built CALL to targetId and wait for its reply. A context caller
// pumps its own queue (servicing re-entrant work) instead of sleeping; a foreign // pumps its own queue (servicing re-entrant work) instead of sleeping; a foreign
// caller blocks on a private reply box. The message is consumed either way. // caller blocks on a private reply box. The message is consumed either way.
@ -797,6 +847,23 @@ static int32_t contextSendBlocking(CalogT *calog, uint64_t targetId, MessageT *m
CalogContextT *caller; CalogContextT *caller;
int32_t status; int32_t status;
// A call that lands on calog's own host queue, made by the exact thread that
// hosts calog, can never be serviced by anyone else -- calogPump(calog) only ever
// runs on that thread, and currentContext may already name a different runtime's
// context (this thread hosts several). Run the call inline instead of enqueuing
// and waiting, or it hangs forever.
if (targetId == CALOG_HOST_ID && pthread_equal(pthread_self(), calog->hostThread)) {
int32_t index;
status = message->fn(message->args, message->argCount, result, message->userData);
if (message->args != NULL) {
for (index = 0; index < message->argCount; index++) {
calogValueFree(&message->args[index]);
}
free(message->args);
}
free(message);
return status;
}
// The token/pump path needs the caller's queue in THIS runtime (its reply routes // The token/pump path needs the caller's queue in THIS runtime (its reply routes
// by the caller's id, resolved in calog). A thread that is foreign to calog -- a // by the caller's id, resolved in calog). A thread that is foreign to calog -- a
// non-calog thread, or one currently hosting a different runtime -- takes the reply // non-calog thread, or one currently hosting a different runtime -- takes the reply
@ -846,6 +913,27 @@ static int32_t contextSendBlocking(CalogT *calog, uint64_t targetId, MessageT *m
} }
// The dispatch shared by every loop that drains a queue (serveLoop, pumpUntil,
// hostDispatch): the three message kinds whose handling never varies by loop. Returns
// whether message->kind was one of them (and so was consumed); the caller still owns
// its own loop-specific kinds (REPLY, ERROR) and the default for anything left over.
static bool dispatchCommon(CalogContextT *context, MessageT *message) {
switch (message->kind) {
case messageCallE:
contextDispatchCall(context, message);
return true;
case messageEvalE:
contextDispatchEval(context, message);
return true;
case messageReleaseE:
contextDispatchRelease(message);
return true;
default:
return false;
}
}
static void enqueueRaw(CalogContextT *context, MessageT *message) { static void enqueueRaw(CalogContextT *context, MessageT *message) {
pthread_mutex_lock(&context->queueMutex); pthread_mutex_lock(&context->queueMutex);
message->next = NULL; message->next = NULL;
@ -861,19 +949,12 @@ static void enqueueRaw(CalogContextT *context, MessageT *message) {
static void hostDispatch(CalogT *calog, MessageT *message) { static void hostDispatch(CalogT *calog, MessageT *message) {
switch (message->kind) { if (message->kind == messageErrorE) {
case messageCallE:
contextDispatchCall(calog->hostContext, message);
break;
case messageReleaseE:
contextDispatchRelease(message);
break;
case messageErrorE:
contextDispatchError(calog, message); contextDispatchError(calog, message);
break; return;
default: }
if (!dispatchCommon(calog->hostContext, message)) {
messageFree(message); messageFree(message);
break;
} }
} }
@ -1008,45 +1089,34 @@ static int32_t pumpUntil(CalogContextT *context, uint64_t token, int32_t *outSta
context->stash = message; context->stash = message;
continue; continue;
} }
if (message->kind == messageCallE) {
contextDispatchCall(context, message);
continue;
}
if (message->kind == messageEvalE) {
contextDispatchEval(context, message);
continue;
}
if (message->kind == messageReleaseE) {
contextDispatchRelease(message);
continue;
}
if (message->kind == messageErrorE) { if (message->kind == messageErrorE) {
contextDispatchError(context->broker, message); contextDispatchError(context->broker, message);
continue; continue;
} }
context->shuttingDown = true; if (!dispatchCommon(context, message)) {
free(message); messageFree(message);
}
} }
} }
// Push a freed slot index onto the recycle freelist. Called under calog->ctxMutex. On // Push a freed slot index onto the recycle freelist. Called under calog->ctxMutex. On
// OOM the index is simply not recycled (a leaked slot, never a correctness fault). // OOM the index is simply not recycled (a leaked slot, never a correctness fault) --
static int32_t registryFreePush(CalogT *calog, int64_t index) { // void because neither caller has anything to act on.
static void registryFreePush(CalogT *calog, int64_t index) {
if (calog->ctxFreeCount == calog->ctxFreeCap) { if (calog->ctxFreeCount == calog->ctxFreeCap) {
int64_t newCap; int64_t newCap;
int64_t *grown; int64_t *grown;
newCap = (calog->ctxFreeCap == 0) ? CONTEXT_INITIAL_CONTEXTS : calog->ctxFreeCap * CALOG_GROWTH_FACTOR; newCap = (calog->ctxFreeCap == 0) ? CONTEXT_INITIAL_CONTEXTS : calog->ctxFreeCap * CALOG_GROWTH_FACTOR;
grown = (int64_t *)realloc(calog->ctxFree, (size_t)newCap * sizeof(int64_t)); grown = (int64_t *)realloc(calog->ctxFree, (size_t)newCap * sizeof(int64_t));
if (grown == NULL) { if (grown == NULL) {
return calogErrOomE; return;
} }
calog->ctxFree = grown; calog->ctxFree = grown;
calog->ctxFreeCap = newCap; calog->ctxFreeCap = newCap;
} }
calog->ctxFree[calog->ctxFreeCount] = index; calog->ctxFree[calog->ctxFreeCount] = index;
calog->ctxFreeCount++; calog->ctxFreeCount++;
return calogOkE;
} }
@ -1078,29 +1148,21 @@ static CalogContextT *registryResolveLocked(CalogT *calog, uint64_t id) {
static void serveLoop(CalogContextT *context) { static void serveLoop(CalogContextT *context) {
for (;;) { for (;;) {
MessageT *message; MessageT *message;
// messageDequeue returns NULL once the queue is empty and contextRequestShutdown
// has flipped shuttingDown -- there is no SHUTDOWN message kind to look for.
message = messageDequeue(context); message = messageDequeue(context);
if (message == NULL) { if (message == NULL) {
return; return;
} }
if (message->kind == messageShutdownE) { if (!dispatchCommon(context, message)) {
context->shuttingDown = true;
free(message);
continue;
}
if (message->kind == messageCallE) {
contextDispatchCall(context, message);
continue;
}
if (message->kind == messageEvalE) {
contextDispatchEval(context, message);
continue;
}
if (message->kind == messageReleaseE) {
contextDispatchRelease(message);
continue;
}
messageFree(message); messageFree(message);
} }
if (atomic_load(&context->retireRequested)) {
// A native in the eval that just finished called calogCurrentRetire (taskExit): the
// eval ran to completion normally; now stop so this context's thread can exit.
contextRequestShutdown(context);
}
}
} }
@ -1143,5 +1205,8 @@ static void *threadMain(void *arg) {
context->engine->destroyInterpreter(context->interp); context->engine->destroyInterpreter(context->interp);
} }
currentContext = NULL; currentContext = NULL;
// Last thing the thread does: publish that it has exited, so a reaper (calogTaskReap on
// the host) can pthread_join it without blocking and then free the context.
atomic_store(&context->finished, true);
return NULL; return NULL;
} }

View file

@ -24,14 +24,11 @@
#include "quickjs.h" #include "quickjs.h"
#include <math.h>
#include <stdint.h> #include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define JS_ERR_MSG_CAP 256
struct CalogJsT { struct CalogJsT {
JSRuntime *rt; JSRuntime *rt;
JSContext *ctx; JSContext *ctx;
@ -348,7 +345,7 @@ static void jsForeignFinalize(JSRuntime *rt, JSValueConst val) {
static JSValue jsFromValue(JSContext *ctx, const CalogValueT *value, int32_t depth) { static JSValue jsFromValue(JSContext *ctx, const CalogValueT *value, int32_t depth) {
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return JS_ThrowRangeError(ctx, "value nesting too deep"); return JS_ThrowRangeError(ctx, "value nesting too deep");
} }
switch (value->type) { switch (value->type) {
@ -371,7 +368,7 @@ static JSValue jsFromValue(JSContext *ctx, const CalogValueT *value, int32_t dep
aggregate = value->as.agg; aggregate = value->as.agg;
// A JS array holds only a sequence; anything keyed (or an explicit map) // A JS array holds only a sequence; anything keyed (or an explicit map)
// becomes an object. // becomes an object.
asObject = aggregate->pairCount > 0 || aggregate->kind == calogMapE; asObject = calogAggIsKeyed(aggregate);
container = asObject ? JS_NewObject(ctx) : JS_NewArray(ctx); container = asObject ? JS_NewObject(ctx) : JS_NewArray(ctx);
if (JS_IsException(container)) { if (JS_IsException(container)) {
return container; return container;
@ -488,6 +485,10 @@ static int jsResolveOwnProperty(JSContext *ctx, JSPropertyDescriptor *desc, JSVa
desc->value = jsFromValue(ctx, &result, 0); desc->value = jsFromValue(ctx, &result, 0);
desc->getter = JS_UNDEFINED; desc->getter = JS_UNDEFINED;
desc->setter = JS_UNDEFINED; desc->setter = JS_UNDEFINED;
if (JS_IsException(desc->value)) {
calogValueFree(&result);
return -1;
}
} }
calogValueFree(&result); calogValueFree(&result);
return 1; return 1;
@ -496,7 +497,7 @@ static int jsResolveOwnProperty(JSContext *ctx, JSPropertyDescriptor *desc, JSVa
static int32_t jsToValue(JSContext *ctx, JSValueConst val, CalogValueT *out, int32_t depth) { static int32_t jsToValue(JSContext *ctx, JSValueConst val, CalogValueT *out, int32_t depth) {
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
if (JS_IsNull(val) || JS_IsUndefined(val)) { if (JS_IsNull(val) || JS_IsUndefined(val)) {
@ -521,11 +522,7 @@ static int32_t jsToValue(JSContext *ctx, JSValueConst val, CalogValueT *out, int
return calogErrTypeE; return calogErrTypeE;
} }
// An integral value in int64 range round-trips as an integer; otherwise a real. // An integral value in int64 range round-trips as an integer; otherwise a real.
if (isfinite(number) && number == floor(number) && number >= -9223372036854775808.0 && number < 9223372036854775808.0) { calogValueFromDouble(out, number);
calogValueInt(out, (int64_t)number);
} else {
calogValueReal(out, number);
}
return calogOkE; return calogOkE;
} }
if (JS_IsString(val)) { if (JS_IsString(val)) {
@ -608,6 +605,7 @@ static int32_t jsToValue(JSContext *ctx, JSValueConst val, CalogValueT *out, int
CalogValueT keyValue; CalogValueT keyValue;
CalogValueT valValue; CalogValueT valValue;
const char *keyStr; const char *keyStr;
size_t keyLen;
JSValue v; JSValue v;
v = JS_GetProperty(ctx, val, tab[index].atom); v = JS_GetProperty(ctx, val, tab[index].atom);
if (JS_IsException(v)) { if (JS_IsException(v)) {
@ -620,13 +618,13 @@ static int32_t jsToValue(JSContext *ctx, JSValueConst val, CalogValueT *out, int
if (status != calogOkE) { if (status != calogOkE) {
break; break;
} }
keyStr = JS_AtomToCString(ctx, tab[index].atom); keyStr = JS_AtomToCStringLen(ctx, &keyLen, tab[index].atom);
if (keyStr == NULL) { if (keyStr == NULL) {
calogValueFree(&valValue); calogValueFree(&valValue);
status = calogErrOomE; status = calogErrOomE;
break; break;
} }
status = calogValueString(&keyValue, keyStr, (int64_t)strlen(keyStr)); status = calogValueString(&keyValue, keyStr, (int64_t)keyLen);
JS_FreeCString(ctx, keyStr); JS_FreeCString(ctx, keyStr);
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(&valValue); calogValueFree(&valValue);
@ -659,7 +657,6 @@ static JSValue jsTrampoline(JSContext *ctx, JSValueConst this_val, int argc, JSV
JSValue jsResult; JSValue jsResult;
int32_t index; int32_t index;
int32_t status; int32_t status;
char message[JS_ERR_MSG_CAP];
(void)this_val; (void)this_val;
(void)magic; (void)magic;
@ -695,9 +692,11 @@ static JSValue jsTrampoline(JSContext *ctx, JSValueConst this_val, int argc, JSV
free(args); free(args);
JS_FreeCString(ctx, name); JS_FreeCString(ctx, name);
if (status != calogOkE) { if (status != calogOkE) {
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : "native call failed"); // JS_ThrowTypeError copies its formatted message immediately, so it is safe to
// throw first and free the result after (no need to buffer the message first).
jsResult = JS_ThrowTypeError(ctx, "%s", (result.type == calogStringE) ? result.as.s.bytes : "native call failed");
calogValueFree(&result); calogValueFree(&result);
return JS_ThrowTypeError(ctx, "%s", message); return jsResult;
} }
jsResult = jsFromValue(ctx, &result, 0); jsResult = jsFromValue(ctx, &result, 0);
calogValueFree(&result); calogValueFree(&result);

View file

@ -10,6 +10,7 @@
#define _POSIX_C_SOURCE 200809L #define _POSIX_C_SOURCE 200809L
#include "luaAdapter.h" #include "luaAdapter.h"
#include "adapterBinding.h"
#include <lua.h> #include <lua.h>
#include <lauxlib.h> #include <lauxlib.h>
@ -18,20 +19,9 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define LUA_BINDING_INITIAL 8
#define LUA_FUNCTION_META "calogFunctionValue" #define LUA_FUNCTION_META "calogFunctionValue"
#define LUA_MARSHAL_SLOTS 8 #define LUA_MARSHAL_SLOTS 8
// An exposed native is bound by broker + name, not by a captured fn pointer, so
// the trampoline dispatches through calogCall and honors the actor route hook:
// a native owned by another context is marshalled to its thread instead of being
// run (wrongly) on this state's thread. With no hook installed, calogCall runs
// it inline -- identical to the single-threaded path.
typedef struct BindingT {
CalogT *broker;
char *name;
} BindingT;
typedef struct LuaExportT { typedef struct LuaExportT {
CalogLuaT *context; CalogLuaT *context;
int32_t ref; int32_t ref;
@ -46,16 +36,78 @@ struct CalogLuaT {
int32_t bindingCap; int32_t bindingCap;
}; };
// Shared shape for the two things a marshalled call can dispatch to: a CalogFnT
// (calogFnInvoke) or a broker binding (calogCall). luaCallDispatch runs the
// marshal/dispatch/cleanup pipeline once and calls back through this.
typedef int32_t (*LuaDispatchFnT)(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
static int luaCallDispatch(lua_State *L, int argBase, LuaDispatchFnT dispatch, void *userData, const char *noun, const char *failMessage);
static int32_t luaCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t luaCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void luaCallableRelease(CalogFnT *callable); static void luaCallableRelease(CalogFnT *callable);
static CalogLuaT *luaContextOf(lua_State *L); static CalogLuaT *luaContextOf(lua_State *L);
static int32_t luaExportAt(CalogLuaT *context, int idx, CalogFnT **out); static int32_t luaExportAt(CalogLuaT *context, int idx, CalogFnT **out);
static int luaFunctionCall(lua_State *L); static int luaFunctionCall(lua_State *L);
static int32_t luaFunctionCallDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
static int luaFunctionGc(lua_State *L); static int luaFunctionGc(lua_State *L);
static int luaGlobalResolve(lua_State *L); static int luaGlobalResolve(lua_State *L);
static int32_t luaPushValueDepth(lua_State *L, const CalogValueT *value, int32_t depth); static int32_t luaPushValueDepth(lua_State *L, const CalogValueT *value, int32_t depth);
static int32_t luaToValueDepth(lua_State *L, int idx, CalogValueT *out, int32_t depth); static int32_t luaToValueDepth(lua_State *L, int idx, CalogValueT *out, int32_t depth);
static int luaTrampoline(lua_State *L); static int luaTrampoline(lua_State *L);
static int32_t luaTrampolineDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
// Shared marshal/dispatch/cleanup pipeline for both call sites that cross into Lua's
// stack-based calling convention: a Lua function-value userdata being invoked, and a
// broker native reached through a light-userdata trampoline upvalue. argBase is the
// stack index of the first argument (differs between the two call sites); dispatch and
// userData supply the actual call; noun and failMessage name the caller in diagnostics.
static int luaCallDispatch(lua_State *L, int argBase, LuaDispatchFnT dispatch, void *userData, const char *noun, const char *failMessage) {
CalogValueT *args;
CalogValueT result;
int32_t argCount;
int32_t index;
int32_t status;
argCount = (int32_t)(lua_gettop(L) - argBase + 1);
args = NULL;
if (argCount > 0) {
args = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (args == NULL) {
return luaL_error(L, "out of memory marshalling %s args", noun);
}
}
for (index = 0; index < argCount; index++) {
status = luaToValueDepth(L, argBase + index, &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
return luaL_error(L, "failed to marshal %s argument %d", noun, index + 1);
}
}
status = dispatch(userData, args, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
size_t len;
const char *message;
message = (result.type == calogStringE) ? result.as.s.bytes : failMessage;
len = (result.type == calogStringE) ? (size_t)result.as.s.length : strlen(message);
lua_pushlstring(L, message, len);
calogValueFree(&result);
return lua_error(L);
}
status = luaPushValueDepth(L, &result, 0);
calogValueFree(&result);
if (status != calogOkE) {
return luaL_error(L, "failed to marshal %s result", noun);
}
return 1;
}
static int32_t luaCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t luaCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
@ -148,19 +200,13 @@ int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId) {
void calogLuaDestroy(CalogLuaT *context) { void calogLuaDestroy(CalogLuaT *context) {
int32_t index;
if (context == NULL) { if (context == NULL) {
return; return;
} }
if (context->L != NULL) { if (context->L != NULL) {
lua_close(context->L); lua_close(context->L);
} }
for (index = 0; index < context->bindingCount; index++) { adapterBindingDestroyAll(context->bindings, context->bindingCount);
free(context->bindings[index]->name);
free(context->bindings[index]);
}
free(context->bindings);
free(context); free(context);
} }
@ -195,12 +241,17 @@ static int32_t luaExportAt(CalogLuaT *context, int idx, CalogFnT **out) {
L = context->L; L = context->L;
*out = NULL; *out = NULL;
export = (LuaExportT *)malloc(sizeof(*export)); // Take the ref before the malloc: luaL_ref can longjmp (LUA_ERRMEM growing the
if (export == NULL) { // registry table), and luaL_unref cannot allocate, so this ordering guarantees
return calogErrOomE; // nothing is stranded if the ref itself fails, and export is freed cleanly if the
} // malloc after it fails instead.
lua_pushvalue(L, idx); lua_pushvalue(L, idx);
ref = luaL_ref(L, LUA_REGISTRYINDEX); ref = luaL_ref(L, LUA_REGISTRYINDEX);
export = (LuaExportT *)malloc(sizeof(*export));
if (export == NULL) {
luaL_unref(L, LUA_REGISTRYINDEX, ref);
return calogErrOomE;
}
export->context = context; export->context = context;
export->ref = ref; export->ref = ref;
status = calogFnCreate(out, context->broker, luaCallableInvoke, export, luaCallableRelease, context->ctxId); status = calogFnCreate(out, context->broker, luaCallableInvoke, export, luaCallableRelease, context->ctxId);
@ -214,40 +265,15 @@ static int32_t luaExportAt(CalogLuaT *context, int idx, CalogFnT **out) {
int32_t calogLuaExpose(CalogLuaT *context, const char *name) { int32_t calogLuaExpose(CalogLuaT *context, const char *name) {
CalogEntryT *entry;
BindingT *binding; BindingT *binding;
lua_State *L; lua_State *L;
int32_t status;
L = context->L; L = context->L;
entry = calogLookup(context->broker, name); status = adapterBindingExpose(context->broker, name, &context->bindings, &context->bindingCount, &context->bindingCap, &binding);
if (entry == NULL) { if (status != calogOkE) {
return calogErrNotFoundE; return status;
} }
binding = (BindingT *)malloc(sizeof(*binding));
if (binding == NULL) {
return calogErrOomE;
}
binding->broker = context->broker;
binding->name = strdup(name);
if (binding->name == NULL) {
free(binding);
return calogErrOomE;
}
if (context->bindingCount == context->bindingCap) {
int32_t newCap;
BindingT **resized;
newCap = (context->bindingCap == 0) ? LUA_BINDING_INITIAL : context->bindingCap * CALOG_GROWTH_FACTOR;
resized = (BindingT **)realloc(context->bindings, (size_t)newCap * sizeof(BindingT *));
if (resized == NULL) {
free(binding->name);
free(binding);
return calogErrOomE;
}
context->bindings = resized;
context->bindingCap = newCap;
}
context->bindings[context->bindingCount] = binding;
context->bindingCount++;
lua_pushlightuserdata(L, binding); lua_pushlightuserdata(L, binding);
lua_pushcclosure(L, luaTrampoline, 1); lua_pushcclosure(L, luaTrampoline, 1);
@ -258,52 +284,14 @@ int32_t calogLuaExpose(CalogLuaT *context, const char *name) {
static int luaFunctionCall(lua_State *L) { static int luaFunctionCall(lua_State *L) {
CalogFnT *callable; CalogFnT *callable;
CalogValueT *args;
CalogValueT result;
int32_t argCount;
int32_t index;
int32_t status;
callable = *(CalogFnT **)lua_touserdata(L, 1); callable = *(CalogFnT **)lua_touserdata(L, 1);
argCount = (int32_t)(lua_gettop(L) - 1); return luaCallDispatch(L, 2, luaFunctionCallDispatch, callable, "function-value", "function value failed");
args = NULL; }
if (argCount > 0) {
args = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (args == NULL) { static int32_t luaFunctionCallDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result) {
return luaL_error(L, "out of memory marshalling function-value args"); return calogFnInvoke((CalogFnT *)userData, args, argCount, result);
}
}
for (index = 0; index < argCount; index++) {
status = luaToValueDepth(L, index + 2, &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
return luaL_error(L, "failed to marshal function-value argument %d", index + 1);
}
}
status = calogFnInvoke(callable, args, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
size_t len;
const char *message;
message = (result.type == calogStringE) ? result.as.s.bytes : "function value failed";
len = (result.type == calogStringE) ? (size_t)result.as.s.length : strlen(message);
lua_pushlstring(L, message, len);
calogValueFree(&result);
return lua_error(L);
}
status = luaPushValueDepth(L, &result, 0);
calogValueFree(&result);
if (status != calogOkE) {
return luaL_error(L, "failed to marshal function-value result");
}
return 1;
} }
@ -357,7 +345,7 @@ static int32_t luaPushValueDepth(lua_State *L, const CalogValueT *value, int32_t
int64_t index; int64_t index;
int tbl; int tbl;
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
if (!lua_checkstack(L, LUA_MARSHAL_SLOTS)) { if (!lua_checkstack(L, LUA_MARSHAL_SLOTS)) {
@ -446,7 +434,7 @@ static int32_t luaToValueDepth(lua_State *L, int idx, CalogValueT *out, int32_t
int absIdx; int absIdx;
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
absIdx = lua_absindex(L, idx); absIdx = lua_absindex(L, idx);
@ -541,12 +529,10 @@ static int32_t luaToValueDepth(lua_State *L, int idx, CalogValueT *out, int32_t
} }
lua_pop(L, 1); lua_pop(L, 1);
} }
if (aggregate->pairCount == 0) { // Created above as calogListE; only reclassify if keyed entries were
aggregate->kind = calogListE; // actually collected.
} else if (aggregate->arrayCount == 0) { if (aggregate->pairCount > 0) {
aggregate->kind = calogMapE; aggregate->kind = (aggregate->arrayCount == 0) ? calogMapE : calogBothE;
} else {
aggregate->kind = calogBothE;
} }
calogValueAgg(out, aggregate); calogValueAgg(out, aggregate);
return calogOkE; return calogOkE;
@ -571,50 +557,15 @@ static int32_t luaToValueDepth(lua_State *L, int idx, CalogValueT *out, int32_t
static int luaTrampoline(lua_State *L) { static int luaTrampoline(lua_State *L) {
BindingT *binding; BindingT *binding;
CalogValueT *args;
CalogValueT result;
int32_t argCount;
int32_t index;
int32_t status;
binding = (BindingT *)lua_touserdata(L, lua_upvalueindex(1)); binding = (BindingT *)lua_touserdata(L, lua_upvalueindex(1));
argCount = (int32_t)lua_gettop(L); return luaCallDispatch(L, 1, luaTrampolineDispatch, binding, "native", "native call failed");
args = NULL; }
if (argCount > 0) {
args = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (args == NULL) { static int32_t luaTrampolineDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result) {
return luaL_error(L, "out of memory marshalling native args"); BindingT *binding;
}
} binding = (BindingT *)userData;
for (index = 0; index < argCount; index++) { return calogCall(binding->broker, binding->name, args, argCount, result);
status = luaToValueDepth(L, index + 1, &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
return luaL_error(L, "failed to marshal native argument %d", index + 1);
}
}
status = calogCall(binding->broker, binding->name, args, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
size_t len;
const char *message;
message = (result.type == calogStringE) ? result.as.s.bytes : "native call failed";
len = (result.type == calogStringE) ? (size_t)result.as.s.length : strlen(message);
lua_pushlstring(L, message, len);
calogValueFree(&result);
return lua_error(L);
}
status = luaPushValueDepth(L, &result, 0);
calogValueFree(&result);
if (status != calogOkE) {
return luaL_error(L, "failed to marshal native result");
}
return 1;
} }

View file

@ -30,6 +30,15 @@ typedef struct MyBasicRoutineT {
mb_value_t routine; mb_value_t routine;
} MyBasicRoutineT; } MyBasicRoutineT;
// A foreign (host-side) callable handed INTO a my-basic script -- wrapped in a refcounted
// usertype-ref whose dtor releases the callable when the last reference drops. A script invokes
// it with calogInvoke(fn, ...args). The tag lets calogInvoke recognise our own refs.
#define MB_FOREIGN_TAG 0x43414c47u
typedef struct MbForeignFnT {
uint32_t tag;
CalogFnT *callable;
} MbForeignFnT;
struct CalogMyBasicT { struct CalogMyBasicT {
struct mb_interpreter_t *bas; struct mb_interpreter_t *bas;
CalogT *broker; CalogT *broker;
@ -39,6 +48,10 @@ struct CalogMyBasicT {
int32_t bankCount; int32_t bankCount;
}; };
// Serializes my-basic context lifecycle (mb_init/mb_dispose + mbContextCount); the engine
// holds it, via calogMyBasicLifecycleLock, around create/destroy. NOT held during execution,
// so scripts run concurrently. Lock and counter live together so the invariant is local.
static pthread_mutex_t mbLifecycleLock = PTHREAD_MUTEX_INITIALIZER;
static int32_t mbContextCount = 0; static int32_t mbContextCount = 0;
static int32_t mbAggregateToCollDepth(CalogMyBasicT *context, void **l, const CalogAggT *aggregate, mb_value_t *out, int32_t depth); static int32_t mbAggregateToCollDepth(CalogMyBasicT *context, void **l, const CalogAggT *aggregate, mb_value_t *out, int32_t depth);
@ -46,10 +59,16 @@ static int32_t mbCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT
static void mbCallableRelease(CalogFnT *callable); static void mbCallableRelease(CalogFnT *callable);
static int32_t mbDictToAggregate(CalogMyBasicT *context, void **l, mb_value_t coll, CalogValueT *out, int32_t depth); static int32_t mbDictToAggregate(CalogMyBasicT *context, void **l, mb_value_t coll, CalogValueT *out, int32_t depth);
static int mbDispatch(int32_t slot, struct mb_interpreter_t *s, void **l); static int mbDispatch(int32_t slot, struct mb_interpreter_t *s, void **l);
static int mbDynamicFuncHandler(struct mb_interpreter_t *s, void **l, const char *name);
static void *mbForeignFnClone(struct mb_interpreter_t *s, void *p);
static void mbForeignFnDtor(struct mb_interpreter_t *s, void *p);
static int mbForeignInvokeNative(struct mb_interpreter_t *s, void **l);
static int32_t mbFromValueDepth(CalogMyBasicT *context, void **l, const CalogValueT *value, mb_value_t *out, int32_t depth); static int32_t mbFromValueDepth(CalogMyBasicT *context, void **l, const CalogValueT *value, mb_value_t *out, int32_t depth);
static int mbInvokeWithOpenBracket(CalogMyBasicT *context, struct mb_interpreter_t *s, void **l, CalogFnT *callable);
static int32_t mbListToAggregate(CalogMyBasicT *context, void **l, mb_value_t coll, CalogValueT *out, int32_t depth); static int32_t mbListToAggregate(CalogMyBasicT *context, void **l, mb_value_t coll, CalogValueT *out, int32_t depth);
static int mbPrinter(struct mb_interpreter_t *s, const char *format, ...); static int mbPrinter(struct mb_interpreter_t *s, const char *format, ...);
static void mbReleaseSetValue(CalogMyBasicT *context, mb_value_t value); static void mbReleaseSetValue(CalogMyBasicT *context, mb_value_t value);
static void mbReportError(CalogMyBasicT *context, const char *stage);
static void mbRetainBeforeSet(CalogMyBasicT *context, void **l, mb_value_t value); static void mbRetainBeforeSet(CalogMyBasicT *context, void **l, mb_value_t value);
static int32_t mbToValueDepth(CalogMyBasicT *context, void **l, const mb_value_t *value, CalogValueT *out, int32_t depth); static int32_t mbToValueDepth(CalogMyBasicT *context, void **l, const mb_value_t *value, CalogValueT *out, int32_t depth);
static int32_t mbWrapRoutine(CalogMyBasicT *context, mb_value_t routine, CalogFnT **out); static int32_t mbWrapRoutine(CalogMyBasicT *context, mb_value_t routine, CalogFnT **out);
@ -316,7 +335,11 @@ MB_TRAMPOLINE(254)
MB_TRAMPOLINE(255) MB_TRAMPOLINE(255)
#undef MB_TRAMPOLINE #undef MB_TRAMPOLINE
static const mb_func_t mbTrampTable[MB_BANK_SIZE] = { // Unsized on purpose: the _Static_assert below ties this table's element count
// back to MB_BANK_SIZE, so the hand-listed MB_TRAMPOLINE bank above, this table,
// and the compile-time constant cannot silently drift apart (raising MB_BANK_SIZE
// without adding trampolines used to compile cleanly with NULL-filled tail slots).
static const mb_func_t mbTrampTable[] = {
mbTramp0, mbTramp1, mbTramp2, mbTramp3, mbTramp4, mbTramp5, mbTramp6, mbTramp7, mbTramp0, mbTramp1, mbTramp2, mbTramp3, mbTramp4, mbTramp5, mbTramp6, mbTramp7,
mbTramp8, mbTramp9, mbTramp10, mbTramp11, mbTramp12, mbTramp13, mbTramp14, mbTramp15, mbTramp8, mbTramp9, mbTramp10, mbTramp11, mbTramp12, mbTramp13, mbTramp14, mbTramp15,
mbTramp16, mbTramp17, mbTramp18, mbTramp19, mbTramp20, mbTramp21, mbTramp22, mbTramp23, mbTramp16, mbTramp17, mbTramp18, mbTramp19, mbTramp20, mbTramp21, mbTramp22, mbTramp23,
@ -351,19 +374,19 @@ static const mb_func_t mbTrampTable[MB_BANK_SIZE] = {
mbTramp248, mbTramp249, mbTramp250, mbTramp251, mbTramp252, mbTramp253, mbTramp254, mbTramp255 mbTramp248, mbTramp249, mbTramp250, mbTramp251, mbTramp252, mbTramp253, mbTramp254, mbTramp255
}; };
_Static_assert(sizeof(mbTrampTable) / sizeof(mbTrampTable[0]) == MB_BANK_SIZE, "mbTrampTable must have exactly MB_BANK_SIZE entries");
static int32_t mbAggregateToCollDepth(CalogMyBasicT *context, void **l, const CalogAggT *aggregate, mb_value_t *out, int32_t depth) { static int32_t mbAggregateToCollDepth(CalogMyBasicT *context, void **l, const CalogAggT *aggregate, mb_value_t *out, int32_t depth) {
mb_value_t coll; mb_value_t coll;
int32_t status; int32_t status;
int64_t index; int64_t index;
bool asList;
mb_make_nil(coll); mb_make_nil(coll);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
asList = (aggregate->pairCount == 0 && aggregate->kind != calogMapE); coll.type = calogAggIsKeyed(aggregate) ? MB_DT_DICT : MB_DT_LIST;
coll.type = asList ? MB_DT_LIST : MB_DT_DICT;
if (mb_init_coll(context->bas, l, &coll) != MB_FUNC_OK) { if (mb_init_coll(context->bas, l, &coll) != MB_FUNC_OK) {
return calogErrOomE; return calogErrOomE;
} }
@ -401,6 +424,7 @@ static int32_t mbAggregateToCollDepth(CalogMyBasicT *context, void **l, const Ca
mb_dispose_value(context->bas, coll); mb_dispose_value(context->bas, coll);
return status; return status;
} }
mbRetainBeforeSet(context, l, key);
mbRetainBeforeSet(context, l, element); mbRetainBeforeSet(context, l, element);
mb_set_coll(context->bas, l, coll, key, element); mb_set_coll(context->bas, l, coll, key, element);
mbReleaseSetValue(context, key); mbReleaseSetValue(context, key);
@ -574,7 +598,10 @@ static int mbDispatch(int32_t slot, struct mb_interpreter_t *s, void **l) {
newCap = (capacity == 0) ? MB_INITIAL_ARGS : capacity * CALOG_GROWTH_FACTOR; newCap = (capacity == 0) ? MB_INITIAL_ARGS : capacity * CALOG_GROWTH_FACTOR;
grown = (CalogValueT *)realloc(args, (size_t)newCap * sizeof(CalogValueT)); grown = (CalogValueT *)realloc(args, (size_t)newCap * sizeof(CalogValueT));
if (grown == NULL) { if (grown == NULL) {
if (popped.type == MB_DT_LIST || popped.type == MB_DT_DICT) { // popped was never handed to the marshaller, so any collection or
// routine scope it owns is ours to free here (same rule as the
// refused-argument path below), or it leaks.
if (popped.type == MB_DT_LIST || popped.type == MB_DT_DICT || popped.type == MB_DT_ROUTINE) {
mb_dispose_value(context->bas, popped); mb_dispose_value(context->bas, popped);
} }
code = mb_raise_error(s, l, SE_RN_OUT_OF_MEMORY, MB_FUNC_ERR); code = mb_raise_error(s, l, SE_RN_OUT_OF_MEMORY, MB_FUNC_ERR);
@ -619,7 +646,6 @@ static int mbDispatch(int32_t slot, struct mb_interpreter_t *s, void **l) {
calogValueFree(&args[index]); calogValueFree(&args[index]);
} }
free(args); free(args);
args = NULL;
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(&result); calogValueFree(&result);
@ -647,6 +673,119 @@ cleanupArgs:
} }
// [calog fork hook] Resolve a bare name called like a function -- exportedFn(args) -- against the
// export registry (case-insensitively, since my-basic uppercases identifiers at parse time) and,
// if it is an export, consume the (args) and invoke it. Returns MB_FUNC_IGNORE when the name is
// not an export, so the interpreter raises its usual "invalid identifier usage" error.
static int mbDynamicFuncHandler(struct mb_interpreter_t *s, void **l, const char *name) {
CalogMyBasicT *context;
void *userData;
CalogValueT nameArg;
CalogValueT resolved;
int code;
userData = NULL;
mb_get_userdata(s, &userData);
context = (CalogMyBasicT *)userData;
// Resolve without touching the pending argument list, so a miss leaves the stack untouched.
if (calogValueString(&nameArg, name, (int64_t)strlen(name)) != calogOkE) {
return MB_FUNC_IGNORE;
}
calogValueNil(&resolved);
code = calogCall(context->broker, "__calogExportResolveFold", &nameArg, 1, &resolved);
calogValueFree(&nameArg);
if (code != calogOkE || resolved.type != calogFnE) {
calogValueFree(&resolved);
return MB_FUNC_IGNORE;
}
code = mb_attempt_open_bracket(s, l);
if (code != MB_FUNC_OK) {
calogValueFree(&resolved);
return code;
}
code = mbInvokeWithOpenBracket(context, s, l, resolved.as.fn);
calogValueFree(&resolved);
return code;
}
static void *mbForeignFnClone(struct mb_interpreter_t *s, void *p) {
MbForeignFnT *src;
MbForeignFnT *dup;
(void)s;
src = (MbForeignFnT *)p;
dup = (MbForeignFnT *)malloc(sizeof(*dup));
if (dup == NULL) {
return NULL;
}
dup->tag = src->tag;
dup->callable = src->callable;
calogFnRetain(src->callable);
return dup;
}
static void mbForeignFnDtor(struct mb_interpreter_t *s, void *p) {
MbForeignFnT *holder;
(void)s;
holder = (MbForeignFnT *)p;
if (holder != NULL) {
calogFnRelease(holder->callable);
free(holder);
}
}
// calogInvoke(fn, ...args) -> result: invoke a foreign (host-side) callable handed into this
// script as a value (see mbFromValueDepth's calogFnE case). fn is the usertype-ref wrapper; the
// rest are marshalled to the callable and its result marshalled back. Arg handling mirrors
// mbDispatch.
static int mbForeignInvokeNative(struct mb_interpreter_t *s, void **l) {
CalogMyBasicT *context;
MbForeignFnT *holder;
void *userData;
void *refData;
mb_value_t fnValue;
int code;
userData = NULL;
mb_get_userdata(s, &userData);
context = (CalogMyBasicT *)userData;
code = mb_attempt_open_bracket(s, l);
if (code != MB_FUNC_OK) {
return code;
}
// First argument: the foreign callable, a usertype-ref this adapter produced.
if (!mb_has_arg(s, l)) {
return mb_raise_error(s, l, MB_NATIVE_ERROR, MB_FUNC_ERR);
}
mb_make_nil(fnValue);
code = mb_pop_value(s, l, &fnValue);
if (code != MB_FUNC_OK) {
return code;
}
refData = NULL;
if (fnValue.type != MB_DT_USERTYPE_REF || mb_get_ref_value(s, l, fnValue, &refData) != MB_FUNC_OK) {
mb_dispose_value(context->bas, fnValue);
return mb_raise_error(s, l, MB_NATIVE_ERROR, MB_FUNC_ERR);
}
holder = (MbForeignFnT *)refData;
if (holder == NULL || holder->tag != MB_FOREIGN_TAG) {
mb_dispose_value(context->bas, fnValue);
return mb_raise_error(s, l, MB_NATIVE_ERROR, MB_FUNC_ERR);
}
// The '(' is consumed and the leading fn argument popped; the shared helper handles the
// rest. Keep fnValue alive across the invoke (it holds the callable), then release it.
code = mbInvokeWithOpenBracket(context, s, l, holder->callable);
mb_dispose_value(context->bas, fnValue);
return code;
}
static int32_t mbFromValueDepth(CalogMyBasicT *context, void **l, const CalogValueT *value, mb_value_t *out, int32_t depth) { static int32_t mbFromValueDepth(CalogMyBasicT *context, void **l, const CalogValueT *value, mb_value_t *out, int32_t depth) {
char *duplicate; char *duplicate;
@ -675,13 +814,119 @@ static int32_t mbFromValueDepth(CalogMyBasicT *context, void **l, const CalogVal
return calogOkE; return calogOkE;
case calogAggE: case calogAggE:
return mbAggregateToCollDepth(context, l, value->as.agg, out, depth); return mbAggregateToCollDepth(context, l, value->as.agg, out, depth);
case calogFnE: case calogFnE: {
return calogErrUnsupportedE; MbForeignFnT *holder;
holder = (MbForeignFnT *)malloc(sizeof(*holder));
if (holder == NULL) {
return calogErrOomE;
}
holder->tag = MB_FOREIGN_TAG;
holder->callable = value->as.fn;
calogFnRetain(value->as.fn);
// Wrap the host callable in a refcounted usertype-ref; mbForeignFnDtor releases the
// retain when the last reference drops. A script calls it with calogInvoke(fn, ...).
if (mb_make_ref_value(context->bas, holder, out, mbForeignFnDtor, mbForeignFnClone, NULL, NULL, NULL) != MB_FUNC_OK) {
calogFnRelease(value->as.fn);
free(holder);
return calogErrOomE;
}
return calogOkE;
}
} }
return calogErrTypeE; return calogErrTypeE;
} }
// Shared by mbForeignInvokeNative and mbDynamicFuncHandler: with '(' already consumed, pop the
// remaining arguments, consume ')', invoke `callable` (borrowed -- the caller owns the reference),
// and push the marshalled result. Returns an MB_FUNC_* code. Mirrors mbDispatch's arg discipline.
static int mbInvokeWithOpenBracket(CalogMyBasicT *context, struct mb_interpreter_t *s, void **l, CalogFnT *callable) {
CalogValueT *args;
CalogValueT result;
mb_value_t pushValue;
void **savedL;
int32_t argCount;
int32_t capacity;
int32_t index;
int32_t status;
int code;
args = NULL;
argCount = 0;
capacity = 0;
while (mb_has_arg(s, l)) {
mb_value_t popped;
mb_make_nil(popped);
code = mb_pop_value(s, l, &popped);
if (code != MB_FUNC_OK) {
goto cleanup;
}
if (argCount == capacity) {
int32_t newCap;
CalogValueT *grown;
newCap = (capacity == 0) ? MB_INITIAL_ARGS : capacity * CALOG_GROWTH_FACTOR;
grown = (CalogValueT *)realloc(args, (size_t)newCap * sizeof(CalogValueT));
if (grown == NULL) {
if (popped.type == MB_DT_LIST || popped.type == MB_DT_DICT || popped.type == MB_DT_ROUTINE) {
mb_dispose_value(context->bas, popped);
}
code = mb_raise_error(s, l, SE_RN_OUT_OF_MEMORY, MB_FUNC_ERR);
goto cleanup;
}
args = grown;
capacity = newCap;
}
status = mbToValueDepth(context, l, &popped, &args[argCount], 0);
if (popped.type == MB_DT_LIST || popped.type == MB_DT_DICT) {
mb_dispose_value(context->bas, popped);
}
if (status != calogOkE) {
if (popped.type == MB_DT_ROUTINE) {
mb_dispose_value(context->bas, popped);
}
code = mb_raise_error(s, l, MB_NATIVE_ERROR, MB_FUNC_ERR);
goto cleanup;
}
argCount++;
}
code = mb_attempt_close_bracket(s, l);
if (code != MB_FUNC_OK) {
goto cleanup;
}
// Publish currentL so a callable that re-enters a BASIC routine finds this serving frame,
// then invoke (the actor layer routes to the callable's owner).
savedL = context->currentL;
context->currentL = l;
status = calogFnInvoke(callable, args, argCount, &result);
context->currentL = savedL;
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
calogValueFree(&result);
return mb_raise_error(s, l, MB_NATIVE_ERROR, MB_FUNC_ERR);
}
mb_make_nil(pushValue);
status = mbFromValueDepth(context, l, &result, &pushValue, 0);
calogValueFree(&result);
if (status != calogOkE) {
return mb_raise_error(s, l, MB_NATIVE_ERROR, MB_FUNC_ERR);
}
if (pushValue.type == MB_DT_STRING) {
return mb_push_string(s, l, pushValue.value.string);
}
return mb_push_value(s, l, pushValue);
cleanup:
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
return code;
}
static int32_t mbListToAggregate(CalogMyBasicT *context, void **l, mb_value_t coll, CalogValueT *out, int32_t depth) { static int32_t mbListToAggregate(CalogMyBasicT *context, void **l, mb_value_t coll, CalogValueT *out, int32_t depth) {
CalogAggT *aggregate; CalogAggT *aggregate;
int32_t status; int32_t status;
@ -742,6 +987,20 @@ static void mbReleaseSetValue(CalogMyBasicT *context, mb_value_t value) {
} }
static void mbReportError(CalogMyBasicT *context, const char *stage) {
const char *desc;
mb_error_e err;
unsigned short row;
unsigned short col;
row = 0;
col = 0;
err = mb_get_last_error(context->bas, NULL, NULL, &row, &col);
desc = mb_get_error_desc(err);
fprintf(stderr, "my-basic %s error: %s (row %u, col %u)\n", stage, desc, row, col);
}
static void mbRetainBeforeSet(CalogMyBasicT *context, void **l, mb_value_t value) { static void mbRetainBeforeSet(CalogMyBasicT *context, void **l, mb_value_t value) {
// mb_set_coll stores a LIST/DICT by pointer WITHOUT adding a reference, so the // mb_set_coll stores a LIST/DICT by pointer WITHOUT adding a reference, so the
// parent would not own it and would leak it on teardown. Add the reference // parent would not own it and would leak it on teardown. Add the reference
@ -759,7 +1018,7 @@ static int32_t mbToValueDepth(CalogMyBasicT *context, void **l, const mb_value_t
int32_t status; int32_t status;
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
switch (value->type) { switch (value->type) {
@ -848,6 +1107,12 @@ int32_t calogMyBasicCreate(CalogMyBasicT **out, CalogT *broker, uint64_t ctxId)
mbContextCount++; mbContextCount++;
mb_set_userdata(bas, context); mb_set_userdata(bas, context);
mb_set_printer(bas, mbPrinter); mb_set_printer(bas, mbPrinter);
// A host callable handed into a script (mbFromValueDepth's calogFnE case) becomes a
// usertype-ref; calogInvoke(fn, ...args) is how a BASIC script calls it.
mb_register_func(bas, "calogInvoke", mbForeignInvokeNative);
// Let a bare name that is otherwise undefined but called like a function resolve to an
// export -- so exportedFn(args) works without an explicit calogCall (like the hook engines).
mb_set_dynamic_func_handler(bas, mbDynamicFuncHandler);
*out = context; *out = context;
return calogOkE; return calogOkE;
} }
@ -920,17 +1185,27 @@ int32_t calogMyBasicExpose(CalogMyBasicT *context, const char *name) {
} }
void calogMyBasicLifecycleLock(void) {
pthread_mutex_lock(&mbLifecycleLock);
}
void calogMyBasicLifecycleUnlock(void) {
pthread_mutex_unlock(&mbLifecycleLock);
}
int32_t calogMyBasicRun(CalogMyBasicT *context, const char *source) { int32_t calogMyBasicRun(CalogMyBasicT *context, const char *source) {
int code; int code;
code = mb_load_string(context->bas, source, true); code = mb_load_string(context->bas, source, true);
if (code != MB_FUNC_OK) { if (code != MB_FUNC_OK) {
fprintf(stderr, "my-basic load error: %d\n", code); mbReportError(context, "load");
return calogErrArgE; return calogErrArgE;
} }
code = mb_run(context->bas, true); code = mb_run(context->bas, true);
if (code != MB_FUNC_OK) { if (code != MB_FUNC_OK) {
fprintf(stderr, "my-basic run error: %d\n", code); mbReportError(context, "run");
return calogErrArgE; return calogErrArgE;
} }
return calogOkE; return calogOkE;

View file

@ -21,6 +21,11 @@ int32_t calogMyBasicCreate(CalogMyBasicT **out, CalogT *broker, uint64_t ctxId);
void calogMyBasicDestroy(CalogMyBasicT *context); void calogMyBasicDestroy(CalogMyBasicT *context);
int32_t calogMyBasicExportRoutine(CalogMyBasicT *context, const char *routineName, CalogFnT **out); int32_t calogMyBasicExportRoutine(CalogMyBasicT *context, const char *routineName, CalogFnT **out);
int32_t calogMyBasicExpose(CalogMyBasicT *context, const char *name); int32_t calogMyBasicExpose(CalogMyBasicT *context, const char *name);
// Serialize my-basic context lifecycle: the engine holds this around create/destroy so the
// adapter-owned mb_init/mb_dispose singletons and the context refcount stay consistent. NOT
// held during script execution.
void calogMyBasicLifecycleLock(void);
void calogMyBasicLifecycleUnlock(void);
int32_t calogMyBasicRun(CalogMyBasicT *context, const char *source); int32_t calogMyBasicRun(CalogMyBasicT *context, const char *source);
#endif #endif

View file

@ -24,10 +24,6 @@ static void mybasicEngineDestroy(void *interp);
static int32_t mybasicEngineRun(void *interp, const char *source, CalogValueT *result); static int32_t mybasicEngineRun(void *interp, const char *source, CalogValueT *result);
static void mybasicExposeVisitor(const CalogEntryT *entry, void *ud); static void mybasicExposeVisitor(const CalogEntryT *entry, void *ud);
// Serializes my-basic context lifecycle (mb_init/mb_dispose + the shared refcount); it
// is NOT held during execution, so scripts run concurrently.
static pthread_mutex_t mbLifecycleLock = PTHREAD_MUTEX_INITIALIZER;
static const char *const mybasicExtensions[] = { "bas", NULL }; static const char *const mybasicExtensions[] = { "bas", NULL };
const CalogEngineT calogMyBasicEngine = { const CalogEngineT calogMyBasicEngine = {
@ -44,21 +40,21 @@ static int32_t mybasicEngineCreate(CalogContextT *context, void **interpOut) {
int32_t status; int32_t status;
*interpOut = NULL; *interpOut = NULL;
pthread_mutex_lock(&mbLifecycleLock); calogMyBasicLifecycleLock();
status = calogMyBasicCreate(&mb, calogContextBroker(context), calogContextId(context)); status = calogMyBasicCreate(&mb, calogContextBroker(context), calogContextId(context));
if (status == calogOkE) { if (status == calogOkE) {
calogForEach(calogContextBroker(context), mybasicExposeVisitor, mb); calogForEach(calogContextBroker(context), mybasicExposeVisitor, mb);
*interpOut = mb; *interpOut = mb;
} }
pthread_mutex_unlock(&mbLifecycleLock); calogMyBasicLifecycleUnlock();
return status; return status;
} }
static void mybasicEngineDestroy(void *interp) { static void mybasicEngineDestroy(void *interp) {
pthread_mutex_lock(&mbLifecycleLock); calogMyBasicLifecycleLock();
calogMyBasicDestroy((CalogMyBasicT *)interp); calogMyBasicDestroy((CalogMyBasicT *)interp);
pthread_mutex_unlock(&mbLifecycleLock); calogMyBasicLifecycleUnlock();
} }

View file

@ -13,7 +13,9 @@
// //
// Error model: a native failure raises a Scheme error via s7_error. calogS7Run wraps the // Error model: a native failure raises a Scheme error via s7_error. calogS7Run wraps the
// source in (catch #t (lambda () (eval-string ...)) handler) so both read and run errors // source in (catch #t (lambda () (eval-string ...)) handler) so both read and run errors
// surface as a value rather than aborting. // report through calogS7Run's status rather than aborting; the handler records the failure
// out-of-band (context->errorFlag) instead of an in-band sentinel value, so a script that
// legitimately returns matching data is never mistaken for a caught error.
#define _POSIX_C_SOURCE 200809L #define _POSIX_C_SOURCE 200809L
@ -26,16 +28,24 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define S7_ERR_CAP 256
#define S7_CONTEXT_VAR "*calog-context*" #define S7_CONTEXT_VAR "*calog-context*"
#define S7_ERROR_TAG "*calog-error*"
// Wrapper source for calogS7Run: escaped script text goes in the single %s, the catch
// handler is the hidden global function below rather than in-band data, so success/failure
// is tracked out-of-band (context->errorFlag) and cannot collide with a script's own return
// value.
#define S7_RUN_WRAP_FMT "(catch #t (lambda () (eval-string \"%s\")) %%calog-catch)"
// Wrapper source for calogS7Expose: a variadic forwarder to the generic dispatcher.
#define S7_EXPOSE_FMT "(define (%s . a) (apply %%calog-call \"%s\" a))"
struct CalogS7T { struct CalogS7T {
s7_scheme *sc; s7_scheme *sc;
CalogT *broker; CalogT *broker;
uint64_t ctxId; uint64_t ctxId;
s7_int fnType; // c-object type for a foreign CalogFnT pushed into Scheme (applicable + freed) s7_int fnType; // c-object type for a foreign CalogFnT pushed into Scheme (applicable + freed)
bool errorFlag; // set by s7CatchHandler when calogS7Run's script raised
char errorText[CALOG_ERR_MSG_CAP];
}; };
// Backs a CalogFnT exported from this interpreter: the owning context, the protected // Backs a CalogFnT exported from this interpreter: the owning context, the protected
@ -48,12 +58,16 @@ typedef struct S7ExportT {
static int32_t s7CallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t s7CallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void s7CallableRelease(CalogFnT *callable); static void s7CallableRelease(CalogFnT *callable);
static s7_pointer s7CatchHandler(s7_scheme *sc, s7_pointer args);
static s7_pointer s7DispatchNative(s7_scheme *sc, s7_pointer args); static s7_pointer s7DispatchNative(s7_scheme *sc, s7_pointer args);
static char *s7EscapeSource(const char *source); static char *s7EscapeSource(const char *source);
static int32_t s7ExportValue(CalogS7T *context, s7_pointer proc, CalogFnT **out); static int32_t s7ExportValue(CalogS7T *context, s7_pointer proc, CalogFnT **out);
static s7_pointer s7FinishInvoke(s7_scheme *sc, CalogS7T *context, CalogValueT *cargs, int32_t argCount, int32_t status, CalogValueT *result, const char *failureMessage);
static s7_pointer s7ForeignApply(s7_scheme *sc, s7_pointer args); static s7_pointer s7ForeignApply(s7_scheme *sc, s7_pointer args);
static void s7ForeignFree(void *value); static void s7ForeignFree(void *value);
static s7_pointer s7FromValue(CalogS7T *context, const CalogValueT *value, int32_t depth); static s7_pointer s7FromValue(CalogS7T *context, const CalogValueT *value, int32_t depth);
static int32_t s7MarshalArgs(CalogS7T *context, s7_pointer rest, int32_t argCount, CalogValueT **outArgs);
static s7_pointer s7MarshalError(s7_scheme *sc, int32_t status, const char *oomMessage, const char *typeMessage);
static s7_pointer s7ResolveUnbound(s7_scheme *sc, s7_pointer args); static s7_pointer s7ResolveUnbound(s7_scheme *sc, s7_pointer args);
static int32_t s7ToValue(CalogS7T *context, s7_pointer obj, CalogValueT *out, int32_t depth); static int32_t s7ToValue(CalogS7T *context, s7_pointer obj, CalogValueT *out, int32_t depth);
@ -77,6 +91,15 @@ int32_t calogS7Create(CalogS7T **out, CalogT *broker, uint64_t ctxId) {
// single generic native dispatcher. // single generic native dispatcher.
s7_define_variable(context->sc, S7_CONTEXT_VAR, s7_make_c_pointer(context->sc, context)); s7_define_variable(context->sc, S7_CONTEXT_VAR, s7_make_c_pointer(context->sc, context));
s7_define_function(context->sc, "%calog-call", s7DispatchNative, 1, 0, true, "calog native dispatch"); s7_define_function(context->sc, "%calog-call", s7DispatchNative, 1, 0, true, "calog native dispatch");
// Hidden catch handler used by calogS7Run: a real function (not in-band Scheme data),
// so a script's own return value can never be confused with a caught error.
s7_define_function(context->sc, "%calog-catch", s7CatchHandler, 2, 0, false, "calog catch handler");
// Hidden wrapper used by s7CallableInvoke: catches a raising exported callback so its
// failure can be reported through calogFnInvoke's status instead of being lost.
s7_eval_c_string(context->sc,
"(define (%calog-invoke proc a) "
"(catch #t (lambda () (cons #t (apply proc a))) "
"(lambda (tag . info) (cons #f (object->string (cons tag info))))))");
// A c-object type for a foreign function pushed into Scheme: applicable via its ref // A c-object type for a foreign function pushed into Scheme: applicable via its ref
// hook (so (f 2 3) invokes it) and released by its free hook. // hook (so (f 2 3) invokes it) and released by its free hook.
context->fnType = s7_make_c_type(context->sc, "calog-fn"); context->fnType = s7_make_c_type(context->sc, "calog-fn");
@ -100,22 +123,37 @@ static int32_t s7CallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT
CalogS7T *context; CalogS7T *context;
s7_scheme *sc; s7_scheme *sc;
s7_pointer argList; s7_pointer argList;
s7_pointer ret; s7_pointer holder;
s7_pointer outcome;
s7_pointer message;
unsigned int loc;
int32_t index; int32_t index;
export = (S7ExportT *)userData; export = (S7ExportT *)userData;
context = export->context; context = export->context;
sc = context->sc; sc = context->sc;
calogValueNil(result); calogValueNil(result);
// Build the argument list back-to-front so evaluation order is preserved. // Build the argument list back-to-front so evaluation order is preserved; the holder
// is gc-protected and updated after every cons so the growing list stays reachable
// while a nested s7FromValue call allocates.
holder = s7_cons(sc, s7_nil(sc), s7_nil(sc));
loc = s7_gc_protect(sc, holder);
argList = s7_nil(sc); argList = s7_nil(sc);
for (index = argCount - 1; index >= 0; index--) { for (index = argCount - 1; index >= 0; index--) {
s7_pointer element; s7_pointer element;
element = s7FromValue(context, &args[index], 0); element = s7FromValue(context, &args[index], 0);
argList = s7_cons(sc, element, argList); argList = s7_cons(sc, element, argList);
s7_set_car(holder, argList);
} }
ret = s7_call(sc, export->proc, argList); s7_gc_unprotect_at(sc, loc);
return s7ToValue(context, ret, result, 0); // %calog-invoke wraps the call in its own catch, so a raising callback comes back as
// (#f . message) instead of an uncaught error object indistinguishable from a value.
outcome = s7_call(sc, s7_name_to_value(sc, "%calog-invoke"), s7_list(sc, 2, export->proc, argList));
if (s7_is_eq(s7_car(outcome), s7_f(sc))) {
message = s7_cdr(outcome);
return calogFail(result, calogErrArgE, s7_is_string(message) ? s7_string(message) : "s7 callback failed");
}
return s7ToValue(context, s7_cdr(outcome), result, 0);
} }
@ -128,6 +166,31 @@ static void s7CallableRelease(CalogFnT *callable) {
} }
// Registered as the catch handler in S7_RUN_WRAP_FMT: (tag . info) is whatever the caught
// error raised. Recording the failure out-of-band (rather than returning an in-band
// sentinel value) means a script that legitimately returns matching data is never mistaken
// for a caught error.
static s7_pointer s7CatchHandler(s7_scheme *sc, s7_pointer args) {
CalogS7T *context;
s7_pointer type;
s7_pointer info;
char *text;
context = (CalogS7T *)s7_c_pointer(s7_name_to_value(sc, S7_CONTEXT_VAR));
type = s7_car(args);
info = s7_cadr(args);
context->errorFlag = true;
text = s7_object_to_c_string(sc, s7_cons(sc, type, info));
if (text != NULL) {
snprintf(context->errorText, sizeof(context->errorText), "%s", text);
free(text);
} else {
context->errorText[0] = '\0';
}
return s7_unspecified(sc);
}
void calogS7Destroy(CalogS7T *context) { void calogS7Destroy(CalogS7T *context) {
if (context == NULL) { if (context == NULL) {
return; return;
@ -139,59 +202,102 @@ void calogS7Destroy(CalogS7T *context) {
} }
static s7_pointer s7DispatchNative(s7_scheme *sc, s7_pointer args) { // Common tail for s7DispatchNative and s7ForeignApply: free the marshalled C args, then
CalogS7T *context; // convert the invocation outcome -- an s7_error on failure (using failureMessage as a
const char *name; // fallback when the result carries no string), or the marshalled Scheme value on success.
s7_pointer rest; static s7_pointer s7FinishInvoke(s7_scheme *sc, CalogS7T *context, CalogValueT *cargs, int32_t argCount, int32_t status, CalogValueT *result, const char *failureMessage) {
s7_pointer node;
CalogValueT *cargs;
CalogValueT result;
s7_pointer sresult; s7_pointer sresult;
int argCount; int32_t index;
int index;
int32_t status;
char message[S7_ERR_CAP];
context = (CalogS7T *)s7_c_pointer(s7_name_to_value(sc, S7_CONTEXT_VAR));
name = s7_string(s7_car(args));
rest = s7_cdr(args);
argCount = s7_list_length(sc, rest);
cargs = NULL;
if (argCount > 0) {
cargs = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (cargs == NULL) {
return s7_error(sc, s7_make_symbol(sc, "memory-error"), s7_make_string(sc, "out of memory marshalling native arguments"));
}
}
node = rest;
for (index = 0; index < argCount; index++) {
status = s7ToValue(context, s7_car(node), &cargs[index], 0);
if (status != calogOkE) {
int cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&cargs[cleanup]);
}
free(cargs);
return s7_error(sc, s7_make_symbol(sc, "wrong-type-arg"), s7_make_string(sc, "failed to marshal a native argument"));
}
node = s7_cdr(node);
}
status = calogCall(context->broker, name, cargs, argCount, &result);
for (index = 0; index < argCount; index++) { for (index = 0; index < argCount; index++) {
calogValueFree(&cargs[index]); calogValueFree(&cargs[index]);
} }
free(cargs); free(cargs);
if (status != calogOkE) { if (status != calogOkE) {
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : "native call failed"); s7_pointer message;
calogValueFree(&result); message = s7_make_string(sc, (result->type == calogStringE) ? result->as.s.bytes : failureMessage);
return s7_error(sc, s7_make_symbol(sc, "calog-error"), s7_make_string(sc, message)); calogValueFree(result);
return s7_error(sc, s7_make_symbol(sc, "calog-error"), message);
} }
sresult = s7FromValue(context, &result, 0); sresult = s7FromValue(context, result, 0);
calogValueFree(&result); calogValueFree(result);
return sresult; return sresult;
} }
// Marshal a Scheme list (rest, of length argCount) into a freshly allocated CalogValueT
// array shared by s7DispatchNative and s7ForeignApply. On success *outArgs is the array
// (NULL when argCount is 0); on failure *outArgs is NULL and any elements already
// marshalled have been freed.
static int32_t s7MarshalArgs(CalogS7T *context, s7_pointer rest, int32_t argCount, CalogValueT **outArgs) {
CalogValueT *cargs;
s7_pointer node;
int32_t index;
int32_t status;
*outArgs = NULL;
if (argCount == 0) {
return calogOkE;
}
cargs = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (cargs == NULL) {
return calogErrOomE;
}
node = rest;
for (index = 0; index < argCount; index++) {
status = s7ToValue(context, s7_car(node), &cargs[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&cargs[cleanup]);
}
free(cargs);
return status;
}
node = s7_cdr(node);
}
*outArgs = cargs;
return calogOkE;
}
// Raise the s7 error matching an s7MarshalArgs failure status.
static s7_pointer s7MarshalError(s7_scheme *sc, int32_t status, const char *oomMessage, const char *typeMessage) {
if (status == calogErrOomE) {
return s7_error(sc, s7_make_symbol(sc, "memory-error"), s7_make_string(sc, oomMessage));
}
return s7_error(sc, s7_make_symbol(sc, "wrong-type-arg"), s7_make_string(sc, typeMessage));
}
static s7_pointer s7DispatchNative(s7_scheme *sc, s7_pointer args) {
CalogS7T *context;
s7_pointer nameObj;
const char *name;
s7_pointer rest;
CalogValueT *cargs;
CalogValueT result;
int32_t argCount;
int32_t status;
context = (CalogS7T *)s7_c_pointer(s7_name_to_value(sc, S7_CONTEXT_VAR));
nameObj = s7_car(args);
if (!s7_is_string(nameObj)) {
return s7_error(sc, s7_make_symbol(sc, "wrong-type-arg"), s7_make_string(sc, "calog native dispatch expects a name string"));
}
name = s7_string(nameObj);
rest = s7_cdr(args);
argCount = (int32_t)s7_list_length(sc, rest);
status = s7MarshalArgs(context, rest, argCount, &cargs);
if (status != calogOkE) {
return s7MarshalError(sc, status, "out of memory marshalling native arguments", "failed to marshal a native argument");
}
calogValueNil(&result);
status = calogCall(context->broker, name, cargs, argCount, &result);
return s7FinishInvoke(sc, context, cargs, argCount, status, &result, "native call failed");
}
// Escape a source string into the body of a Scheme string literal (backslash and // Escape a source string into the body of a Scheme string literal (backslash and
// double-quote get a leading backslash). Caller frees. // double-quote get a leading backslash). Caller frees.
static char *s7EscapeSource(const char *source) { static char *s7EscapeSource(const char *source) {
@ -267,13 +373,15 @@ int32_t calogS7Expose(CalogS7T *context, const char *name) {
if (entry == NULL) { if (entry == NULL) {
return calogErrNotFoundE; return calogErrNotFoundE;
} }
// Define a variadic wrapper that forwards to the generic dispatcher by name. // Define a variadic wrapper that forwards to the generic dispatcher by name. Size off
size = strlen(name) * 2 + 64; // sizeof(S7_EXPOSE_FMT) (not a hand-picked constant) so a future edit to the format
// string can't silently outgrow a stale slack value.
size = strlen(name) * 2 + sizeof(S7_EXPOSE_FMT);
define = (char *)malloc(size); define = (char *)malloc(size);
if (define == NULL) { if (define == NULL) {
return calogErrOomE; return calogErrOomE;
} }
snprintf(define, size, "(define (%s . a) (apply %%calog-call \"%s\" a))", name, name); snprintf(define, size, S7_EXPOSE_FMT, name, name);
s7_eval_c_string(context->sc, define); s7_eval_c_string(context->sc, define);
free(define); free(define);
return calogOkE; return calogOkE;
@ -286,52 +394,22 @@ static s7_pointer s7ForeignApply(s7_scheme *sc, s7_pointer args) {
CalogS7T *context; CalogS7T *context;
CalogFnT *callable; CalogFnT *callable;
s7_pointer rest; s7_pointer rest;
s7_pointer node;
CalogValueT *cargs; CalogValueT *cargs;
CalogValueT result; CalogValueT result;
s7_pointer sresult; int32_t argCount;
int argCount;
int index;
int32_t status; int32_t status;
context = (CalogS7T *)s7_c_pointer(s7_name_to_value(sc, S7_CONTEXT_VAR)); context = (CalogS7T *)s7_c_pointer(s7_name_to_value(sc, S7_CONTEXT_VAR));
callable = (CalogFnT *)s7_c_object_value(s7_car(args)); callable = (CalogFnT *)s7_c_object_value(s7_car(args));
rest = s7_cdr(args); rest = s7_cdr(args);
argCount = s7_list_length(sc, rest); argCount = (int32_t)s7_list_length(sc, rest);
cargs = NULL; status = s7MarshalArgs(context, rest, argCount, &cargs);
if (argCount > 0) {
cargs = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (cargs == NULL) {
return s7_error(sc, s7_make_symbol(sc, "memory-error"), s7_make_string(sc, "out of memory marshalling function-value args"));
}
}
node = rest;
for (index = 0; index < argCount; index++) {
status = s7ToValue(context, s7_car(node), &cargs[index], 0);
if (status != calogOkE) { if (status != calogOkE) {
int cleanup; return s7MarshalError(sc, status, "out of memory marshalling function-value args", "failed to marshal a function-value argument");
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&cargs[cleanup]);
}
free(cargs);
return s7_error(sc, s7_make_symbol(sc, "wrong-type-arg"), s7_make_string(sc, "failed to marshal a function-value argument"));
}
node = s7_cdr(node);
} }
calogValueNil(&result);
status = calogFnInvoke(callable, cargs, argCount, &result); status = calogFnInvoke(callable, cargs, argCount, &result);
for (index = 0; index < argCount; index++) { return s7FinishInvoke(sc, context, cargs, argCount, status, &result, "function value failed");
calogValueFree(&cargs[index]);
}
free(cargs);
if (status != calogOkE) {
s7_pointer message;
message = s7_make_string(sc, (result.type == calogStringE) ? result.as.s.bytes : "function value failed");
calogValueFree(&result);
return s7_error(sc, s7_make_symbol(sc, "calog-error"), message);
}
sresult = s7FromValue(context, &result, 0);
calogValueFree(&result);
return sresult;
} }
@ -350,7 +428,7 @@ static s7_pointer s7FromValue(CalogS7T *context, const CalogValueT *value, int32
s7_scheme *sc; s7_scheme *sc;
sc = context->sc; sc = context->sc;
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return s7_unspecified(sc); return s7_unspecified(sc);
} }
switch (value->type) { switch (value->type) {
@ -371,11 +449,16 @@ static s7_pointer s7FromValue(CalogS7T *context, const CalogValueT *value, int32
// Anything keyed (or an explicit map) becomes a hash-table (applicable, so a // Anything keyed (or an explicit map) becomes a hash-table (applicable, so a
// script reads (user "name")) with the sequence part at integer keys; a pure // script reads (user "name")) with the sequence part at integer keys; a pure
// sequence becomes a Scheme list. // sequence becomes a Scheme list.
if (aggregate->pairCount > 0 || aggregate->kind == calogMapE) { if (calogAggIsKeyed(aggregate)) {
s7_pointer table; s7_pointer table;
s7_int size; s7_int size;
unsigned int loc;
size = (s7_int)(aggregate->arrayCount + aggregate->pairCount); size = (s7_int)(aggregate->arrayCount + aggregate->pairCount);
table = s7_make_hash_table(sc, size > 0 ? size : 1); table = s7_make_hash_table(sc, size > 0 ? size : 1);
// Protect the table across the whole build: a large aggregate can allocate
// past the GC's implicit "recent allocations" window, and the table's own
// cell (allocated first) would otherwise be the first thing swept.
loc = s7_gc_protect(sc, table);
for (index = 0; index < aggregate->arrayCount; index++) { for (index = 0; index < aggregate->arrayCount; index++) {
s7_hash_table_set(sc, table, s7_make_integer(sc, (s7_int)index), s7_hash_table_set(sc, table, s7_make_integer(sc, (s7_int)index),
s7FromValue(context, &aggregate->array[index], depth + 1)); s7FromValue(context, &aggregate->array[index], depth + 1));
@ -394,15 +477,25 @@ static s7_pointer s7FromValue(CalogS7T *context, const CalogValueT *value, int32
s7_hash_table_set(sc, table, keyObj, s7_hash_table_set(sc, table, keyObj,
s7FromValue(context, &aggregate->pairs[index].value, depth + 1)); s7FromValue(context, &aggregate->pairs[index].value, depth + 1));
} }
s7_gc_unprotect_at(sc, loc);
return table; return table;
} else { } else {
s7_pointer list; s7_pointer list;
s7_pointer holder;
unsigned int loc;
// The growing list's head moves every iteration, so root it through a
// protected holder pair updated after each cons rather than protecting a
// pointer that goes stale.
holder = s7_cons(sc, s7_nil(sc), s7_nil(sc));
loc = s7_gc_protect(sc, holder);
list = s7_nil(sc); list = s7_nil(sc);
for (index = aggregate->arrayCount - 1; index >= 0; index--) { for (index = aggregate->arrayCount - 1; index >= 0; index--) {
s7_pointer element; s7_pointer element;
element = s7FromValue(context, &aggregate->array[index], depth + 1); element = s7FromValue(context, &aggregate->array[index], depth + 1);
list = s7_cons(sc, element, list); list = s7_cons(sc, element, list);
s7_set_car(holder, list);
} }
s7_gc_unprotect_at(sc, loc);
return list; return list;
} }
} }
@ -424,30 +517,30 @@ int32_t calogS7Run(CalogS7T *context, const char *source) {
char *escaped; char *escaped;
char *wrapped; char *wrapped;
size_t size; size_t size;
s7_pointer result;
sc = context->sc; sc = context->sc;
escaped = s7EscapeSource(source); escaped = s7EscapeSource(source);
if (escaped == NULL) { if (escaped == NULL) {
return calogErrOomE; return calogErrOomE;
} }
size = strlen(escaped) + 128; // Size off sizeof(S7_RUN_WRAP_FMT) (not a hand-picked constant) so a future edit to the
// wrapper text can't silently outgrow a stale slack value.
size = strlen(escaped) + sizeof(S7_RUN_WRAP_FMT);
wrapped = (char *)malloc(size); wrapped = (char *)malloc(size);
if (wrapped == NULL) { if (wrapped == NULL) {
free(escaped); free(escaped);
return calogErrOomE; return calogErrOomE;
} }
// Catch read- and run-time errors so a bad script reports rather than aborts. // Catch read- and run-time errors so a bad script reports rather than aborts; the
snprintf(wrapped, size, // handler is s7CatchHandler, which records failure in context->errorFlag rather than an
"(catch #t (lambda () (eval-string \"%s\")) (lambda (tag . info) (cons '%s (object->string (cons tag info)))))", // in-band value, so a script's own return data can never be mistaken for a caught error.
escaped, S7_ERROR_TAG); snprintf(wrapped, size, S7_RUN_WRAP_FMT, escaped);
free(escaped); free(escaped);
result = s7_eval_c_string(sc, wrapped); context->errorFlag = false;
s7_eval_c_string(sc, wrapped);
free(wrapped); free(wrapped);
if (s7_is_pair(result) && s7_is_eq(s7_car(result), s7_make_symbol(sc, S7_ERROR_TAG))) { if (context->errorFlag) {
s7_pointer message; fprintf(stderr, "s7 error: %s\n", context->errorText);
message = s7_cdr(result);
fprintf(stderr, "s7 error: %s\n", s7_is_string(message) ? s7_string(message) : "(unknown)");
return calogErrArgE; return calogErrArgE;
} }
return calogOkE; return calogOkE;
@ -490,7 +583,7 @@ static int32_t s7ToValue(CalogS7T *context, s7_pointer obj, CalogValueT *out, in
sc = context->sc; sc = context->sc;
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
if (s7_is_boolean(obj)) { if (s7_is_boolean(obj)) {

View file

@ -11,6 +11,7 @@
#define _POSIX_C_SOURCE 200809L #define _POSIX_C_SOURCE 200809L
#include "squirrelAdapter.h" #include "squirrelAdapter.h"
#include "adapterBinding.h"
#include <squirrel.h> #include <squirrel.h>
@ -19,19 +20,9 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define SQUIRREL_BINDING_INITIAL 8
#define SQUIRREL_STACK_SIZE 1024 #define SQUIRREL_STACK_SIZE 1024
#define SQUIRREL_MARSHAL_SLOTS 8 #define SQUIRREL_MARSHAL_SLOTS 8
// An exposed native is bound by broker + name (not a captured fn pointer) so the
// trampoline dispatches through calogCall and honors the actor route hook: a
// native owned by another context is marshalled to its thread rather than run on
// this VM's thread. With no hook installed, calogCall runs it inline.
typedef struct BindingT {
CalogT *broker;
char *name;
} BindingT;
// Backs a CalogFnT exported from this VM: a pinned (sq_addref'd) closure object // Backs a CalogFnT exported from this VM: a pinned (sq_addref'd) closure object
// plus the owning context whose VM must run the invoke and the eventual release. // plus the owning context whose VM must run the invoke and the eventual release.
typedef struct SquirrelExportT { typedef struct SquirrelExportT {
@ -48,17 +39,88 @@ struct CalogSquirrelT {
int32_t bindingCap; int32_t bindingCap;
}; };
// Shared shape for the two things a marshalled call can dispatch to: a CalogFnT
// (calogFnInvoke) or a broker binding (calogCall). squirrelCallDispatch runs the
// marshal/dispatch/cleanup pipeline once and calls back through this.
typedef int32_t (*SquirrelDispatchFnT)(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
static SQInteger squirrelCallDispatch(HSQUIRRELVM v, SquirrelDispatchFnT dispatch, void *userData, const char *noun, const char *failMessage);
static int32_t squirrelCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t squirrelCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void squirrelCallableRelease(CalogFnT *callable); static void squirrelCallableRelease(CalogFnT *callable);
static void squirrelCompileError(HSQUIRRELVM v, const SQChar *desc, const SQChar *source, SQInteger line, SQInteger column);
static CalogSquirrelT *squirrelContextOf(HSQUIRRELVM v); static CalogSquirrelT *squirrelContextOf(HSQUIRRELVM v);
static void squirrelErrorPrint(HSQUIRRELVM v, const SQChar *format, ...);
static int32_t squirrelExportAt(CalogSquirrelT *context, SQInteger idx, CalogFnT **out); static int32_t squirrelExportAt(CalogSquirrelT *context, SQInteger idx, CalogFnT **out);
static SQInteger squirrelForeignCall(HSQUIRRELVM v); static SQInteger squirrelForeignCall(HSQUIRRELVM v);
static int32_t squirrelForeignDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
static SQInteger squirrelForeignRelease(SQUserPointer p, SQInteger size); static SQInteger squirrelForeignRelease(SQUserPointer p, SQInteger size);
static void squirrelPrint(HSQUIRRELVM v, const SQChar *format, ...); static void squirrelPrint(HSQUIRRELVM v, const SQChar *format, ...);
static int32_t squirrelPushValueDepth(HSQUIRRELVM v, const CalogValueT *value, int32_t depth); static int32_t squirrelPushValueDepth(HSQUIRRELVM v, const CalogValueT *value, int32_t depth);
static void squirrelReportRuntimeError(HSQUIRRELVM v, const char *phase);
static SQInteger squirrelResolveGet(HSQUIRRELVM v); static SQInteger squirrelResolveGet(HSQUIRRELVM v);
static int32_t squirrelToValueDepth(HSQUIRRELVM v, SQInteger idx, CalogValueT *out, int32_t depth); static int32_t squirrelToValueDepth(HSQUIRRELVM v, SQInteger idx, CalogValueT *out, int32_t depth);
static SQInteger squirrelTrampoline(HSQUIRRELVM v); static SQInteger squirrelTrampoline(HSQUIRRELVM v);
static int32_t squirrelTrampolineDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
// Shared marshal/dispatch/cleanup pipeline for both call sites that cross into a
// Squirrel native closure: a foreign CalogFnT pushed into Squirrel and invoked as a
// native closure, and a broker native reached through the shared trampoline. Both
// closures have exactly one free variable pushed after the call arguments (per
// sq_newclosure(v, fn, 1)), so 'this' is always at 1, the arguments at 2..top-1, and
// the free variable at top -- argBase is fixed at 2 for both. dispatch and userData
// supply the actual call; noun and failMessage name the caller in diagnostics.
static SQInteger squirrelCallDispatch(HSQUIRRELVM v, SquirrelDispatchFnT dispatch, void *userData, const char *noun, const char *failMessage) {
CalogValueT *args;
CalogValueT result;
char message[CALOG_ERR_MSG_CAP];
SQInteger top;
int32_t argCount;
int32_t index;
int32_t status;
top = sq_gettop(v);
argCount = (int32_t)(top - 2); // 'this' at 1, args at 2..top-1, free variable at top
args = NULL;
if (argCount > 0) {
args = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (args == NULL) {
snprintf(message, sizeof(message), "out of memory marshalling %s args", noun);
return sq_throwerror(v, message);
}
}
for (index = 0; index < argCount; index++) {
status = squirrelToValueDepth(v, index + 2, &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
snprintf(message, sizeof(message), "failed to marshal a %s argument", noun);
return sq_throwerror(v, message);
}
}
status = dispatch(userData, args, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
const char *detail;
detail = (result.type == calogStringE) ? result.as.s.bytes : failMessage;
sq_throwerror(v, detail);
calogValueFree(&result);
return SQ_ERROR;
}
status = squirrelPushValueDepth(v, &result, 0);
calogValueFree(&result);
if (status != calogOkE) {
snprintf(message, sizeof(message), "failed to marshal a %s result", noun);
return sq_throwerror(v, message);
}
return 1;
}
static int32_t squirrelCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) { static int32_t squirrelCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
@ -103,6 +165,15 @@ static void squirrelCallableRelease(CalogFnT *callable) {
} }
// Registered with sq_setcompilererrorhandler so a failed sq_compilebuffer surfaces the
// actual syntax error (message, source name, line, column) instead of a bare fixed
// string -- mirrors the detail the Lua adapter gets for free from lua_tostring.
static void squirrelCompileError(HSQUIRRELVM v, const SQChar *desc, const SQChar *source, SQInteger line, SQInteger column) {
(void)v;
fprintf(stderr, "squirrel compile error: %s (%s:%lld:%lld)\n", desc, source, (long long)line, (long long)column);
}
int32_t calogSquirrelCreate(CalogSquirrelT **out, CalogT *broker, uint64_t ctxId) { int32_t calogSquirrelCreate(CalogSquirrelT **out, CalogT *broker, uint64_t ctxId) {
CalogSquirrelT *context; CalogSquirrelT *context;
HSQUIRRELVM v; HSQUIRRELVM v;
@ -121,7 +192,8 @@ int32_t calogSquirrelCreate(CalogSquirrelT **out, CalogT *broker, uint64_t ctxId
context->broker = broker; context->broker = broker;
context->ctxId = ctxId; context->ctxId = ctxId;
sq_setforeignptr(v, context); sq_setforeignptr(v, context);
sq_setprintfunc(v, squirrelPrint, squirrelPrint); sq_setprintfunc(v, squirrelPrint, squirrelErrorPrint);
sq_setcompilererrorhandler(v, squirrelCompileError);
// Install a root-table delegate whose _get resolves a missing name against the export // Install a root-table delegate whose _get resolves a missing name against the export
// registry, so an exported function is callable by its bare name. Native slots are own // registry, so an exported function is callable by its bare name. Native slots are own
// slots of the root table, so _get fires only for names the root does not already have. // slots of the root table, so _get fires only for names the root does not already have.
@ -138,19 +210,13 @@ int32_t calogSquirrelCreate(CalogSquirrelT **out, CalogT *broker, uint64_t ctxId
void calogSquirrelDestroy(CalogSquirrelT *context) { void calogSquirrelDestroy(CalogSquirrelT *context) {
int32_t index;
if (context == NULL) { if (context == NULL) {
return; return;
} }
if (context->v != NULL) { if (context->v != NULL) {
sq_close(context->v); sq_close(context->v);
} }
for (index = 0; index < context->bindingCount; index++) { adapterBindingDestroyAll(context->bindings, context->bindingCount);
free(context->bindings[index]->name);
free(context->bindings[index]);
}
free(context->bindings);
free(context); free(context);
} }
@ -217,40 +283,15 @@ static int32_t squirrelExportAt(CalogSquirrelT *context, SQInteger idx, CalogFnT
int32_t calogSquirrelExpose(CalogSquirrelT *context, const char *name) { int32_t calogSquirrelExpose(CalogSquirrelT *context, const char *name) {
CalogEntryT *entry;
BindingT *binding; BindingT *binding;
HSQUIRRELVM v; HSQUIRRELVM v;
int32_t status;
v = context->v; v = context->v;
entry = calogLookup(context->broker, name); status = adapterBindingExpose(context->broker, name, &context->bindings, &context->bindingCount, &context->bindingCap, &binding);
if (entry == NULL) { if (status != calogOkE) {
return calogErrNotFoundE; return status;
} }
binding = (BindingT *)malloc(sizeof(*binding));
if (binding == NULL) {
return calogErrOomE;
}
binding->broker = context->broker;
binding->name = strdup(name);
if (binding->name == NULL) {
free(binding);
return calogErrOomE;
}
if (context->bindingCount == context->bindingCap) {
int32_t newCap;
BindingT **resized;
newCap = (context->bindingCap == 0) ? SQUIRREL_BINDING_INITIAL : context->bindingCap * CALOG_GROWTH_FACTOR;
resized = (BindingT **)realloc(context->bindings, (size_t)newCap * sizeof(BindingT *));
if (resized == NULL) {
free(binding->name);
free(binding);
return calogErrOomE;
}
context->bindings = resized;
context->bindingCap = newCap;
}
context->bindings[context->bindingCount] = binding;
context->bindingCount++;
sq_pushroottable(v); sq_pushroottable(v);
sq_pushstring(v, name, -1); sq_pushstring(v, name, -1);
@ -268,68 +309,29 @@ int32_t calogSquirrelExpose(CalogSquirrelT *context, const char *name) {
static SQInteger squirrelForeignCall(HSQUIRRELVM v) { static SQInteger squirrelForeignCall(HSQUIRRELVM v) {
CalogFnT *callable; CalogFnT *callable;
SQUserPointer ptr; SQUserPointer ptr;
CalogValueT *args;
CalogValueT result;
SQInteger top; SQInteger top;
int32_t argCount;
int32_t index;
int32_t status;
top = sq_gettop(v); top = sq_gettop(v);
sq_getuserdata(v, top, &ptr, NULL); sq_getuserdata(v, top, &ptr, NULL);
callable = *(CalogFnT **)ptr; callable = *(CalogFnT **)ptr;
argCount = (int32_t)(top - 2); // 'this' at 1, args at 2..top-1, free variable at top return squirrelCallDispatch(v, squirrelForeignDispatch, callable, "function-value", "function value failed");
if (argCount < 0) {
argCount = 0;
}
args = NULL;
if (argCount > 0) {
args = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (args == NULL) {
return sq_throwerror(v, _SC("out of memory marshalling function-value args"));
}
}
for (index = 0; index < argCount; index++) {
status = squirrelToValueDepth(v, index + 2, &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
return sq_throwerror(v, _SC("failed to marshal a function-value argument"));
}
}
status = calogFnInvoke(callable, args, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
const SQChar *message;
message = (result.type == calogStringE) ? result.as.s.bytes : _SC("function value failed");
sq_throwerror(v, message);
calogValueFree(&result);
return SQ_ERROR;
}
status = squirrelPushValueDepth(v, &result, 0);
calogValueFree(&result);
if (status != calogOkE) {
return sq_throwerror(v, _SC("failed to marshal a function-value result"));
}
return 1;
} }
// Release hook on that userdata: drop the reference the push took. static int32_t squirrelForeignDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result) {
return calogFnInvoke((CalogFnT *)userData, args, argCount, result);
}
// Release hook on that userdata: drop the reference the push took. squirrelPushValueDepth
// always installs a live CalogFnT pointer before this hook is armed (case calogFnE below),
// so callable is never NULL here.
static SQInteger squirrelForeignRelease(SQUserPointer p, SQInteger size) { static SQInteger squirrelForeignRelease(SQUserPointer p, SQInteger size) {
CalogFnT *callable; CalogFnT *callable;
(void)size; (void)size;
callable = *(CalogFnT **)p; callable = *(CalogFnT **)p;
if (callable != NULL) {
calogFnRelease(callable); calogFnRelease(callable);
}
return 1; return 1;
} }
@ -344,8 +346,20 @@ static void squirrelPrint(HSQUIRRELVM v, const SQChar *format, ...) {
} }
// The VM's error-print channel (registered as the errfunc half of sq_setprintfunc):
// unlike squirrelPrint, this writes to stderr.
static void squirrelErrorPrint(HSQUIRRELVM v, const SQChar *format, ...) {
va_list args;
(void)v;
va_start(args, format);
vfprintf(stderr, format, args);
va_end(args);
}
static int32_t squirrelPushValueDepth(HSQUIRRELVM v, const CalogValueT *value, int32_t depth) { static int32_t squirrelPushValueDepth(HSQUIRRELVM v, const CalogValueT *value, int32_t depth) {
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
// The VM stack does not auto-grow on push (only EnterFrame resizes it), so // The VM stack does not auto-grow on push (only EnterFrame resizes it), so
@ -375,13 +389,11 @@ static int32_t squirrelPushValueDepth(HSQUIRRELVM v, const CalogValueT *value, i
SQInteger baseTop; SQInteger baseTop;
int32_t status; int32_t status;
int64_t index; int64_t index;
bool asTable;
aggregate = value->as.agg; aggregate = value->as.agg;
baseTop = sq_gettop(v); baseTop = sq_gettop(v);
// A Squirrel array holds only a sequence; anything with keyed entries // A Squirrel array holds only a sequence; anything with keyed entries
// (or an explicit map) must become a table. // (or an explicit map) must become a table.
asTable = aggregate->pairCount > 0 || aggregate->kind == calogMapE; if (!calogAggIsKeyed(aggregate)) {
if (!asTable) {
sq_newarray(v, 0); sq_newarray(v, 0);
for (index = 0; index < aggregate->arrayCount; index++) { for (index = 0; index < aggregate->arrayCount; index++) {
status = squirrelPushValueDepth(v, &aggregate->array[index], depth + 1); status = squirrelPushValueDepth(v, &aggregate->array[index], depth + 1);
@ -454,13 +466,14 @@ int32_t calogSquirrelRun(CalogSquirrelT *context, const char *source) {
v = context->v; v = context->v;
baseTop = sq_gettop(v); baseTop = sq_gettop(v);
if (SQ_FAILED(sq_compilebuffer(v, source, (SQInteger)strlen(source), _SC("calog"), SQTrue))) { if (SQ_FAILED(sq_compilebuffer(v, source, (SQInteger)strlen(source), _SC("calog"), SQTrue))) {
fprintf(stderr, "squirrel compile error\n"); // squirrelCompileError (installed on this VM) has already written the detailed
// syntax error to stderr.
sq_settop(v, baseTop); sq_settop(v, baseTop);
return calogErrArgE; return calogErrArgE;
} }
sq_pushroottable(v); sq_pushroottable(v);
if (SQ_FAILED(sq_call(v, 1, SQFalse, SQTrue))) { if (SQ_FAILED(sq_call(v, 1, SQFalse, SQTrue))) {
fprintf(stderr, "squirrel run error\n"); squirrelReportRuntimeError(v, "squirrel run error");
sq_settop(v, baseTop); sq_settop(v, baseTop);
return calogErrArgE; return calogErrArgE;
} }
@ -469,6 +482,22 @@ int32_t calogSquirrelRun(CalogSquirrelT *context, const char *source) {
} }
// Fetch and print the VM's last error (set by sq_throwerror on any native/script
// failure, including the broker error strings squirrelTrampoline/squirrelForeignCall
// throw), so a fire-and-forget script's uncaught error is not silently discarded.
static void squirrelReportRuntimeError(HSQUIRRELVM v, const char *phase) {
const SQChar *detail;
detail = NULL;
sq_getlasterror(v);
if (sq_gettype(v, -1) == OT_STRING) {
sq_getstring(v, -1, &detail);
}
fprintf(stderr, "%s: %s\n", phase, (detail != NULL) ? detail : "(no error detail available)");
sq_pop(v, 1);
}
static SQInteger squirrelResolveGet(HSQUIRRELVM v) { static SQInteger squirrelResolveGet(HSQUIRRELVM v) {
CalogSquirrelT *context; CalogSquirrelT *context;
CalogValueT nameArg; CalogValueT nameArg;
@ -506,7 +535,7 @@ static int32_t squirrelToValueDepth(HSQUIRRELVM v, SQInteger idx, CalogValueT *o
SQInteger absIdx; SQInteger absIdx;
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
absIdx = (idx < 0) ? sq_gettop(v) + idx + 1 : idx; absIdx = (idx < 0) ? sq_gettop(v) + idx + 1 : idx;
@ -618,12 +647,7 @@ static int32_t squirrelToValueDepth(HSQUIRRELVM v, SQInteger idx, CalogValueT *o
static SQInteger squirrelTrampoline(HSQUIRRELVM v) { static SQInteger squirrelTrampoline(HSQUIRRELVM v) {
BindingT *binding; BindingT *binding;
SQUserPointer ptr; SQUserPointer ptr;
CalogValueT *args;
CalogValueT result;
SQInteger top; SQInteger top;
int32_t argCount;
int32_t index;
int32_t status;
// The binding pointer is the closure's single free variable, which the VM // The binding pointer is the closure's single free variable, which the VM
// pushes onto the stack after the call arguments -- so it sits at the top. // pushes onto the stack after the call arguments -- so it sits at the top.
@ -631,44 +655,13 @@ static SQInteger squirrelTrampoline(HSQUIRRELVM v) {
top = sq_gettop(v); top = sq_gettop(v);
sq_getuserpointer(v, top, &ptr); sq_getuserpointer(v, top, &ptr);
binding = (BindingT *)ptr; binding = (BindingT *)ptr;
argCount = (int32_t)(top - 2); return squirrelCallDispatch(v, squirrelTrampolineDispatch, binding, "native", "native call failed");
if (argCount < 0) { }
argCount = 0;
}
args = NULL; static int32_t squirrelTrampolineDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result) {
if (argCount > 0) { BindingT *binding;
args = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (args == NULL) { binding = (BindingT *)userData;
return sq_throwerror(v, _SC("out of memory marshalling native args")); return calogCall(binding->broker, binding->name, args, argCount, result);
}
}
for (index = 0; index < argCount; index++) {
status = squirrelToValueDepth(v, index + 2, &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
return sq_throwerror(v, _SC("failed to marshal native argument"));
}
}
status = calogCall(binding->broker, binding->name, args, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
if (status != calogOkE) {
const SQChar *message;
message = (result.type == calogStringE) ? result.as.s.bytes : _SC("native call failed");
sq_throwerror(v, message);
calogValueFree(&result);
return SQ_ERROR;
}
status = squirrelPushValueDepth(v, &result, 0);
calogValueFree(&result);
if (status != calogOkE) {
return sq_throwerror(v, _SC("failed to marshal native result"));
}
return 1;
} }

View file

@ -10,6 +10,7 @@
#include "calogInternal.h" #include "calogInternal.h"
#include <math.h>
#include <stdatomic.h> #include <stdatomic.h>
#include <stdint.h> #include <stdint.h>
#include <stdlib.h> #include <stdlib.h>
@ -29,17 +30,17 @@ struct CalogFnT {
}; };
static int32_t aggregateCopyDepth(CalogAggT **out, const CalogAggT *src, int32_t depth); static int32_t aggregateCopyDepth(CalogAggT **out, const CalogAggT *src, int32_t depth);
static int32_t ensureCapacity(void **buffer, int64_t *cap, int64_t needed, size_t elemSize);
static int32_t valueCopyDepth(CalogValueT *dst, const CalogValueT *src, int32_t depth); static int32_t valueCopyDepth(CalogValueT *dst, const CalogValueT *src, int32_t depth);
static int32_t aggregateCopyDepth(CalogAggT **out, const CalogAggT *src, int32_t depth) { static int32_t aggregateCopyDepth(CalogAggT **out, const CalogAggT *src, int32_t depth) {
CalogAggT *copy; CalogAggT *copy;
void *buffer;
int32_t status; int32_t status;
int64_t index; int64_t index;
*out = NULL; *out = NULL;
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
status = calogAggCreate(&copy, src->kind); status = calogAggCreate(&copy, src->kind);
@ -74,13 +75,20 @@ static int32_t aggregateCopyDepth(CalogAggT **out, const CalogAggT *src, int32_t
calogAggFree(copy); calogAggFree(copy);
return status; return status;
} }
status = calogAggSet(copy, &key, &value); // Source pairs are unique by construction, so append directly instead of via
// calogAggSet, whose per-insert duplicate scan made this copy O(n^2).
buffer = copy->pairs;
status = calogGrow(&buffer, &copy->pairCap, copy->pairCount + 1, sizeof(CalogPairT));
if (status != calogOkE) { if (status != calogOkE) {
calogValueFree(&key); calogValueFree(&key);
calogValueFree(&value); calogValueFree(&value);
calogAggFree(copy); calogAggFree(copy);
return status; return status;
} }
copy->pairs = (CalogPairT *)buffer;
calogValueMove(&copy->pairs[copy->pairCount].key, &key);
calogValueMove(&copy->pairs[copy->pairCount].value, &value);
copy->pairCount++;
} }
*out = copy; *out = copy;
return calogOkE; return calogOkE;
@ -132,14 +140,19 @@ CalogValueT *calogAggGet(CalogAggT *aggregate, const CalogValueT *key) {
} }
bool calogAggIsKeyed(const CalogAggT *aggregate) {
return aggregate->pairCount > 0 || aggregate->kind == calogMapE;
}
int32_t calogAggPush(CalogAggT *aggregate, CalogValueT *value) { int32_t calogAggPush(CalogAggT *aggregate, CalogValueT *value) {
void *buffer; void *buffer;
int32_t status; int32_t status;
// Pun through a real void* lvalue, never (void **)&typedPointer, to avoid a // Pun through a real void* lvalue, never (void **)&typedPointer, to avoid a
// strict-aliasing violation when ensureCapacity stores the resized pointer. // strict-aliasing violation when calogGrow stores the resized pointer.
buffer = aggregate->array; buffer = aggregate->array;
status = ensureCapacity(&buffer, &aggregate->arrayCap, aggregate->arrayCount + 1, sizeof(CalogValueT)); status = calogGrow(&buffer, &aggregate->arrayCap, aggregate->arrayCount + 1, sizeof(CalogValueT));
if (status != calogOkE) { if (status != calogOkE) {
return status; return status;
} }
@ -164,7 +177,7 @@ int32_t calogAggSet(CalogAggT *aggregate, CalogValueT *key, CalogValueT *value)
} }
} }
buffer = aggregate->pairs; buffer = aggregate->pairs;
status = ensureCapacity(&buffer, &aggregate->pairCap, aggregate->pairCount + 1, sizeof(CalogPairT)); status = calogGrow(&buffer, &aggregate->pairCap, aggregate->pairCount + 1, sizeof(CalogPairT));
if (status != calogOkE) { if (status != calogOkE) {
return status; return status;
} }
@ -197,8 +210,8 @@ int32_t calogFnCreate(CalogFnT **out, CalogT *runtime, CalogNativeFnT fn, void *
int32_t calogFnFromNative(CalogFnT **out, CalogT *calog, CalogNativeFnT fn, void *userData) { int32_t calogFnFromNative(CalogFnT **out, CalogT *calog, CalogNativeFnT fn, void *userData) {
// A host-owned callable (owner id 0 = host): no engine handle to release. // A host-owned callable (owner id CALOG_HOST_ID = host): no engine handle to release.
return calogFnCreate(out, calog, fn, userData, NULL, 0); return calogFnCreate(out, calog, fn, userData, NULL, CALOG_HOST_ID);
} }
@ -296,7 +309,7 @@ void *calogFnUserData(const CalogFnT *callable) {
} }
static int32_t ensureCapacity(void **buffer, int64_t *cap, int64_t needed, size_t elemSize) { int32_t calogGrow(void **buffer, int64_t *cap, int64_t needed, size_t elemSize) {
int64_t newCap; int64_t newCap;
void *resized; void *resized;
@ -437,6 +450,15 @@ void calogValueFn(CalogValueT *value, CalogFnT *callable) {
} }
void calogValueFromDouble(CalogValueT *out, double number) {
if (isfinite(number) && number == floor(number) && number >= CALOG_INT64_MIN_DOUBLE && number < CALOG_INT64_MAX_DOUBLE) {
calogValueInt(out, (int64_t)number);
} else {
calogValueReal(out, number);
}
}
void calogValueInt(CalogValueT *value, int64_t i) { void calogValueInt(CalogValueT *value, int64_t i) {
value->type = calogIntE; value->type = calogIntE;
value->as.i = i; value->as.i = i;
@ -510,3 +532,84 @@ const char *calogTypeName(CalogTypeE type) {
} }
return "unknown"; return "unknown";
} }
// ---- shared library helpers (see calogInternal.h) ----
int32_t calogMapSetBool(CalogAggT *map, const char *key, bool flag) {
CalogValueT keyValue;
CalogValueT boolValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
calogValueBool(&boolValue, flag);
status = calogAggSet(map, &keyValue, &boolValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
}
return status;
}
int32_t calogMapSetInt(CalogAggT *map, const char *key, int64_t value) {
CalogValueT keyValue;
CalogValueT intValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
calogValueInt(&intValue, value);
status = calogAggSet(map, &keyValue, &intValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
}
return status;
}
int32_t calogMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length) {
CalogValueT keyValue;
CalogValueT stringValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
status = calogValueString(&stringValue, bytes, length);
if (status != calogOkE) {
calogValueFree(&keyValue);
return status;
}
status = calogAggSet(map, &keyValue, &stringValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
calogValueFree(&stringValue);
}
return status;
}
void calogRegistryRelease(pthread_mutex_t *initMutex, int32_t *refCount, void (*freeAll)(void)) {
pthread_mutex_lock(initMutex);
if (*refCount > 0) {
(*refCount)--;
}
if (*refCount <= 0) {
freeAll();
}
pthread_mutex_unlock(initMutex);
}
void calogRegistryRetain(pthread_mutex_t *initMutex, int32_t *refCount) {
pthread_mutex_lock(initMutex);
(*refCount)++;
pthread_mutex_unlock(initMutex);
}

View file

@ -17,13 +17,11 @@
#include "wren.h" #include "wren.h"
#include <math.h>
#include <stdint.h> #include <stdint.h>
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#define WREN_ERR_CAP 256
#define WREN_MAX_CALL_ARITY 16 #define WREN_MAX_CALL_ARITY 16
struct CalogWrenT { struct CalogWrenT {
@ -41,18 +39,32 @@ typedef struct WrenExportT {
WrenHandle *fn; WrenHandle *fn;
} WrenExportT; } WrenExportT;
// Bundles the (broker, name) pair Calog.call(name, args) needs so wrenInvokeShared can
// reach either a native call or a function-value call through one callback signature.
typedef struct WrenDispatchThunkT {
CalogT *broker;
const char *name;
} WrenDispatchThunkT;
// Common shape of "invoke the marshalled args against a target" used by wrenDispatch
// (target is a WrenDispatchThunkT) and wrenForeignInvoke (target is a CalogFnT).
typedef int32_t (*WrenInvokeFn)(void *target, CalogValueT *cargs, int32_t argCount, CalogValueT *result);
static int32_t wrenCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData); static int32_t wrenCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void wrenCallableRelease(CalogFnT *callable); static void wrenCallableRelease(CalogFnT *callable);
static WrenHandle *wrenCallHandle(CalogWrenT *context, int arity); static WrenHandle *wrenCallHandle(CalogWrenT *context, int arity);
static WrenForeignClassMethods wrenBindClass(WrenVM *vm, const char *module, const char *className); static WrenForeignClassMethods wrenBindClass(WrenVM *vm, const char *module, const char *className);
static WrenForeignMethodFn wrenBindMethod(WrenVM *vm, const char *module, const char *className, bool isStatic, const char *signature); static WrenForeignMethodFn wrenBindMethod(WrenVM *vm, const char *module, const char *className, bool isStatic, const char *signature);
static void wrenDispatch(WrenVM *vm); static void wrenDispatch(WrenVM *vm);
static int32_t wrenDispatchThunk(void *target, CalogValueT *cargs, int32_t argCount, CalogValueT *result);
static void wrenError(WrenVM *vm, WrenErrorType type, const char *module, int line, const char *message); static void wrenError(WrenVM *vm, WrenErrorType type, const char *module, int line, const char *message);
static int32_t wrenExportSlot(CalogWrenT *context, int slot, CalogFnT **out); static int32_t wrenExportSlot(CalogWrenT *context, int slot, CalogFnT **out);
static void wrenForeignAllocate(WrenVM *vm); static void wrenForeignAllocate(WrenVM *vm);
static void wrenForeignFinalize(void *data); static void wrenForeignFinalize(void *data);
static void wrenForeignInvoke(WrenVM *vm); static void wrenForeignInvoke(WrenVM *vm);
static void wrenFromValue(CalogWrenT *context, const CalogValueT *value, int slot, int32_t depth); static int32_t wrenForeignInvokeThunk(void *target, CalogValueT *cargs, int32_t argCount, CalogValueT *result);
static int32_t wrenFromValue(CalogWrenT *context, const CalogValueT *value, int slot, int32_t depth);
static void wrenInvokeShared(CalogWrenT *context, int listSlot, int scratchSlot, WrenInvokeFn invoke, void *target, const char *oomMsg, const char *marshalMsg, const char *failMsg);
static int32_t wrenToValue(CalogWrenT *context, int slot, CalogValueT *out, int32_t depth); static int32_t wrenToValue(CalogWrenT *context, int slot, CalogValueT *out, int32_t depth);
static void wrenWrite(WrenVM *vm, const char *text); static void wrenWrite(WrenVM *vm, const char *text);
@ -118,7 +130,11 @@ static int32_t wrenCallableInvoke(CalogValueT *args, int32_t argCount, CalogValu
wrenEnsureSlots(vm, argCount + 1); wrenEnsureSlots(vm, argCount + 1);
wrenSetSlotHandle(vm, 0, export->fn); // receiver is the function wrenSetSlotHandle(vm, 0, export->fn); // receiver is the function
for (index = 0; index < argCount; index++) { for (index = 0; index < argCount; index++) {
wrenFromValue(context, &args[index], index + 1, 0); int32_t marshalStatus;
marshalStatus = wrenFromValue(context, &args[index], index + 1, 0);
if (marshalStatus != calogOkE) {
return calogFail(result, marshalStatus, "argument nesting too deep");
}
} }
free(context->errorMsg); free(context->errorMsg);
context->errorMsg = NULL; context->errorMsg = NULL;
@ -217,56 +233,28 @@ static WrenForeignMethodFn wrenBindMethod(WrenVM *vm, const char *module, const
static void wrenDispatch(WrenVM *vm) { static void wrenDispatch(WrenVM *vm) {
CalogWrenT *context; CalogWrenT *context;
const char *name; WrenDispatchThunkT thunk;
CalogValueT *cargs;
CalogValueT result;
int argCount;
int index;
int32_t status;
char message[WREN_ERR_CAP];
context = (CalogWrenT *)wrenGetUserData(vm); context = (CalogWrenT *)wrenGetUserData(vm);
// slot 0 = receiver, slot 1 = name, slot 2 = args list. // slot 0 = receiver, slot 1 = name, slot 2 = args list. Wren is built without
name = wrenGetSlotString(vm, 1); // -DDEBUG, so the slot getters below only ASSERT their type (a no-op); a wrong
argCount = wrenGetListCount(vm, 2); // type reinterprets a raw NaN-boxed value as an object pointer, so check first.
cargs = NULL; if (wrenGetSlotType(vm, 1) != WREN_TYPE_STRING || wrenGetSlotType(vm, 2) != WREN_TYPE_LIST) {
if (argCount > 0) { wrenSetSlotString(vm, 0, "Calog.call expects a string name and a list of arguments");
cargs = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (cargs == NULL) {
wrenSetSlotString(vm, 0, "out of memory marshalling native arguments");
wrenAbortFiber(vm, 0); wrenAbortFiber(vm, 0);
return; return;
} }
} thunk.broker = context->broker;
wrenEnsureSlots(vm, 4); // slot 3 is a scratch slot for list elements thunk.name = wrenGetSlotString(vm, 1);
for (index = 0; index < argCount; index++) { wrenInvokeShared(context, 2, 3, wrenDispatchThunk, &thunk, "out of memory marshalling native arguments", "failed to marshal a native argument", "native call failed");
wrenGetListElement(vm, 2, index, 3); }
status = wrenToValue(context, 3, &cargs[index], 0);
if (status != calogOkE) {
int cleanup; static int32_t wrenDispatchThunk(void *target, CalogValueT *cargs, int32_t argCount, CalogValueT *result) {
for (cleanup = 0; cleanup < index; cleanup++) { WrenDispatchThunkT *thunk;
calogValueFree(&cargs[cleanup]);
} thunk = (WrenDispatchThunkT *)target;
free(cargs); return calogCall(thunk->broker, thunk->name, cargs, argCount, result);
wrenSetSlotString(vm, 0, "failed to marshal a native argument");
wrenAbortFiber(vm, 0);
return;
}
}
status = calogCall(context->broker, name, cargs, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&cargs[index]);
}
free(cargs);
if (status != calogOkE) {
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : "native call failed");
calogValueFree(&result);
wrenSetSlotString(vm, 0, message);
wrenAbortFiber(vm, 0);
return;
}
wrenFromValue(context, &result, 0, 0); // return value in slot 0
calogValueFree(&result);
} }
@ -358,95 +346,74 @@ static void wrenForeignFinalize(void *data) {
static void wrenForeignInvoke(WrenVM *vm) { static void wrenForeignInvoke(WrenVM *vm) {
CalogWrenT *context; CalogWrenT *context;
CalogFnT *callable; CalogFnT *callable;
CalogValueT *cargs;
CalogValueT result;
int argCount;
int index;
int32_t status;
char message[WREN_ERR_CAP];
context = (CalogWrenT *)wrenGetUserData(vm); context = (CalogWrenT *)wrenGetUserData(vm);
// Wren is built without -DDEBUG, so wrenGetListCount only ASSERTs its type (a
// no-op); a non-list argument would reinterpret a raw NaN-boxed value as an
// object pointer, so check first.
if (wrenGetSlotType(vm, 1) != WREN_TYPE_LIST) {
wrenSetSlotString(vm, 0, "CalogFn.call expects a list of arguments");
wrenAbortFiber(vm, 0);
return;
}
callable = *(CalogFnT **)wrenGetSlotForeign(vm, 0); // slot 0 is the CalogFn instance callable = *(CalogFnT **)wrenGetSlotForeign(vm, 0); // slot 0 is the CalogFn instance
argCount = wrenGetListCount(vm, 1); // slot 1 is the argument list if (callable == NULL) {
cargs = NULL; // Reached only via the script-visible CalogFn.new().call([...]); wrenForeignAllocate
if (argCount > 0) { // deliberately leaves a freshly-constructed instance empty.
cargs = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT)); wrenSetSlotString(vm, 0, "empty CalogFn");
if (cargs == NULL) {
wrenSetSlotString(vm, 0, "out of memory marshalling function-value args");
wrenAbortFiber(vm, 0); wrenAbortFiber(vm, 0);
return; return;
} }
} wrenInvokeShared(context, 1, 2, wrenForeignInvokeThunk, callable, "out of memory marshalling function-value args", "failed to marshal a function-value argument", "function value failed");
wrenEnsureSlots(vm, 3); // slot 2 is a scratch slot for list elements
for (index = 0; index < argCount; index++) {
wrenGetListElement(vm, 1, index, 2);
status = wrenToValue(context, 2, &cargs[index], 0);
if (status != calogOkE) {
int cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&cargs[cleanup]);
}
free(cargs);
wrenSetSlotString(vm, 0, "failed to marshal a function-value argument");
wrenAbortFiber(vm, 0);
return;
}
}
status = calogFnInvoke(callable, cargs, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&cargs[index]);
}
free(cargs);
if (status != calogOkE) {
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : "function value failed");
calogValueFree(&result);
wrenSetSlotString(vm, 0, message);
wrenAbortFiber(vm, 0);
return;
}
wrenFromValue(context, &result, 0, 0); // return value in slot 0
calogValueFree(&result);
} }
static void wrenFromValue(CalogWrenT *context, const CalogValueT *value, int slot, int32_t depth) { static int32_t wrenForeignInvokeThunk(void *target, CalogValueT *cargs, int32_t argCount, CalogValueT *result) {
return calogFnInvoke((CalogFnT *)target, cargs, argCount, result);
}
static int32_t wrenFromValue(CalogWrenT *context, const CalogValueT *value, int slot, int32_t depth) {
WrenVM *vm; WrenVM *vm;
vm = context->vm; vm = context->vm;
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
wrenSetSlotNull(vm, slot); return calogErrDepthE;
return;
} }
switch (value->type) { switch (value->type) {
case calogNilE: case calogNilE:
wrenSetSlotNull(vm, slot); wrenSetSlotNull(vm, slot);
return; return calogOkE;
case calogBoolE: case calogBoolE:
wrenSetSlotBool(vm, slot, value->as.b); wrenSetSlotBool(vm, slot, value->as.b);
return; return calogOkE;
case calogIntE: case calogIntE:
// Wren numbers are doubles; magnitudes above 2^53 lose precision. // Wren numbers are doubles; magnitudes above 2^53 lose precision.
wrenSetSlotDouble(vm, slot, (double)value->as.i); wrenSetSlotDouble(vm, slot, (double)value->as.i);
return; return calogOkE;
case calogRealE: case calogRealE:
wrenSetSlotDouble(vm, slot, value->as.r); wrenSetSlotDouble(vm, slot, value->as.r);
return; return calogOkE;
case calogStringE: case calogStringE:
wrenSetSlotBytes(vm, slot, value->as.s.bytes, (size_t)value->as.s.length); wrenSetSlotBytes(vm, slot, value->as.s.bytes, (size_t)value->as.s.length);
return; return calogOkE;
case calogAggE: { case calogAggE: {
CalogAggT *aggregate; CalogAggT *aggregate;
int64_t index; int64_t index;
int32_t status;
aggregate = value->as.agg; aggregate = value->as.agg;
// Anything keyed (or an explicit map) becomes a Wren Map with the sequence // Anything keyed (or an explicit map) becomes a Wren Map with the sequence
// part at numeric keys (a script reads user["name"]); a pure sequence // part at numeric keys (a script reads user["name"]); a pure sequence
// becomes a Wren List. slot+1 = key scratch, slot+2 = value scratch. // becomes a Wren List. slot+1 = key scratch, slot+2 = value scratch.
if (aggregate->pairCount > 0 || aggregate->kind == calogMapE) { if (calogAggIsKeyed(aggregate)) {
wrenSetSlotNewMap(vm, slot); wrenSetSlotNewMap(vm, slot);
wrenEnsureSlots(vm, slot + 3); wrenEnsureSlots(vm, slot + 3);
for (index = 0; index < aggregate->arrayCount; index++) { for (index = 0; index < aggregate->arrayCount; index++) {
wrenSetSlotDouble(vm, slot + 1, (double)index); wrenSetSlotDouble(vm, slot + 1, (double)index);
wrenFromValue(context, &aggregate->array[index], slot + 2, depth + 1); status = wrenFromValue(context, &aggregate->array[index], slot + 2, depth + 1);
if (status != calogOkE) {
return status;
}
wrenSetMapValue(vm, slot, slot + 1, slot + 2); wrenSetMapValue(vm, slot, slot + 1, slot + 2);
} }
for (index = 0; index < aggregate->pairCount; index++) { for (index = 0; index < aggregate->pairCount; index++) {
@ -459,18 +426,24 @@ static void wrenFromValue(CalogWrenT *context, const CalogValueT *value, int slo
} else { } else {
continue; // non-scalar key: drop the pair continue; // non-scalar key: drop the pair
} }
wrenFromValue(context, &aggregate->pairs[index].value, slot + 2, depth + 1); status = wrenFromValue(context, &aggregate->pairs[index].value, slot + 2, depth + 1);
if (status != calogOkE) {
return status;
}
wrenSetMapValue(vm, slot, slot + 1, slot + 2); wrenSetMapValue(vm, slot, slot + 1, slot + 2);
} }
} else { } else {
wrenSetSlotNewList(vm, slot); wrenSetSlotNewList(vm, slot);
wrenEnsureSlots(vm, slot + 2); wrenEnsureSlots(vm, slot + 2);
for (index = 0; index < aggregate->arrayCount; index++) { for (index = 0; index < aggregate->arrayCount; index++) {
wrenFromValue(context, &aggregate->array[index], slot + 1, depth + 1); status = wrenFromValue(context, &aggregate->array[index], slot + 1, depth + 1);
if (status != calogOkE) {
return status;
}
wrenInsertInList(vm, slot, -1, slot + 1); wrenInsertInList(vm, slot, -1, slot + 1);
} }
} }
return; return calogOkE;
} }
case calogFnE: { case calogFnE: {
// Wrap the foreign function in a CalogFn instance (script calls f.call([...])); // Wrap the foreign function in a CalogFn instance (script calls f.call([...]));
@ -481,10 +454,71 @@ static void wrenFromValue(CalogWrenT *context, const CalogValueT *value, int slo
data = (CalogFnT **)wrenSetSlotNewForeign(vm, slot, slot + 1, sizeof(CalogFnT *)); data = (CalogFnT **)wrenSetSlotNewForeign(vm, slot, slot + 1, sizeof(CalogFnT *));
*data = value->as.fn; *data = value->as.fn;
calogFnRetain(value->as.fn); calogFnRetain(value->as.fn);
return; return calogOkE;
} }
} }
wrenSetSlotNull(vm, slot); wrenSetSlotNull(vm, slot);
return calogOkE;
}
// Shared marshal/invoke/error skeleton for Calog.call(name, args) and CalogFn.call(args):
// calloc the argument buffer (aborting the fiber on OOM), marshal each list element to
// CalogValueT (freeing what was already marshalled and aborting on a bad element),
// invoke through the caller-supplied thunk, then marshal the result back into slot 0.
static void wrenInvokeShared(CalogWrenT *context, int listSlot, int scratchSlot, WrenInvokeFn invoke, void *target, const char *oomMsg, const char *marshalMsg, const char *failMsg) {
WrenVM *vm;
CalogValueT *cargs;
CalogValueT result;
int argCount;
int index;
int32_t status;
char message[CALOG_ERR_MSG_CAP];
vm = context->vm;
argCount = wrenGetListCount(vm, listSlot);
cargs = NULL;
if (argCount > 0) {
cargs = (CalogValueT *)calloc((size_t)argCount, sizeof(CalogValueT));
if (cargs == NULL) {
wrenSetSlotString(vm, 0, oomMsg);
wrenAbortFiber(vm, 0);
return;
}
}
wrenEnsureSlots(vm, scratchSlot + 1);
for (index = 0; index < argCount; index++) {
wrenGetListElement(vm, listSlot, index, scratchSlot);
status = wrenToValue(context, scratchSlot, &cargs[index], 0);
if (status != calogOkE) {
int cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&cargs[cleanup]);
}
free(cargs);
wrenSetSlotString(vm, 0, marshalMsg);
wrenAbortFiber(vm, 0);
return;
}
}
status = invoke(target, cargs, argCount, &result);
for (index = 0; index < argCount; index++) {
calogValueFree(&cargs[index]);
}
free(cargs);
if (status != calogOkE) {
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : failMsg);
calogValueFree(&result);
wrenSetSlotString(vm, 0, message);
wrenAbortFiber(vm, 0);
return;
}
status = wrenFromValue(context, &result, 0, 0); // return value in slot 0
calogValueFree(&result);
if (status != calogOkE) {
wrenSetSlotString(vm, 0, "return value nesting too deep");
wrenAbortFiber(vm, 0);
}
} }
@ -511,7 +545,7 @@ static int32_t wrenToValue(CalogWrenT *context, int slot, CalogValueT *out, int3
vm = context->vm; vm = context->vm;
calogValueNil(out); calogValueNil(out);
if (depth > CALOG_MAX_DEPTH) { if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE; return calogErrDepthE;
} }
switch (wrenGetSlotType(vm, slot)) { switch (wrenGetSlotType(vm, slot)) {
@ -523,11 +557,7 @@ static int32_t wrenToValue(CalogWrenT *context, int slot, CalogValueT *out, int3
case WREN_TYPE_NUM: { case WREN_TYPE_NUM: {
double number; double number;
number = wrenGetSlotDouble(vm, slot); number = wrenGetSlotDouble(vm, slot);
if (isfinite(number) && number == floor(number) && number >= -9223372036854775808.0 && number < 9223372036854775808.0) { calogValueFromDouble(out, number);
calogValueInt(out, (int64_t)number);
} else {
calogValueReal(out, number);
}
return calogOkE; return calogOkE;
} }
case WREN_TYPE_STRING: { case WREN_TYPE_STRING: {

View file

@ -34,6 +34,19 @@ Constraint: a callback must be a TOP-LEVEL def/lambda (persistent scope); a rout
local to a function would dangle once that function returns -- the same rule every local to a function would dangle once that function returns -- the same rule every
engine's closures follow. engine's closures follow.
Bare-name resolution for dynamic functions
- mb_dynamic_func_handler / mb_set_dynamic_func_handler(): a new hook (a field on
mb_interpreter_t plus its setter, declared in ourBasic.h). In _calc_expression,
where a bare identifier called like a function -- foo(args) -- that is not a
variable/routine/collection would raise "Invalid identifier usage", the interpreter
now first offers the (uppercased) name to the registered handler. The handler
returns MB_FUNC_OK (handled -- the result is left in the running intermediate
value), MB_FUNC_IGNORE (unknown -- the normal error is then raised), or an error
code; the pending argument list is left untouched on IGNORE. calog uses it to
resolve a bare name to a broker export, so exportedFn(args) works without an
explicit calogCall -- the same convenience the hook engines have. (The adapter
resolves case-insensitively, since my-basic uppercases identifiers at parse time.)
Verified: full calog `make test` (28 binaries, 0 failed) plus the my-basic engine Verified: full calog `make test` (28 binaries, 0 failed) plus the my-basic engine
suite (testMyBasic / testEngineMyBasic / testPolyglot / testLoad / testTask), suite (testMyBasic / testEngineMyBasic / testPolyglot / testLoad / testTask),
ASan/UBSan-clean, with def and lambda callbacks firing via timer/pubsub/export and ASan/UBSan-clean, with def and lambda callbacks firing via timer/pubsub/export and

View file

@ -911,6 +911,7 @@ typedef struct mb_interpreter_t {
mb_print_func_t printer; mb_print_func_t printer;
mb_input_func_t inputer; mb_input_func_t inputer;
mb_import_handler_t import_handler; mb_import_handler_t import_handler;
mb_dynamic_func_handler_t dynamic_func_handler; /* [calog fork] */
} mb_interpreter_t; } mb_interpreter_t;
/* Operations */ /* Operations */
@ -1328,7 +1329,7 @@ static void _resize_dynamic_buffer(_dynamic_buffer_t* buf, size_t es, size_t c);
#define _MB_READ_MEM_TAG_SIZE(t) (*((mb_mem_tag_t*)((char*)(t) - _MB_MEM_TAG_SIZE))) #define _MB_READ_MEM_TAG_SIZE(t) (*((mb_mem_tag_t*)((char*)(t) - _MB_MEM_TAG_SIZE)))
#ifdef MB_ENABLE_ALLOC_STAT #ifdef MB_ENABLE_ALLOC_STAT
/* calog patch (see myBasic.c.orig): _Atomic instead of volatile so the global /* calog patch (see ourBasic.c.upstream): _Atomic instead of volatile so the global
* allocation counter is safe when independent interpreters run on separate threads; * allocation counter is safe when independent interpreters run on separate threads;
* every += / -= / read below is then an atomic op. Lets calog run my-basic contexts in * every += / -= / read below is then an atomic op. Lets calog run my-basic contexts in
* parallel with only lifecycle (mb_init/mb_dispose) serialized. */ * parallel with only lifecycle (mb_init/mb_dispose) serialized. */
@ -4178,8 +4179,28 @@ _var:
} }
#endif /* MB_ENABLE_COLLECTION_LIB */ #endif /* MB_ENABLE_COLLECTION_LIB */
if(_IS_FUNC(_err_or_bracket, _core_open_bracket)) { if(_IS_FUNC(_err_or_bracket, _core_open_bracket)) {
int _dyn_hr = MB_FUNC_IGNORE;
if(s->dynamic_func_handler && _IS_VAR(c)) {
_ls_node_t* _dyn_saved = *l;
*l = ast->prev;
_dyn_hr = s->dynamic_func_handler(s, (void**)l, c->data.variable->name);
if(_dyn_hr == MB_FUNC_IGNORE)
*l = _dyn_saved;
}
if(_dyn_hr == MB_FUNC_OK) {
c = _create_object();
_LAZY_INIT_GLIST;
_ls_pushback(garbage, c);
_public_value_to_internal_object(&running->intermediate_value, c);
_REF(c)
ast = *l;
} else if(_dyn_hr != MB_FUNC_IGNORE) {
result = _dyn_hr;
goto _error;
} else {
_handle_error_on_obj(s, SE_RN_INVALID_ID_USAGE, s->source_file, DON(ast), MB_FUNC_ERR, _error, result); _handle_error_on_obj(s, SE_RN_INVALID_ID_USAGE, s->source_file, DON(ast), MB_FUNC_ERR, _error, result);
} }
}
} while(0); } while(0);
} }
} }
@ -14959,6 +14980,23 @@ _exit:
return result; return result;
} }
/* [calog fork] Set the resolver for a bare name called like a function that is otherwise
undefined (see mb_dynamic_func_handler_t). */
int mb_set_dynamic_func_handler(struct mb_interpreter_t* s, mb_dynamic_func_handler_t h) {
int result = MB_FUNC_OK;
if(!s) {
result = MB_FUNC_ERR;
goto _exit;
}
s->dynamic_func_handler = h;
_exit:
return result;
}
/* Register a string measurer globally */ /* Register a string measurer globally */
int mb_set_string_measurer(mb_string_measure_func_t m) { int mb_set_string_measurer(mb_string_measure_func_t m) {
_mb_strlen_func = m; _mb_strlen_func = m;
@ -19523,13 +19561,26 @@ _exit:
/* [calog fork] Invoke a routine value from an IDLE interpreter (no live call frame). /* [calog fork] Invoke a routine value from an IDLE interpreter (no live call frame).
Supplies the last AST node as a clean return landing so RETURN/ENDDEF have a valid Supplies the last AST node as a clean return landing so RETURN/ENDDEF have a valid
return point, and passes args directly. Used by calog to fire script callbacks return point, and passes args directly. Used by calog to fire script callbacks
(pubsub/timer/export) that were registered earlier. See myBasic.h. */ (pubsub/timer/export) that were registered earlier. See ourBasic.h. */
int mb_eval_routine_cold(struct mb_interpreter_t* s, mb_value_t routine, mb_value_t* args, unsigned argc, mb_value_t* ret) { int mb_eval_routine_cold(struct mb_interpreter_t* s, mb_value_t routine, mb_value_t* args, unsigned argc, mb_value_t* ret) {
_ls_node_t* n; _ls_node_t* mark;
void* land; void* land;
int result;
if(!s) return MB_FUNC_ERR; if(!s) return MB_FUNC_ERR;
n = s->ast; /* s->ast is a sentinel list whose prev always points at the tail (see
while(n && n->next) n = n->next; _ls_pushback), so this is O(1) instead of walking the whole program AST on
land = n; every callback dispatch. */
return mb_eval_routine(s, &land, routine, args, argc, ret); land = _ls_back(s->ast);
/* Remember the sub_stack depth before the call: every error exit inside
_eval_script_routine's push of this landing node (upstream) skips the pop that
only a successful RETURN/ENDDEF performs, which would otherwise leak one node
per failed callback on calog's long-lived interpreter. Trim back down to the
mark on any non-OK result. */
mark = _ls_back(s->sub_stack);
result = mb_eval_routine(s, &land, routine, args, argc, ret);
if(result != MB_FUNC_OK) {
while(_ls_back(s->sub_stack) != mark)
_ls_popback(s->sub_stack);
}
return result;
} }

View file

@ -579,6 +579,11 @@ typedef void (* mb_error_handler_t)(struct mb_interpreter_t*, mb_error_e, const
typedef int (* mb_print_func_t)(struct mb_interpreter_t*, const char*, ...); typedef int (* mb_print_func_t)(struct mb_interpreter_t*, const char*, ...);
typedef int (* mb_input_func_t)(struct mb_interpreter_t*, const char*, char*, int); typedef int (* mb_input_func_t)(struct mb_interpreter_t*, const char*, char*, int);
typedef int (* mb_import_handler_t)(struct mb_interpreter_t*, const char*); typedef int (* mb_import_handler_t)(struct mb_interpreter_t*, const char*);
/* [calog fork] resolver for a bare identifier used as a function call that is not a known
variable/routine/native (e.g. a dynamically-registered export). Returns MB_FUNC_OK if it
handled the call (result left in the running intermediate value), MB_FUNC_IGNORE if the name
is unknown to it (the interpreter then raises its normal error), or an error code. */
typedef int (* mb_dynamic_func_handler_t)(struct mb_interpreter_t*, void**, const char*);
typedef void (* mb_dtor_func_t)(struct mb_interpreter_t*, void*); typedef void (* mb_dtor_func_t)(struct mb_interpreter_t*, void*);
typedef void* (* mb_clone_func_t)(struct mb_interpreter_t*, void*); typedef void* (* mb_clone_func_t)(struct mb_interpreter_t*, void*);
typedef unsigned (* mb_hash_func_t)(struct mb_interpreter_t*, void*); typedef unsigned (* mb_hash_func_t)(struct mb_interpreter_t*, void*);
@ -690,6 +695,7 @@ MBAPI int mb_set_printer(struct mb_interpreter_t* s, mb_print_func_t p);
MBAPI int mb_set_inputer(struct mb_interpreter_t* s, mb_input_func_t p); MBAPI int mb_set_inputer(struct mb_interpreter_t* s, mb_input_func_t p);
MBAPI int mb_set_import_handler(struct mb_interpreter_t* s, mb_import_handler_t h); MBAPI int mb_set_import_handler(struct mb_interpreter_t* s, mb_import_handler_t h);
MBAPI int mb_set_dynamic_func_handler(struct mb_interpreter_t* s, mb_dynamic_func_handler_t h); /* [calog fork] */
MBAPI int mb_set_string_measurer(mb_string_measure_func_t m); MBAPI int mb_set_string_measurer(mb_string_measure_func_t m);
MBAPI int mb_set_memory_manager(mb_memory_allocate_func_t a, mb_memory_free_func_t f); MBAPI int mb_set_memory_manager(mb_memory_allocate_func_t a, mb_memory_free_func_t f);
MBAPI bool_t mb_get_gc_enabled(struct mb_interpreter_t* s); MBAPI bool_t mb_get_gc_enabled(struct mb_interpreter_t* s);