AI_Slop/calog
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

More libraries added. Janet added.

Browse source
This commit is contained in:
Scott Duensing 2026-07-05 19:21:29 -05:00
parent fa51d577ab
commit 7a5744f619
5022 changed files with 1482823 additions and 117 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
.gitignore vendored
View file

@ -11,6 +11,9 @@
/vendor/mruby/build/
/vendor/libssh2/build/
/vendor/tcl/build/
/vendor/xz/build/
/vendor/libarchive/_cmk/
/vendor/pcre2/build/
# ...and mruby's Rake drops a transient per-config lock next to the config (src/mruby/).
/src/mruby/*.lock
@ -33,3 +36,4 @@ core.*
.DS_Store
compile_commands.json
.cache/
/vendor/libxml2/build/

112
API.md
View file

@ -47,6 +47,51 @@ etc. -- see the README and `src/calog.h`.)
| `calogPrint(...values: any)` | Write each argument to stdout, space-separated, with a trailing newline. |
| `calogExit([code: int])` | Tear down the runtime and exit the process with `code` (default `0`). Does not return. |
## archive
Compression and archives over libarchive. All data is binary-safe. One-shot codecs work whole-buffer
in memory; archive read/write use an opaque integer handle. A per-transfer 64 MiB cap guards against a
decompression bomb.
| Function | Description |
| --- | --- |
| `compress(data: string, filter: string [, level: int]) -> string` | Compress `data`. `filter`: `"gzip"`, `"bzip2"`, `"xz"`, `"zstd"`, `"lz4"`, `"compress"`, `"none"`. |
| `decompress(data: string [, filter: string]) -> string` | Decompress; the filter is auto-detected from the magic bytes if omitted. |
| `archiveReadOpen(bytes: string) -> handle` | Open an archive held in memory. |
| `archiveReadOpenFile(path: string) -> handle` | Open an archive streamed from a file. |
| `archiveReadNext(handle) -> map \| nil` | Advance one entry: `{ name, size, mode, mtime, type[, linkname] }`; `nil` at the end. |
| `archiveReadData(handle) -> string` | The current entry's bytes (whole, capped). |
| `archiveReadClose(handle)` | Close and free the reader. |
| `archiveWriteOpen(format: string, filter: string [, level: int]) -> handle` | Start an in-memory archive. `format` is a container name (see Container formats below); `filter` compresses the whole archive, as for `compress`. |
| `archiveWriteEntry(handle, name: string, data: string [, opts: map])` | Add one entry. `opts`: `{ mode: int, mtime: int, type: "file"\|"dir" }`. |
| `archiveWriteFinish(handle) -> string` | Close and return the whole archive; frees the handle. |
**Container formats.** `archiveReadOpen`/`archiveReadOpenFile` auto-detect the format from the byte
stream, so extracting takes no format argument. `archiveWriteOpen` takes one of the names below.
Compression is orthogonal to the container: a `.tar.zst`, for example, is `format` `"tar"` with
`filter` `"zstd"`.
| Format | Read | Write | Notes |
| --- | --- | --- | --- |
| `tar` / `ustar` / `pax` / `gnutar` / `v7tar` | yes | yes | POSIX and GNU tar variants. |
| `cpio` (aliases `newc`, `odc`) | yes | yes | |
| `zip` | yes | yes | |
| `7zip` | yes | yes | |
| `iso9660` | yes | yes | ISO CD/DVD filesystem image. |
| `ar` (alias `arbsd`) | yes | yes | Unix `.a` archive. |
| `mtree` | yes | yes | File-hierarchy listing: entry metadata, no file contents. |
| `warc` | yes | yes | Web archive. |
| `xar` | yes | yes | Extensible archive (Apple `.pkg`); needs vendored libxml2 + OpenSSL. |
| `shar` | no | yes | Self-extracting shell archive. |
| `cab` | yes | no | Microsoft Cabinet; extract only. |
| `lha` / `lzh` | yes | no | LHA; extract only. |
| `rar` / `rar5` | yes | no | RAR is proprietary -- libarchive can extract but not create it. |
Formats marked write `no` are decode-only: the encoder is proprietary or absent. `xar` read and
write are backed by the vendored libxml2 (its TOC is XML) plus OpenSSL (its entry checksums are
MD5/SHA1); both are already in the build. A single unnamed stream can be written with `format`
`"raw"`, but for one-shot compression prefer `compress`/`decompress`.
## crypto
Binary-safe cryptographic primitives over OpenSSL.
@ -63,6 +108,16 @@ Binary-safe cryptographic primitives over OpenSSL.
| `cryptoHexDecode(hexText: string) -> string` | Hex decode (case-insensitive). |
| `cryptoUuid() -> string` | A random RFC 4122 version-4 UUID string. |
## csv
Parse and format RFC 4180 CSV. Cells are strings; quoted cells may contain the delimiter, newlines,
and doubled quotes. Binary-safe.
| Function | Description |
| --- | --- |
| `csvParse(text: string [, delimiter: string]) -> list(list(string))` | Parse CSV into rows of string cells. The optional delimiter is one byte (default `,`). |
| `csvFormat(rows: list(list) [, delimiter: string]) -> string` | Format rows of cells (string/int/real/bool/nil) into CSV (records joined by CRLF); cells with the delimiter, a quote, or a newline are quoted. |
## db
SQL over SQLite, PostgreSQL, and MySQL/MariaDB. Parameters are always **bound** (never
@ -84,7 +139,7 @@ Share a function by name across contexts and engines.
|---|---|
| `calogExport(name: string, fn: fn)` | Publish function `fn` under a global `name`. |
| `calogUnexport(name: string)` | Remove an exported name. |
| `calogCall(name: string, ...args: any) -> any` | Call an exported function by name -- works in **every** engine. (An export is also reachable by its bare name -- `exportedFn(args)` -- on Lua/JS/Squirrel/s7, on Ruby at top level (via `method_missing`), on Tcl (via its `unknown` handler), and on my-basic case-insensitively (it uppercases identifiers); Wren and Berry always need `calogCall`.) |
| `calogCall(name: string, ...args: any) -> any` | Call an exported function by name -- works in **every** engine. (An export is also reachable by its bare name -- `exportedFn(args)` -- on Lua/JS/Squirrel/s7, on Ruby at top level (via `method_missing`), on Tcl (via its `unknown` handler), and on my-basic case-insensitively (it uppercases identifiers); Wren, Berry, and Janet always need `calogCall`.) |
## fs
@ -113,6 +168,23 @@ verifies the server certificate against the system CA store by default.
| `httpGet(url: string) -> map` | GET a URL, following redirects. Returns `{status, body, headers}` (headers keyed by lowercased name). |
| `httpRequest(opts: map) -> map` | `opts` is `{method (default "GET"), url, headers (map), body, insecure (bool), maxRedirects (int, default 16; 0 = don't follow)}`. `insecure=true` skips TLS verification. Returns `{status, body, headers}`. |
## httpd
A polyglot HTTP server: route handlers are function values (in any engine), invoked on their owning
context. Re-registering a route replaces the handler live (hot reload). v1 serves HTTP/1.1 with
`Connection: close`, one request in flight at a time; TLS and WebSocket are not yet included.
| Function | Description |
| --- | --- |
| `httpdListen(port [, opts]) -> serverHandle` | Bind + listen; starts the acceptor. `opts`: `{ host, backlog, maxBody }`. |
| `httpdRoute(serverHandle, method, path, handler)` | Register a handler. `method` `"*"` = any; exact-path match; re-registering replaces (hot reload). |
| `httpdUnroute(serverHandle, method, path)` | Remove a route. |
| `httpdStop(serverHandle)` | Stop accepting, close, release handlers. |
The handler receives a request map `{ method, path, query, headers (map, lowercased names), body }`
and returns a response: a map `{ status (default 200), headers (map), body (string) }`, a bare string
(=> 200 with that body), or nil (=> 204). Bodies are binary-safe.
## json
| Function | Description |
@ -165,6 +237,14 @@ ENet (reliable UDP -- ordered, reliable channels over UDP):
| `enetDisconnect(peerHandle: handle)` | Begin disconnecting a peer. |
| `enetClose(hostHandle: handle)` | Destroy an ENet host. |
## proc
Run a subprocess and capture its output (POSIX only; started with `posix_spawn`, not `fork`).
| Function | Description |
| --- | --- |
| `procRun(argv: list(string) [, opts: map]) -> map` | Spawn `argv[0]`, wait, and return `{ exit: int, stdout: string, stderr: string }` (exit is the negative signal number if killed). `opts`: `{ stdin: string, cwd: string, env: map (string->string; replaces the environment) }`. Blocks the caller's context thread; stdin is fed while stdout/stderr are drained, so a large transfer cannot deadlock. |
## pubsub
Deliver a message to every subscriber of a topic, across contexts and engines. Delivery is
@ -177,6 +257,19 @@ message. Keep publish graphs acyclic.
| `psUnsubscribe(id: int)` | Drop the subscription with that id. |
| `psPublish(topic: string, msg: any) -> int` | Deliver a copy of `msg` to every subscriber; returns how many were invoked. |
## regex
Perl-compatible regular expressions over PCRE2. `flags` is a string of single letters: `i`
case-insensitive, `m` multiline, `s` dot-matches-newline, `x` extended, `g` replace-all (for
`regexReplace`). A non-participating capture group is `nil`. Binary-safe.
| Function | Description |
| --- | --- |
| `regexMatch(pattern, subject [, flags]) -> nil \| list` | First match, as `[fullMatch, group1, group2, ...]`; `nil` if no match. |
| `regexSearch(pattern, subject [, flags]) -> nil \| map` | First match, as `{ match, start, end, groups: list }` (byte offsets); `nil` if no match. |
| `regexReplace(pattern, subject, replacement [, flags]) -> string` | Replace matches; `replacement` uses `$1` / `${name}`. The `g` flag replaces all. |
| `regexSplit(pattern, subject [, flags]) -> list(string)` | Split `subject` on matches of `pattern`. |
## ssh
SSH/SFTP over libssh2. Requires a reachable SSH server. Payloads are binary-safe. A connection
@ -208,7 +301,7 @@ out, or the close blocks until the task next returns to the runtime.
| Function | Description |
|---|---|
| `taskSpawn(engine: string, code: string) -> handle` | Run a code string on a named engine (`"lua"`, `"js"`, `"squirrel"`, `"mybasic"`, `"berry"`, `"s7"`, `"wren"`, `"mruby"`, `"tcl"`). |
| `taskSpawn(engine: string, code: string) -> handle` | Run a code string on a named engine (`"lua"`, `"js"`, `"squirrel"`, `"mybasic"`, `"berry"`, `"s7"`, `"wren"`, `"mruby"`, `"tcl"`, `"janet"`). |
| `taskLoad(baseName: string) -> handle` | Launch a script *file* (engine chosen by extension). |
| `taskEval(handle: handle, code: string)` | Feed more code into a running task (runs on its thread). |
| `taskClose(handle: handle)` | Ask a task to stop (cooperative: sets its shutdown flag, then waits for its thread to exit). |
@ -236,3 +329,18 @@ timer.
| `timerAfter(ms: int, fn: fn) -> int` | Fire `fn` once, `ms` milliseconds from now; returns a timer id. |
| `timerEvery(ms: int, fn: fn) -> int` | Fire `fn` every `ms` milliseconds; returns a timer id. |
| `timerCancel(id: int)` | Stop a pending or repeating timer. |
## xml
Parse and produce XML over vendored libxml2. An element node is a map `{ tag, attr, children }`:
`tag` is the element name; `attr` (present only when the element has attributes) maps attribute names
to string values; and `children` is an ordered list whose items are child element maps or plain
strings (text nodes). Mixed content and all character data are preserved, so `xmlParse` then
`xmlStringify` round-trips the element structure; comments, processing instructions, and the XML
declaration are dropped. Parsing loads no external entities and does no network I/O (no XXE), and
nesting is bounded by the same maximum depth as `json`. Marshalling is binary-safe (UTF-8).
| Function | Description |
|---|---|
| `xmlParse(text: string) -> map` | Parse an XML document; returns its root element node. Malformed XML raises a catchable error. |
| `xmlStringify(node: map) -> string` | Serialize an element node (the shape `xmlParse` returns) back to XML text. |

38
LICENSE.md
View file

@ -70,6 +70,9 @@ libraries below.
`vendor/mruby/` (unmodified upstream; license in `vendor/mruby/LICENSE`, contributors
in `vendor/mruby/AUTHORS`). `libmruby.a` is generated by mruby's own Rake build (needs a
host Ruby + bison at build time); calog links the resulting static archive.
- **Janet 1.41.2** (Lisp) -- MIT License. Copyright (c) 2017-2025 Calvin Rose and
contributors. `vendor/janet/` (single-file amalgamation `janet.c`/`janet.h` +
`janetconf.h`, built from source like s7/Wren). `vendor/janet/LICENSE`.
- **Tcl 9.0.4** (Tool Command Language) -- Tcl/Tk License (BSD-style, permissive; allows
binary-only redistribution, no copyleft). Copyright (c) the Tcl Core Team and contributors.
`vendor/tcl/` (unmodified upstream, trimmed to the core: `generic`/`unix`/`compat`/
@ -106,6 +109,41 @@ All of the following are linked into the default build and into `bin/calog`.
Stenberg, Sara Golemon, The Written Word, Inc., and other contributors (full
list in the file). `vendor/libssh2/COPYING`.
### Compression and archive libraries
The archive library (`libs/calogArchive.c`) is built on libarchive with the codec
backends below. **All are permissive** (BSD / zlib / 0BSD / MIT), so this whole stack keeps
an all-permissive redistributable binary. Each is vendored and built from source; see
each `vendor/<lib>/NOTICE.calog` for what was kept. (libarchive's XAR format also uses the
vendored OpenSSL for its MD5/SHA1 entry checksums; OpenSSL is listed above.)
- **libarchive 3.8.8** -- BSD-2-Clause License. Copyright (c) the libarchive
contributors. `vendor/libarchive/COPYING`.
- **zlib 1.3.1** (DEFLATE / gzip) -- zlib License. Copyright (c) 1995-2024 Jean-loup
Gailly and Mark Adler. `vendor/zlib`. Built to a single `lib/libz.a` used by the
archive stack.
- **bzip2 1.0.8** -- bzip2 License (BSD-style). Copyright (c) 1996-2019 Julian Seward.
`vendor/bzip2/LICENSE`.
- **LZ4 1.10.0** -- BSD-2-Clause License. Copyright (c) 2011-2024 Yann Collet.
`vendor/lz4/LICENSE`.
- **Zstandard 1.5.6** -- BSD-3-Clause License (zstd is dual BSD-3 / GPLv2; the
permissive BSD option applies here). Copyright (c) Meta Platforms, Inc. and
affiliates. `vendor/zstd/LICENSE`.
- **XZ Utils / liblzma 5.6.4** (xz/lzma, and libarchive's 7-zip) -- BSD Zero Clause
License (0BSD), public-domain-equivalent. `vendor/xz`.
- **libxml2 2.13.9** -- MIT License. Copyright (C) 1998-2012 Daniel Veillard. The XML
backend for libarchive's XAR format (read + write) and for the XML library
(`libs/calogXml.c`). Vendored and built static and lean (no python/programs/http/icu/
modules). `vendor/libxml2/Copyright`.
### Text and process utilities
- **PCRE2 10.45** (Perl-compatible regular expressions) -- BSD-3-Clause License with the
PCRE2 exception (permissive; the exception concerns only binary redistribution of the JIT,
which calog does not build). The regex backend for `libs/calogRegex.c`; built 8-bit, static,
no JIT. Copyright (c) 1997-2024 University of Cambridge and Zoltan Herczeg.
`vendor/pcre2/LICENCE.md`. (The CSV and subprocess libraries have no third-party dependency.)
> **Note on the database backends and LGPL.** MariaDB Connector/C is the only
> copyleft dependency in the tree. It is **statically linked** into `bin/calog`,
> which carries the usual LGPL-2.1 obligations for a distributed binary (chiefly,

221
Makefile
View file

@ -27,8 +27,8 @@ LDFLAGS = $(SAN)
# Our headers: src/ plus the per-language subdirs. VPATH lets the object rules find
# a source by name without spelling out its directory.
INC = -Isrc -Isrc/lua -Isrc/mybasic -Isrc/squirrel -Isrc/js -Isrc/berry -Isrc/s7 -Isrc/wren -Isrc/mruby -Isrc/tcl -Ilibs
VPATH = src:src/lua:src/mybasic:src/squirrel:src/js:src/berry:src/s7:src/wren:src/mruby:src/tcl:libs:tests
INC = -Isrc -Isrc/lua -Isrc/mybasic -Isrc/squirrel -Isrc/js -Isrc/berry -Isrc/s7 -Isrc/wren -Isrc/mruby -Isrc/tcl -Isrc/janet -Ilibs
VPATH = src:src/lua:src/mybasic:src/squirrel:src/js:src/berry:src/s7:src/wren:src/mruby:src/tcl:src/janet:libs:tests
# --- vendored our-basic: calog's fork of MY-BASIC (see vendor/ourbasic/NOTICE) ---
MBDIR = vendor/ourbasic
@ -141,6 +141,16 @@ WRENFLAGS = -std=c99 -w -g -O1
WRENOBJ = obj/wren.o
WRENLIBS = -lm
# --- vendored Janet 1.41.2 (Lisp, single-file amalgamation like s7/wren: janet.c + janet.h +
# janetconf.h). Its whole state is a JANET_THREAD_LOCAL janet_vm (__thread on gcc), so one
# independent VM per context thread -- the actor model exactly. Lean build: no net/process/dynamic-
# modules (calog provides IO). Links -lm -lpthread. MIT license. ---
JANETDIR = vendor/janet
JANETINC = -I$(JANETDIR)
JANETFLAGS = -std=c11 -w -g -O2 -DJANET_NO_NET -DJANET_NO_PROCESSES -DJANET_NO_DYNAMIC_MODULES
JANETOBJ = obj/janet.o
JANETLIBS = -lm -lpthread
# --- vendored mruby 4.0.0 (Ruby). UNLIKE every other engine, libmruby.a is GENERATED by mruby's
# own Rake build (needs a host Ruby >=2.5 + bison), not compiled from .c here. The embed gembox
# is the core language + runtime eval (mruby-compiler) + stdlib/math/metaprog, with no executables
@ -206,21 +216,143 @@ obj/wpth_%.o: $(WPTHDIR)/src/%.c | obj
$(WPTHLIB): $(WPTHOBJ) | lib
$(AR) rcs $@ $(WPTHOBJ)
BINS = bin/testBroker bin/testLua bin/testMyBasic bin/testPolyglot bin/testActor \
# --- vendored zlib -> a single lib/libz.a: the one source of truth for DEFLATE in the tree, and
# the gzip/zip backend for the archive/compression library. No configure step -- the shipped
# zconf.h is portable. Built from source like every dependency (permissive zlib license). ---
ZLIBDIR = vendor/zlib
ZLIBINC = -I$(ZLIBDIR)
ZLIBSRC = $(wildcard $(ZLIBDIR)/*.c)
ZLIBOBJ = $(patsubst $(ZLIBDIR)/%.c,obj/z_%.o,$(ZLIBSRC))
ZLIBLIB = lib/libz.a
obj/z_%.o: $(ZLIBDIR)/%.c | obj
$(CC) -std=c11 -w -O2 $(ZLIBINC) -c -o $@ $<
$(ZLIBLIB): $(ZLIBOBJ) | lib
$(AR) rcs $@ $(ZLIBOBJ)
# --- vendored bzip2 -> lib/libbz2.a (bzip2 filter backend). Pure object-rules, no configure. ---
BZIP2DIR = vendor/bzip2
BZIP2INC = -I$(BZIP2DIR)
BZIP2SRC = $(wildcard $(BZIP2DIR)/*.c)
BZIP2OBJ = $(patsubst $(BZIP2DIR)/%.c,obj/bz2_%.o,$(BZIP2SRC))
BZIP2LIB = lib/libbz2.a
obj/bz2_%.o: $(BZIP2DIR)/%.c | obj
$(CC) -std=c11 -w -O2 $(BZIP2INC) -c -o $@ $<
$(BZIP2LIB): $(BZIP2OBJ) | lib
$(AR) rcs $@ $(BZIP2OBJ)
# --- vendored lz4 -> lib/liblz4.a (lz4 filter backend; needs lz4.h + lz4frame.h). Object-rules. ---
LZ4DIR = vendor/lz4
LZ4INC = -I$(LZ4DIR)
LZ4SRC = $(wildcard $(LZ4DIR)/*.c)
LZ4OBJ = $(patsubst $(LZ4DIR)/%.c,obj/l4_%.o,$(LZ4SRC))
LZ4LIB = lib/liblz4.a
obj/l4_%.o: $(LZ4DIR)/%.c | obj
$(CC) -std=c11 -w -O2 $(LZ4INC) -c -o $@ $<
$(LZ4LIB): $(LZ4OBJ) | lib
$(AR) rcs $@ $(LZ4OBJ)
# --- vendored zstd -> lib/libzstd.a (zstd filter backend). Object-rules across its lib subdirs.
# ZSTD_DISABLE_ASM: the x86-64 asm decode path is not vendored. MULTITHREAD left UNSET (single-
# threaded encoder -> tsan-clean; defining it even to 0 would enable worker threads). ---
ZSTDDIR = vendor/zstd
ZSTDINC = -I$(ZSTDDIR)
ZSTDOBJ = $(patsubst $(ZSTDDIR)/common/%.c,obj/zstd_%.o,$(wildcard $(ZSTDDIR)/common/*.c)) $(patsubst $(ZSTDDIR)/compress/%.c,obj/zstd_%.o,$(wildcard $(ZSTDDIR)/compress/*.c)) $(patsubst $(ZSTDDIR)/decompress/%.c,obj/zstd_%.o,$(wildcard $(ZSTDDIR)/decompress/*.c))
ZSTDLIB = lib/libzstd.a
obj/zstd_%.o: $(ZSTDDIR)/common/%.c | obj
$(CC) -std=c11 -w -O2 -DZSTD_DISABLE_ASM $(ZSTDINC) -c -o $@ $<
obj/zstd_%.o: $(ZSTDDIR)/compress/%.c | obj
$(CC) -std=c11 -w -O2 -DZSTD_DISABLE_ASM $(ZSTDINC) -c -o $@ $<
obj/zstd_%.o: $(ZSTDDIR)/decompress/%.c | obj
$(CC) -std=c11 -w -O2 -DZSTD_DISABLE_ASM $(ZSTDINC) -c -o $@ $<
$(ZSTDLIB): $(ZSTDOBJ) | lib
$(AR) rcs $@ $(ZSTDOBJ)
# --- vendored XZ Utils / liblzma -> vendor/xz/build/liblzma.a (xz/lzma filter + 7-zip backend).
# Like Tcl/libssh2, produced by the codec's own CMake OUT OF TREE (source stays pristine); only the
# liblzma target is built (never the xz CLI). ENABLE_THREADS=OFF -> single-threaded, tsan-clean. ---
XZDIR = vendor/xz
XZINC = -I$(XZDIR)/src/liblzma/api
XZLIB = $(XZDIR)/build/liblzma.a
$(XZLIB):
cmake -S $(XZDIR) -B $(XZDIR)/build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
-DENABLE_THREADS=OFF -DENABLE_NLS=OFF -DENABLE_DOXYGEN=OFF >/dev/null
cmake --build $(XZDIR)/build --target liblzma -j >/dev/null
# --- vendored libxml2 -> vendor/libxml2/build/libxml2.a. The XML backend for libarchive's XAR
# format (read AND write) AND for libs/calogXml.c. XAR additionally needs OpenSSL (MD5/SHA1, already
# vendored) and glibc iconv (ENABLE_ICONV in the libarchive block below). Its own CMake OUT OF TREE
# (source pristine); static and lean (no python/programs/http/icu/modules; UTF-8 needs no iconv). The
# generated xmlversion.h lands in build/, the rest of the API in include/, so both are on the path. ---
LIBXML2DIR = vendor/libxml2
LIBXML2INC = -I$(LIBXML2DIR)/include -I$(LIBXML2DIR)/build
LIBXML2LIB = $(LIBXML2DIR)/build/libxml2.a
$(LIBXML2LIB):
cmake -S $(LIBXML2DIR) -B $(LIBXML2DIR)/build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
-DLIBXML2_WITH_PYTHON=OFF -DLIBXML2_WITH_PROGRAMS=OFF -DLIBXML2_WITH_TESTS=OFF \
-DLIBXML2_WITH_HTTP=OFF -DLIBXML2_WITH_FTP=OFF -DLIBXML2_WITH_ICU=OFF \
-DLIBXML2_WITH_LZMA=OFF -DLIBXML2_WITH_ZLIB=OFF -DLIBXML2_WITH_ICONV=OFF \
-DLIBXML2_WITH_MODULES=OFF -DLIBXML2_WITH_CATALOG=OFF -DLIBXML2_WITH_THREADS=ON >/dev/null
cmake --build $(LIBXML2DIR)/build --target LibXml2 -j >/dev/null
# --- vendored libarchive -> the compression/archive backend for libs/calogArchive.c. Built by its
# own CMake OUT OF TREE (like libssh2), static, with each codec pinned to our vendored copy so no
# system codec is ever picked up. CLI tools + tests disabled. XAR (read + write) is enabled via the
# vendored libxml2, vendored OpenSSL (its required MD5/SHA1), and glibc iconv (ENABLE_ICONV -- which
# libarchive gates the libxml2 detection on). LIBXML2_INCLUDE_DIR is a two-entry list: the source API
# headers plus the build dir holding the generated xmlversion.h. ---
LIBARCHIVEDIR = vendor/libarchive
LIBARCHIVEINC = -I$(LIBARCHIVEDIR)/libarchive
LIBARCHIVELIB = $(LIBARCHIVEDIR)/_cmk/libarchive/libarchive.a
$(LIBARCHIVELIB): $(ZLIBLIB) $(BZIP2LIB) $(LZ4LIB) $(ZSTDLIB) $(XZLIB) $(LIBXML2LIB)
cmake -S $(LIBARCHIVEDIR) -B $(LIBARCHIVEDIR)/_cmk -DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=OFF -DENABLE_TEST=OFF -DENABLE_INSTALL=OFF \
-DENABLE_TAR=OFF -DENABLE_CPIO=OFF -DENABLE_CAT=OFF -DENABLE_UNZIP=OFF \
-DENABLE_ACL=OFF -DENABLE_XATTR=OFF -DENABLE_ICONV=ON -DENABLE_LIBB2=OFF \
-DENABLE_LIBXML2=ON "-DLIBXML2_INCLUDE_DIR=$(CURDIR)/vendor/libxml2/include;$(CURDIR)/vendor/libxml2/build" -DLIBXML2_LIBRARY=$(CURDIR)/$(LIBXML2LIB) \
-DENABLE_OPENSSL=ON -DOPENSSL_ROOT_DIR=$(CURDIR)/vendor/openssl -DOPENSSL_INCLUDE_DIR=$(CURDIR)/vendor/openssl/include \
-DOPENSSL_SSL_LIBRARY=$(CURDIR)/vendor/openssl/libssl.a -DOPENSSL_CRYPTO_LIBRARY=$(CURDIR)/vendor/openssl/libcrypto.a \
-DENABLE_LZO=OFF -DENABLE_NETTLE=OFF -DENABLE_MBEDTLS=OFF -DENABLE_PCREPOSIX=OFF -DENABLE_PCRE2POSIX=OFF \
-DENABLE_ZLIB=ON -DZLIB_INCLUDE_DIR=$(CURDIR)/vendor/zlib -DZLIB_LIBRARY=$(CURDIR)/$(ZLIBLIB) \
-DENABLE_BZip2=ON -DBZIP2_INCLUDE_DIR=$(CURDIR)/vendor/bzip2 -DBZIP2_LIBRARIES=$(CURDIR)/$(BZIP2LIB) \
-DENABLE_LZMA=ON -DLIBLZMA_INCLUDE_DIR=$(CURDIR)/vendor/xz/src/liblzma/api -DLIBLZMA_LIBRARY=$(CURDIR)/$(XZLIB) \
-DENABLE_ZSTD=ON -DZSTD_INCLUDE_DIR=$(CURDIR)/vendor/zstd -DZSTD_LIBRARY=$(CURDIR)/$(ZSTDLIB) \
-DENABLE_LZ4=ON -DLZ4_INCLUDE_DIR=$(CURDIR)/vendor/lz4 -DLZ4_LIBRARY=$(CURDIR)/$(LZ4LIB) >/dev/null
cmake --build $(LIBARCHIVEDIR)/_cmk --target archive_static -j >/dev/null
# The whole compression stack for the archive library link line.
# libarchive first (references the rest), then libxml2 + codecs; XAR's MD5/SHA1 resolve against
# $(SSLARCH), which the archive-linked binaries add after this group.
ARCHIVELIBS = $(LIBARCHIVELIB) $(LIBXML2LIB) $(XZLIB) $(ZSTDLIB) $(LZ4LIB) $(BZIP2LIB) $(ZLIBLIB)
# --- vendored PCRE2 10.45 (Perl-compatible regex) -> vendor/pcre2/build/libpcre2-8.a, the regex
# backend for libs/calogRegex.c. Like Tcl/XZ, produced by PCRE2's own CMake OUT OF TREE: pcre2.h +
# config.h are GENERATED from .in templates (baking in version/config), so the source stays pristine
# and the generated pcre2.h a consumer includes is under build/. 8-bit width only, static, NO JIT
# (JIT needs W^X executable memory calog does not want). A consumer defines PCRE2_CODE_UNIT_WIDTH 8
# before <pcre2.h>. License: BSD-3-Clause with exception. ---
PCRE2DIR = vendor/pcre2
PCRE2INC = -I$(PCRE2DIR)/build
PCRE2LIB = $(PCRE2DIR)/build/libpcre2-8.a
$(PCRE2LIB):
cmake -S $(PCRE2DIR) -B $(PCRE2DIR)/build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
-DPCRE2_BUILD_PCRE2_8=ON -DPCRE2_BUILD_PCRE2_16=OFF -DPCRE2_BUILD_PCRE2_32=OFF \
-DPCRE2_SUPPORT_JIT=OFF -DPCRE2_BUILD_TESTS=OFF -DPCRE2_BUILD_PCRE2GREP=OFF >/dev/null
cmake --build $(PCRE2DIR)/build --target pcre2-8-static -j >/dev/null
BINS = bin/testBroker bin/testLua bin/testMyBasic bin/testPolyglot bin/testActor bin/testHooks \
bin/testEngineLua bin/testEngineMyBasic bin/testSquirrel bin/testEngineSquirrel bin/testJs bin/testEngineJs \
bin/testEngineBerry bin/testEngineS7 bin/testEngineWren bin/testEngineMruby bin/testEngineTcl bin/testLoad bin/testDb bin/testNet bin/testTask bin/testExport bin/testJson bin/testTime bin/testFs bin/testCrypto bin/testKv bin/testTimer bin/testPubsub bin/testHttp bin/testHttps bin/embed bin/calog
bin/testEngineBerry bin/testEngineS7 bin/testEngineWren bin/testEngineMruby bin/testEngineTcl bin/testEngineJanet bin/testLoad bin/testDb bin/testNet bin/testTask bin/testExport bin/testJson bin/testXml bin/testTime bin/testFs bin/testCrypto bin/testKv bin/testTimer bin/testPubsub bin/testHttp bin/testHttps bin/testArchive bin/testUtil bin/testSandbox bin/testTrace bin/testHttpd bin/embed bin/calog
all: $(BINS)
# ---- object rules, grouped by flag set (sources resolved via VPATH) ----
# strict C, no threads
STRICTOBJ = obj/broker.o obj/value.o obj/luaEngine.o obj/squirrelEngine.o obj/jsEngine.o obj/berryEngine.o obj/s7Engine.o obj/wrenEngine.o obj/mrubyEngine.o obj/tclEngine.o obj/testBroker.o
STRICTOBJ = obj/broker.o obj/value.o obj/luaEngine.o obj/squirrelEngine.o obj/jsEngine.o obj/berryEngine.o obj/s7Engine.o obj/wrenEngine.o obj/mrubyEngine.o obj/tclEngine.o obj/janetEngine.o obj/testBroker.o
$(STRICTOBJ): obj/%.o: %.c | obj
$(CC) $(COREFLAGS) $(INC) -c -o $@ $<
# strict C, threaded
THREADOBJ = obj/context.o obj/mybasicEngine.o obj/testActor.o obj/testEngineLua.o obj/testEngineSquirrel.o obj/testEngineJs.o obj/testEngineMyBasic.o obj/testEngineBerry.o obj/testEngineS7.o obj/testEngineWren.o obj/testEngineMruby.o obj/testEngineTcl.o obj/calogHandle.o obj/testDb.o obj/testNet.o obj/testTask.o obj/calogExport.o obj/testExport.o obj/calogJson.o obj/testJson.o obj/calogFs.o obj/testFs.o obj/calogTime.o obj/testTime.o obj/calogKv.o obj/testKv.o obj/testCrypto.o obj/calogTimer.o obj/testTimer.o obj/calogPubsub.o obj/testPubsub.o obj/testHttp.o obj/testSsh.o
THREADOBJ = obj/context.o obj/mybasicEngine.o obj/testActor.o obj/testEngineLua.o obj/testEngineSquirrel.o obj/testEngineJs.o obj/testEngineMyBasic.o obj/testEngineBerry.o obj/testEngineS7.o obj/testEngineWren.o obj/testEngineMruby.o obj/testEngineTcl.o obj/testEngineJanet.o obj/calogHandle.o obj/testDb.o obj/testNet.o obj/testTask.o obj/calogExport.o obj/testExport.o obj/calogJson.o obj/testJson.o obj/calogFs.o obj/testFs.o obj/calogTime.o obj/testTime.o obj/calogKv.o obj/testKv.o obj/testCrypto.o obj/calogTimer.o obj/testTimer.o obj/calogPubsub.o obj/testPubsub.o obj/testHttp.o obj/testSsh.o obj/calogCsv.o obj/calogProc.o obj/testSandbox.o obj/testTrace.o obj/calogHttpd.o obj/testHttpd.o obj/testHooks.o
$(THREADOBJ): obj/%.o: %.c | obj
$(CC) $(COREFLAGS) $(INC) -pthread -c -o $@ $<
@ -243,6 +375,21 @@ $(LIBSSH2LIB): | $(LIBSSH2DIR)
obj/calogSsh.o: calogSsh.c | obj
$(CC) $(COREFLAGS) $(INC) $(LIBSSH2INC) -pthread -c -o $@ $<
# calogArchive needs the vendored libarchive headers (outside $(INC)).
obj/calogArchive.o obj/testArchive.o: obj/%.o: %.c | obj
$(CC) $(COREFLAGS) $(INC) $(LIBARCHIVEINC) -pthread -c -o $@ $<
# calogXml needs the vendored libxml2 headers (outside $(INC)): the source API plus the build dir
# with the GENERATED xmlversion.h, so $(LIBXML2LIB) (which runs the CMake that generates it) is an
# order-only prerequisite -- like PCRE2 below, whose pcre2.h is likewise generated.
obj/calogXml.o obj/testXml.o: obj/%.o: %.c | obj $(LIBXML2LIB)
$(CC) $(COREFLAGS) $(INC) $(LIBXML2INC) -pthread -c -o $@ $<
# calogRegex needs the vendored (generated) PCRE2 header under vendor/pcre2/build, so the PCRE2
# build (which generates pcre2.h) is an order-only prerequisite.
obj/calogRegex.o obj/testUtil.o: obj/%.o: %.c | obj $(PCRE2LIB)
$(CC) $(COREFLAGS) $(INC) $(PCRE2INC) -pthread -c -o $@ $<
# relaxed C + Lua headers
LUAADP = obj/luaAdapter.o obj/testLua.o
$(LUAADP): obj/%.o: %.c | obj
@ -273,6 +420,10 @@ S7ADP = obj/s7Adapter.o
$(S7ADP): obj/%.o: %.c | obj
$(CC) $(ADPFLAGS) $(INC) $(S7INC) -c -o $@ $<
JANETADP = obj/janetAdapter.o
$(JANETADP): obj/%.o: %.c | obj
$(CC) $(ADPFLAGS) $(INC) $(JANETINC) -c -o $@ $<
# relaxed C + Wren headers
WRENADP = obj/wrenAdapter.o
$(WRENADP): obj/%.o: %.c | obj
@ -302,7 +453,7 @@ $(NETADP): obj/%.o: %.c | obj
# task library: strict C with every engine name compiled in (it references each engine
# vtable under its CALOG_WITH_* guard), so a task-using binary links all engine archives.
ENGINEDEFS = -DCALOG_WITH_LUA -DCALOG_WITH_JS -DCALOG_WITH_SQUIRREL -DCALOG_WITH_MYBASIC -DCALOG_WITH_BERRY -DCALOG_WITH_S7 -DCALOG_WITH_WREN -DCALOG_WITH_MRUBY -DCALOG_WITH_TCL
ENGINEDEFS = -DCALOG_WITH_LUA -DCALOG_WITH_JS -DCALOG_WITH_SQUIRREL -DCALOG_WITH_MYBASIC -DCALOG_WITH_BERRY -DCALOG_WITH_S7 -DCALOG_WITH_WREN -DCALOG_WITH_MRUBY -DCALOG_WITH_TCL -DCALOG_WITH_JANET
TASKADP = obj/calogTask.o
$(TASKADP): obj/%.o: %.c | obj
$(CC) $(COREFLAGS) $(INC) $(ENGINEDEFS) -pthread -c -o $@ $<
@ -341,6 +492,9 @@ obj/s7.o: $(S7DIR)/s7.c | obj
obj/wren.o: $(WRENDIR)/wren.c | obj
$(CC) $(WRENFLAGS) $(WRENINC) -c -o $@ $<
obj/janet.o: $(JANETDIR)/janet.c | obj
$(CC) $(JANETFLAGS) $(JANETINC) -c -o $@ $<
obj/sqlite3.o: $(SQLITEDIR)/sqlite3.c | obj
$(CC) $(SQLITEFLAGS) $(SQLITEINC) -c -o $@ $<
@ -356,7 +510,7 @@ CALOGLIB = obj/value.o obj/broker.o obj/context.o \
obj/luaAdapter.o obj/luaEngine.o obj/jsAdapter.o obj/jsEngine.o \
obj/squirrelAdapter.o obj/squirrelEngine.o obj/mybasicAdapter.o obj/mybasicEngine.o \
obj/berryAdapter.o obj/berryEngine.o obj/s7Adapter.o obj/s7Engine.o obj/wrenAdapter.o obj/wrenEngine.o \
obj/mrubyAdapter.o obj/mrubyEngine.o obj/tclAdapter.o obj/tclEngine.o
obj/mrubyAdapter.o obj/mrubyEngine.o obj/tclAdapter.o obj/tclEngine.o obj/janetAdapter.o obj/janetEngine.o
lib/libcalog.a: $(CALOGLIB) | lib
ar rcs $@ $^
@ -374,6 +528,8 @@ lib/libs7.a: $(S7OBJ) | lib
ar rcs $@ $^
lib/libwren.a: $(WRENOBJ) | lib
ar rcs $@ $^
lib/libjanet.a: $(JANETOBJ) | lib
ar rcs $@ $^
lib/libsqlite3.a: $(SQLITEOBJ) | lib
ar rcs $@ $^
lib/libenet.a: $(ENETOBJ) | lib
@ -386,6 +542,9 @@ bin/testBroker: obj/testBroker.o lib/libcalog.a | bin
bin/testActor: obj/testActor.o lib/libcalog.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^
bin/testHooks: obj/testHooks.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testLua: obj/testLua.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -o $@ $^ $(LUALIBS)
@ -466,6 +625,9 @@ bin/testEngineMruby: obj/testEngineMruby.o lib/libcalog.a $(MRUBYLIB) | bin
bin/testEngineTcl: obj/testEngineTcl.o lib/libcalog.a $(TCLLIB) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(TCLLIBS)
bin/testEngineJanet: obj/testEngineJanet.o lib/libcalog.a lib/libjanet.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(JANETLIBS)
# example host: embeds calog + JS via calog.h only, linking the two archives.
obj/embed.o: examples/embed.c src/calog.h | obj
$(CC) $(COREFLAGS) -Isrc -pthread -c -o $@ $<
@ -480,9 +642,9 @@ bin/embed: obj/embed.o lib/libcalog.a lib/libquickjs.a | bin
# binary can drop the MySQL backend (see LICENSE.md). libssh2 precedes OpenSSL (it depends on
# it); SSLARCH follows the archives that reference it.
bin/calog: obj/calogMain.o \
obj/calogCrypto.o obj/calogDbFull.o obj/calogExport.o obj/calogFs.o obj/calogHttp.o obj/calogJson.o obj/calogKv.o obj/calogNet.o obj/calogPubsub.o obj/calogSsh.o obj/calogTask.o obj/calogTime.o obj/calogTimer.o obj/calogHandle.o \
lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a $(MRUBYLIB) $(TCLLIB) \
lib/libsqlite3.a lib/libenet.a $(LIBSSH2LIB) $(DBARCH) | bin
obj/calogArchive.o obj/calogCrypto.o obj/calogCsv.o obj/calogDbFull.o obj/calogExport.o obj/calogFs.o obj/calogHttp.o obj/calogHttpd.o obj/calogJson.o obj/calogKv.o obj/calogNet.o obj/calogProc.o obj/calogPubsub.o obj/calogRegex.o obj/calogSsh.o obj/calogTask.o obj/calogTime.o obj/calogTimer.o obj/calogXml.o obj/calogHandle.o \
lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a lib/libjanet.a $(MRUBYLIB) $(TCLLIB) \
lib/libsqlite3.a lib/libenet.a $(LIBSSH2LIB) $(ARCHIVELIBS) $(PCRE2LIB) $(DBARCH) | bin
$(CC) $(LDFLAGS) $(RELEASELD) -pthread -o $@ $(filter-out $(DBARCH),$^) -Wl,--start-group $(PGARCHIVES) -Wl,--end-group $(MYSQLARCH) $(SSLARCH) $(CXXLIB) $(DLLIB) -lm -lpthread $(SOCKETLIBS)
# ---- release build (fewest system dependencies while keeping DNS) --------------------
@ -532,11 +694,11 @@ static: bin/calogStatic
obj/testLoad.o: tests/testLoad.c src/calog.h | obj
$(CC) $(COREFLAGS) -Isrc -pthread $(ENGINEDEFS) -c -o $@ $<
bin/testLoad: obj/testLoad.o lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a $(MRUBYLIB) $(TCLLIB) | bin
bin/testLoad: obj/testLoad.o lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a lib/libjanet.a $(MRUBYLIB) $(TCLLIB) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(CXXLIB) -lm $(MRUBYLIBS) $(TCLLIBS)
# task library test: the task lib names every engine, so (like testLoad) it links them all.
bin/testTask: obj/testTask.o $(TASKADP) obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a $(MRUBYLIB) $(TCLLIB) | bin
bin/testTask: obj/testTask.o $(TASKADP) obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libquickjs.a lib/libsquirrel.a lib/libmybasic.a lib/libberry.a lib/libs7.a lib/libwren.a lib/libjanet.a $(MRUBYLIB) $(TCLLIB) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(CXXLIB) -lm $(MRUBYLIBS) $(TCLLIBS)
# export library test: publish from Lua, call by bare name from Lua + via callExport from JS.
@ -546,6 +708,9 @@ bin/testExport: obj/testExport.o obj/calogExport.o lib/libcalog.a lib/liblua.a l
bin/testJson: obj/testJson.o obj/calogJson.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testXml: obj/testXml.o obj/calogXml.o lib/libcalog.a lib/liblua.a $(LIBXML2LIB) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testTime: obj/testTime.o obj/calogTime.o lib/libcalog.a lib/liblua.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
@ -567,6 +732,21 @@ bin/testPubsub: obj/testPubsub.o obj/calogPubsub.o lib/libcalog.a lib/liblua.a |
bin/testHttp: obj/testHttp.o obj/calogHttp.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(DLLIB)
bin/testArchive: obj/testArchive.o obj/calogArchive.o obj/calogHandle.o lib/libcalog.a lib/liblua.a $(ARCHIVELIBS) $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(DLLIB)
bin/testUtil: obj/testUtil.o obj/calogCsv.o obj/calogProc.o obj/calogRegex.o lib/libcalog.a lib/liblua.a $(PCRE2LIB) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testSandbox: obj/testSandbox.o lib/libcalog.a lib/liblua.a lib/libquickjs.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testTrace: obj/testTrace.o obj/calogExport.o lib/libcalog.a lib/liblua.a lib/libquickjs.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testHttpd: obj/testHttpd.o obj/calogHttpd.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libquickjs.a | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS)
bin/testHttps: obj/testHttps.o obj/calogHttp.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
$(CC) $(LDFLAGS) -pthread -o $@ $^ $(LUALIBS) $(DLLIB)
@ -585,8 +765,8 @@ obj bin lib:
test: all
./bin/testBroker && ./bin/testLua && ./bin/testMyBasic && ./bin/testPolyglot && \
./bin/testActor && ./bin/testEngineLua && ./bin/testEngineMyBasic && ./bin/testSquirrel && ./bin/testEngineSquirrel && \
./bin/testJs && ./bin/testEngineJs && ./bin/testEngineBerry && ./bin/testEngineS7 && ./bin/testEngineWren && ./bin/testEngineMruby && ./bin/testEngineTcl && ./bin/testLoad && ./bin/testDb && ./bin/testNet && ./bin/testTask && ./bin/testExport && ./bin/testJson && ./bin/testTime && ./bin/testFs && ./bin/testCrypto && ./bin/testKv && ./bin/testTimer && ./bin/testPubsub && ./bin/testHttp && ./bin/testHttps
./bin/testActor && ./bin/testHooks && ./bin/testEngineLua && ./bin/testEngineMyBasic && ./bin/testSquirrel && ./bin/testEngineSquirrel && \
./bin/testJs && ./bin/testEngineJs && ./bin/testEngineBerry && ./bin/testEngineS7 && ./bin/testEngineWren && ./bin/testEngineMruby && ./bin/testEngineTcl && ./bin/testEngineJanet && ./bin/testLoad && ./bin/testDb && ./bin/testNet && ./bin/testTask && ./bin/testExport && ./bin/testJson && ./bin/testXml && ./bin/testTime && ./bin/testFs && ./bin/testCrypto && ./bin/testKv && ./bin/testTimer && ./bin/testPubsub && ./bin/testHttp && ./bin/testHttps && ./bin/testArchive && ./bin/testUtil && ./bin/testSandbox && ./bin/testTrace && ./bin/testHttpd
# ThreadSanitizer build of the actor core and the Lua engine path (cannot combine
# with ASan). Recompiled from source under TSan; the vendored Lua objects are
@ -688,6 +868,15 @@ tsantcl: $(TCLLIB) | bin obj
$(TCLLIB) $(TCLLIBS)
setarch -R ./bin/testEngineTclTsan
# ThreadSanitizer build of the Janet engine path (janet.o linked un-sanitized). This is the GATE-A
# race check: N thread-local janet_vm VMs on N threads must not alias -- the concurrent-contexts
# test is the gate.
tsanjanet: lib/libjanet.a | bin obj
$(CC) $(TSANFLAGS) $(INC) $(JANETINC) -o bin/testEngineJanetTsan \
tests/testEngineJanet.c src/janet/janetEngine.c src/janet/janetAdapter.c $(TSANCORE) \
lib/libjanet.a $(JANETLIBS)
setarch -R ./bin/testEngineJanetTsan
# ThreadSanitizer build of the concurrent libraries (timer's background thread, pubsub's
# callback fan-out, kv's shared store), each over a Lua context so callbacks marshal across
# threads. The unsanitized liblua.a links in fine (TSan instruments only the calog code).
@ -705,4 +894,4 @@ clean:
-include $(wildcard obj/*.d)
-include $(wildcard obj/rel/*.d)
.PHONY: all test tsan tsansq tsanjs tsanmb tsanberry tsans7 tsanwren tsanmruby tsantcl tsanlibs release clean
.PHONY: all test tsan tsansq tsanjs tsanmb tsanberry tsans7 tsanwren tsanmruby tsantcl tsanjanet tsanlibs release clean

51
PORTING.md
View file

@ -125,6 +125,42 @@ Verified results (this repo): musl -- Lua, QuickJS, my-basic, Squirrel, and `tes
winpthreads. Since the Windows binaries are the *same source* that passes on musl, the build is strong
evidence; full run-verification needs a Windows host or wine.
## Cross-compiling the archive stack with zig (`tools/crossArchive.sh`)
The compression/archive library (libarchive + its five codecs: zlib, bzip2, lz4, zstd, xz/liblzma)
is the one vendored stack that leans on CMake, so it gets its own cross-build script. Same idea as
`crossBuild.sh`, same single zig toolchain:
```
ZIG=/path/to/zig ./tools/crossArchive.sh
```
It builds all five codecs plus libarchive for **musl** (fully static, RUN on the host) and **Windows
x64** (a `testArchive.exe` verified as a PE), driving each through calog's real `testArchive`
(compress/decompress round-trips through every codec). CMake cross-compilation with zig works by
wrapping `zig cc` as a single-binary compiler (`build/cross/bin/zcc-*`) plus a toolchain file. The
key asymmetry:
- **musl toolchain does NOT set `CMAKE_SYSTEM_NAME`.** A musl-static binary runs on this Linux host,
so the build stays "native" (`CMAKE_CROSSCOMPILING` false) and CMake's `try_run` feature-detection
executes normally. zig supplies its own musl sysroot, so no host headers leak in.
- **Windows toolchain sets `CMAKE_SYSTEM_NAME Windows`** -> cross mode. libarchive's single
`CHECK_C_SOURCE_RUNS` is already guarded on `CMAKE_CROSSCOMPILING` (it degrades to a warning) and
xz has no run-checks, so nothing needs pre-seeding.
Two target-specific gotchas the script handles:
- **zlib on musl** needs `-DHAVE_UNISTD_H` so it includes `<unistd.h>` for `lseek`. zlib's
`./configure` normally sets this; the object-rule build (no configure) must pass it, and musl --
unlike glibc -- does not declare `lseek` without the include. Only zlib is affected.
- **libarchive on Windows** pulls in system libraries the POSIX build does not: CNG crypto
(`-lbcrypt`) for its AES/PBKDF2, and XmlLite + OLE (`-lxmllite -lole32`) for XAR, which libarchive
auto-substitutes for the libxml2/expat we disable. The link adds
`-lbcrypt -lxmllite -lole32 -ladvapi32 -lcrypt32 -lshlwapi`.
Everything lands under `build/cross/` (gitignored); the vendored sources and the native
`vendor/*/build` / `_cmk` trees the normal `make` uses are never touched.
## Status
- **Platform abstraction** (`calogPlatform.h`) and the net/ssh/http conversion: **done**, verified
@ -143,7 +179,14 @@ evidence; full run-verification needs a Windows host or wine.
Mach-O executables for both **x86_64 and arm64** (zig's libSystem stub; no Apple SDK needed for
calog's libc surface). Run-verification needs a Mac. The C code and Makefile hooks are all in place
(`libc++`, no `-ldl`, SIGPIPE, native pthreads + BSD sockets).
- The heavy vendored libraries (OpenSSL, Tcl, mruby, libssh2, MariaDB, PostgreSQL) still need a
per-target build to reach a *full-featured* calog on macOS/Windows; the cross-build proves the core
broker + engines + socket layer, which is where all the portability risk lived. No code rewrite is
expected -- the OS surface is only threads + sockets + DNS, all abstracted.
- **Archive stack** (libarchive + zlib/bzip2/lz4/zstd/xz): **musl run-verified, Windows
build-verified** via `tools/crossArchive.sh` (zig). The musl `testArchive` is fully static and
passes all 21 checks on the host (real compress/decompress through every codec); the Windows build
is a valid `testArchive.exe` (PE32+) linked against vendored winpthreads + the CNG/XmlLite system
libs. This was the one CMake-heavy vendored stack, so it is the strongest cross-build evidence
after the core. The XAR format (native builds enable it via libxml2 + OpenSSL + iconv) is **not**
in the cross build -- it is a native-only feature until those three are cross-built too.
- The remaining heavy vendored libraries (OpenSSL, Tcl, mruby, libssh2, MariaDB, PostgreSQL) still
need a per-target build to reach a *full-featured* calog on macOS/Windows; the cross-build proves
the core broker + engines + socket + archive layers, which is where all the portability risk lived.
No code rewrite is expected -- the OS surface is only threads + sockets + DNS, all abstracted.

15
README.md
View file

@ -4,7 +4,7 @@
calog embeds scripting into a C or C++ program. You register your native C functions
**once**, and they become callable from **any** embedded engine -- Lua, JavaScript, Scheme,
Squirrel, Wren, Berry, Ruby, Tcl, or MY-BASIC -- with values passed through a single canonical type. Add a new
Squirrel, Wren, Berry, Ruby, Tcl, Janet, or MY-BASIC -- with values passed through a single canonical type. Add a new
scripting language to your app by linking one more archive; your native functions don't
change.
@ -19,7 +19,7 @@ calogContextEval(ctx, "hostLog('hello from JavaScript, ' + 6 * 7)");
```
Swap `&calogJsEngine` for `&calogLuaEngine`, `&calogSquirrelEngine`,
`&calogS7Engine`, `&calogWrenEngine`, `&calogBerryEngine`, `&calogMrubyEngine`, `&calogTclEngine`, or `&calogMyBasicEngine` -- nothing else changes.
`&calogS7Engine`, `&calogWrenEngine`, `&calogBerryEngine`, `&calogMrubyEngine`, `&calogTclEngine`, `&calogJanetEngine`, or `&calogMyBasicEngine` -- nothing else changes.
---
@ -49,6 +49,16 @@ Swap `&calogJsEngine` for `&calogLuaEngine`, `&calogSquirrelEngine`,
function, fix a bug, or add a feature while the process keeps running and keeps its state --
no restart. Re-export the fixed function and every caller, in every language, picks up the
new version on the next call.
- **A polyglot web server.** The `httpd` library serves HTTP where each route handler is a
script function -- so `/pay` can be Ruby, `/render` Lua, `/rules` Scheme, all sharing helpers
via `calogExport`, and you fix a bug by re-registering a handler while it keeps serving. It is
the export + hot-reload + actor model in one library.
- **Sandbox untrusted scripts.** Open a context with `calogContextOpenLimited` and a
`CalogLimitsT` policy: a **native allow-list** (which registered natives the script may
call -- enforced for every engine at the one dispatch choke point), a **wall-clock budget**
(a runaway loop is retired -- Lua and JavaScript), and a **memory cap** (Lua and JavaScript,
via their per-VM allocators). Each context is already its own thread, so a limited plugin
can misbehave without taking down the host.
- **Many runtimes.** Independent `CalogT` runtimes coexist in one process; one host
thread can drive several.
- **Load by filename.** `calogContextLoad(calog, "config")` finds `config.lua` /
@ -76,6 +86,7 @@ Swap `&calogJsEngine` for `&calogLuaEngine`, `&calogSquirrelEngine`,
| Berry | `calogBerryEngine` | `.be` | scalars + callbacks |
| Ruby | `calogMrubyEngine` | `.rb` | mruby; 64-bit ints; built by mruby's Rake |
| Tcl | `calogTclEngine` | `.tcl` | Tcl 9; 64-bit ints; string-typed values |
| Janet | `calogJanetEngine` | `.janet` | Lisp; thread-local VM; doubles (2^53) |
A native can return a keyed `CalogValueT` record and **every** engine reads its fields with
native syntax (`user.name`, `user["name"]`, `(user "name")`), and a script can hand a

1
examples/embed.c
View file

@ -45,7 +45,6 @@ int main(void) {
nanosleep(&tick, NULL);
}
calogContextClose(ctx);
calogDestroy(calog);
return 0;
}

2
examples/scripts/README.md
View file

@ -35,6 +35,8 @@ Conventions every example follows:
| `berry.be` | Berry (Python-like): a `def`, a list built with a `for`, a map, then json + uuid |
| `ruby.rb` | Ruby (mruby): a Range, `reduce` with a block, `select(&:even?)`, a Hash, then json + sha256 |
| `tcl.tcl` | Tcl 9: `set`/`expr`, a `foreach`, a list and a `dict`, then json + sha256 |
| `janet.janet` | Janet (Lisp): `def`/`fn`, first-class `map`, an array and a table, then json + sha256 |
| `servers/httpServer.lua` | HTTP server: route handlers as script functions, a response map, hot-reloadable |
| `scheme.scm` | s7 Scheme: a recursive factorial, list build + map, string ops (all in one `(begin ...)`) |
| `wren.wren` | Wren: a class, a `List`, and every native reached via `Calog.call(name, [args])` |

19
examples/scripts/languages/janet.janet Normal file
View file

@ -0,0 +1,19 @@
# A guided tour of Janet (a Lisp) running on calog.
# Demonstrates: def/fn, a first-class function passed to map, an array and a table, then the
# json + cryptoHashSha256 natives. Janet integers are doubles (exact to 2^53). Exports from other
# scripts are reached with (calogCall "name" ...) -- Janet has no bare-name resolution hook.
# run: bin/calog examples/scripts/languages/janet.janet
(def nums [1 2 3 4 5])
(def total (reduce + 0 nums))
(def doubled (string/join (map (fn [x] (string (* x 2))) nums) " "))
(print "nums total = " total ", doubled = " doubled)
(def user @{"name" "ada" "age" 36})
(print "user as JSON: " (jsonStringify user))
(def parsed (jsonParse `{"lang":"janet","ints":[10,20,30]}`))
(print "parsed lang = " (get parsed "lang") ", second int = " (get (get parsed "ints") 1))
(print "sha256(calog) = " (cryptoHashSha256 "calog"))
(calogExit 0)

23
examples/scripts/servers/httpServer.lua Normal file
View file

@ -0,0 +1,23 @@
-- A polyglot HTTP server. Each route handler is a script function value; re-registering a route
-- hot-swaps it live while the server keeps serving. Handlers here are Lua, but a route handler can
-- be a function from ANY calog engine (and routes can share helpers via calogExport). The script
-- does not call calogExit, so its context stays alive to serve requests.
-- run: bin/calog examples/scripts/servers/httpServer.lua
-- then: curl localhost:8080/ curl localhost:8080/hello 'curl localhost:8080/echo?x=1'
local srv = httpdListen(8080)
httpdRoute(srv, "GET", "/", function(req)
return { status = 200, headers = { ["Content-Type"] = "text/plain" },
body = "calog httpd -- try /hello or /echo?name=ada\n" }
end)
httpdRoute(srv, "GET", "/hello", function(req)
return "hello from a Lua handler\n"
end)
httpdRoute(srv, "GET", "/echo", function(req)
return jsonStringify({ method = req.method, path = req.path, query = req.query })
end)
calogPrint("listening on http://localhost:8080")

5
examples/staticDemo.c
View file

@ -76,8 +76,6 @@ int main(void) {
if (ctx == NULL) {
printf("context open failed\n");
calogDestroy(calog);
calogDbShutdown();
calogNetShutdown();
return 1;
}
calogContextEval(ctx,
@ -98,10 +96,7 @@ int main(void) {
}
calogPump(calog);
calogContextClose(ctx);
calogDestroy(calog);
calogDbShutdown();
calogNetShutdown();
if (atomic_load(&reportedTotal) != 55) {
printf("FAILED: expected 55, got %lld\n", (long long)atomic_load(&reportedTotal));

704
libs/calogArchive.c Normal file
View file

@ -0,0 +1,704 @@
// calogArchive.c -- calog compression + archive library (see calogArchive.h), built on libarchive.
//
// Two capabilities: one-shot raw codecs (compress/decompress a whole buffer in memory) and archive
// read/write over the shared typed handle table. All data is binary-safe; a per-transfer cap
// (ARC_MAX) guards against a decompression bomb. Every native is inline (runs on the calling
// context's thread). libarchive needs no global init; the process-wide lib struct only owns a
// refcounted handle table, exactly like calogNet.
#include "calogArchive.h"
#include "calogHandle.h"
#include "calogInternal.h"
#include <pthread.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <archive.h>
#include <archive_entry.h>
// Handle kinds, distinct across the registry so a read handle passed to a write native (or vice
// versa) fails to resolve.
#define ARC_TYPE_READ 1u
#define ARC_TYPE_WRITE 2u
// Upper bound on a single one-shot buffer / entry read, so a hostile input cannot exhaust memory.
#define ARC_MAX (64 * 1024 * 1024)
// A growable byte buffer backing the in-memory write sink.
typedef struct ArcBufT {
char *data;
size_t len;
size_t cap;
} ArcBufT;
// A read handle: the libarchive reader plus, for an in-memory source, the owned copy it streams
// from (libarchive does not copy the source buffer, so it must outlive the reader).
typedef struct ArcReadT {
struct archive *archive;
void *memory;
} ArcReadT;
// A write handle: the libarchive writer feeding the growable in-memory sink.
typedef struct ArcWriteT {
struct archive *archive;
ArcBufT sink;
} ArcWriteT;
// Process-wide state shared by every runtime that registers the natives.
typedef struct ArcLibT {
CalogHandleTableT *handles;
int32_t refCount;
} ArcLibT;
typedef struct ArcNativeT {
const char *name;
CalogNativeFnT fn;
} ArcNativeT;
static pthread_mutex_t gArcMutex = PTHREAD_MUTEX_INITIALIZER;
static ArcLibT *gArcLib = NULL;
static int32_t arcBufAppend(ArcBufT *buffer, const void *bytes, size_t length);
static void arcCloser(uint32_t type, void *resource);
static int32_t arcCompress(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcDecompress(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcEntryTypeName(mode_t filetype, const char **out);
static CalogValueT *arcMapField(CalogAggT *map, const char *name);
static int32_t arcReadClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcReadData(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcReadNext(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcReadOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcReadOpenFile(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcStore(ArcLibT *lib, uint32_t type, void *resource, CalogValueT *result);
static la_ssize_t arcWriteSink(struct archive *archive, void *client, const void *buffer, size_t length);
static int32_t arcWriteEntry(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcWriteFinish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t arcWriteOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static const ArcNativeT gArcNatives[] = {
{ "compress", arcCompress },
{ "decompress", arcDecompress },
{ "archiveReadOpen", arcReadOpen },
{ "archiveReadOpenFile", arcReadOpenFile },
{ "archiveReadNext", arcReadNext },
{ "archiveReadData", arcReadData },
{ "archiveReadClose", arcReadClose },
{ "archiveWriteOpen", arcWriteOpen },
{ "archiveWriteEntry", arcWriteEntry },
{ "archiveWriteFinish", arcWriteFinish },
};
int32_t calogArchiveRegister(CalogT *calog) {
ArcLibT *lib;
int32_t status;
size_t index;
pthread_mutex_lock(&gArcMutex);
if (gArcLib == NULL) {
ArcLibT *created;
created = (ArcLibT *)calloc(1, sizeof(*created));
if (created == NULL) {
pthread_mutex_unlock(&gArcMutex);
return calogErrOomE;
}
created->handles = calogHandleTableCreate();
if (created->handles == NULL) {
free(created);
pthread_mutex_unlock(&gArcMutex);
return calogErrOomE;
}
gArcLib = created;
}
gArcLib->refCount++;
lib = gArcLib;
pthread_mutex_unlock(&gArcMutex);
status = calogOkE;
for (index = 0; index < sizeof(gArcNatives) / sizeof(gArcNatives[0]); index++) {
status = calogRegisterInline(calog, gArcNatives[index].name, gArcNatives[index].fn, lib);
if (status != calogOkE) {
break;
}
}
if (status != calogOkE) {
calogArchiveShutdown();
return status;
}
return calogAtDestroy(calog, calogArchiveShutdown, calogDestroyAfterContextsE);
}
void calogArchiveShutdown(void) {
pthread_mutex_lock(&gArcMutex);
if (gArcLib == NULL) {
pthread_mutex_unlock(&gArcMutex);
return;
}
gArcLib->refCount--;
if (gArcLib->refCount <= 0) {
calogHandleTableDestroy(gArcLib->handles, arcCloser);
free(gArcLib);
gArcLib = NULL;
}
pthread_mutex_unlock(&gArcMutex);
}
static int32_t arcBufAppend(ArcBufT *buffer, const void *bytes, size_t length) {
if (buffer->len + length > buffer->cap) {
size_t wanted;
char *grown;
wanted = buffer->cap ? buffer->cap * 2 : 4096;
while (wanted < buffer->len + length) {
wanted *= 2;
}
grown = (char *)realloc(buffer->data, wanted);
if (grown == NULL) {
return calogErrOomE;
}
buffer->data = grown;
buffer->cap = wanted;
}
memcpy(buffer->data + buffer->len, bytes, length);
buffer->len += length;
return calogOkE;
}
static void arcCloser(uint32_t type, void *resource) {
switch (type) {
case ARC_TYPE_READ: {
ArcReadT *reader;
reader = (ArcReadT *)resource;
if (reader->archive != NULL) {
archive_read_free(reader->archive);
}
free(reader->memory);
free(reader);
break;
}
case ARC_TYPE_WRITE: {
ArcWriteT *writer;
writer = (ArcWriteT *)resource;
if (writer->archive != NULL) {
archive_write_free(writer->archive);
}
free(writer->sink.data);
free(writer);
break;
}
default:
break;
}
}
static int32_t arcCompress(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
struct archive *archive;
struct archive_entry *entry;
ArcBufT sink;
const char *filter;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 2 || argCount > 3 || args[0].type != calogStringE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "compress expects (data, filter [, level])");
}
if (argCount == 3 && args[2].type != calogIntE) {
return calogFail(result, calogErrArgE, "compress: level must be an integer");
}
filter = args[1].as.s.bytes;
archive = archive_write_new();
if (archive == NULL) {
return calogFail(result, calogErrOomE, "compress: out of memory");
}
archive_write_set_format_raw(archive);
// No block padding for a one-shot codec: the output is exactly the filtered stream, so a
// reader (e.g. zstd/lz4) does not mistake trailing NUL padding for a corrupt second frame.
archive_write_set_bytes_per_block(archive, 1);
archive_write_set_bytes_in_last_block(archive, 1);
if (strcmp(filter, "none") != 0 && archive_write_add_filter_by_name(archive, filter) != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, archive_error_string(archive));
archive_write_free(archive);
return status;
}
if (argCount == 3) {
char option[64];
snprintf(option, sizeof(option), "%s:compression-level=%lld", filter, (long long)args[2].as.i);
archive_write_set_options(archive, option); // best effort; unknown option is ignored
}
memset(&sink, 0, sizeof(sink));
if (archive_write_open2(archive, &sink, NULL, arcWriteSink, NULL, NULL) != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, archive_error_string(archive));
archive_write_free(archive);
free(sink.data);
return status;
}
entry = archive_entry_new();
if (entry == NULL) {
archive_write_free(archive);
free(sink.data);
return calogFail(result, calogErrOomE, "compress: out of memory");
}
archive_entry_set_pathname(entry, "data");
archive_entry_set_filetype(entry, AE_IFREG);
archive_entry_set_size(entry, args[0].as.s.length);
status = calogOkE;
if (archive_write_header(archive, entry) != ARCHIVE_OK ||
archive_write_data(archive, args[0].as.s.bytes, (size_t)args[0].as.s.length) < 0) {
status = calogFail(result, calogErrArgE, archive_error_string(archive));
}
archive_entry_free(entry);
if (status == calogOkE && archive_write_close(archive) != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, archive_error_string(archive));
}
archive_write_free(archive);
if (status == calogOkE) {
status = calogValueString(result, sink.data ? sink.data : "", (int64_t)sink.len);
}
free(sink.data);
return status;
}
static int32_t arcDecompress(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
struct archive *archive;
struct archive_entry *entry;
ArcBufT out;
int code;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 1 || argCount > 2 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "decompress expects (data [, filter])");
}
archive = archive_read_new();
if (archive == NULL) {
return calogFail(result, calogErrOomE, "decompress: out of memory");
}
archive_read_support_filter_all(archive);
archive_read_support_format_raw(archive);
if (archive_read_open_memory(archive, args[0].as.s.bytes, (size_t)args[0].as.s.length) != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, archive_error_string(archive));
archive_read_free(archive);
return status;
}
code = archive_read_next_header(archive, &entry);
if (code != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, code == ARCHIVE_EOF ? "decompress: empty input" : archive_error_string(archive));
archive_read_free(archive);
return status;
}
memset(&out, 0, sizeof(out));
status = calogOkE;
for (;;) {
char chunk[65536];
la_ssize_t got;
got = archive_read_data(archive, chunk, sizeof(chunk));
if (got < 0) {
status = calogFail(result, calogErrArgE, archive_error_string(archive));
break;
}
if (got == 0) {
break;
}
if (out.len + (size_t)got > ARC_MAX) {
status = calogFail(result, calogErrRangeE, "decompress: output exceeds the size cap");
break;
}
if (arcBufAppend(&out, chunk, (size_t)got) != calogOkE) {
status = calogFail(result, calogErrOomE, "decompress: out of memory");
break;
}
}
archive_read_free(archive);
if (status == calogOkE) {
status = calogValueString(result, out.data ? out.data : "", (int64_t)out.len);
}
free(out.data);
return status;
}
static int32_t arcEntryTypeName(mode_t filetype, const char **out) {
switch (filetype) {
case AE_IFREG: *out = "file"; return calogOkE;
case AE_IFDIR: *out = "dir"; return calogOkE;
case AE_IFLNK: *out = "symlink"; return calogOkE;
case AE_IFSOCK: *out = "socket"; return calogOkE;
case AE_IFCHR: *out = "chardev"; return calogOkE;
case AE_IFBLK: *out = "blockdev"; return calogOkE;
case AE_IFIFO: *out = "fifo"; return calogOkE;
default: *out = "unknown"; return calogOkE;
}
}
// Look up a string-keyed field in a map (borrowed pointer into the map, or NULL).
static CalogValueT *arcMapField(CalogAggT *map, const char *name) {
CalogValueT key;
CalogValueT *field;
if (calogValueString(&key, name, (int64_t)strlen(name)) != calogOkE) {
return NULL;
}
field = calogAggGet(map, &key);
calogValueFree(&key);
return field;
}
static int32_t arcReadClose(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcReadT *reader;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "archiveReadClose expects (handle)");
}
reader = (ArcReadT *)calogHandleRemove(lib->handles, args[0].as.i, ARC_TYPE_READ);
if (reader == NULL) {
return calogFail(result, calogErrArgE, "archiveReadClose: invalid handle");
}
arcCloser(ARC_TYPE_READ, reader);
return calogOkE;
}
static int32_t arcReadData(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcReadT *reader;
ArcBufT out;
int32_t status;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "archiveReadData expects (handle)");
}
reader = (ArcReadT *)calogHandleGet(lib->handles, args[0].as.i, ARC_TYPE_READ);
if (reader == NULL) {
return calogFail(result, calogErrArgE, "archiveReadData: invalid handle");
}
memset(&out, 0, sizeof(out));
status = calogOkE;
for (;;) {
char chunk[65536];
la_ssize_t got;
got = archive_read_data(reader->archive, chunk, sizeof(chunk));
if (got < 0) {
status = calogFail(result, calogErrArgE, archive_error_string(reader->archive));
break;
}
if (got == 0) {
break;
}
if (out.len + (size_t)got > ARC_MAX) {
status = calogFail(result, calogErrRangeE, "archiveReadData: entry exceeds the size cap");
break;
}
if (arcBufAppend(&out, chunk, (size_t)got) != calogOkE) {
status = calogFail(result, calogErrOomE, "archiveReadData: out of memory");
break;
}
}
if (status == calogOkE) {
status = calogValueString(result, out.data ? out.data : "", (int64_t)out.len);
}
free(out.data);
return status;
}
static int32_t arcReadNext(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcReadT *reader;
struct archive_entry *entry;
CalogAggT *map;
const char *name;
const char *typeName;
int code;
int32_t status;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "archiveReadNext expects (handle)");
}
reader = (ArcReadT *)calogHandleGet(lib->handles, args[0].as.i, ARC_TYPE_READ);
if (reader == NULL) {
return calogFail(result, calogErrArgE, "archiveReadNext: invalid handle");
}
code = archive_read_next_header(reader->archive, &entry);
if (code == ARCHIVE_EOF) {
return calogOkE; // nil signals the end of the archive
}
if (code != ARCHIVE_OK && code != ARCHIVE_WARN) {
return calogFail(result, calogErrArgE, archive_error_string(reader->archive));
}
status = calogAggCreate(&map, calogMapE);
if (status != calogOkE) {
return calogFail(result, status, "archiveReadNext: out of memory");
}
name = archive_entry_pathname(entry);
arcEntryTypeName(archive_entry_filetype(entry), &typeName);
status = calogMapSetStr(map, "name", name ? name : "", name ? (int64_t)strlen(name) : 0);
if (status == calogOkE) {
status = calogMapSetInt(map, "size", (int64_t)archive_entry_size(entry));
}
if (status == calogOkE) {
status = calogMapSetInt(map, "mode", (int64_t)archive_entry_perm(entry));
}
if (status == calogOkE) {
status = calogMapSetInt(map, "mtime", (int64_t)archive_entry_mtime(entry));
}
if (status == calogOkE) {
status = calogMapSetStr(map, "type", typeName, (int64_t)strlen(typeName));
}
if (status == calogOkE && archive_entry_filetype(entry) == AE_IFLNK) {
const char *target;
target = archive_entry_symlink(entry);
if (target != NULL) {
status = calogMapSetStr(map, "linkname", target, (int64_t)strlen(target));
}
}
if (status != calogOkE) {
calogAggFree(map);
return calogFail(result, status, "archiveReadNext: failed to build the entry");
}
calogValueAgg(result, map);
return calogOkE;
}
static int32_t arcReadOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcReadT *reader;
void *copy;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "archiveReadOpen expects (bytes)");
}
reader = (ArcReadT *)calloc(1, sizeof(*reader));
if (reader == NULL) {
return calogFail(result, calogErrOomE, "archiveReadOpen: out of memory");
}
// libarchive streams the source lazily and does NOT copy it, so keep an owned copy alive.
copy = malloc(args[0].as.s.length > 0 ? (size_t)args[0].as.s.length : 1);
if (copy == NULL) {
free(reader);
return calogFail(result, calogErrOomE, "archiveReadOpen: out of memory");
}
memcpy(copy, args[0].as.s.bytes, (size_t)args[0].as.s.length);
reader->memory = copy;
reader->archive = archive_read_new();
if (reader->archive == NULL) {
free(copy);
free(reader);
return calogFail(result, calogErrOomE, "archiveReadOpen: out of memory");
}
archive_read_support_filter_all(reader->archive);
archive_read_support_format_all(reader->archive);
if (archive_read_open_memory(reader->archive, copy, (size_t)args[0].as.s.length) != ARCHIVE_OK) {
int32_t status;
status = calogFail(result, calogErrArgE, archive_error_string(reader->archive));
arcCloser(ARC_TYPE_READ, reader);
return status;
}
return arcStore(lib, ARC_TYPE_READ, reader, result);
}
static int32_t arcReadOpenFile(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcReadT *reader;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "archiveReadOpenFile expects (path)");
}
reader = (ArcReadT *)calloc(1, sizeof(*reader));
if (reader == NULL) {
return calogFail(result, calogErrOomE, "archiveReadOpenFile: out of memory");
}
reader->archive = archive_read_new();
if (reader->archive == NULL) {
free(reader);
return calogFail(result, calogErrOomE, "archiveReadOpenFile: out of memory");
}
archive_read_support_filter_all(reader->archive);
archive_read_support_format_all(reader->archive);
if (archive_read_open_filename(reader->archive, args[0].as.s.bytes, 65536) != ARCHIVE_OK) {
int32_t status;
status = calogFail(result, calogErrArgE, archive_error_string(reader->archive));
arcCloser(ARC_TYPE_READ, reader);
return status;
}
return arcStore(lib, ARC_TYPE_READ, reader, result);
}
// Wrap a resource in a handle-table entry, transferring ownership. On failure the resource is
// freed via arcCloser. Sets result to the new integer handle on success.
static int32_t arcStore(ArcLibT *lib, uint32_t type, void *resource, CalogValueT *result) {
int64_t handle;
handle = calogHandleAdd(lib->handles, type, resource);
if (handle == 0) {
arcCloser(type, resource);
return calogFail(result, calogErrOomE, "out of memory");
}
calogValueInt(result, handle);
return calogOkE;
}
static la_ssize_t arcWriteSink(struct archive *archive, void *client, const void *buffer, size_t length) {
(void)archive;
if (arcBufAppend((ArcBufT *)client, buffer, length) != calogOkE) {
return -1;
}
return (la_ssize_t)length;
}
static int32_t arcWriteEntry(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcWriteT *writer;
struct archive_entry *entry;
int32_t status;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount < 3 || argCount > 4 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogStringE) {
return calogFail(result, calogErrArgE, "archiveWriteEntry expects (handle, name, data [, opts])");
}
if (argCount == 4 && args[3].type != calogAggE) {
return calogFail(result, calogErrArgE, "archiveWriteEntry: opts must be a map");
}
writer = (ArcWriteT *)calogHandleGet(lib->handles, args[0].as.i, ARC_TYPE_WRITE);
if (writer == NULL) {
return calogFail(result, calogErrArgE, "archiveWriteEntry: invalid handle");
}
entry = archive_entry_new();
if (entry == NULL) {
return calogFail(result, calogErrOomE, "archiveWriteEntry: out of memory");
}
archive_entry_set_pathname(entry, args[1].as.s.bytes);
archive_entry_set_filetype(entry, AE_IFREG);
archive_entry_set_perm(entry, 0644);
archive_entry_set_size(entry, args[2].as.s.length);
if (argCount == 4) {
CalogValueT *field;
field = arcMapField(args[3].as.agg,"mode");
if (field != NULL && field->type == calogIntE) {
archive_entry_set_perm(entry, (mode_t)field->as.i);
}
field = arcMapField(args[3].as.agg,"mtime");
if (field != NULL && field->type == calogIntE) {
archive_entry_set_mtime(entry, (time_t)field->as.i, 0);
}
field = arcMapField(args[3].as.agg,"type");
if (field != NULL && field->type == calogStringE && strcmp(field->as.s.bytes, "dir") == 0) {
archive_entry_set_filetype(entry, AE_IFDIR);
archive_entry_set_size(entry, 0);
}
}
status = calogOkE;
if (archive_write_header(writer->archive, entry) != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, archive_error_string(writer->archive));
} else if (args[2].as.s.length > 0 && archive_write_data(writer->archive, args[2].as.s.bytes, (size_t)args[2].as.s.length) < 0) {
status = calogFail(result, calogErrArgE, archive_error_string(writer->archive));
}
archive_entry_free(entry);
return status;
}
static int32_t arcWriteFinish(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcWriteT *writer;
int32_t status;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "archiveWriteFinish expects (handle)");
}
writer = (ArcWriteT *)calogHandleRemove(lib->handles, args[0].as.i, ARC_TYPE_WRITE);
if (writer == NULL) {
return calogFail(result, calogErrArgE, "archiveWriteFinish: invalid handle");
}
status = calogOkE;
if (archive_write_close(writer->archive) != ARCHIVE_OK) {
status = calogFail(result, calogErrArgE, archive_error_string(writer->archive));
}
if (status == calogOkE) {
status = calogValueString(result, writer->sink.data ? writer->sink.data : "", (int64_t)writer->sink.len);
}
arcCloser(ARC_TYPE_WRITE, writer);
return status;
}
static int32_t arcWriteOpen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
ArcLibT *lib;
ArcWriteT *writer;
const char *format;
const char *filter;
lib = (ArcLibT *)userData;
calogValueNil(result);
if (argCount < 2 || argCount > 3 || args[0].type != calogStringE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "archiveWriteOpen expects (format, filter [, level])");
}
if (argCount == 3 && args[2].type != calogIntE) {
return calogFail(result, calogErrArgE, "archiveWriteOpen: level must be an integer");
}
// "tar" is the friendly name for pax-restricted, libarchive's portable modern default.
format = strcmp(args[0].as.s.bytes, "tar") == 0 ? "paxr" : args[0].as.s.bytes;
filter = args[1].as.s.bytes;
writer = (ArcWriteT *)calloc(1, sizeof(*writer));
if (writer == NULL) {
return calogFail(result, calogErrOomE, "archiveWriteOpen: out of memory");
}
writer->archive = archive_write_new();
if (writer->archive == NULL) {
free(writer);
return calogFail(result, calogErrOomE, "archiveWriteOpen: out of memory");
}
if (archive_write_set_format_by_name(writer->archive, format) != ARCHIVE_OK) {
int32_t status;
status = calogFail(result, calogErrArgE, archive_error_string(writer->archive));
arcCloser(ARC_TYPE_WRITE, writer);
return status;
}
if (strcmp(filter, "none") != 0 && archive_write_add_filter_by_name(writer->archive, filter) != ARCHIVE_OK) {
int32_t status;
status = calogFail(result, calogErrArgE, archive_error_string(writer->archive));
arcCloser(ARC_TYPE_WRITE, writer);
return status;
}
if (argCount == 3) {
char option[64];
snprintf(option, sizeof(option), "%s:compression-level=%lld", filter, (long long)args[2].as.i);
archive_write_set_options(writer->archive, option); // best effort
}
if (archive_write_open2(writer->archive, &writer->sink, NULL, arcWriteSink, NULL, NULL) != ARCHIVE_OK) {
int32_t status;
status = calogFail(result, calogErrArgE, archive_error_string(writer->archive));
arcCloser(ARC_TYPE_WRITE, writer);
return status;
}
return arcStore(lib, ARC_TYPE_WRITE, writer, result);
}

41
libs/calogArchive.h Normal file
View file

@ -0,0 +1,41 @@
// calogArchive.h -- calog compression + archive library, built on libarchive.
//
// Two orthogonal capabilities, available to every engine. All data is a binary-safe string
// (embedded NULs preserved); a per-transfer cap (ARC_MAX) guards against a decompression bomb.
//
// One-shot raw codecs (whole buffer, in memory):
// compress(data, filter [, level]) filter: "gzip"|"bzip2"|"xz"|"zstd"|"lz4"|"compress"|"none"
// decompress(data [, filter]) filter auto-detected from the magic bytes if omitted
//
// Archive read (over the shared typed handle table):
// archiveReadOpen(bytes) -> handle (whole archive already in memory)
// archiveReadOpenFile(path) -> handle (streams from disk)
// archiveReadNext(handle) -> entry map | nil at end
// entry: { name, size, mode, mtime, type[, linkname] }
// archiveReadData(handle) -> bytes (current entry's data, whole, capped at ARC_MAX)
// archiveReadClose(handle)
//
// Archive write:
// archiveWriteOpen(format, filter [, level]) -> handle
// format: "tar"|"pax"|"ustar"|"gnutar"|"cpio"|"zip"|"7zip"|"iso9660"|"ar"|"mtree"
// filter: "none"|"gzip"|"bzip2"|"xz"|"zstd"|"lz4"|"compress"
// archiveWriteEntry(handle, name, data [, opts]) opts map: { mode, mtime, type }
// archiveWriteFinish(handle) -> bytes (closes + returns the whole archive; frees the handle)
//
// The natives are INLINE (they run on the calling context's thread, stalling only that actor).
// Each archive handle is used by exactly one context at a time (its owner) and closed by exactly
// one context -- calog does not serialize a single handle across contexts. libarchive needs no
// process-wide init; register only builds the handle table, shutdown tears it down on last use.
#ifndef CALOG_ARCHIVE_H
#define CALOG_ARCHIVE_H
#include "calog.h"
// Register the archive/compression natives on a runtime. Idempotent across runtimes.
int32_t calogArchiveRegister(CalogT *calog);
// Release the shared handle table once the last registered runtime unregisters.
void calogArchiveShutdown(void);
#endif

320
libs/calogCsv.c Normal file
View file

@ -0,0 +1,320 @@
// calogCsv.c -- calog CSV library (see calogCsv.h). A hand-written RFC 4180 parser/formatter, no
// dependency (the grammar is small, mirroring the in-tree JSON codec). Natives are inline.
#include "calogCsv.h"
#include "calogInternal.h"
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// A growable byte buffer for one cell / the output.
typedef struct CsvBufT {
char *data;
size_t len;
size_t cap;
} CsvBufT;
static int32_t csvBufByte(CsvBufT *buffer, char byte);
static int32_t csvBufBytes(CsvBufT *buffer, const char *bytes, size_t length);
static int32_t csvCellToText(const CalogValueT *cell, CsvBufT *out, char delimiter);
static int32_t csvCommitRow(CalogAggT *rows, CalogAggT **rowPtr);
static int32_t csvFormat(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t csvParse(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t csvPushField(CalogAggT *row, CsvBufT *field);
int32_t calogCsvRegister(CalogT *calog) {
int32_t status;
status = calogRegisterInline(calog, "csvParse", csvParse, NULL);
if (status != calogOkE) {
return status;
}
return calogRegisterInline(calog, "csvFormat", csvFormat, NULL);
}
static int32_t csvBufByte(CsvBufT *buffer, char byte) {
return csvBufBytes(buffer, &byte, 1);
}
static int32_t csvBufBytes(CsvBufT *buffer, const char *bytes, size_t length) {
if (buffer->len + length > buffer->cap) {
size_t wanted;
char *grown;
wanted = buffer->cap ? buffer->cap * 2 : 128;
while (wanted < buffer->len + length) {
wanted *= 2;
}
grown = (char *)realloc(buffer->data, wanted);
if (grown == NULL) {
return calogErrOomE;
}
buffer->data = grown;
buffer->cap = wanted;
}
memcpy(buffer->data + buffer->len, bytes, length);
buffer->len += length;
return calogOkE;
}
// Render one cell into `out`, quoting per RFC 4180 if it contains the delimiter, a quote, CR, or LF.
static int32_t csvCellToText(const CalogValueT *cell, CsvBufT *out, char delimiter) {
char scratch[64];
const char *bytes;
int64_t length;
bool needQuote;
int64_t i;
switch (cell->type) {
case calogNilE:
return calogOkE;
case calogBoolE:
bytes = cell->as.b ? "true" : "false";
length = (int64_t)strlen(bytes);
break;
case calogIntE:
length = snprintf(scratch, sizeof(scratch), "%lld", (long long)cell->as.i);
bytes = scratch;
break;
case calogRealE:
length = snprintf(scratch, sizeof(scratch), "%g", cell->as.r);
bytes = scratch;
break;
case calogStringE:
bytes = cell->as.s.bytes;
length = cell->as.s.length;
break;
default:
return calogErrTypeE;
}
needQuote = false;
for (i = 0; i < length; i++) {
char c;
c = bytes[i];
if (c == delimiter || c == '"' || c == '\n' || c == '\r') {
needQuote = true;
break;
}
}
if (!needQuote) {
return csvBufBytes(out, bytes, (size_t)length);
}
if (csvBufByte(out, '"') != calogOkE) {
return calogErrOomE;
}
for (i = 0; i < length; i++) {
if (bytes[i] == '"' && csvBufByte(out, '"') != calogOkE) {
return calogErrOomE;
}
if (csvBufByte(out, bytes[i]) != calogOkE) {
return calogErrOomE;
}
}
return csvBufByte(out, '"');
}
static int32_t csvFormat(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
CsvBufT out;
char delimiter;
int64_t rowIndex;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 1 || argCount > 2 || args[0].type != calogAggE) {
return calogFail(result, calogErrArgE, "csvFormat expects (rows [, delimiter])");
}
delimiter = ',';
if (argCount == 2) {
if (args[1].type != calogStringE || args[1].as.s.length != 1) {
return calogFail(result, calogErrArgE, "csvFormat: delimiter must be a one-byte string");
}
delimiter = args[1].as.s.bytes[0];
}
memset(&out, 0, sizeof(out));
status = calogOkE;
for (rowIndex = 0; rowIndex < args[0].as.agg->arrayCount; rowIndex++) {
CalogValueT *row;
int64_t cellIndex;
if (rowIndex > 0 && csvBufBytes(&out, "\r\n", 2) != calogOkE) {
status = calogErrOomE;
break;
}
row = &args[0].as.agg->array[rowIndex];
if (row->type != calogAggE) {
status = calogFail(result, calogErrArgE, "csvFormat: each row must be a list of cells");
break;
}
for (cellIndex = 0; cellIndex < row->as.agg->arrayCount; cellIndex++) {
if (cellIndex > 0 && csvBufByte(&out, delimiter) != calogOkE) {
status = calogErrOomE;
break;
}
status = csvCellToText(&row->as.agg->array[cellIndex], &out, delimiter);
if (status != calogOkE) {
break;
}
}
if (status != calogOkE) {
break;
}
}
if (status == calogOkE) {
status = calogValueString(result, out.data ? out.data : "", (int64_t)out.len);
} else if (result->type != calogStringE) {
(void)calogFail(result, status, "csvFormat: failed to build output");
}
free(out.data);
return status;
}
// Wrap *rowPtr, push it into rows, and replace *rowPtr with a fresh empty row (NULL on failure so
// the caller's cleanup never double-frees a row already owned by rows).
static int32_t csvCommitRow(CalogAggT *rows, CalogAggT **rowPtr) {
CalogValueT rowValue;
int32_t status;
calogValueAgg(&rowValue, *rowPtr);
*rowPtr = NULL;
status = calogAggPush(rows, &rowValue);
if (status != calogOkE) {
calogValueFree(&rowValue);
return status;
}
status = calogAggCreate(rowPtr, calogListE);
if (status != calogOkE) {
*rowPtr = NULL;
}
return status;
}
// Push the accumulated field as a string cell into row, then reset the field.
static int32_t csvPushField(CalogAggT *row, CsvBufT *field) {
CalogValueT cell;
int32_t status;
status = calogValueString(&cell, field->data ? field->data : "", (int64_t)field->len);
if (status != calogOkE) {
return status;
}
status = calogAggPush(row, &cell);
if (status != calogOkE) {
calogValueFree(&cell);
return status;
}
field->len = 0;
return calogOkE;
}
static int32_t csvParse(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
CalogAggT *rows;
CalogAggT *row;
CsvBufT field;
const char *text;
char delimiter;
int64_t length;
int64_t i;
bool inQuotes;
bool lineHasContent;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 1 || argCount > 2 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "csvParse expects (text [, delimiter])");
}
delimiter = ',';
if (argCount == 2) {
if (args[1].type != calogStringE || args[1].as.s.length != 1) {
return calogFail(result, calogErrArgE, "csvParse: delimiter must be a one-byte string");
}
delimiter = args[1].as.s.bytes[0];
}
status = calogAggCreate(&rows, calogListE);
if (status != calogOkE) {
return calogFail(result, status, "csvParse: out of memory");
}
row = NULL;
status = calogAggCreate(&row, calogListE);
if (status != calogOkE) {
calogAggFree(rows);
return calogFail(result, status, "csvParse: out of memory");
}
text = args[0].as.s.bytes;
length = args[0].as.s.length;
memset(&field, 0, sizeof(field));
inQuotes = false;
lineHasContent = false;
i = 0;
while (status == calogOkE && i < length) {
char c;
c = text[i];
if (inQuotes) {
if (c == '"') {
if (i + 1 < length && text[i + 1] == '"') {
status = csvBufByte(&field, '"');
i += 2;
} else {
inQuotes = false;
i++;
}
} else {
status = csvBufByte(&field, c);
i++;
}
continue;
}
if (c == '"' && field.len == 0) {
inQuotes = true;
lineHasContent = true;
i++;
} else if (c == delimiter) {
lineHasContent = true;
status = csvPushField(row, &field);
i++;
} else if (c == '\n' || c == '\r') {
if (c == '\r' && i + 1 < length && text[i + 1] == '\n') {
i++;
}
if (lineHasContent) {
status = csvPushField(row, &field);
if (status == calogOkE) {
status = csvCommitRow(rows, &row);
}
lineHasContent = false;
}
i++;
} else {
lineHasContent = true;
status = csvBufByte(&field, c);
i++;
}
}
// Flush a final row that had no trailing newline.
if (status == calogOkE && lineHasContent) {
status = csvPushField(row, &field);
if (status == calogOkE) {
status = csvCommitRow(rows, &row);
}
}
free(field.data);
if (row != NULL) {
calogAggFree(row); // an unused fresh row (trailing newline) or a partial row on error
}
if (status != calogOkE) {
calogAggFree(rows);
return calogFail(result, status, "csvParse: out of memory");
}
calogValueAgg(result, rows);
return calogOkE;
}

20
libs/calogCsv.h Normal file
View file

@ -0,0 +1,20 @@
// calogCsv.h -- calog CSV library: parse and format RFC 4180 comma-separated values.
//
// csvParse(text: string [, delimiter: string]) -> list(list(string))
// Parse CSV into rows of string cells. Quoted cells ("...") may contain the delimiter,
// newlines, and doubled quotes (""). The optional delimiter is one byte (default ",").
// csvFormat(rows: list(list) [, delimiter: string]) -> string
// Format rows of cells into CSV (records joined by CRLF). A cell is a string, int, real,
// bool, or nil; any cell containing the delimiter, a quote, or a newline is quoted.
//
// Pure and dependency-free; the natives are inline (they run on the calling context's thread).
#ifndef CALOG_CSV_H
#define CALOG_CSV_H
#include "calog.h"
// Register the CSV natives on a runtime. Idempotent across runtimes (no shared state).
int32_t calogCsvRegister(CalogT *calog);
#endif

3
libs/calogDb.c
View file

@ -141,9 +141,10 @@ int32_t calogDbRegister(CalogT *calog) {
// Roll back the refCount taken above so a partially-registered runtime does not
// keep the process-wide registry alive past its own shutdown.
calogDbShutdown();
}
return status;
}
return calogAtDestroy(calog, calogDbShutdown, calogDestroyAfterContextsE);
}
void calogDbShutdown(void) {

2
libs/calogExport.c
View file

@ -55,7 +55,7 @@ int32_t calogExportRegister(CalogT *calog) {
// Case-insensitive variant: my-basic uppercases every identifier at parse time, so its
// bare-name resolver matches an export regardless of the case it was registered with.
calogRegisterInline(calog, "__calogExportResolveFold", exportResolveFold, NULL);
return calogOkE;
return calogAtDestroy(calog, calogExportShutdown, calogDestroyAfterContextsE);
}

806
libs/calogHttpd.c Normal file
View file

@ -0,0 +1,806 @@
// calogHttpd.c -- calog polyglot HTTP server (see calogHttpd.h). Each server owns an acceptor
// thread; per request it parses the message, finds the matching route, invokes that route's handler
// (a CalogFnT, so the call marshals to the handler's owning context thread and runs there), and
// writes the response. Routes are guarded by a mutex; re-registering replaces the handler live.
// v1: HTTP/1.1, Connection: close, one request in flight at a time. No TLS or WebSocket yet.
#define _GNU_SOURCE
#include "calogHttpd.h"
#include "calogHandle.h"
#include "calogInternal.h"
#include "calogPlatform.h"
#include <pthread.h>
#include <stdatomic.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#define HTTPD_TYPE_SERVER 1u
#define HTTPD_HEADER_MAX (64 * 1024)
#define HTTPD_BODY_MAX (16 * 1024 * 1024)
typedef struct HttpdBufT {
char *data;
size_t len;
size_t cap;
} HttpdBufT;
typedef struct RouteT {
char *method; // uppercase, or "*" for any
char *path; // exact match (query stripped)
CalogFnT *handler;
struct RouteT *next;
} RouteT;
typedef struct ServerT {
CalogSocketT listenFd;
pthread_t acceptor;
bool acceptorStarted;
pthread_mutex_t routesMutex;
RouteT *routes;
_Atomic bool stop;
int64_t maxBody;
} ServerT;
typedef struct HttpdLibT {
CalogHandleTableT *handles;
int32_t refCount;
} HttpdLibT;
typedef struct HttpdNativeT {
const char *name;
CalogNativeFnT fn;
} HttpdNativeT;
static pthread_mutex_t gHttpdMutex = PTHREAD_MUTEX_INITIALIZER;
static HttpdLibT *gHttpdLib = NULL;
static void *httpdAcceptor(void *arg);
static int32_t httpdBufAppend(HttpdBufT *buffer, const void *bytes, size_t length);
static void httpdCloser(uint32_t type, void *resource);
static const char *httpdFindHeader(const char *headers, size_t headerLen, const char *name, size_t *valueLen);
static CalogFnT *httpdMatchRoute(ServerT *server, const char *method, const char *path);
static int32_t httpdListen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t httpdParseRequest(ServerT *server, const char *buffer, size_t headerLen, const char *body, size_t bodyLen, CalogValueT *out);
static bool httpdReadRequest(ServerT *server, CalogSocketT fd, HttpdBufT *buffer, size_t *headerLen, size_t *bodyLen);
static const char *httpdReason(int64_t status);
static int32_t httpdRoute(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void httpdServe(ServerT *server, CalogSocketT fd);
static int32_t httpdStop(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t httpdUnroute(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void httpdWriteResponse(CalogSocketT fd, const CalogValueT *response);
static void httpdWriteStatus(HttpdBufT *out, int64_t status, int64_t bodyLen);
static const HttpdNativeT gHttpdNatives[] = {
{ "httpdListen", httpdListen },
{ "httpdRoute", httpdRoute },
{ "httpdUnroute", httpdUnroute },
{ "httpdStop", httpdStop },
};
int32_t calogHttpdRegister(CalogT *calog) {
HttpdLibT *lib;
int32_t status;
size_t index;
pthread_mutex_lock(&gHttpdMutex);
if (gHttpdLib == NULL) {
HttpdLibT *created;
if (calogPlatformNetInit() != 0) {
pthread_mutex_unlock(&gHttpdMutex);
return calogErrUnsupportedE;
}
created = (HttpdLibT *)calloc(1, sizeof(*created));
if (created == NULL) {
calogPlatformNetShutdown();
pthread_mutex_unlock(&gHttpdMutex);
return calogErrOomE;
}
created->handles = calogHandleTableCreate();
if (created->handles == NULL) {
free(created);
calogPlatformNetShutdown();
pthread_mutex_unlock(&gHttpdMutex);
return calogErrOomE;
}
gHttpdLib = created;
}
gHttpdLib->refCount++;
lib = gHttpdLib;
pthread_mutex_unlock(&gHttpdMutex);
status = calogOkE;
for (index = 0; index < sizeof(gHttpdNatives) / sizeof(gHttpdNatives[0]); index++) {
status = calogRegisterInline(calog, gHttpdNatives[index].name, gHttpdNatives[index].fn, lib);
if (status != calogOkE) {
break;
}
}
if (status != calogOkE) {
calogHttpdShutdown();
return status;
}
return calogAtDestroy(calog, calogHttpdShutdown, calogDestroyBeforeContextsE);
}
void calogHttpdShutdown(void) {
pthread_mutex_lock(&gHttpdMutex);
if (gHttpdLib == NULL) {
pthread_mutex_unlock(&gHttpdMutex);
return;
}
gHttpdLib->refCount--;
if (gHttpdLib->refCount <= 0) {
calogHandleTableDestroy(gHttpdLib->handles, httpdCloser);
calogPlatformNetShutdown();
free(gHttpdLib);
gHttpdLib = NULL;
}
pthread_mutex_unlock(&gHttpdMutex);
}
// The acceptor thread: accept a connection, serve it to completion, repeat, until stopped. The
// listen socket is non-blocking and gated by poll() with a short timeout, so the loop notices `stop`
// promptly -- closing the socket from another thread does NOT reliably wake a thread in accept().
static void *httpdAcceptor(void *arg) {
ServerT *server;
server = (ServerT *)arg;
while (!atomic_load(&server->stop)) {
struct pollfd pfd;
CalogSocketT client;
pfd.fd = server->listenFd;
pfd.events = POLLIN;
pfd.revents = 0;
if (calogPoll(&pfd, 1, 200) <= 0) {
continue; // timeout or error -> re-check stop
}
client = accept(server->listenFd, NULL, NULL);
if (client == CALOG_INVALID_SOCKET) {
continue;
}
httpdServe(server, client);
calogSockClose(client);
}
return NULL;
}
static int32_t httpdBufAppend(HttpdBufT *buffer, const void *bytes, size_t length) {
if (buffer->len + length > buffer->cap) {
size_t wanted;
char *grown;
wanted = buffer->cap ? buffer->cap * 2 : 4096;
while (wanted < buffer->len + length) {
wanted *= 2;
}
grown = (char *)realloc(buffer->data, wanted);
if (grown == NULL) {
return calogErrOomE;
}
buffer->data = grown;
buffer->cap = wanted;
}
memcpy(buffer->data + buffer->len, bytes, length);
buffer->len += length;
return calogOkE;
}
static void httpdCloser(uint32_t type, void *resource) {
ServerT *server;
RouteT *route;
if (type != HTTPD_TYPE_SERVER) {
return;
}
server = (ServerT *)resource;
atomic_store(&server->stop, true);
if (server->acceptorStarted) {
pthread_join(server->acceptor, NULL); // exits within one poll timeout of stop
}
if (server->listenFd != CALOG_INVALID_SOCKET) {
calogSockClose(server->listenFd);
server->listenFd = CALOG_INVALID_SOCKET;
}
route = server->routes;
while (route != NULL) {
RouteT *next;
next = route->next;
calogFnRelease(route->handler);
free(route->method);
free(route->path);
free(route);
route = next;
}
pthread_mutex_destroy(&server->routesMutex);
free(server);
}
// Case-insensitive lookup of a header value within the raw header block (NUL-free scan bounded by
// headerLen). Returns a pointer to the value (trimmed of leading spaces) + its length, or NULL.
static const char *httpdFindHeader(const char *headers, size_t headerLen, const char *name, size_t *valueLen) {
size_t nameLen;
size_t i;
nameLen = strlen(name);
for (i = 0; i + nameLen + 1 < headerLen; i++) {
if ((i == 0 || headers[i - 1] == '\n') && strncasecmp(headers + i, name, nameLen) == 0 && headers[i + nameLen] == ':') {
size_t v;
size_t end;
v = i + nameLen + 1;
while (v < headerLen && (headers[v] == ' ' || headers[v] == '\t')) {
v++;
}
end = v;
while (end < headerLen && headers[end] != '\r' && headers[end] != '\n') {
end++;
}
*valueLen = end - v;
return headers + v;
}
}
*valueLen = 0;
return NULL;
}
static int32_t httpdListen(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
HttpdLibT *lib;
ServerT *server;
struct addrinfo hints;
struct addrinfo *res;
struct addrinfo *rp;
CalogSocketT fd;
char portBuffer[8];
int64_t port;
int64_t handle;
int yes;
lib = (HttpdLibT *)userData;
calogValueNil(result);
if (argCount < 1 || argCount > 2 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "httpdListen expects (port [, opts])");
}
if (argCount == 2 && args[1].type != calogAggE) {
return calogFail(result, calogErrArgE, "httpdListen: opts must be a map");
}
port = args[0].as.i;
if (port < 0 || port > 65535) {
return calogFail(result, calogErrArgE, "httpdListen: port out of range");
}
memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_INET;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE;
snprintf(portBuffer, sizeof(portBuffer), "%u", (unsigned int)port);
if (getaddrinfo(NULL, portBuffer, &hints, &res) != 0) {
return calogFail(result, calogErrArgE, "httpdListen: could not resolve the bind address");
}
fd = CALOG_INVALID_SOCKET;
yes = 1;
for (rp = res; rp != NULL; rp = rp->ai_next) {
fd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
if (fd == CALOG_INVALID_SOCKET) {
continue;
}
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (const char *)&yes, sizeof(yes));
if (bind(fd, rp->ai_addr, (socklen_t)rp->ai_addrlen) == 0 && listen(fd, 64) == 0) {
break;
}
calogSockClose(fd);
fd = CALOG_INVALID_SOCKET;
}
freeaddrinfo(res);
if (fd == CALOG_INVALID_SOCKET) {
return calogFail(result, calogErrArgE, "httpdListen: could not bind/listen on the port");
}
server = (ServerT *)calloc(1, sizeof(*server));
if (server == NULL) {
calogSockClose(fd);
return calogFail(result, calogErrOomE, "httpdListen: out of memory");
}
calogSockSetNonblock(fd, 1); // non-blocking + poll-gated accept, so the acceptor can be stopped
server->listenFd = fd;
server->maxBody = HTTPD_BODY_MAX;
pthread_mutex_init(&server->routesMutex, NULL);
handle = calogHandleAdd(lib->handles, HTTPD_TYPE_SERVER, server);
if (handle == 0) {
calogSockClose(fd);
pthread_mutex_destroy(&server->routesMutex);
free(server);
return calogFail(result, calogErrOomE, "httpdListen: out of memory");
}
if (pthread_create(&server->acceptor, NULL, httpdAcceptor, server) != 0) {
calogHandleRemove(lib->handles, handle, HTTPD_TYPE_SERVER);
httpdCloser(HTTPD_TYPE_SERVER, server);
return calogFail(result, calogErrUnsupportedE, "httpdListen: could not start the acceptor thread");
}
server->acceptorStarted = true;
calogValueInt(result, handle);
return calogOkE;
}
// Find the handler for method+path (RETAINED, so it survives a concurrent unroute), or NULL.
static CalogFnT *httpdMatchRoute(ServerT *server, const char *method, const char *path) {
RouteT *route;
CalogFnT *handler;
handler = NULL;
pthread_mutex_lock(&server->routesMutex);
for (route = server->routes; route != NULL; route = route->next) {
if ((strcmp(route->method, "*") == 0 || strcmp(route->method, method) == 0) && strcmp(route->path, path) == 0) {
handler = route->handler;
calogFnRetain(handler);
break;
}
}
pthread_mutex_unlock(&server->routesMutex);
return handler;
}
// Parse the request line + headers + body into a request map { method, path, query, headers, body }.
static int32_t httpdParseRequest(ServerT *server, const char *buffer, size_t headerLen, const char *body, size_t bodyLen, CalogValueT *out) {
CalogAggT *map;
CalogAggT *headers;
const char *p;
const char *lineEnd;
const char *methodEnd;
const char *target;
const char *targetEnd;
const char *query;
size_t i;
int32_t status;
(void)server;
calogValueNil(out);
// Request line: METHOD SP TARGET SP HTTP/x.y
p = buffer;
lineEnd = memchr(buffer, '\n', headerLen);
if (lineEnd == NULL) {
return calogErrArgE;
}
methodEnd = memchr(p, ' ', (size_t)(lineEnd - p));
if (methodEnd == NULL) {
return calogErrArgE;
}
target = methodEnd + 1;
targetEnd = memchr(target, ' ', (size_t)(lineEnd - target));
if (targetEnd == NULL) {
targetEnd = lineEnd;
}
query = memchr(target, '?', (size_t)(targetEnd - target));
status = calogAggCreate(&map, calogMapE);
if (status != calogOkE) {
return status;
}
status = calogMapSetStr(map, "method", p, (int64_t)(methodEnd - p));
if (status == calogOkE) {
const char *pathEnd;
pathEnd = query != NULL ? query : targetEnd;
status = calogMapSetStr(map, "path", target, (int64_t)(pathEnd - target));
}
if (status == calogOkE) {
status = calogMapSetStr(map, "query", query != NULL ? query + 1 : "", query != NULL ? (int64_t)(targetEnd - query - 1) : 0);
}
if (status == calogOkE) {
status = calogMapSetStr(map, "body", bodyLen > 0 ? body : "", (int64_t)bodyLen);
}
if (status != calogOkE) {
calogAggFree(map);
return status;
}
// Headers: each "Name: Value" line after the request line; names lowercased into a map.
status = calogAggCreate(&headers, calogMapE);
if (status != calogOkE) {
calogAggFree(map);
return status;
}
i = (size_t)(lineEnd - buffer) + 1;
while (i < headerLen) {
const char *colon;
const char *nl;
size_t lineLen;
char nameLower[128];
size_t nameLen;
size_t v;
size_t valueEnd;
size_t k;
nl = memchr(buffer + i, '\n', headerLen - i);
lineLen = nl != NULL ? (size_t)(nl - (buffer + i)) : (headerLen - i);
if (lineLen == 0 || (lineLen == 1 && buffer[i] == '\r')) {
break;
}
colon = memchr(buffer + i, ':', lineLen);
if (colon != NULL) {
nameLen = (size_t)(colon - (buffer + i));
if (nameLen >= sizeof(nameLower)) {
nameLen = sizeof(nameLower) - 1;
}
for (k = 0; k < nameLen; k++) {
char c;
c = buffer[i + k];
nameLower[k] = (c >= 'A' && c <= 'Z') ? (char)(c - 'A' + 'a') : c;
}
nameLower[nameLen] = '\0';
v = (size_t)(colon - buffer) + 1;
while (v < i + lineLen && (buffer[v] == ' ' || buffer[v] == '\t')) {
v++;
}
valueEnd = i + lineLen;
while (valueEnd > v && (buffer[valueEnd - 1] == '\r' || buffer[valueEnd - 1] == ' ')) {
valueEnd--;
}
status = calogMapSetStr(headers, nameLower, buffer + v, (int64_t)(valueEnd - v));
if (status != calogOkE) {
calogAggFree(headers);
calogAggFree(map);
return status;
}
}
if (nl == NULL) {
break;
}
i += lineLen + 1;
}
{
CalogValueT headersKey;
CalogValueT headersValue;
calogValueAgg(&headersValue, headers);
if (calogValueString(&headersKey, "headers", 7) != calogOkE) {
calogValueFree(&headersValue);
calogAggFree(map);
return calogErrOomE;
}
status = calogAggSet(map, &headersKey, &headersValue);
if (status != calogOkE) {
calogAggFree(map);
return status;
}
}
calogValueAgg(out, map);
return calogOkE;
}
// Read the request headers (until CRLFCRLF) then the body (per Content-Length). Returns false on a
// closed/oversized/malformed request.
static bool httpdReadRequest(ServerT *server, CalogSocketT fd, HttpdBufT *buffer, size_t *headerLen, size_t *bodyLen) {
const char *marker;
const char *value;
size_t valueLen;
size_t headerEnd;
int64_t contentLength;
size_t have;
marker = NULL;
while (buffer->len < HTTPD_HEADER_MAX) {
char chunk[8192];
ssize_t got;
got = recv(fd, chunk, sizeof(chunk), 0);
if (got <= 0) {
return false;
}
if (httpdBufAppend(buffer, chunk, (size_t)got) != calogOkE) {
return false;
}
marker = memmem(buffer->data, buffer->len, "\r\n\r\n", 4);
if (marker != NULL) {
break;
}
}
if (marker == NULL) {
return false;
}
headerEnd = (size_t)(marker - buffer->data) + 4;
*headerLen = headerEnd;
contentLength = 0;
value = httpdFindHeader(buffer->data, headerEnd, "content-length", &valueLen);
if (value != NULL) {
char tmp[24];
if (valueLen >= sizeof(tmp)) {
return false;
}
memcpy(tmp, value, valueLen);
tmp[valueLen] = '\0';
contentLength = strtoll(tmp, NULL, 10);
if (contentLength < 0 || contentLength > server->maxBody) {
return false;
}
}
have = buffer->len - headerEnd; // body bytes already read past the header
while ((int64_t)have < contentLength) {
char chunk[8192];
ssize_t got;
got = recv(fd, chunk, sizeof(chunk), 0);
if (got <= 0) {
return false;
}
if (httpdBufAppend(buffer, chunk, (size_t)got) != calogOkE) {
return false;
}
have += (size_t)got;
}
*bodyLen = (size_t)contentLength;
return true;
}
static const char *httpdReason(int64_t status) {
switch (status) {
case 200: return "OK";
case 201: return "Created";
case 204: return "No Content";
case 301: return "Moved Permanently";
case 302: return "Found";
case 400: return "Bad Request";
case 401: return "Unauthorized";
case 403: return "Forbidden";
case 404: return "Not Found";
case 405: return "Method Not Allowed";
case 500: return "Internal Server Error";
default: return "Status";
}
}
static int32_t httpdRoute(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
HttpdLibT *lib;
ServerT *server;
RouteT *route;
RouteT *existing;
char *method;
char *path;
size_t i;
lib = (HttpdLibT *)userData;
calogValueNil(result);
if (argCount != 4 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogStringE || args[3].type != calogFnE) {
return calogFail(result, calogErrArgE, "httpdRoute expects (server, method, path, handler)");
}
server = (ServerT *)calogHandleGet(lib->handles, args[0].as.i, HTTPD_TYPE_SERVER);
if (server == NULL) {
return calogFail(result, calogErrArgE, "httpdRoute: invalid server handle");
}
method = strdup(args[1].as.s.bytes);
path = strdup(args[2].as.s.bytes);
if (method == NULL || path == NULL) {
free(method);
free(path);
return calogFail(result, calogErrOomE, "httpdRoute: out of memory");
}
for (i = 0; method[i] != '\0'; i++) {
if (method[i] >= 'a' && method[i] <= 'z') {
method[i] = (char)(method[i] - 'a' + 'A');
}
}
pthread_mutex_lock(&server->routesMutex);
// Re-registering the same method+path replaces the handler live (hot reload).
for (existing = server->routes; existing != NULL; existing = existing->next) {
if (strcmp(existing->method, method) == 0 && strcmp(existing->path, path) == 0) {
calogFnRelease(existing->handler);
calogFnRetain(args[3].as.fn);
existing->handler = args[3].as.fn;
pthread_mutex_unlock(&server->routesMutex);
free(method);
free(path);
return calogOkE;
}
}
route = (RouteT *)calloc(1, sizeof(*route));
if (route == NULL) {
pthread_mutex_unlock(&server->routesMutex);
free(method);
free(path);
return calogFail(result, calogErrOomE, "httpdRoute: out of memory");
}
calogFnRetain(args[3].as.fn);
route->method = method;
route->path = path;
route->handler = args[3].as.fn;
route->next = server->routes;
server->routes = route;
pthread_mutex_unlock(&server->routesMutex);
return calogOkE;
}
// Serve one connection: read + parse the request, invoke the matching handler on its owning context
// thread, write the response.
static void httpdServe(ServerT *server, CalogSocketT fd) {
HttpdBufT buffer;
CalogValueT request;
CalogValueT response;
CalogFnT *handler;
CalogValueT *method;
CalogValueT *path;
size_t headerLen;
size_t bodyLen;
CalogValueT methodKey;
CalogValueT pathKey;
memset(&buffer, 0, sizeof(buffer));
if (!httpdReadRequest(server, fd, &buffer, &headerLen, &bodyLen)) {
free(buffer.data);
return;
}
if (httpdParseRequest(server, buffer.data, headerLen, buffer.data + headerLen, bodyLen, &request) != calogOkE) {
free(buffer.data);
return;
}
free(buffer.data);
// Look up the handler by the parsed method + path.
method = NULL;
path = NULL;
if (calogValueString(&methodKey, "method", 6) == calogOkE) {
method = calogAggGet(request.as.agg, &methodKey);
calogValueFree(&methodKey);
}
if (calogValueString(&pathKey, "path", 4) == calogOkE) {
path = calogAggGet(request.as.agg, &pathKey);
calogValueFree(&pathKey);
}
handler = (method != NULL && path != NULL) ? httpdMatchRoute(server, method->as.s.bytes, path->as.s.bytes) : NULL;
if (handler == NULL) {
static const char notFound[] = "HTTP/1.1 404 Not Found\r\nContent-Length: 9\r\nConnection: close\r\n\r\nNot Found";
send(fd, notFound, sizeof(notFound) - 1, CALOG_MSG_NOSIGNAL);
calogValueFree(&request);
return;
}
calogValueNil(&response);
if (calogFnInvoke(handler, &request, 1, &response) != calogOkE) {
static const char err[] = "HTTP/1.1 500 Internal Server Error\r\nContent-Length: 21\r\nConnection: close\r\n\r\nInternal Server Error";
send(fd, err, sizeof(err) - 1, CALOG_MSG_NOSIGNAL);
} else {
httpdWriteResponse(fd, &response);
}
calogFnRelease(handler);
calogValueFree(&response);
calogValueFree(&request);
}
static int32_t httpdStop(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
HttpdLibT *lib;
ServerT *server;
lib = (HttpdLibT *)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "httpdStop expects (server)");
}
server = (ServerT *)calogHandleRemove(lib->handles, args[0].as.i, HTTPD_TYPE_SERVER);
if (server == NULL) {
return calogFail(result, calogErrArgE, "httpdStop: invalid server handle");
}
httpdCloser(HTTPD_TYPE_SERVER, server);
return calogOkE;
}
static int32_t httpdUnroute(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
HttpdLibT *lib;
ServerT *server;
RouteT **link;
char method[16];
size_t i;
lib = (HttpdLibT *)userData;
calogValueNil(result);
if (argCount != 3 || args[0].type != calogIntE || args[1].type != calogStringE || args[2].type != calogStringE) {
return calogFail(result, calogErrArgE, "httpdUnroute expects (server, method, path)");
}
server = (ServerT *)calogHandleGet(lib->handles, args[0].as.i, HTTPD_TYPE_SERVER);
if (server == NULL) {
return calogFail(result, calogErrArgE, "httpdUnroute: invalid server handle");
}
snprintf(method, sizeof(method), "%s", args[1].as.s.bytes);
for (i = 0; method[i] != '\0'; i++) {
if (method[i] >= 'a' && method[i] <= 'z') {
method[i] = (char)(method[i] - 'a' + 'A');
}
}
pthread_mutex_lock(&server->routesMutex);
for (link = &server->routes; *link != NULL; link = &(*link)->next) {
RouteT *route;
route = *link;
if (strcmp(route->method, method) == 0 && strcmp(route->path, args[2].as.s.bytes) == 0) {
*link = route->next;
calogFnRelease(route->handler);
free(route->method);
free(route->path);
free(route);
break;
}
}
pthread_mutex_unlock(&server->routesMutex);
return calogOkE;
}
// Turn a handler's return value into an HTTP response and write it. nil -> 204; a string -> 200 with
// that body; a map -> { status (default 200), headers (map), body (string) }.
static void httpdWriteResponse(CalogSocketT fd, const CalogValueT *response) {
HttpdBufT out;
const char *body;
int64_t bodyLen;
int64_t status;
memset(&out, 0, sizeof(out));
body = "";
bodyLen = 0;
status = 200;
if (response->type == calogNilE) {
status = 204;
} else if (response->type == calogStringE) {
body = response->as.s.bytes;
bodyLen = response->as.s.length;
} else if (response->type == calogAggE && calogAggIsKeyed(response->as.agg)) {
CalogValueT key;
CalogValueT *field;
if (calogValueString(&key, "status", 6) == calogOkE) {
field = calogAggGet(response->as.agg, &key);
calogValueFree(&key);
if (field != NULL && field->type == calogIntE) {
status = field->as.i;
}
}
if (calogValueString(&key, "body", 4) == calogOkE) {
field = calogAggGet(response->as.agg, &key);
calogValueFree(&key);
if (field != NULL && field->type == calogStringE) {
body = field->as.s.bytes;
bodyLen = field->as.s.length;
}
}
httpdWriteStatus(&out, status, bodyLen);
// Custom headers, if any (Content-Length + Connection are set by httpdWriteStatus).
if (calogValueString(&key, "headers", 7) == calogOkE) {
field = calogAggGet(response->as.agg, &key);
calogValueFree(&key);
if (field != NULL && field->type == calogAggE && calogAggIsKeyed(field->as.agg)) {
int64_t h;
for (h = 0; h < field->as.agg->pairCount; h++) {
if (field->as.agg->pairs[h].key.type == calogStringE && field->as.agg->pairs[h].value.type == calogStringE) {
httpdBufAppend(&out, field->as.agg->pairs[h].key.as.s.bytes, (size_t)field->as.agg->pairs[h].key.as.s.length);
httpdBufAppend(&out, ": ", 2);
httpdBufAppend(&out, field->as.agg->pairs[h].value.as.s.bytes, (size_t)field->as.agg->pairs[h].value.as.s.length);
httpdBufAppend(&out, "\r\n", 2);
}
}
}
}
httpdBufAppend(&out, "\r\n", 2);
httpdBufAppend(&out, body, (size_t)bodyLen);
send(fd, out.data, out.len, CALOG_MSG_NOSIGNAL);
free(out.data);
return;
}
httpdWriteStatus(&out, status, bodyLen);
httpdBufAppend(&out, "\r\n", 2);
httpdBufAppend(&out, body, (size_t)bodyLen);
send(fd, out.data, out.len, CALOG_MSG_NOSIGNAL);
free(out.data);
}
// Write the status line + the always-present Content-Length and Connection: close headers.
static void httpdWriteStatus(HttpdBufT *out, int64_t status, int64_t bodyLen) {
char header[128];
int n;
n = snprintf(header, sizeof(header), "HTTP/1.1 %lld %s\r\nContent-Length: %lld\r\nConnection: close\r\n", (long long)status, httpdReason(status), (long long)bodyLen);
if (n > 0) {
httpdBufAppend(out, header, (size_t)n);
}
}

33
libs/calogHttpd.h Normal file
View file

@ -0,0 +1,33 @@
// calogHttpd.h -- calog polyglot HTTP server: serve routes whose handlers are script functions.
//
// A script registers route handlers (function values, in ANY engine); the server accepts
// connections on its own thread and, per request, invokes the matching handler ON ITS OWNING
// CONTEXT THREAD (via the callable machinery), then writes the returned response. Because a handler
// is an ordinary calog function value, re-registering a route replaces it live -- hot reload -- and
// different routes can be written in different languages and still share helpers via calogExport.
//
// httpdListen(port [, optsMap]) -> serverHandle opts: { host, backlog, maxBody }
// httpdRoute(serverHandle, method, path, handler) method "*" = any; exact-path match; re-register replaces
// httpdUnroute(serverHandle, method, path)
// httpdStop(serverHandle) stop accepting, close, release handlers
//
// The handler receives a request map { method, path, query, headers (map, lowercased names), body }
// and returns a response: a map { status (default 200), headers (map), body (string) }, a bare
// string (=> 200 with that body), or nil (=> 204 No Content). Bodies are binary-safe.
//
// v1 serves HTTP/1.1 with Connection: close and one request in flight at a time (a handler runs to
// completion before the next connection is accepted). TLS (https) and WebSocket are not yet included.
#ifndef CALOG_HTTPD_H
#define CALOG_HTTPD_H
#include "calog.h"
// Register the httpd natives on a runtime. Idempotent across runtimes.
int32_t calogHttpdRegister(CalogT *calog);
// Stop every server and release route handlers (once the last registered runtime unregisters).
// Call while the handler-owning contexts are still alive, before calogDestroy.
void calogHttpdShutdown(void);
#endif

2
libs/calogKv.c
View file

@ -56,7 +56,7 @@ int32_t calogKvRegister(CalogT *calog) {
calogRegisterInline(calog, "kvHas", kvHas, NULL);
calogRegisterInline(calog, "kvDelete", kvDelete, NULL);
calogRegisterInline(calog, "kvKeys", kvKeys, NULL);
return calogOkE;
return calogAtDestroy(calog, calogKvShutdown, calogDestroyAfterContextsE);
}

2
libs/calogNet.c
View file

@ -148,7 +148,7 @@ int32_t calogNetRegister(CalogT *calog) {
calogNetShutdown();
return status;
}
return calogOkE;
return calogAtDestroy(calog, calogNetShutdown, calogDestroyAfterContextsE);
}

383
libs/calogProc.c Normal file
View file

@ -0,0 +1,383 @@
// calogProc.c -- calog subprocess library (see calogProc.h). Spawns a child with posix_spawn (NOT
// fork -- calog runs many pthreads), feeds its stdin while draining stdout/stderr through one poll
// loop so a large transfer cannot deadlock, and returns its exit code + captured output. POSIX-only;
// on Windows procRun reports "unsupported" rather than shipping a half-working implementation.
#define _GNU_SOURCE
#include "calogProc.h"
#include "calogInternal.h"
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#ifndef _WIN32
#include <errno.h>
#include <fcntl.h>
#include <poll.h>
#include <spawn.h>
#include <sys/wait.h>
#include <unistd.h>
extern char **environ;
#endif
// Upper bound on captured stdout/stderr, so a runaway child cannot exhaust memory.
#define PROC_MAX (64 * 1024 * 1024)
typedef struct ProcBufT {
char *data;
size_t len;
size_t cap;
} ProcBufT;
static int32_t procBufAppend(ProcBufT *buffer, const void *bytes, size_t length);
static CalogValueT *procOpt(CalogAggT *opts, const char *name);
static int32_t procRun(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
int32_t calogProcRegister(CalogT *calog) {
return calogRegisterInline(calog, "procRun", procRun, NULL);
}
static int32_t procBufAppend(ProcBufT *buffer, const void *bytes, size_t length) {
if (buffer->len + length > buffer->cap) {
size_t wanted;
char *grown;
wanted = buffer->cap ? buffer->cap * 2 : 4096;
while (wanted < buffer->len + length) {
wanted *= 2;
}
grown = (char *)realloc(buffer->data, wanted);
if (grown == NULL) {
return calogErrOomE;
}
buffer->data = grown;
buffer->cap = wanted;
}
memcpy(buffer->data + buffer->len, bytes, length);
buffer->len += length;
return calogOkE;
}
static CalogValueT *procOpt(CalogAggT *opts, const char *name) {
CalogValueT key;
CalogValueT *field;
if (opts == NULL) {
return NULL;
}
if (calogValueString(&key, name, (int64_t)strlen(name)) != calogOkE) {
return NULL;
}
field = calogAggGet(opts, &key);
calogValueFree(&key);
return field;
}
#ifdef _WIN32
static int32_t procRun(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
calogValueNil(result);
return calogFail(result, calogErrUnsupportedE, "procRun: subprocess spawning is not supported on Windows in this build");
}
#else
// Build a result map { exit, stdout, stderr } and hand ownership to result.
static int32_t procResult(CalogValueT *result, int32_t exitCode, ProcBufT *out, ProcBufT *err) {
CalogAggT *map;
int32_t status;
status = calogAggCreate(&map, calogMapE);
if (status != calogOkE) {
return calogFail(result, status, "procRun: out of memory");
}
status = calogMapSetInt(map, "exit", exitCode);
if (status == calogOkE) {
status = calogMapSetStr(map, "stdout", out->data ? out->data : "", (int64_t)out->len);
}
if (status == calogOkE) {
status = calogMapSetStr(map, "stderr", err->data ? err->data : "", (int64_t)err->len);
}
if (status != calogOkE) {
calogAggFree(map);
return calogFail(result, status, "procRun: failed to build the result");
}
calogValueAgg(result, map);
return calogOkE;
}
static int32_t procRun(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
posix_spawn_file_actions_t actions;
ProcBufT outBuf;
ProcBufT errBuf;
char **argv;
char **builtEnv;
char **envp;
const char *stdinBytes;
const char *cwd;
CalogValueT *field;
int64_t stdinLen;
int64_t stdinPos;
int64_t argCountList;
int64_t i;
int inPipe[2];
int outPipe[2];
int errPipe[2];
int spawnRc;
int waitStatus;
int32_t exitCode;
int32_t status;
pid_t pid;
bool actionsReady;
(void)userData;
calogValueNil(result);
if (argCount < 1 || argCount > 2 || args[0].type != calogAggE || calogAggIsKeyed(args[0].as.agg)) {
return calogFail(result, calogErrArgE, "procRun expects (argvList [, opts])");
}
if (argCount == 2 && (args[1].type != calogAggE || !calogAggIsKeyed(args[1].as.agg))) {
return calogFail(result, calogErrArgE, "procRun: opts must be a map");
}
argCountList = args[0].as.agg->arrayCount;
if (argCountList < 1) {
return calogFail(result, calogErrArgE, "procRun: argvList must have at least the program");
}
argv = (char **)calloc((size_t)argCountList + 1, sizeof(char *));
if (argv == NULL) {
return calogFail(result, calogErrOomE, "procRun: out of memory");
}
for (i = 0; i < argCountList; i++) {
if (args[0].as.agg->array[i].type != calogStringE) {
free(argv);
return calogFail(result, calogErrArgE, "procRun: every argv element must be a string");
}
argv[i] = args[0].as.agg->array[i].as.s.bytes;
}
stdinBytes = NULL;
stdinLen = 0;
cwd = NULL;
builtEnv = NULL;
envp = environ;
if (argCount == 2) {
field = procOpt(args[1].as.agg, "stdin");
if (field != NULL && field->type == calogStringE) {
stdinBytes = field->as.s.bytes;
stdinLen = field->as.s.length;
}
field = procOpt(args[1].as.agg, "cwd");
if (field != NULL && field->type == calogStringE) {
cwd = field->as.s.bytes;
}
field = procOpt(args[1].as.agg, "env");
if (field != NULL && field->type == calogAggE && calogAggIsKeyed(field->as.agg)) {
CalogAggT *envMap;
int64_t envCount;
int64_t e;
envMap = field->as.agg;
envCount = envMap->pairCount;
builtEnv = (char **)calloc((size_t)envCount + 1, sizeof(char *));
if (builtEnv == NULL) {
free(argv);
return calogFail(result, calogErrOomE, "procRun: out of memory");
}
for (e = 0; e < envCount; e++) {
const char *k;
const char *v;
size_t klen;
size_t vlen;
if (envMap->pairs[e].key.type != calogStringE || envMap->pairs[e].value.type != calogStringE) {
continue;
}
k = envMap->pairs[e].key.as.s.bytes;
v = envMap->pairs[e].value.as.s.bytes;
klen = (size_t)envMap->pairs[e].key.as.s.length;
vlen = (size_t)envMap->pairs[e].value.as.s.length;
builtEnv[e] = (char *)malloc(klen + vlen + 2);
if (builtEnv[e] == NULL) {
continue;
}
memcpy(builtEnv[e], k, klen);
builtEnv[e][klen] = '=';
memcpy(builtEnv[e] + klen + 1, v, vlen);
builtEnv[e][klen + 1 + vlen] = '\0';
}
envp = builtEnv;
}
}
if (pipe(inPipe) != 0 || pipe(outPipe) != 0 || pipe(errPipe) != 0) {
free(argv);
if (builtEnv != NULL) {
for (i = 0; builtEnv[i] != NULL; i++) {
free(builtEnv[i]);
}
free(builtEnv);
}
return calogFail(result, calogErrArgE, "procRun: could not create pipes");
}
posix_spawn_file_actions_init(&actions);
actionsReady = true;
if (cwd != NULL) {
posix_spawn_file_actions_addchdir_np(&actions, cwd);
}
posix_spawn_file_actions_adddup2(&actions, inPipe[0], 0);
posix_spawn_file_actions_adddup2(&actions, outPipe[1], 1);
posix_spawn_file_actions_adddup2(&actions, errPipe[1], 2);
posix_spawn_file_actions_addclose(&actions, inPipe[1]);
posix_spawn_file_actions_addclose(&actions, outPipe[0]);
posix_spawn_file_actions_addclose(&actions, errPipe[0]);
spawnRc = posix_spawnp(&pid, argv[0], &actions, NULL, argv, envp);
posix_spawn_file_actions_destroy(&actions);
(void)actionsReady;
// Parent keeps the outer ends: write inPipe[1], read outPipe[0]/errPipe[0].
close(inPipe[0]);
close(outPipe[1]);
close(errPipe[1]);
free(argv);
if (builtEnv != NULL) {
for (i = 0; builtEnv[i] != NULL; i++) {
free(builtEnv[i]);
}
free(builtEnv);
}
if (spawnRc != 0) {
close(inPipe[1]);
close(outPipe[0]);
close(errPipe[0]);
return calogFail(result, calogErrArgE, strerror(spawnRc));
}
fcntl(inPipe[1], F_SETFL, fcntl(inPipe[1], F_GETFL, 0) | O_NONBLOCK);
fcntl(outPipe[0], F_SETFL, fcntl(outPipe[0], F_GETFL, 0) | O_NONBLOCK);
fcntl(errPipe[0], F_SETFL, fcntl(errPipe[0], F_GETFL, 0) | O_NONBLOCK);
memset(&outBuf, 0, sizeof(outBuf));
memset(&errBuf, 0, sizeof(errBuf));
stdinPos = 0;
if (stdinLen == 0) {
close(inPipe[1]);
inPipe[1] = -1;
}
status = calogOkE;
while (inPipe[1] >= 0 || outPipe[0] >= 0 || errPipe[0] >= 0) {
struct pollfd fds[3];
int nfds;
int inIdx;
int outIdx;
int errIdx;
nfds = 0;
inIdx = -1;
outIdx = -1;
errIdx = -1;
if (inPipe[1] >= 0) {
fds[nfds].fd = inPipe[1];
fds[nfds].events = POLLOUT;
inIdx = nfds++;
}
if (outPipe[0] >= 0) {
fds[nfds].fd = outPipe[0];
fds[nfds].events = POLLIN;
outIdx = nfds++;
}
if (errPipe[0] >= 0) {
fds[nfds].fd = errPipe[0];
fds[nfds].events = POLLIN;
errIdx = nfds++;
}
if (poll(fds, (nfds_t)nfds, -1) < 0) {
if (errno == EINTR) {
continue;
}
status = calogFail(result, calogErrArgE, strerror(errno));
break;
}
if (inIdx >= 0 && (fds[inIdx].revents & (POLLOUT | POLLERR | POLLHUP))) {
ssize_t wrote;
wrote = write(inPipe[1], stdinBytes + stdinPos, (size_t)(stdinLen - stdinPos));
if (wrote > 0) {
stdinPos += wrote;
}
if (wrote < 0 && errno != EAGAIN && errno != EWOULDBLOCK && errno != EINTR) {
close(inPipe[1]);
inPipe[1] = -1;
} else if (stdinPos >= stdinLen) {
close(inPipe[1]);
inPipe[1] = -1;
}
}
if (outIdx >= 0 && (fds[outIdx].revents & (POLLIN | POLLERR | POLLHUP))) {
char chunk[65536];
ssize_t got;
got = read(outPipe[0], chunk, sizeof(chunk));
if (got > 0) {
if (outBuf.len + (size_t)got > PROC_MAX || procBufAppend(&outBuf, chunk, (size_t)got) != calogOkE) {
status = calogFail(result, calogErrRangeE, "procRun: stdout exceeds the size cap");
close(outPipe[0]);
outPipe[0] = -1;
}
} else if (got == 0 || (got < 0 && errno != EAGAIN && errno != EWOULDBLOCK && errno != EINTR)) {
close(outPipe[0]);
outPipe[0] = -1;
}
}
if (errIdx >= 0 && (fds[errIdx].revents & (POLLIN | POLLERR | POLLHUP))) {
char chunk[65536];
ssize_t got;
got = read(errPipe[0], chunk, sizeof(chunk));
if (got > 0) {
if (errBuf.len + (size_t)got > PROC_MAX || procBufAppend(&errBuf, chunk, (size_t)got) != calogOkE) {
status = calogFail(result, calogErrRangeE, "procRun: stderr exceeds the size cap");
close(errPipe[0]);
errPipe[0] = -1;
}
} else if (got == 0 || (got < 0 && errno != EAGAIN && errno != EWOULDBLOCK && errno != EINTR)) {
close(errPipe[0]);
errPipe[0] = -1;
}
}
if (status != calogOkE) {
break;
}
}
if (inPipe[1] >= 0) {
close(inPipe[1]);
}
if (outPipe[0] >= 0) {
close(outPipe[0]);
}
if (errPipe[0] >= 0) {
close(errPipe[0]);
}
while (waitpid(pid, &waitStatus, 0) < 0 && errno == EINTR) {
continue;
}
if (status != calogOkE) {
free(outBuf.data);
free(errBuf.data);
return status;
}
if (WIFEXITED(waitStatus)) {
exitCode = (int32_t)WEXITSTATUS(waitStatus);
} else if (WIFSIGNALED(waitStatus)) {
exitCode = -(int32_t)WTERMSIG(waitStatus);
} else {
exitCode = -1;
}
status = procResult(result, exitCode, &outBuf, &errBuf);
free(outBuf.data);
free(errBuf.data);
return status;
}
#endif

23
libs/calogProc.h Normal file
View file

@ -0,0 +1,23 @@
// calogProc.h -- calog subprocess library: run a child process, capture its output.
//
// procRun(argv: list(string) [, opts: map]) -> map{ exit: int, stdout: string, stderr: string }
// Spawn argv[0] with the given arguments, wait for it, and return its exit code (or the
// negative signal number if it was killed) plus its captured stdout/stderr (binary-safe).
// opts: { stdin: string (fed to the child), cwd: string (working directory), env: map
// (string->string; REPLACES the environment -- omit to inherit the parent's) }.
//
// The child is started with posix_spawn (NOT fork -- calog runs many pthreads, and fork in a
// multithreaded process is a well-known hazard). The native is inline and blocks the calling
// context's own thread until the child exits, feeding stdin while draining stdout/stderr through a
// single poll loop so a large transfer cannot deadlock. Subprocess spawning is POSIX-only; on
// Windows the native returns an "unsupported" error rather than a half-working implementation.
#ifndef CALOG_PROC_H
#define CALOG_PROC_H
#include "calog.h"
// Register the subprocess native on a runtime. Idempotent across runtimes (no shared state).
int32_t calogProcRegister(CalogT *calog);
#endif

2
libs/calogPubsub.c
View file

@ -59,7 +59,7 @@ int32_t calogPubsubRegister(CalogT *calog) {
calogRegisterInline(calog, "psSubscribe", pubsubSubscribe, NULL);
calogRegisterInline(calog, "psUnsubscribe", pubsubUnsubscribe, NULL);
calogRegisterInline(calog, "psPublish", pubsubPublish, NULL);
return calogOkE;
return calogAtDestroy(calog, calogPubsubShutdown, calogDestroyAfterContextsE);
}

398
libs/calogRegex.c Normal file
View file

@ -0,0 +1,398 @@
// calogRegex.c -- calog regex library (see calogRegex.h), over PCRE2 (8-bit, interpreted, no JIT).
// A pattern is compiled per call (no cache in v1); subjects/results are binary-safe. Natives inline.
#include "calogRegex.h"
#include "calogInternal.h"
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#define PCRE2_CODE_UNIT_WIDTH 8
#include <pcre2.h>
static int32_t regexCompile(const CalogValueT *patternArg, const char *flags, int64_t flagsLen, bool *global, pcre2_code **out, CalogValueT *result);
static int32_t regexGroups(const char *subject, PCRE2_SIZE *ovector, uint32_t count, bool includeFull, CalogAggT **out);
static int32_t regexMatch(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static uint32_t regexOptions(const char *flags, int64_t flagsLen, bool *global);
static int32_t regexReplace(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t regexSearch(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t regexSplit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
int32_t calogRegexRegister(CalogT *calog) {
int32_t status;
status = calogRegisterInline(calog, "regexMatch", regexMatch, NULL);
if (status == calogOkE) {
status = calogRegisterInline(calog, "regexSearch", regexSearch, NULL);
}
if (status == calogOkE) {
status = calogRegisterInline(calog, "regexReplace", regexReplace, NULL);
}
if (status == calogOkE) {
status = calogRegisterInline(calog, "regexSplit", regexSplit, NULL);
}
return status;
}
static uint32_t regexOptions(const char *flags, int64_t flagsLen, bool *global) {
uint32_t options;
int64_t i;
options = 0;
*global = false;
for (i = 0; i < flagsLen; i++) {
switch (flags[i]) {
case 'i': options |= PCRE2_CASELESS; break;
case 'm': options |= PCRE2_MULTILINE; break;
case 's': options |= PCRE2_DOTALL; break;
case 'x': options |= PCRE2_EXTENDED; break;
case 'g': *global = true; break;
default: break;
}
}
return options;
}
// Compile args[argIndex 0] as the pattern with the parsed flags; on error set result and return it.
static int32_t regexCompile(const CalogValueT *patternArg, const char *flags, int64_t flagsLen, bool *global, pcre2_code **out, CalogValueT *result) {
pcre2_code *code;
uint32_t options;
int errorCode;
PCRE2_SIZE errorOffset;
*out = NULL;
options = regexOptions(flags, flagsLen, global);
code = pcre2_compile((PCRE2_SPTR)patternArg->as.s.bytes, (PCRE2_SIZE)patternArg->as.s.length, options, &errorCode, &errorOffset, NULL);
if (code == NULL) {
PCRE2_UCHAR message[256];
pcre2_get_error_message(errorCode, message, sizeof(message));
return calogFail(result, calogErrArgE, (const char *)message);
}
*out = code;
return calogOkE;
}
// Build a list of the captured groups from the ovector (group 0 = whole match included only when
// includeFull). A non-participating group (offset PCRE2_UNSET) becomes nil.
static int32_t regexGroups(const char *subject, PCRE2_SIZE *ovector, uint32_t count, bool includeFull, CalogAggT **out) {
CalogAggT *list;
uint32_t start;
uint32_t i;
int32_t status;
*out = NULL;
status = calogAggCreate(&list, calogListE);
if (status != calogOkE) {
return status;
}
start = includeFull ? 0u : 1u;
for (i = start; i < count; i++) {
CalogValueT value;
if (ovector[2 * i] == PCRE2_UNSET) {
calogValueNil(&value);
} else {
PCRE2_SIZE from;
PCRE2_SIZE to;
from = ovector[2 * i];
to = ovector[2 * i + 1];
status = calogValueString(&value, subject + from, (int64_t)(to - from));
if (status != calogOkE) {
calogAggFree(list);
return status;
}
}
status = calogAggPush(list, &value);
if (status != calogOkE) {
calogValueFree(&value);
calogAggFree(list);
return status;
}
}
*out = list;
return calogOkE;
}
static int32_t regexMatch(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
pcre2_code *code;
pcre2_match_data *matchData;
CalogAggT *groups;
const char *flags;
int64_t flagsLen;
bool global;
int rc;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 2 || argCount > 3 || args[0].type != calogStringE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexMatch expects (pattern, subject [, flags])");
}
if (argCount == 3 && args[2].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexMatch: flags must be a string");
}
flags = argCount == 3 ? args[2].as.s.bytes : "";
flagsLen = argCount == 3 ? args[2].as.s.length : 0;
status = regexCompile(&args[0], flags, flagsLen, &global, &code, result);
if (status != calogOkE) {
return status;
}
matchData = pcre2_match_data_create_from_pattern(code, NULL);
if (matchData == NULL) {
pcre2_code_free(code);
return calogFail(result, calogErrOomE, "regexMatch: out of memory");
}
rc = pcre2_match(code, (PCRE2_SPTR)args[1].as.s.bytes, (PCRE2_SIZE)args[1].as.s.length, 0, 0, matchData, NULL);
if (rc < 0) {
pcre2_match_data_free(matchData);
pcre2_code_free(code);
return calogOkE; // PCRE2_ERROR_NOMATCH (and any other failure) -> nil
}
status = regexGroups(args[1].as.s.bytes, pcre2_get_ovector_pointer(matchData), (uint32_t)rc, true, &groups);
pcre2_match_data_free(matchData);
pcre2_code_free(code);
if (status != calogOkE) {
return calogFail(result, status, "regexMatch: out of memory");
}
calogValueAgg(result, groups);
return calogOkE;
}
static int32_t regexReplace(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
pcre2_code *code;
PCRE2_UCHAR *output;
PCRE2_SIZE outLen;
const char *flags;
int64_t flagsLen;
bool global;
uint32_t options;
int rc;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 3 || argCount > 4 || args[0].type != calogStringE || args[1].type != calogStringE || args[2].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexReplace expects (pattern, subject, replacement [, flags])");
}
if (argCount == 4 && args[3].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexReplace: flags must be a string");
}
flags = argCount == 4 ? args[3].as.s.bytes : "";
flagsLen = argCount == 4 ? args[3].as.s.length : 0;
status = regexCompile(&args[0], flags, flagsLen, &global, &code, result);
if (status != calogOkE) {
return status;
}
options = PCRE2_SUBSTITUTE_EXTENDED | PCRE2_SUBSTITUTE_OVERFLOW_LENGTH;
if (global) {
options |= PCRE2_SUBSTITUTE_GLOBAL;
}
// First pass: OVERFLOW_LENGTH makes PCRE2 report the needed length in outLen if the buffer is
// too small (a guess start avoids a second call for the common case).
outLen = (PCRE2_SIZE)args[1].as.s.length + 256;
output = (PCRE2_UCHAR *)malloc(outLen);
if (output == NULL) {
pcre2_code_free(code);
return calogFail(result, calogErrOomE, "regexReplace: out of memory");
}
rc = pcre2_substitute(code, (PCRE2_SPTR)args[1].as.s.bytes, (PCRE2_SIZE)args[1].as.s.length, 0, options, NULL, NULL,
(PCRE2_SPTR)args[2].as.s.bytes, (PCRE2_SIZE)args[2].as.s.length, output, &outLen);
if (rc == PCRE2_ERROR_NOMEMORY) {
PCRE2_UCHAR *grown;
grown = (PCRE2_UCHAR *)realloc(output, outLen);
if (grown == NULL) {
free(output);
pcre2_code_free(code);
return calogFail(result, calogErrOomE, "regexReplace: out of memory");
}
output = grown;
rc = pcre2_substitute(code, (PCRE2_SPTR)args[1].as.s.bytes, (PCRE2_SIZE)args[1].as.s.length, 0, options, NULL, NULL,
(PCRE2_SPTR)args[2].as.s.bytes, (PCRE2_SIZE)args[2].as.s.length, output, &outLen);
}
pcre2_code_free(code);
if (rc < 0) {
PCRE2_UCHAR message[256];
pcre2_get_error_message(rc, message, sizeof(message));
free(output);
return calogFail(result, calogErrArgE, (const char *)message);
}
status = calogValueString(result, (const char *)output, (int64_t)outLen);
free(output);
return status;
}
static int32_t regexSearch(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
pcre2_code *code;
pcre2_match_data *matchData;
CalogAggT *map;
CalogAggT *groups;
PCRE2_SIZE *ovector;
const char *flags;
int64_t flagsLen;
bool global;
int rc;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 2 || argCount > 3 || args[0].type != calogStringE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexSearch expects (pattern, subject [, flags])");
}
if (argCount == 3 && args[2].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexSearch: flags must be a string");
}
flags = argCount == 3 ? args[2].as.s.bytes : "";
flagsLen = argCount == 3 ? args[2].as.s.length : 0;
status = regexCompile(&args[0], flags, flagsLen, &global, &code, result);
if (status != calogOkE) {
return status;
}
matchData = pcre2_match_data_create_from_pattern(code, NULL);
if (matchData == NULL) {
pcre2_code_free(code);
return calogFail(result, calogErrOomE, "regexSearch: out of memory");
}
rc = pcre2_match(code, (PCRE2_SPTR)args[1].as.s.bytes, (PCRE2_SIZE)args[1].as.s.length, 0, 0, matchData, NULL);
if (rc < 0) {
pcre2_match_data_free(matchData);
pcre2_code_free(code);
return calogOkE; // no match -> nil
}
ovector = pcre2_get_ovector_pointer(matchData);
status = regexGroups(args[1].as.s.bytes, ovector, (uint32_t)rc, false, &groups);
if (status != calogOkE) {
pcre2_match_data_free(matchData);
pcre2_code_free(code);
return calogFail(result, status, "regexSearch: out of memory");
}
status = calogAggCreate(&map, calogMapE);
if (status == calogOkE) {
status = calogMapSetStr(map, "match", args[1].as.s.bytes + ovector[0], (int64_t)(ovector[1] - ovector[0]));
}
if (status == calogOkE) {
status = calogMapSetInt(map, "start", (int64_t)ovector[0]);
}
if (status == calogOkE) {
status = calogMapSetInt(map, "end", (int64_t)ovector[1]);
}
if (status == calogOkE) {
CalogValueT groupsKey;
CalogValueT groupsValue;
calogValueAgg(&groupsValue, groups);
if (calogValueString(&groupsKey, "groups", 6) == calogOkE) {
status = calogAggSet(map, &groupsKey, &groupsValue); // takes ownership of key + value
} else {
calogValueFree(&groupsValue);
status = calogErrOomE;
}
} else {
calogAggFree(groups);
}
pcre2_match_data_free(matchData);
pcre2_code_free(code);
if (status != calogOkE) {
calogAggFree(map);
return calogFail(result, status, "regexSearch: failed to build the result");
}
calogValueAgg(result, map);
return calogOkE;
}
static int32_t regexSplit(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
pcre2_code *code;
pcre2_match_data *matchData;
CalogAggT *pieces;
const char *subject;
const char *flags;
int64_t subjectLen;
int64_t flagsLen;
PCRE2_SIZE offset;
bool global;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount < 2 || argCount > 3 || args[0].type != calogStringE || args[1].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexSplit expects (pattern, subject [, flags])");
}
if (argCount == 3 && args[2].type != calogStringE) {
return calogFail(result, calogErrArgE, "regexSplit: flags must be a string");
}
flags = argCount == 3 ? args[2].as.s.bytes : "";
flagsLen = argCount == 3 ? args[2].as.s.length : 0;
status = regexCompile(&args[0], flags, flagsLen, &global, &code, result);
if (status != calogOkE) {
return status;
}
matchData = pcre2_match_data_create_from_pattern(code, NULL);
if (matchData == NULL) {
pcre2_code_free(code);
return calogFail(result, calogErrOomE, "regexSplit: out of memory");
}
status = calogAggCreate(&pieces, calogListE);
if (status != calogOkE) {
pcre2_match_data_free(matchData);
pcre2_code_free(code);
return calogFail(result, status, "regexSplit: out of memory");
}
subject = args[1].as.s.bytes;
subjectLen = args[1].as.s.length;
offset = 0;
while (status == calogOkE) {
PCRE2_SIZE *ovector;
PCRE2_SIZE matchStart;
PCRE2_SIZE matchEnd;
CalogValueT piece;
int rc;
rc = pcre2_match(code, (PCRE2_SPTR)subject, (PCRE2_SIZE)subjectLen, offset, 0, matchData, NULL);
if (rc < 0) {
break;
}
ovector = pcre2_get_ovector_pointer(matchData);
matchStart = ovector[0];
matchEnd = ovector[1];
if (matchEnd == matchStart) {
// Empty match: advance one byte so the split terminates rather than looping.
if (matchStart >= (PCRE2_SIZE)subjectLen) {
break;
}
offset = matchStart + 1;
continue;
}
status = calogValueString(&piece, subject + offset, (int64_t)(matchStart - offset));
if (status == calogOkE) {
status = calogAggPush(pieces, &piece);
if (status != calogOkE) {
calogValueFree(&piece);
}
}
offset = matchEnd;
}
if (status == calogOkE) {
CalogValueT tail;
status = calogValueString(&tail, subject + offset, (int64_t)(subjectLen - (int64_t)offset));
if (status == calogOkE) {
status = calogAggPush(pieces, &tail);
if (status != calogOkE) {
calogValueFree(&tail);
}
}
}
pcre2_match_data_free(matchData);
pcre2_code_free(code);
if (status != calogOkE) {
calogAggFree(pieces);
return calogFail(result, status, "regexSplit: out of memory");
}
calogValueAgg(result, pieces);
return calogOkE;
}

21
libs/calogRegex.h Normal file
View file

@ -0,0 +1,21 @@
// calogRegex.h -- calog regular-expression library over PCRE2 (Perl-compatible).
//
// regexMatch(pattern, subject [, flags]) -> nil | list(fullMatch, group1, ...)
// regexSearch(pattern, subject [, flags]) -> nil | map{ match, start, end, groups:list }
// regexReplace(pattern, subject, replacement [, flags]) -> string (replacement: $1 / ${name}; "g" = all)
// regexSplit(pattern, subject [, flags]) -> list(string)
//
// flags is a string of single-letter options: "i" case-insensitive, "m" multiline (^/$ per line),
// "s" dot-matches-newline, "x" extended (ignore whitespace), and "g" replace-all (regexReplace only).
// A non-participating capture group is nil. Subjects and results are binary-safe; the natives are
// inline (they run on the calling context's thread). Matching is interpreted (JIT is not built).
#ifndef CALOG_REGEX_H
#define CALOG_REGEX_H
#include "calog.h"
// Register the regex natives on a runtime. Idempotent across runtimes (no shared state).
int32_t calogRegexRegister(CalogT *calog);
#endif

2
libs/calogSsh.c
View file

@ -141,7 +141,7 @@ int32_t calogSshRegister(CalogT *calog) {
calogRegisterInline(calog, "sftpStat", sftpStat, gSshLib);
calogRegisterInline(calog, "sftpRemove", sftpRemove, gSshLib);
calogRegisterInline(calog, "sftpMkdir", sftpMkdir, gSshLib);
return calogOkE;
return calogAtDestroy(calog, calogSshShutdown, calogDestroyAfterContextsE);
}

5
libs/calogTask.c
View file

@ -75,6 +75,9 @@ static const TaskEngineT gTaskEngines[] = {
#endif
#ifdef CALOG_WITH_TCL
{ "tcl", &calogTclEngine },
#endif
#ifdef CALOG_WITH_JANET
{ "janet", &calogJanetEngine },
#endif
{ NULL, NULL }
};
@ -123,7 +126,7 @@ int32_t calogTaskRegister(CalogT *calog) {
calogRegisterInline(calog, "taskExit", taskExit, gTaskLib);
calogRegisterInline(calog, "taskSelf", taskSelf, gTaskLib);
calogRegisterInline(calog, "taskCount", taskCount, gTaskLib);
return calogOkE;
return calogAtDestroy(calog, calogTaskShutdown, calogDestroyAfterContextsE);
}

2
libs/calogTimer.c
View file

@ -76,7 +76,7 @@ int32_t calogTimerRegister(CalogT *calog) {
calogRegisterInline(calog, "timerAfter", timerAfterNative, NULL);
calogRegisterInline(calog, "timerEvery", timerEveryNative, NULL);
calogRegisterInline(calog, "timerCancel", timerCancelNative, NULL);
return calogOkE;
return calogAtDestroy(calog, calogTimerShutdown, calogDestroyBeforeContextsE);
}

618
libs/calogXml.c Normal file
View file

@ -0,0 +1,618 @@
// calogXml.c -- calog XML library (see calogXml.h). Parsing rides vendored libxml2 (SAX) and builds
// an element tree in calog's aggregate model; serializing walks that tree back to XML text. The two
// natives are pure functions of their arguments; libxml2 keeps process-global state, initialized in
// calogXmlRegister and released in calogXmlShutdown.
#define _POSIX_C_SOURCE 200809L
#include "calogXml.h"
#include "calogInternal.h"
#include <libxml/parser.h>
#include <libxml/xmlerror.h>
#include <limits.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define XML_BUF_INITIAL 64
// A growable output byte buffer, used both for serialization and for coalescing a run of
// character data before it becomes one text child.
typedef struct XmlBufT {
char *bytes;
size_t length;
size_t cap;
} XmlBufT;
// One open element while parsing: the child list being filled, the attribute map, and any
// character data seen so far that has not yet been committed as a text child.
typedef struct XmlFrameT {
CalogAggT *children;
CalogAggT *attr;
XmlBufT text;
} XmlFrameT;
// The whole parse in flight. frames[0, depth) are the open elements (outermost first). status is
// sticky: once set, every handler becomes a no-op and the failure is reported. root holds the tree.
typedef struct XmlParseT {
XmlFrameT *frames;
int32_t depth;
int32_t status;
CalogValueT root;
bool haveRoot;
char error[192];
bool haveError;
} XmlParseT;
static int32_t xmlBufEnsure(XmlBufT *buf, size_t extra);
static void xmlBufFree(XmlBufT *buf);
static int32_t xmlBufPutBytes(XmlBufT *buf, const char *bytes, size_t length);
static int32_t xmlBufPutChar(XmlBufT *buf, char c);
static int32_t xmlBufPutEscaped(XmlBufT *buf, const char *bytes, int64_t length, bool inAttribute);
static void xmlCharData(void *userData, const xmlChar *text, int length);
static int32_t xmlEncodeElement(XmlBufT *buf, CalogAggT *element, int32_t depth);
static void xmlEndElement(void *userData, const xmlChar *name);
static void xmlErrorSink(void *userData, const xmlError *error);
static void xmlFrameFree(XmlFrameT *frame);
static CalogValueT *xmlMapGet(CalogAggT *map, const char *name);
static int32_t xmlParseNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t xmlPushText(CalogAggT *children, XmlBufT *text);
static int32_t xmlSetAgg(CalogAggT *map, const char *key, CalogAggT *child);
static int32_t xmlSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length);
static void xmlStartElement(void *userData, const xmlChar *name, const xmlChar **atts);
static void xmlStop(XmlParseT *state, int32_t status);
static int32_t xmlStringifyNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
int32_t calogXmlRegister(CalogT *calog) {
// libxml2 self-initializes, thread-safely (mutex-guarded), when a parser context is created, so
// there is no explicit init to do and nothing process-global to tear down: its per-thread state
// is released when each context thread exits.
calogRegisterInline(calog, "xmlParse", xmlParseNative, NULL);
calogRegisterInline(calog, "xmlStringify", xmlStringifyNative, NULL);
return calogOkE;
}
static int32_t xmlBufEnsure(XmlBufT *buf, size_t extra) {
size_t need;
size_t cap;
char *grown;
need = buf->length + extra;
if (need <= buf->cap) {
return calogOkE;
}
cap = (buf->cap == 0) ? XML_BUF_INITIAL : buf->cap;
while (cap < need) {
cap *= CALOG_GROWTH_FACTOR;
}
grown = (char *)realloc(buf->bytes, cap);
if (grown == NULL) {
return calogErrOomE;
}
buf->bytes = grown;
buf->cap = cap;
return calogOkE;
}
static void xmlBufFree(XmlBufT *buf) {
free(buf->bytes);
buf->bytes = NULL;
buf->length = 0;
buf->cap = 0;
}
static int32_t xmlBufPutBytes(XmlBufT *buf, const char *bytes, size_t length) {
int32_t status;
if (length == 0) {
return calogOkE;
}
status = xmlBufEnsure(buf, length);
if (status != calogOkE) {
return status;
}
memcpy(buf->bytes + buf->length, bytes, length);
buf->length += length;
return calogOkE;
}
static int32_t xmlBufPutChar(XmlBufT *buf, char c) {
return xmlBufPutBytes(buf, &c, 1);
}
static int32_t xmlBufPutEscaped(XmlBufT *buf, const char *bytes, int64_t length, bool inAttribute) {
int64_t index;
int32_t status;
for (index = 0; index < length; index++) {
unsigned char c;
c = (unsigned char)bytes[index];
switch (c) {
case '&':
status = xmlBufPutBytes(buf, "&amp;", 5);
break;
case '<':
status = xmlBufPutBytes(buf, "&lt;", 4);
break;
case '>':
status = xmlBufPutBytes(buf, "&gt;", 4);
break;
case '"':
status = inAttribute ? xmlBufPutBytes(buf, "&quot;", 6) : xmlBufPutChar(buf, '"');
break;
// XML rewrites literal whitespace on re-parse: attribute-value normalization turns a raw
// TAB/newline into a space, and end-of-line handling turns a raw CR into a newline. Emit
// them as numeric references so the exact bytes survive a parse -> stringify -> parse.
case '\t':
status = inAttribute ? xmlBufPutBytes(buf, "&#9;", 4) : xmlBufPutChar(buf, '\t');
break;
case '\n':
status = inAttribute ? xmlBufPutBytes(buf, "&#10;", 5) : xmlBufPutChar(buf, '\n');
break;
case '\r':
status = xmlBufPutBytes(buf, "&#13;", 5);
break;
default:
status = xmlBufPutChar(buf, (char)c);
break;
}
if (status != calogOkE) {
return status;
}
}
return calogOkE;
}
static void xmlCharData(void *userData, const xmlChar *text, int length) {
XmlParseT *state;
int32_t status;
state = (XmlParseT *)userData;
if (state->status != calogOkE) {
return;
}
if (state->depth == 0 || length <= 0) {
return; // character data outside the root (libxml2 only delivers whitespace here)
}
status = xmlBufPutBytes(&state->frames[state->depth - 1].text, (const char *)text, (size_t)length);
if (status != calogOkE) {
xmlStop(state, status);
}
}
static int32_t xmlEncodeElement(XmlBufT *buf, CalogAggT *element, int32_t depth) {
CalogValueT *tag;
CalogValueT *attr;
CalogValueT *children;
int32_t status;
int64_t index;
if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE;
}
tag = xmlMapGet(element, "tag");
if (tag == NULL || tag->type != calogStringE) {
return calogErrTypeE;
}
status = xmlBufPutChar(buf, '<');
if (status != calogOkE) {
return status;
}
status = xmlBufPutBytes(buf, tag->as.s.bytes, (size_t)tag->as.s.length);
if (status != calogOkE) {
return status;
}
attr = xmlMapGet(element, "attr");
if (attr != NULL && attr->type == calogAggE) {
CalogAggT *map;
map = attr->as.agg;
for (index = 0; index < map->pairCount; index++) {
CalogValueT *key;
CalogValueT *value;
key = &map->pairs[index].key;
value = &map->pairs[index].value;
if (key->type != calogStringE || value->type != calogStringE) {
return calogErrTypeE;
}
status = xmlBufPutChar(buf, ' ');
if (status != calogOkE) {
return status;
}
status = xmlBufPutBytes(buf, key->as.s.bytes, (size_t)key->as.s.length);
if (status != calogOkE) {
return status;
}
status = xmlBufPutBytes(buf, "=\"", 2);
if (status != calogOkE) {
return status;
}
status = xmlBufPutEscaped(buf, value->as.s.bytes, value->as.s.length, true);
if (status != calogOkE) {
return status;
}
status = xmlBufPutChar(buf, '"');
if (status != calogOkE) {
return status;
}
}
}
children = xmlMapGet(element, "children");
if (children == NULL || children->type != calogAggE || children->as.agg->arrayCount == 0) {
return xmlBufPutBytes(buf, "/>", 2); // empty element -> self-closing tag
}
status = xmlBufPutChar(buf, '>');
if (status != calogOkE) {
return status;
}
for (index = 0; index < children->as.agg->arrayCount; index++) {
CalogValueT *child;
child = &children->as.agg->array[index];
if (child->type == calogStringE) {
status = xmlBufPutEscaped(buf, child->as.s.bytes, child->as.s.length, false);
} else if (child->type == calogAggE) {
status = xmlEncodeElement(buf, child->as.agg, depth + 1);
} else {
return calogErrTypeE;
}
if (status != calogOkE) {
return status;
}
}
status = xmlBufPutBytes(buf, "</", 2);
if (status != calogOkE) {
return status;
}
status = xmlBufPutBytes(buf, tag->as.s.bytes, (size_t)tag->as.s.length);
if (status != calogOkE) {
return status;
}
return xmlBufPutChar(buf, '>');
}
static void xmlEndElement(void *userData, const xmlChar *name) {
XmlParseT *state;
XmlFrameT *top;
CalogAggT *element;
CalogValueT elementValue;
int32_t status;
state = (XmlParseT *)userData;
if (state->status != calogOkE) {
return;
}
top = &state->frames[state->depth - 1];
status = xmlPushText(top->children, &top->text);
if (status != calogOkE) {
xmlStop(state, status);
return;
}
xmlBufFree(&top->text); // this element is closing: no more character data joins it
status = calogAggCreate(&element, calogMapE);
if (status != calogOkE) {
xmlStop(state, status);
return;
}
status = xmlSetStr(element, "tag", (const char *)name, (int64_t)strlen((const char *)name));
if (status != calogOkE) {
calogAggFree(element);
xmlStop(state, status);
return;
}
if (top->attr != NULL && top->attr->pairCount > 0) {
status = xmlSetAgg(element, "attr", top->attr);
top->attr = NULL; // consumed by element (or freed by xmlSetAgg on failure)
if (status != calogOkE) {
calogAggFree(element);
xmlStop(state, status);
return;
}
} else {
if (top->attr != NULL) {
calogAggFree(top->attr);
}
top->attr = NULL;
}
status = xmlSetAgg(element, "children", top->children);
top->children = NULL; // consumed by element (or freed by xmlSetAgg on failure)
if (status != calogOkE) {
calogAggFree(element);
xmlStop(state, status);
return;
}
state->depth--; // the frame's resources are now owned by element
calogValueAgg(&elementValue, element);
if (state->depth > 0) {
status = calogAggPush(state->frames[state->depth - 1].children, &elementValue);
if (status != calogOkE) {
calogValueFree(&elementValue);
xmlStop(state, status);
return;
}
} else {
calogValueMove(&state->root, &elementValue);
state->haveRoot = true;
}
}
static void xmlErrorSink(void *userData, const xmlError *error) {
XmlParseT *state;
state = (XmlParseT *)userData;
if (error == NULL || error->level < XML_ERR_ERROR) {
return; // ignore warnings; only real errors fail the parse
}
if (state->status == calogOkE) {
state->status = calogErrArgE;
}
if (!state->haveError && error->message != NULL) {
size_t length;
snprintf(state->error, sizeof(state->error), "xmlParse: %s", error->message);
length = strlen(state->error);
while (length > 0 && (state->error[length - 1] == '\n' || state->error[length - 1] == '\r')) {
length--;
state->error[length] = '\0';
}
state->haveError = true;
}
}
static void xmlFrameFree(XmlFrameT *frame) {
if (frame->children != NULL) {
calogAggFree(frame->children);
frame->children = NULL;
}
if (frame->attr != NULL) {
calogAggFree(frame->attr);
frame->attr = NULL;
}
xmlBufFree(&frame->text);
}
static CalogValueT *xmlMapGet(CalogAggT *map, const char *name) {
CalogValueT key;
CalogValueT *found;
if (calogValueString(&key, name, (int64_t)strlen(name)) != calogOkE) {
return NULL;
}
found = calogAggGet(map, &key);
calogValueFree(&key);
return found;
}
static int32_t xmlParseNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
XmlParseT state;
xmlSAXHandler sax;
xmlParserCtxtPtr ctxt;
int32_t index;
(void)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "xmlParse expects (text)");
}
if (args[0].as.s.length > INT_MAX) {
return calogFail(result, calogErrArgE, "xmlParse: input too large");
}
memset(&state, 0, sizeof(state));
state.frames = (XmlFrameT *)calloc((size_t)CALOG_MAX_DEPTH, sizeof(XmlFrameT));
if (state.frames == NULL) {
return calogFail(result, calogErrOomE, "xmlParse: out of memory");
}
memset(&sax, 0, sizeof(sax));
sax.initialized = 0; // SAX1 element callbacks (qnames, no namespace splitting)
sax.startElement = xmlStartElement;
sax.endElement = xmlEndElement;
sax.characters = xmlCharData;
sax.cdataBlock = xmlCharData; // treat CDATA content as ordinary text
ctxt = xmlCreatePushParserCtxt(&sax, &state, NULL, 0, NULL);
if (ctxt == NULL) {
free(state.frames);
return calogFail(result, calogErrOomE, "xmlParse: out of memory");
}
xmlCtxtSetErrorHandler(ctxt, xmlErrorSink, &state);
// NOENT substitutes predefined and internal entities so attribute values decode (the SAX
// interface otherwise reports "&" in an attribute as "&#38;"); NO_XXE and NONET then block all
// external content and network access, so no entity reference can reach a local file or URL --
// XXE-safe. NOCDATA delivers CDATA content through the characters callback.
xmlCtxtUseOptions(ctxt, XML_PARSE_NONET | XML_PARSE_NOCDATA | XML_PARSE_NOENT | XML_PARSE_NO_XXE);
xmlParseChunk(ctxt, args[0].as.s.bytes, (int)args[0].as.s.length, 1);
xmlFreeParserCtxt(ctxt);
if (state.status == calogOkE && !state.haveRoot) {
state.status = calogErrArgE;
state.haveError = false; // fall through to the "no root" message below
}
if (state.status != calogOkE) {
const char *message;
if (state.haveError) {
message = state.error;
} else if (state.status == calogErrDepthE) {
message = "xmlParse: nesting exceeds the maximum depth";
} else if (state.status == calogErrOomE) {
message = "xmlParse: out of memory";
} else {
message = "xmlParse: no root element";
}
for (index = 0; index < state.depth; index++) {
xmlFrameFree(&state.frames[index]);
}
if (state.haveRoot) {
calogValueFree(&state.root);
}
free(state.frames);
return calogFail(result, state.status, message);
}
free(state.frames);
calogValueMove(result, &state.root);
return calogOkE;
}
static int32_t xmlPushText(CalogAggT *children, XmlBufT *text) {
CalogValueT value;
int32_t status;
if (text->length == 0) {
return calogOkE;
}
status = calogValueString(&value, text->bytes, (int64_t)text->length);
if (status != calogOkE) {
return status;
}
status = calogAggPush(children, &value);
if (status != calogOkE) {
calogValueFree(&value);
return status;
}
text->length = 0; // committed; keep the buffer for any following run of text
return calogOkE;
}
static int32_t xmlSetAgg(CalogAggT *map, const char *key, CalogAggT *child) {
CalogValueT keyValue;
CalogValueT childValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
calogAggFree(child);
return status;
}
calogValueAgg(&childValue, child);
status = calogAggSet(map, &keyValue, &childValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
calogValueFree(&childValue); // frees child too
}
return status;
}
static int32_t xmlSetStr(CalogAggT *map, const char *key, const char *bytes, int64_t length) {
CalogValueT keyValue;
CalogValueT stringValue;
int32_t status;
status = calogValueString(&keyValue, key, (int64_t)strlen(key));
if (status != calogOkE) {
return status;
}
status = calogValueString(&stringValue, bytes, length);
if (status != calogOkE) {
calogValueFree(&keyValue);
return status;
}
status = calogAggSet(map, &keyValue, &stringValue);
if (status != calogOkE) {
calogValueFree(&keyValue);
calogValueFree(&stringValue);
}
return status;
}
static void xmlStartElement(void *userData, const xmlChar *name, const xmlChar **atts) {
XmlParseT *state;
XmlFrameT *frame;
CalogAggT *children;
CalogAggT *attr;
int32_t status;
int32_t index;
(void)name;
state = (XmlParseT *)userData;
if (state->status != calogOkE) {
return;
}
if (state->depth > 0) {
// Character data seen before this child becomes a text node preceding it.
status = xmlPushText(state->frames[state->depth - 1].children, &state->frames[state->depth - 1].text);
if (status != calogOkE) {
xmlStop(state, status);
return;
}
}
if (state->depth >= CALOG_MAX_DEPTH) {
xmlStop(state, calogErrDepthE);
return;
}
status = calogAggCreate(&children, calogListE);
if (status != calogOkE) {
xmlStop(state, status);
return;
}
status = calogAggCreate(&attr, calogMapE);
if (status != calogOkE) {
calogAggFree(children);
xmlStop(state, status);
return;
}
if (atts != NULL) {
for (index = 0; atts[index] != NULL; index += 2) {
status = xmlSetStr(attr, (const char *)atts[index], (const char *)atts[index + 1],
(int64_t)strlen((const char *)atts[index + 1]));
if (status != calogOkE) {
calogAggFree(children);
calogAggFree(attr);
xmlStop(state, status);
return;
}
}
}
frame = &state->frames[state->depth];
frame->children = children;
frame->attr = attr;
frame->text.bytes = NULL;
frame->text.length = 0;
frame->text.cap = 0;
state->depth++;
}
static void xmlStop(XmlParseT *state, int32_t status) {
if (state->status == calogOkE) {
state->status = status;
}
// Handlers guard on state->status, so subsequent SAX callbacks become no-ops from here.
}
static int32_t xmlStringifyNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
XmlBufT buf;
int32_t status;
(void)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogAggE) {
return calogFail(result, calogErrArgE, "xmlStringify expects (element map)");
}
buf.bytes = NULL;
buf.length = 0;
buf.cap = 0;
status = xmlEncodeElement(&buf, args[0].as.agg, 0);
if (status != calogOkE) {
xmlBufFree(&buf);
return calogFail(result, status, "xmlStringify: value is not a valid XML element");
}
status = calogValueString(result, (buf.bytes != NULL) ? buf.bytes : "", (int64_t)buf.length);
xmlBufFree(&buf);
return status;
}

30
libs/calogXml.h Normal file
View file

@ -0,0 +1,30 @@
// calogXml.h -- calog XML library.
//
// Bridges XML text and calog's value/aggregate model, so any engine can parse and produce
// XML without an engine-specific library:
// xmlParse(text) -> element (the document's root element node)
// xmlStringify(node) -> XML text
//
// An element node is a map:
// { "tag": string, "attr": { name: value, ... }, "children": [ node | text, ... ] }
// where each child is either a nested element map or a plain string (a text node). The "attr"
// key is present only when the element has attributes. Mixed content and all character data
// (including whitespace between elements) are preserved, so parse -> stringify is lossless for
// element structure; comments, processing instructions, and the XML declaration are dropped.
//
// Parsing uses vendored libxml2 (SAX) with networking off and no external-entity loading (no
// XXE). Nesting is bounded by CALOG_MAX_DEPTH both ways. Marshalling is binary-safe (UTF-8). The
// natives are INLINE (pure computation, no shared state): libxml2 self-initializes thread-safely
// when a parser context is created and frees its per-thread state on thread exit, so there is
// nothing to shut down.
#ifndef CALOG_XML_H
#define CALOG_XML_H
#include "calog.h"
// Register the XML natives (xmlParse, xmlStringify) on a runtime. Stateless: safe to call on any
// number of runtimes, and there is no matching shutdown.
int32_t calogXmlRegister(CalogT *calog);
#endif

61
src/calog.h
View file

@ -115,7 +115,11 @@ struct CalogAggT {
typedef int32_t (*CalogNativeFnT)(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
// Delivered on the host thread (during calogPump) when a fire-and-forget script
// fails: contextId is the failing context, message is the engine's error text.
// fails: contextId is the failing context, message is the engine's error text. When the failure
// crossed context/engine boundaries (a script calling another script's exported function, whose
// callback then failed), the message is prefixed with the chain of hops it unwound through, e.g.
// "[lua ctx 2] [janet ctx 1] original error" -- so a cross-boundary error reads clearly with no
// change to this handler's signature.
typedef void (*CalogErrorFnT)(uint64_t contextId, const char *message, void *userData);
// Per-engine interpreter lifecycle + execution. Every hook runs on the context's OWN
@ -138,6 +142,36 @@ void calogDestroy(CalogT *calog);
void calogPump(CalogT *calog); // run pending script->native calls on THIS thread
void calogSetErrorHandler(CalogT *calog, CalogErrorFnT fn, void *userData);
// ---- library lifecycle hooks ----
// A library registers these from its ...Register (at setup, before any context starts, like native
// registration) so it can clean up automatically -- an embedder never calls a per-library shutdown
// by hand. A runtime-shutdown hook runs in one of two phases relative to context teardown:
typedef enum CalogDestroyPhaseE {
// Run after every context thread has been joined (the default): free process-global state once
// no script can touch it -- e.g. handle tables, cached connections.
calogDestroyAfterContextsE = 0,
// Run while contexts are still alive: stop a background thread that invokes context-owned
// callbacks (an accept loop, a timer) before those contexts can be torn down under it.
calogDestroyBeforeContextsE = 1
} CalogDestroyPhaseE;
typedef void (*CalogDestroyHookFnT)(void);
// Runs on the context's OWN thread when it opens (init) and when it closes (shutdown), so it may
// use calogCurrentId() and set up thread-local state. Fires for engine contexts, not the host.
typedef void (*CalogContextHookFnT)(CalogContextT *context, void *userData);
// Register a hook that calogDestroy runs in the given phase. Returns calogErrOomE on allocation
// failure. Registration is setup-only (before contexts start).
int32_t calogAtDestroy(CalogT *calog, CalogDestroyHookFnT fn, CalogDestroyPhaseE phase);
// Register per-context hooks; either may be NULL. userData is passed through to both. Setup-only.
int32_t calogAtContext(CalogT *calog, CalogContextHookFnT init, CalogContextHookFnT shutdown, void *userData);
// Format the CALLING thread's current cross-context call chain (newest first, e.g.
// "lua ctx 2 <- janet ctx 1") into buffer, returning the bytes written. Call it from within a
// native to see how a nested cross-engine call reached it. (Post-mortem, a failed cross-context
// call's error message already carries the same chain as "[engine ctx N] ..." tags -- see above.)
int32_t calogLastTrace(char *buffer, size_t size);
// ---- natives ----
// A name beginning with "__" is INTERNAL: it is callable via calogCall but is NOT exposed
// into any engine's global namespace, so scripts can neither see nor shadow it.
@ -179,6 +213,27 @@ void calogFnRelease(CalogFnT *fn);
// ---- contexts + engines ----
CalogContextT *calogContextOpen(CalogT *calog, const CalogEngineT *engine); // create + start; NULL on failure
// Per-context resource limits for calogContextOpenLimited (sandboxing untrusted scripts).
// A zero/NULL field means "no limit". The wall-clock budget is measured from context open, and the
// allow-list gates which registered natives the script may call.
//
// COVERAGE (honest, per engine):
// allowList -- enforced for EVERY engine (checked in the broker's one dispatch choke point).
// wallClockMillis -- enforced for Lua and JavaScript (their periodic hooks check the deadline). A
// script blocked in a long-running C native is not preempted (inherent). Other
// engines ignore it.
// memoryBytes -- enforced for Lua and JavaScript only (per-VM allocators). The other engines use
// a process-global allocator and ignore it.
typedef struct CalogLimitsT {
int64_t memoryBytes; // 0 = unlimited (Lua + JS only)
int64_t wallClockMillis; // 0 = unlimited (Lua + JS only)
const char *const *allowList; // NULL = all natives permitted; else a NULL-terminated list of allowed names
} CalogLimitsT;
// Like calogContextOpen, but the context enforces the given limits. limits may be NULL (unlimited,
// identical to calogContextOpen). The limits are copied; the caller need not keep them.
CalogContextT *calogContextOpenLimited(CalogT *calog, const CalogEngineT *engine, const CalogLimitsT *limits);
// Make an engine available to calogContextLoad. Call at setup, before loading;
// registration order is the search priority.
void calogRegisterEngine(CalogT *calog, const CalogEngineT *engine);
@ -205,6 +260,7 @@ extern const CalogEngineT calogS7Engine;
extern const CalogEngineT calogWrenEngine;
extern const CalogEngineT calogMrubyEngine;
extern const CalogEngineT calogTclEngine;
extern const CalogEngineT calogJanetEngine;
// Register the built-in engines selected at build time, so calogContextLoad works
// without hand-registering each. Define CALOG_WITH_LUA / _JS / _SQUIRREL / _MYBASIC (in
@ -239,6 +295,9 @@ static inline void calogRegisterBuiltinEngines(CalogT *calog) {
#endif
#ifdef CALOG_WITH_TCL
calogRegisterEngine(calog, &calogTclEngine);
#endif
#ifdef CALOG_WITH_JANET
calogRegisterEngine(calog, &calogJanetEngine);
#endif
(void)calog; // no-op if the build selected no engines
}

37
src/calogInternal.h
View file

@ -51,6 +51,18 @@ typedef int32_t (*CalogRouteFnT)(CalogT *calog, CalogEntryT *entry, CalogValueT
typedef int32_t (*CalogInvokeHookT)(CalogFnT *fn, CalogValueT *args, int32_t argCount, CalogValueT *result);
typedef void (*CalogReleaseHookT)(CalogFnT *fn);
// Library lifecycle hook records (registered via calogAtDestroy / calogAtContext; setup-only).
typedef struct CalogDestroyHookEntryT {
CalogDestroyHookFnT fn;
CalogDestroyPhaseE phase;
} CalogDestroyHookEntryT;
typedef struct CalogContextHookEntryT {
CalogContextHookFnT init;
CalogContextHookFnT shutdown;
void *userData;
} CalogContextHookEntryT;
// The runtime. Owns the native-function registry AND the active-context registry, so
// a host's contexts are its property -- calogDestroy closes every open one. Both
// registries grow dynamically with no preset limit (context ids are 64-bit: a 32-bit
@ -83,6 +95,14 @@ struct CalogT {
const CalogEngineT **engines;
int64_t engineCount;
int64_t engineCap;
// library lifecycle hooks (calogAtDestroy / calogAtContext); written setup-only, so read
// without a lock -- destroy hooks at teardown, context hooks on each context's own thread.
CalogDestroyHookEntryT *destroyHooks;
int64_t destroyHookCount;
int64_t destroyHookCap;
CalogContextHookEntryT *contextHooks;
int64_t contextHookCount;
int64_t contextHookCap;
};
// ---- registry (calogCreate/calogDestroy compose these with the actor layer) ----
@ -110,6 +130,23 @@ CalogT *calogContextBroker(const CalogContextT *context);
void *calogContextInterp(CalogContextT *context);
bool calogContextRegistered(CalogT *runtime, uint64_t ctxId);
// ---- per-context resource limits (sandboxing) ----
// The mutable state a limited context enforces on its OWN thread: memUsed is charged by the engine's
// allocator (only Lua and QuickJS can do this cleanly), and deadlineMs is a monotonic-clock deadline
// the engine's periodic hook checks. Touched only on the context thread, so no atomics are needed.
typedef struct CalogLimitStateT {
int64_t memUsed; // bytes currently charged against the cap
int64_t memCap; // 0 = unlimited
uint64_t deadlineMs; // 0 = no wall-clock deadline; else a calogMonotonicMillis() value
} CalogLimitStateT;
// The limit state an engine adapter installs its allocator/hook against, or NULL if the context is
// unlimited (so the common case installs nothing).
CalogLimitStateT *calogContextLimitState(CalogContextT *context);
// Monotonic milliseconds -- for computing a deadline at open and checking it in the engine hooks.
uint64_t calogMonotonicMillis(void);
// ---- shared helpers (one source of truth; value.c) ----
// A formatted engine error message is copied into a stack buffer of this size before the

31
src/calogMain.c
View file

@ -23,19 +23,25 @@
#include "calog.h"
#include "calogArchive.h"
#include "calogCrypto.h"
#include "calogCsv.h"
#include "calogDb.h"
#include "calogExport.h"
#include "calogFs.h"
#include "calogHttp.h"
#include "calogHttpd.h"
#include "calogJson.h"
#include "calogKv.h"
#include "calogNet.h"
#include "calogProc.h"
#include "calogPubsub.h"
#include "calogRegex.h"
#include "calogSsh.h"
#include "calogTask.h"
#include "calogTime.h"
#include "calogTimer.h"
#include "calogXml.h"
#include <errno.h>
#include <signal.h>
@ -95,7 +101,8 @@ static const CalogEngineT *const gEngines[] = {
&calogS7Engine,
&calogWrenEngine,
&calogMrubyEngine,
&calogTclEngine
&calogTclEngine,
&calogJanetEngine
};
// The built-in libraries, in registration order. One source of truth: main() registers every
@ -105,19 +112,25 @@ static const struct {
const char *name;
int32_t (*reg)(CalogT *calog);
} gLibraries[] = {
{ "archive", calogArchiveRegister },
{ "crypto", calogCryptoRegister },
{ "csv", calogCsvRegister },
{ "db", calogDbRegister },
{ "export", calogExportRegister },
{ "fs", calogFsRegister },
{ "http", calogHttpRegister },
{ "httpd", calogHttpdRegister },
{ "json", calogJsonRegister },
{ "kv", calogKvRegister },
{ "net", calogNetRegister },
{ "proc", calogProcRegister },
{ "pubsub", calogPubsubRegister },
{ "regex", calogRegexRegister },
{ "ssh", calogSshRegister },
{ "task", calogTaskRegister },
{ "time", calogTimeRegister },
{ "timer", calogTimerRegister }
{ "timer", calogTimerRegister },
{ "xml", calogXmlRegister }
};
// s7 interns a small, bounded set of "permanent" strings it never reclaims (an s7 trait, not a
@ -539,17 +552,11 @@ int main(int argc, char **argv) {
calogPump(calog);
calogTaskReap();
// Release cross-context references while the contexts are still alive, tear the runtime
// down (joining every context thread), then free the registries that outlive it.
calogExportShutdown();
calogPubsubShutdown();
calogTimerShutdown();
calogKvShutdown();
// Tear the runtime down. Each library registered its own shutdown via calogAtDestroy, so
// calogDestroy runs them in the right phase -- background-thread libraries (httpd, timer) while
// contexts are still alive, the rest after every context thread is joined -- with none called
// by hand here.
calogDestroy(calog);
calogDbShutdown();
calogNetShutdown();
calogSshShutdown();
calogTaskShutdown();
free(gLaunched);
return (int)atomic_load(&gExitCode);

311
src/context.c
View file

@ -36,6 +36,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#define CONTEXT_INITIAL_CONTEXTS 8
#define CONTEXT_INITIAL_ENGINES 4
@ -101,6 +102,10 @@ struct CalogContextT {
_Atomic bool finished; // set by threadMain just before the thread returns; lets any thread safely reap it
bool started;
bool interpDead; // set (under ctxMutex) once the interpreter is torn down
CalogLimitStateT limits; // memCap/deadlineMs/memUsed, enforced by the engine adapter
char **allow; // sorted allowed-native names, or NULL = every native permitted
int32_t allowCount;
bool limited; // any limit active (adapter installs an allocator/hook; allow checked)
};
// currentContext is the ONLY process-global actor state -- and it is thread-local: it
@ -109,6 +114,21 @@ struct CalogContextT {
// handle instead.
static _Thread_local CalogContextT *currentContext = NULL;
// Cross-boundary trace: a thread-local stack of the cross-context callable hops the CALLING thread
// is currently inside (each entry = the context a callable is owned by). tracePush/Pop bracket every
// actorInvokeCallable; calogLastTrace formats it for live introspection. Post-mortem, the error
// STRING itself carries the chain -- traceEnrich prepends the owner's "[engine ctx N]" tag as an
// error unwinds out of each boundary, so the tag accumulates across the reply and reaches onError.
#define CALOG_TRACE_MAX 32
typedef struct TraceFrameT {
CalogT *runtime;
uint64_t ctxId;
} TraceFrameT;
static _Thread_local TraceFrameT traceFrames[CALOG_TRACE_MAX];
static _Thread_local int32_t traceDepth = 0;
static int32_t actorInvokeCallable(CalogFnT *callable, CalogValueT *args, int32_t argCount, CalogValueT *result);
static void actorReleaseCallable(CalogFnT *callable);
static int32_t actorRoute(CalogT *calog, CalogEntryT *entry, CalogValueT *args, int32_t argCount, CalogValueT *result);
@ -118,6 +138,7 @@ static void contextDispatchError(CalogT *calog, MessageT *message);
static void contextDispatchEval(CalogContextT *context, MessageT *message);
static void contextDispatchRelease(MessageT *message);
static void contextDrainQueue(CalogContextT *context);
static void contextFreeAllow(CalogContextT *context);
static int32_t contextEnqueue(CalogT *calog, uint64_t targetId, MessageT *message);
static int32_t contextPostRelease(CalogT *calog, uint64_t targetId, CalogFnT *callable);
static void contextReply(CalogT *calog, MessageT *message, int32_t status, CalogValueT *result);
@ -125,10 +146,12 @@ static void contextRequestShutdown(CalogContextT *context);
static int32_t contextSendBlocking(CalogT *calog, uint64_t targetId, MessageT *message, CalogValueT *result);
static bool dispatchCommon(CalogContextT *context, MessageT *message);
static void enqueueRaw(CalogContextT *context, MessageT *message);
static int32_t hookGrow(void **array, int64_t count, int64_t *cap, size_t elemSize);
static void hostDispatch(CalogT *calog, MessageT *message);
static uint64_t idCompose(int64_t index, uint32_t generation);
static uint32_t idGeneration(uint64_t id);
static int64_t idIndex(uint64_t id);
static int limitNameCompare(const void *a, const void *b);
static MessageT *messageDequeue(CalogContextT *context);
static void messageFree(MessageT *message);
static bool onOwnerThread(CalogT *runtime, uint64_t owner);
@ -137,6 +160,10 @@ static int32_t pumpUntil(CalogContextT *context, uint64_t token, int32_t
static void registryFreePush(CalogT *calog, int64_t index);
static CalogContextT *registryResolveLocked(CalogT *calog, uint64_t id);
static void serveLoop(CalogContextT *context);
static void traceEnrich(CalogT *runtime, uint64_t ctxId, CalogValueT *result);
static const char *traceEngineName(CalogT *runtime, uint64_t ctxId);
static void tracePop(void);
static void tracePush(CalogT *runtime, uint64_t ctxId);
static MessageT *tryDequeue(CalogContextT *context);
static void *threadMain(void *arg);
@ -171,21 +198,119 @@ static bool onOwnerThread(CalogT *runtime, uint64_t owner) {
}
static void tracePush(CalogT *runtime, uint64_t ctxId) {
if (traceDepth >= 0 && traceDepth < CALOG_TRACE_MAX) {
traceFrames[traceDepth].runtime = runtime;
traceFrames[traceDepth].ctxId = ctxId;
}
traceDepth++; // still counts past the array so the matching pop stays balanced
}
static void tracePop(void) {
if (traceDepth > 0) {
traceDepth--;
}
}
static const char *traceEngineName(CalogT *runtime, uint64_t ctxId) {
CalogContextT *context;
const char *name;
if (ctxId == CALOG_HOST_ID) {
return "host";
}
name = "?";
pthread_mutex_lock(&runtime->ctxMutex);
context = registryResolveLocked(runtime, ctxId);
if (context != NULL && context->engine != NULL) {
name = context->engine->name;
}
pthread_mutex_unlock(&runtime->ctxMutex);
return name;
}
// Prepend "[engine ctx N] " to an error result as it unwinds out of a cross-context callable, so
// the chain accumulates in the string that crosses the reply back to the caller (and to onError).
static void traceEnrich(CalogT *runtime, uint64_t ctxId, CalogValueT *result) {
char prefix[96];
char *combined;
const char *engine;
size_t prefixLen;
size_t oldLen;
if (result->type != calogStringE) {
return;
}
engine = traceEngineName(runtime, ctxId);
prefixLen = (size_t)snprintf(prefix, sizeof(prefix), "[%s ctx %lld] ", engine, (long long)idIndex(ctxId));
if (prefixLen >= sizeof(prefix)) {
prefixLen = sizeof(prefix) - 1;
}
oldLen = (size_t)result->as.s.length;
combined = (char *)malloc(prefixLen + oldLen + 1);
if (combined == NULL) {
return;
}
memcpy(combined, prefix, prefixLen);
memcpy(combined + prefixLen, result->as.s.bytes, oldLen);
combined[prefixLen + oldLen] = '\0';
calogValueFree(result);
calogValueString(result, combined, (int64_t)(prefixLen + oldLen));
free(combined);
}
int32_t calogLastTrace(char *buffer, size_t size) {
int32_t written;
int32_t depth;
int32_t i;
if (size == 0) {
return 0;
}
buffer[0] = '\0';
written = 0;
depth = traceDepth < CALOG_TRACE_MAX ? traceDepth : CALOG_TRACE_MAX;
for (i = depth - 1; i >= 0; i--) {
const char *engine;
int n;
engine = traceEngineName(traceFrames[i].runtime, traceFrames[i].ctxId);
n = snprintf(buffer + written, size - (size_t)written, "%s%s ctx %lld", written > 0 ? " <- " : "", engine, (long long)idIndex(traceFrames[i].ctxId));
if (n < 0 || (size_t)n >= size - (size_t)written) {
break;
}
written += n;
}
return written;
}
static int32_t actorInvokeCallable(CalogFnT *callable, CalogValueT *args, int32_t argCount, CalogValueT *result) {
CalogNativeFnT fn;
CalogT *runtime;
uint64_t owner;
int32_t status;
// Inline when already on the owner's thread; otherwise marshal to the owner (in the
// callable's own runtime). A callable's fn + userData are exactly a native call, so
// the CALL machinery carries it unchanged.
// Inline when already on the owner's thread; otherwise marshal to the owner (in the callable's
// own runtime). A callable's fn + userData are exactly a native call, so the CALL machinery
// carries it unchanged. The boundary hop is traced so a cross-context failure reads clearly.
runtime = calogFnRuntime(callable);
owner = calogFnOwner(callable);
tracePush(runtime, owner);
if (onOwnerThread(runtime, owner)) {
CalogNativeFnT fn;
fn = calogFnNative(callable);
return fn(args, argCount, result, calogFnUserData(callable));
status = fn(args, argCount, result, calogFnUserData(callable));
} else {
status = contextDispatch(runtime, owner, calogFnNative(callable), calogFnUserData(callable), args, argCount, result);
}
return contextDispatch(runtime, owner, calogFnNative(callable), calogFnUserData(callable), args, argCount, result);
tracePop();
if (status != calogOkE) {
traceEnrich(runtime, owner, result);
}
return status;
}
@ -210,6 +335,14 @@ static void actorReleaseCallable(CalogFnT *callable) {
static int32_t actorRoute(CalogT *calog, CalogEntryT *entry, CalogValueT *args, int32_t argCount, CalogValueT *result) {
// Sandboxing: a limited context (allow != NULL -- only a limited SCRIPT context ever sets it;
// the host context never does) may call only the natives on its allow-list. currentContext is
// the CALLING context, so this gates every engine uniformly through the one dispatch choke point.
if (currentContext != NULL && currentContext->allow != NULL &&
bsearch(&entry->name, currentContext->allow, (size_t)currentContext->allowCount, sizeof(char *), limitNameCompare) == NULL) {
calogValueNil(result);
return calogFail(result, calogErrUnsupportedE, "native not permitted by this context's allow-list");
}
// An inline native, or any call already on this runtime's host thread, runs inline;
// otherwise the native is marshalled to the host thread (serviced by calogPump).
// currentContext alone is not enough: it is a single per-thread slot, so a thread
@ -258,6 +391,7 @@ void calogActorShutdown(CalogT *calog) {
calog->ctxSlots[index].context = NULL;
pthread_mutex_unlock(&calog->ctxMutex);
contextDrainQueue(context);
contextFreeAllow(context);
pthread_mutex_destroy(&context->queueMutex);
pthread_cond_destroy(&context->queueCond);
free(context);
@ -319,18 +453,129 @@ CalogT *calogCreate(void) {
}
#define CALOG_HOOK_INITIAL_CAP 4
static int32_t hookGrow(void **array, int64_t count, int64_t *cap, size_t elemSize) {
void *grown;
int64_t newCap;
if (count < *cap) {
return calogOkE;
}
newCap = (*cap == 0) ? CALOG_HOOK_INITIAL_CAP : *cap * CALOG_GROWTH_FACTOR;
grown = realloc(*array, (size_t)newCap * elemSize);
if (grown == NULL) {
return calogErrOomE;
}
*array = grown;
*cap = newCap;
return calogOkE;
}
int32_t calogAtContext(CalogT *calog, CalogContextHookFnT init, CalogContextHookFnT shutdown, void *userData) {
int32_t status;
if (init == NULL && shutdown == NULL) {
return calogOkE;
}
status = hookGrow((void **)&calog->contextHooks, calog->contextHookCount, &calog->contextHookCap, sizeof(*calog->contextHooks));
if (status != calogOkE) {
return status;
}
calog->contextHooks[calog->contextHookCount].init = init;
calog->contextHooks[calog->contextHookCount].shutdown = shutdown;
calog->contextHooks[calog->contextHookCount].userData = userData;
calog->contextHookCount++;
return calogOkE;
}
int32_t calogAtDestroy(CalogT *calog, CalogDestroyHookFnT fn, CalogDestroyPhaseE phase) {
int32_t status;
if (fn == NULL) {
return calogOkE;
}
status = hookGrow((void **)&calog->destroyHooks, calog->destroyHookCount, &calog->destroyHookCap, sizeof(*calog->destroyHooks));
if (status != calogOkE) {
return status;
}
calog->destroyHooks[calog->destroyHookCount].fn = fn;
calog->destroyHooks[calog->destroyHookCount].phase = phase;
calog->destroyHookCount++;
return calogOkE;
}
void calogDestroy(CalogT *calog) {
int64_t index;
if (calog == NULL) {
return;
}
// Before-context destroy hooks: a library with a background thread that invokes context
// callbacks stops it here, while those contexts are still alive. Reverse registration order.
for (index = calog->destroyHookCount - 1; index >= 0; index--) {
if (calog->destroyHooks[index].phase == calogDestroyBeforeContextsE) {
calog->destroyHooks[index].fn();
}
}
calogActorShutdown(calog);
// After-context destroy hooks: free process-global state now that no context thread survives.
for (index = calog->destroyHookCount - 1; index >= 0; index--) {
if (calog->destroyHooks[index].phase == calogDestroyAfterContextsE) {
calog->destroyHooks[index].fn();
}
}
free(calog->destroyHooks);
free(calog->contextHooks);
calogBrokerDestroy(calog);
}
// Create + start a context in one step (design: no owned-native registration happens
// between the two, so there is nothing to do in between). Returns NULL on failure.
uint64_t calogMonotonicMillis(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint64_t)ts.tv_sec * 1000u + (uint64_t)(ts.tv_nsec / 1000000);
}
CalogLimitStateT *calogContextLimitState(CalogContextT *context) {
return context->limited ? &context->limits : NULL;
}
static int limitNameCompare(const void *a, const void *b) {
return strcmp(*(const char *const *)a, *(const char *const *)b);
}
static void contextFreeAllow(CalogContextT *context) {
int32_t i;
if (context->allow == NULL) {
return;
}
for (i = 0; i < context->allowCount; i++) {
free(context->allow[i]);
}
free(context->allow);
context->allow = NULL;
context->allowCount = 0;
}
CalogContextT *calogContextOpen(CalogT *broker, const CalogEngineT *engine) {
return calogContextOpenLimited(broker, engine, NULL);
}
CalogContextT *calogContextOpenLimited(CalogT *broker, const CalogEngineT *engine, const CalogLimitsT *limits) {
CalogContextT *context;
int64_t index;
uint32_t generation;
@ -343,6 +588,41 @@ CalogContextT *calogContextOpen(CalogT *broker, const CalogEngineT *engine) {
context->engine = engine;
pthread_mutex_init(&context->queueMutex, NULL);
pthread_cond_init(&context->queueCond, NULL);
// Sandboxing: copy the policy BEFORE the thread starts. createInterpreter, which reads the limit
// state to install its allocator/hook, runs on the new thread, and pthread_create orders it
// after these writes. A NULL policy leaves the context unlimited.
if (limits != NULL) {
if (limits->memoryBytes > 0) {
context->limits.memCap = limits->memoryBytes;
context->limited = true;
}
if (limits->wallClockMillis > 0) {
context->limits.deadlineMs = calogMonotonicMillis() + (uint64_t)limits->wallClockMillis;
context->limited = true;
}
if (limits->allowList != NULL) {
int32_t count;
int32_t i;
count = 0;
while (limits->allowList[count] != NULL) {
count++;
}
context->allow = (char **)calloc((size_t)(count > 0 ? count : 1), sizeof(char *));
if (context->allow == NULL) {
goto fail;
}
for (i = 0; i < count; i++) {
context->allow[i] = strdup(limits->allowList[i]);
if (context->allow[i] == NULL) {
context->allowCount = i;
goto fail;
}
}
context->allowCount = count;
qsort(context->allow, (size_t)count, sizeof(char *), limitNameCompare);
context->limited = true;
}
}
pthread_mutex_lock(&broker->ctxMutex);
if (broker->ctxFreeCount > 0) {
@ -386,8 +666,9 @@ CalogContextT *calogContextOpen(CalogT *broker, const CalogEngineT *engine) {
return context;
fail:
// Shared teardown for every failure above: nothing beyond the queue's own
// mutex/cond and the context shell has been allocated at any of these points.
// Shared teardown for every failure above: nothing beyond the queue's own mutex/cond, the
// copied allow-list, and the context shell has been allocated at any of these points.
contextFreeAllow(context);
pthread_mutex_destroy(&context->queueMutex);
pthread_cond_destroy(&context->queueCond);
free(context);
@ -520,6 +801,7 @@ void calogContextClose(CalogContextT *context) {
}
pthread_mutex_unlock(&broker->ctxMutex);
contextDrainQueue(context);
contextFreeAllow(context);
pthread_mutex_destroy(&context->queueMutex);
pthread_cond_destroy(&context->queueCond);
free(context);
@ -1187,13 +1469,26 @@ static MessageT *tryDequeue(CalogContextT *context) {
static void *threadMain(void *arg) {
CalogContextT *context;
int64_t index;
context = (CalogContextT *)arg;
currentContext = context;
if (context->engine != NULL && context->engine->createInterpreter != NULL) {
context->engine->createInterpreter(context, &context->interp);
}
// Per-context library init, on this context's own thread now that the interpreter exists.
for (index = 0; index < context->broker->contextHookCount; index++) {
if (context->broker->contextHooks[index].init != NULL) {
context->broker->contextHooks[index].init(context, context->broker->contextHooks[index].userData);
}
}
serveLoop(context);
// Per-context library shutdown (reverse order), while the context + interpreter are still valid.
for (index = context->broker->contextHookCount - 1; index >= 0; index--) {
if (context->broker->contextHooks[index].shutdown != NULL) {
context->broker->contextHooks[index].shutdown(context, context->broker->contextHooks[index].userData);
}
}
// Mark the interpreter dead BEFORE destroying it, so registryResolveLocked stops handing
// this context out: a callable this context owns that is released from now on (e.g. one
// published via the export library and dropped after the context unloads) finalizes by

637
src/janet/janetAdapter.c Normal file
View file

@ -0,0 +1,637 @@
// janetAdapter.c -- Janet engine adapter for the broker.
//
// Each calog context owns a thread-local Janet VM (janet_init/janet_deinit on the context thread;
// the whole Janet state is a JANET_THREAD_LOCAL janet_vm, so N contexts are N independent VMs).
// Values marshal both ways through CalogValueT. Janet has no separate integer engine type at the
// script level worth preserving past 2^53, so calog ints cross as doubles (janet_wrap_number) --
// like Lua/JS/Wren -- and a number egresses through the canonical double classifier. Strings are
// length-prefixed and binary-safe; nil round-trips.
//
// Callables ride on Janet's callable-abstract mechanism (JanetAbstractType.call): a Janet C function
// carries no user data and so cannot bind a native name, so an exposed native (gNativeType) and a
// foreign CalogFnT pushed in (gForeignType) are each a small abstract whose .call handler dispatches
// (calogCall by name, or calogFnInvoke) and whose .gc handler frees the binding. A Janet function
// handed OUT becomes a CalogFnT that janet_gcroots the function so the GC keeps it alive until the
// callable is released (janetScriptRelease, on the owner thread).
//
// Error model: a failed native/marshal calls janet_panic (a longjmp out of the C frame back to the
// nearest fiber). Because janet_panic never returns, every handler frees the heap CalogValueT
// arguments it owns and copies any message into a stack buffer BEFORE panicking. Janet only collects
// at VM instruction boundaries, so C-side marshalling never races the GC; the one value that must
// survive across VM calls (a Janet function out) is explicitly gcrooted.
#define _POSIX_C_SOURCE 200809L
#include "janetAdapter.h"
#include "janet.h"
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct CalogJanetT {
JanetTable *env;
CalogT *broker;
uint64_t ctxId;
};
// Backs an exposed broker native: an abstract whose .call dispatches through calogCall by name.
typedef struct JanetNativeT {
CalogJanetT *context;
char *name;
} JanetNativeT;
// Backs a foreign CalogFnT pushed into this VM: an abstract whose .call invokes the retained fn.
typedef struct JanetForeignT {
CalogJanetT *context;
CalogFnT *fn;
} JanetForeignT;
// Backs a CalogFnT exported from this VM: the owning context and the gcrooted Janet function.
typedef struct JanetScriptT {
CalogJanetT *context;
JanetFunction *function;
} JanetScriptT;
static Janet janetForeignCall(void *p, int32_t argc, Janet *argv);
static int janetForeignGc(void *data, size_t len);
static void janetFreeArgs(CalogValueT *args, int32_t argCount);
static int32_t janetFromValue(CalogJanetT *context, const CalogValueT *value, Janet *out, int32_t depth);
static int32_t janetMarshalArgs(CalogJanetT *context, int32_t argc, Janet *argv, CalogValueT **out);
static Janet janetNativeCall(void *p, int32_t argc, Janet *argv);
static int janetNativeGc(void *data, size_t len);
static int32_t janetScriptInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void janetScriptRelease(CalogFnT *callable);
static int32_t janetToValue(CalogJanetT *context, Janet value, CalogValueT *out, int32_t depth);
static int32_t janetWrapFunction(CalogJanetT *context, JanetFunction *function, CalogFnT **out);
// Callable abstracts. Defined after the prototypes so the designated initializers can name the
// handlers. Not registered with janet_register_abstract_type: calling and GC need no registration
// (only marshalling would), and the pointer identity is enough to recognize a foreign value on the
// way back out.
static const JanetAbstractType gForeignType = {
.name = "calog/foreign",
.gc = janetForeignGc,
.call = janetForeignCall
};
static const JanetAbstractType gNativeType = {
.name = "calog/native",
.gc = janetNativeGc,
.call = janetNativeCall
};
int32_t calogJanetCreate(CalogJanetT **out, CalogT *broker, uint64_t ctxId) {
CalogJanetT *context;
*out = NULL;
context = (CalogJanetT *)calloc(1, sizeof(*context));
if (context == NULL) {
return calogErrOomE;
}
janet_init();
// The core env is memoized and gcrooted internally by Janet, so the defs installed by
// calogJanetExpose survive collection for the VM's lifetime.
context->env = janet_core_env(NULL);
context->broker = broker;
context->ctxId = ctxId;
*out = context;
return calogOkE;
}
void calogJanetDestroy(CalogJanetT *context) {
if (context == NULL) {
return;
}
// Tears down the thread-local VM, firing every remaining abstract's .gc handler: exposed
// natives free their name, live foreign callables release their CalogFnT.
janet_deinit();
free(context);
}
int32_t calogJanetExport(CalogJanetT *context, const char *name, CalogFnT **out) {
Janet resolved;
JanetBindingType binding;
*out = NULL;
resolved = janet_wrap_nil();
binding = janet_resolve(context->env, janet_csymbol(name), &resolved);
if (binding == JANET_BINDING_NONE || janet_type(resolved) != JANET_FUNCTION) {
return calogErrNotFoundE;
}
return janetWrapFunction(context, janet_unwrap_function(resolved), out);
}
int32_t calogJanetExpose(CalogJanetT *context, const char *name) {
JanetNativeT *native;
char *copy;
if (calogLookup(context->broker, name) == NULL) {
return calogErrNotFoundE;
}
// Copy the name first so the abstract is fully formed the instant it exists (its .gc handler
// frees this copy). janet_abstract aborts rather than returns NULL on OOM.
copy = strdup(name);
if (copy == NULL) {
return calogErrOomE;
}
native = (JanetNativeT *)janet_abstract(&gNativeType, sizeof(JanetNativeT));
native->context = context;
native->name = copy;
janet_def(context->env, name, janet_wrap_abstract(native), NULL);
return calogOkE;
}
// Call target for a foreign CalogFnT pushed into this VM (gForeignType.call). p is the abstract's
// JanetForeignT payload; the arguments are argv[0..argc).
static Janet janetForeignCall(void *p, int32_t argc, Janet *argv) {
JanetForeignT *foreign;
CalogJanetT *context;
CalogValueT *args;
CalogValueT result;
Janet out;
int32_t status;
foreign = (JanetForeignT *)p;
context = foreign->context;
args = NULL;
if (janetMarshalArgs(context, argc, argv, &args) != calogOkE) {
janet_panic("failed to marshal a function-value argument");
}
status = calogFnInvoke(foreign->fn, args, argc, &result);
janetFreeArgs(args, argc);
if (status != calogOkE) {
char message[CALOG_ERR_MSG_CAP];
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : "function value failed");
calogValueFree(&result);
janet_panic(message);
}
status = janetFromValue(context, &result, &out, 0);
calogValueFree(&result);
if (status != calogOkE) {
janet_panic("failed to marshal the function-value result");
}
return out;
}
static int janetForeignGc(void *data, size_t len) {
JanetForeignT *foreign;
(void)len;
foreign = (JanetForeignT *)data;
calogFnRelease(foreign->fn);
return 0;
}
static void janetFreeArgs(CalogValueT *args, int32_t argCount) {
int32_t index;
if (args == NULL) {
return;
}
for (index = 0; index < argCount; index++) {
calogValueFree(&args[index]);
}
free(args);
}
static int32_t janetFromValue(CalogJanetT *context, const CalogValueT *value, Janet *out, int32_t depth) {
*out = janet_wrap_nil();
if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE;
}
switch (value->type) {
case calogNilE:
return calogOkE;
case calogBoolE:
*out = janet_wrap_boolean(value->as.b ? 1 : 0);
return calogOkE;
case calogIntE:
// Doubles, not wrap_integer: preserve calog's 64-bit ints up to 2^53 (the shared
// ceiling of every double-number engine).
*out = janet_wrap_number((double)value->as.i);
return calogOkE;
case calogRealE:
*out = janet_wrap_number(value->as.r);
return calogOkE;
case calogStringE:
*out = janet_wrap_string(janet_string((const uint8_t *)value->as.s.bytes, (int32_t)value->as.s.length));
return calogOkE;
case calogAggE: {
CalogAggT *aggregate;
int64_t index;
int32_t status;
Janet child;
aggregate = value->as.agg;
// No GC runs while this executes (no bytecode), so the partially built container and
// the child in hand are safe without rooting.
if (calogAggIsKeyed(aggregate)) {
JanetTable *table;
table = janet_table((int32_t)(aggregate->arrayCount + aggregate->pairCount));
for (index = 0; index < aggregate->arrayCount; index++) {
status = janetFromValue(context, &aggregate->array[index], &child, depth + 1);
if (status != calogOkE) {
return status;
}
janet_table_put(table, janet_wrap_number((double)index), child);
}
for (index = 0; index < aggregate->pairCount; index++) {
Janet key;
status = janetFromValue(context, &aggregate->pairs[index].key, &key, depth + 1);
if (status != calogOkE) {
return status;
}
status = janetFromValue(context, &aggregate->pairs[index].value, &child, depth + 1);
if (status != calogOkE) {
return status;
}
janet_table_put(table, key, child);
}
*out = janet_wrap_table(table);
return calogOkE;
}
JanetArray *array;
array = janet_array((int32_t)aggregate->arrayCount);
for (index = 0; index < aggregate->arrayCount; index++) {
status = janetFromValue(context, &aggregate->array[index], &child, depth + 1);
if (status != calogOkE) {
return status;
}
janet_array_push(array, child);
}
*out = janet_wrap_array(array);
return calogOkE;
}
case calogFnE: {
JanetForeignT *foreign;
foreign = (JanetForeignT *)janet_abstract(&gForeignType, sizeof(JanetForeignT));
foreign->context = context;
foreign->fn = value->as.fn;
calogFnRetain(value->as.fn);
*out = janet_wrap_abstract(foreign);
return calogOkE;
}
}
return calogErrTypeE;
}
static int32_t janetMarshalArgs(CalogJanetT *context, int32_t argc, Janet *argv, CalogValueT **out) {
CalogValueT *args;
int32_t index;
int32_t status;
*out = NULL;
if (argc <= 0) {
return calogOkE;
}
args = (CalogValueT *)calloc((size_t)argc, sizeof(CalogValueT));
if (args == NULL) {
return calogErrOomE;
}
for (index = 0; index < argc; index++) {
status = janetToValue(context, argv[index], &args[index], 0);
if (status != calogOkE) {
int32_t cleanup;
for (cleanup = 0; cleanup < index; cleanup++) {
calogValueFree(&args[cleanup]);
}
free(args);
return status;
}
}
*out = args;
return calogOkE;
}
// Call target for an exposed broker native (gNativeType.call). p is the abstract's JanetNativeT
// payload; the arguments are argv[0..argc).
static Janet janetNativeCall(void *p, int32_t argc, Janet *argv) {
JanetNativeT *native;
CalogJanetT *context;
CalogValueT *args;
CalogValueT result;
Janet out;
int32_t status;
native = (JanetNativeT *)p;
context = native->context;
args = NULL;
if (janetMarshalArgs(context, argc, argv, &args) != calogOkE) {
janet_panic("failed to marshal a native argument");
}
status = calogCall(context->broker, native->name, args, argc, &result);
janetFreeArgs(args, argc);
if (status != calogOkE) {
char message[CALOG_ERR_MSG_CAP];
snprintf(message, sizeof(message), "%s", (result.type == calogStringE) ? result.as.s.bytes : "native call failed");
calogValueFree(&result);
janet_panic(message);
}
status = janetFromValue(context, &result, &out, 0);
calogValueFree(&result);
if (status != calogOkE) {
janet_panic("failed to marshal the native result");
}
return out;
}
static int janetNativeGc(void *data, size_t len) {
JanetNativeT *native;
(void)len;
native = (JanetNativeT *)data;
free(native->name);
return 0;
}
static int32_t janetScriptInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
JanetScriptT *holder;
CalogJanetT *context;
Janet *argv;
Janet out;
JanetSignal signal;
int32_t index;
int32_t status;
holder = (JanetScriptT *)userData;
context = holder->context;
calogValueNil(result);
argv = NULL;
if (argCount > 0) {
argv = (Janet *)calloc((size_t)argCount, sizeof(Janet));
if (argv == NULL) {
return calogFail(result, calogErrOomE, "out of memory invoking janet callback");
}
}
// No GC runs while argv is built (no bytecode); janet_pcall copies argv into the fiber stack
// before running, so the values are rooted for the duration of the call.
for (index = 0; index < argCount; index++) {
status = janetFromValue(context, &args[index], &argv[index], 0);
if (status != calogOkE) {
free(argv);
return calogFail(result, status, "failed to marshal a janet callback argument");
}
}
out = janet_wrap_nil();
signal = janet_pcall(holder->function, argCount, argv, &out, NULL);
free(argv);
if (signal != JANET_SIGNAL_OK) {
const char *message;
message = (janet_type(out) == JANET_STRING) ? (const char *)janet_unwrap_string(out) : "janet callback failed";
return calogFail(result, calogErrArgE, message);
}
return janetToValue(context, out, result, 0);
}
static void janetScriptRelease(CalogFnT *callable) {
JanetScriptT *holder;
holder = (JanetScriptT *)calogFnUserData(callable);
// Runs on the owner thread, so the thread-local gc root list is this VM's.
janet_gcunroot(janet_wrap_function(holder->function));
free(holder);
}
static int32_t janetToValue(CalogJanetT *context, Janet value, CalogValueT *out, int32_t depth) {
calogValueNil(out);
if (depth >= CALOG_MAX_DEPTH) {
return calogErrDepthE;
}
switch (janet_type(value)) {
case JANET_NIL:
return calogOkE;
case JANET_BOOLEAN:
calogValueBool(out, janet_unwrap_boolean(value) != 0);
return calogOkE;
case JANET_NUMBER:
// Canonical double classification: an exact integer becomes an int, else a real.
calogValueFromDouble(out, janet_unwrap_number(value));
return calogOkE;
case JANET_STRING:
case JANET_SYMBOL:
case JANET_KEYWORD: {
JanetString bytes;
bytes = janet_unwrap_string(value);
return calogValueString(out, (const char *)bytes, (int64_t)janet_string_length(bytes));
}
case JANET_BUFFER: {
JanetBuffer *buffer;
buffer = janet_unwrap_buffer(value);
return calogValueString(out, (const char *)buffer->data, (int64_t)buffer->count);
}
case JANET_ARRAY: {
JanetArray *array;
CalogAggT *aggregate;
int32_t index;
int32_t status;
array = janet_unwrap_array(value);
status = calogAggCreate(&aggregate, calogListE);
if (status != calogOkE) {
return status;
}
for (index = 0; index < array->count; index++) {
CalogValueT element;
status = janetToValue(context, array->data[index], &element, depth + 1);
if (status != calogOkE) {
calogAggFree(aggregate);
return status;
}
status = calogAggPush(aggregate, &element);
if (status != calogOkE) {
calogValueFree(&element);
calogAggFree(aggregate);
return status;
}
}
calogValueAgg(out, aggregate);
return calogOkE;
}
case JANET_TUPLE: {
const Janet *tuple;
CalogAggT *aggregate;
int32_t count;
int32_t index;
int32_t status;
tuple = janet_unwrap_tuple(value);
count = janet_tuple_length(tuple);
status = calogAggCreate(&aggregate, calogListE);
if (status != calogOkE) {
return status;
}
for (index = 0; index < count; index++) {
CalogValueT element;
status = janetToValue(context, tuple[index], &element, depth + 1);
if (status != calogOkE) {
calogAggFree(aggregate);
return status;
}
status = calogAggPush(aggregate, &element);
if (status != calogOkE) {
calogValueFree(&element);
calogAggFree(aggregate);
return status;
}
}
calogValueAgg(out, aggregate);
return calogOkE;
}
case JANET_TABLE: {
JanetTable *table;
CalogAggT *aggregate;
int32_t index;
int32_t status;
table = janet_unwrap_table(value);
status = calogAggCreate(&aggregate, calogMapE);
if (status != calogOkE) {
return status;
}
for (index = 0; index < table->capacity; index++) {
CalogValueT key;
CalogValueT val;
if (janet_checktype(table->data[index].key, JANET_NIL)) {
continue;
}
status = janetToValue(context, table->data[index].key, &key, depth + 1);
if (status != calogOkE) {
calogAggFree(aggregate);
return status;
}
status = janetToValue(context, table->data[index].value, &val, depth + 1);
if (status != calogOkE) {
calogValueFree(&key);
calogAggFree(aggregate);
return status;
}
status = calogAggSet(aggregate, &key, &val);
if (status != calogOkE) {
calogValueFree(&key);
calogValueFree(&val);
calogAggFree(aggregate);
return status;
}
}
calogValueAgg(out, aggregate);
return calogOkE;
}
case JANET_STRUCT: {
const JanetKV *st;
CalogAggT *aggregate;
int32_t capacity;
int32_t index;
int32_t status;
st = janet_unwrap_struct(value);
capacity = janet_struct_capacity(st);
status = calogAggCreate(&aggregate, calogMapE);
if (status != calogOkE) {
return status;
}
for (index = 0; index < capacity; index++) {
CalogValueT key;
CalogValueT val;
if (janet_checktype(st[index].key, JANET_NIL)) {
continue;
}
status = janetToValue(context, st[index].key, &key, depth + 1);
if (status != calogOkE) {
calogAggFree(aggregate);
return status;
}
status = janetToValue(context, st[index].value, &val, depth + 1);
if (status != calogOkE) {
calogValueFree(&key);
calogAggFree(aggregate);
return status;
}
status = calogAggSet(aggregate, &key, &val);
if (status != calogOkE) {
calogValueFree(&key);
calogValueFree(&val);
calogAggFree(aggregate);
return status;
}
}
calogValueAgg(out, aggregate);
return calogOkE;
}
case JANET_FUNCTION: {
CalogFnT *callable;
int32_t status;
status = janetWrapFunction(context, janet_unwrap_function(value), &callable);
if (status != calogOkE) {
return status;
}
calogValueFn(out, callable);
return calogOkE;
}
case JANET_ABSTRACT: {
void *abstract;
abstract = janet_unwrap_abstract(value);
// Our own foreign callable coming back out: return the wrapped fn, retained.
if (janet_abstract_type(abstract) == &gForeignType) {
JanetForeignT *foreign;
foreign = (JanetForeignT *)abstract;
calogFnRetain(foreign->fn);
calogValueFn(out, foreign->fn);
return calogOkE;
}
return calogOkE; // a foreign abstract has no calog analogue; stays nil
}
default:
return calogOkE; // fiber, cfunction, pointer: no analogue; stays nil
}
}
// Wrap a Janet function as a host-visible CalogFnT. The function is pinned as a GC root until the
// callable's last reference drops (janetScriptRelease unroots it).
static int32_t janetWrapFunction(CalogJanetT *context, JanetFunction *function, CalogFnT **out) {
JanetScriptT *holder;
int32_t status;
*out = NULL;
holder = (JanetScriptT *)malloc(sizeof(*holder));
if (holder == NULL) {
return calogErrOomE;
}
holder->context = context;
holder->function = function;
janet_gcroot(janet_wrap_function(function));
status = calogFnCreate(out, context->broker, janetScriptInvoke, holder, janetScriptRelease, context->ctxId);
if (status != calogOkE) {
janet_gcunroot(janet_wrap_function(function));
free(holder);
return status;
}
return calogOkE;
}
int32_t calogJanetRun(CalogJanetT *context, const char *source) {
Janet out;
int flags;
out = janet_wrap_nil();
flags = janet_dobytes(context->env, (const uint8_t *)source, (int32_t)strlen(source), "calog", &out);
if (flags != 0) {
// janet_dobytes already printed the error + a stack trace to stderr (janet_eprintf), so
// do not print it again here; just report failure through the single error channel.
return calogErrArgE;
}
return calogOkE;
}

41
src/janet/janetAdapter.h Normal file
View file

@ -0,0 +1,41 @@
// janetAdapter.h -- Janet engine adapter for the broker (see janetAdapter.c).
//
// Each calog context runs its own thread-local Janet VM (janet_init/janet_deinit on the context
// thread; Janet's whole state is a JANET_THREAD_LOCAL janet_vm, so N contexts = N independent VMs
// with no shared state -- the actor model exactly). Values marshal both ways through CalogValueT.
//
// Callables ride on Janet's callable-abstract mechanism (JanetAbstractType.call), because a Janet
// C function carries no user data and so cannot bind a native name: an exposed native and a foreign
// CalogFnT are each a small abstract value whose .call handler dispatches (calogCall by name, or
// calogFnInvoke). A Janet function handed OUT becomes a CalogFnT that janet_gcroots the function so
// the GC keeps it alive until the callable is released. Janet integers are doubles (2^53 exact,
// like Lua/JS/Wren); Janet strings are length-prefixed and binary-safe. nil round-trips.
//
// Bare-name export resolution is NOT provided: Janet resolves symbols at compile time and has no
// runtime unbound-symbol hook (unlike Tcl's `unknown` or s7's *unbound-variable-hook*), so exports
// are reached with the (calogCall "name" ...) native.
#ifndef CALOG_JANET_ADAPTER_H
#define CALOG_JANET_ADAPTER_H
#include "calog.h"
#include "calogInternal.h"
typedef struct CalogJanetT CalogJanetT;
// Create a Janet VM for this context (call on the context's own thread). *out owns the VM.
int32_t calogJanetCreate(CalogJanetT **out, CalogT *broker, uint64_t ctxId);
// Tear down the VM (call on the context's own thread).
void calogJanetDestroy(CalogJanetT *context);
// Wrap a top-level Janet function named `name` as a CalogFnT (retained), or return an error.
int32_t calogJanetExport(CalogJanetT *context, const char *name, CalogFnT **out);
// Bind a broker native named `name` as a callable in the Janet environment.
int32_t calogJanetExpose(CalogJanetT *context, const char *name);
// Evaluate Janet source in this context's VM.
int32_t calogJanetRun(CalogJanetT *context, const char *source);
#endif

61
src/janet/janetEngine.c Normal file
View file

@ -0,0 +1,61 @@
// janetEngine.c -- bridges the Janet adapter to the actor layer's CalogEngineT vtable.
//
// Every hook runs on the owning context's thread (createInterpreter, runSource, destroyInterpreter),
// confining the Janet VM to one thread -- which is exactly Janet's model, since its whole state is a
// JANET_THREAD_LOCAL janet_vm. createInterpreter builds the VM and exposes the registered natives;
// runSource evaluates a script and reports failure through the single error channel (result).
#include "calogInternal.h"
#include "janetAdapter.h"
static int32_t janetEngineCreate(CalogContextT *context, void **interpOut);
static void janetEngineDestroy(void *interp);
static int32_t janetEngineRun(void *interp, const char *source, CalogValueT *result);
static void janetExposeVisitor(const CalogEntryT *entry, void *ud);
static const char *const janetExtensions[] = { "janet", NULL };
const CalogEngineT calogJanetEngine = {
"janet",
janetExtensions,
janetEngineCreate,
janetEngineDestroy,
janetEngineRun
};
static int32_t janetEngineCreate(CalogContextT *context, void **interpOut) {
CalogJanetT *janet;
int32_t status;
*interpOut = NULL;
status = calogJanetCreate(&janet, calogContextBroker(context), calogContextId(context));
if (status != calogOkE) {
return status;
}
calogForEach(calogContextBroker(context), janetExposeVisitor, janet);
*interpOut = janet;
return calogOkE;
}
static void janetEngineDestroy(void *interp) {
calogJanetDestroy((CalogJanetT *)interp);
}
static int32_t janetEngineRun(void *interp, const char *source, CalogValueT *result) {
int32_t status;
calogValueNil(result);
status = calogJanetRun((CalogJanetT *)interp, source);
if (status != calogOkE) {
return calogFail(result, status, "janet script failed");
}
return calogOkE;
}
static void janetExposeVisitor(const CalogEntryT *entry, void *ud) {
calogJanetExpose((CalogJanetT *)ud, entry->name);
}

30
src/js/jsAdapter.c
View file

@ -36,6 +36,7 @@ struct CalogJsT {
uint64_t ctxId;
JSClassID fnClassId; // class for a foreign CalogFnT pushed into JS (callable + finalized)
JSClassID resolverClassId; // class for the exotic prototype that resolves bare export names
CalogLimitStateT *limits; // sandbox limits (mem/time), or NULL if the context is unlimited
};
// Backs a CalogFnT exported from this context: the retained JS function value (kept
@ -50,6 +51,7 @@ static void jsCallableRelease(CalogFnT *callable);
static int32_t jsExportValue(CalogJsT *context, JSValueConst fn, CalogFnT **out);
static JSValue jsForeignCall(JSContext *ctx, JSValueConst funcObj, JSValueConst thisVal, int argc, JSValueConst *argv, int flags);
static void jsForeignFinalize(JSRuntime *rt, JSValueConst val);
static int jsInterrupt(JSRuntime *rt, void *opaque);
static JSValue jsFromValue(JSContext *ctx, const CalogValueT *value, int32_t depth);
static int jsResolveOwnProperty(JSContext *ctx, JSPropertyDescriptor *desc, JSValueConst obj, JSAtom prop);
static int32_t jsToValue(JSContext *ctx, JSValueConst val, CalogValueT *out, int32_t depth);
@ -60,7 +62,7 @@ static JSValue jsTrampoline(JSContext *ctx, JSValueConst this_val, int argc, JSV
static JSClassExoticMethods jsResolverExotic = { .get_own_property = jsResolveOwnProperty };
int32_t calogJsCreate(CalogJsT **out, CalogT *broker, uint64_t ctxId) {
int32_t calogJsCreate(CalogJsT **out, CalogT *broker, uint64_t ctxId, CalogLimitStateT *limits) {
CalogJsT *context;
*out = NULL;
@ -73,6 +75,11 @@ int32_t calogJsCreate(CalogJsT **out, CalogT *broker, uint64_t ctxId) {
free(context);
return calogErrOomE;
}
// A memory-capped context uses QuickJS's own exact per-runtime limit (it fails allocations
// cleanly, so JS_NewContext below simply returns NULL if the cap is below the base runtime).
if (limits != NULL && limits->memCap > 0) {
JS_SetMemoryLimit(context->rt, (size_t)limits->memCap);
}
context->ctx = JS_NewContext(context->rt);
if (context->ctx == NULL) {
JS_FreeRuntime(context->rt);
@ -81,6 +88,12 @@ int32_t calogJsCreate(CalogJsT **out, CalogT *broker, uint64_t ctxId) {
}
context->broker = broker;
context->ctxId = ctxId;
context->limits = limits;
// A time-limited context checks the wall-clock deadline from the interrupt handler (fired by the
// interpreter periodically), recovering the context through the runtime opaque set just below.
if (limits != NULL && limits->deadlineMs > 0) {
JS_SetInterruptHandler(context->rt, jsInterrupt, context);
}
JS_SetContextOpaque(context->ctx, context);
// A callable, finalized class wraps a foreign CalogFnT pushed into JS. The runtime
// opaque lets the finalizer (which gets only the runtime) recover the context.
@ -450,6 +463,21 @@ int32_t calogJsRun(CalogJsT *context, const char *source) {
}
// The periodic interrupt handler for a time-limited context: retire the context and abort the
// running script (non-zero return) once the wall-clock deadline has passed.
static int jsInterrupt(JSRuntime *rt, void *opaque) {
CalogJsT *context;
(void)rt;
context = (CalogJsT *)opaque;
if (context->limits != NULL && context->limits->deadlineMs != 0 && calogMonotonicMillis() >= context->limits->deadlineMs) {
calogCurrentRetire();
return 1;
}
return 0;
}
static int jsResolveOwnProperty(JSContext *ctx, JSPropertyDescriptor *desc, JSValueConst obj, JSAtom prop) {
CalogJsT *context;
CalogValueT nameArg;

2
src/js/jsAdapter.h
View file

@ -17,7 +17,7 @@
typedef struct CalogJsT CalogJsT;
int32_t calogJsCreate(CalogJsT **out, CalogT *broker, uint64_t ctxId);
int32_t calogJsCreate(CalogJsT **out, CalogT *broker, uint64_t ctxId, CalogLimitStateT *limits);
void calogJsDestroy(CalogJsT *context);
int32_t calogJsExport(CalogJsT *context, const char *globalName, CalogFnT **out);
int32_t calogJsExpose(CalogJsT *context, const char *name);

2
src/js/jsEngine.c
View file

@ -29,7 +29,7 @@ static int32_t jsEngineCreate(CalogContextT *context, void **interpOut) {
int32_t status;
*interpOut = NULL;
status = calogJsCreate(&jc, calogContextBroker(context), calogContextId(context));
status = calogJsCreate(&jc, calogContextBroker(context), calogContextId(context), calogContextLimitState(context));
if (status != calogOkE) {
return status;
}

57
src/lua/luaAdapter.c
View file

@ -34,6 +34,7 @@ struct CalogLuaT {
BindingT **bindings;
int32_t bindingCount;
int32_t bindingCap;
CalogLimitStateT *limits; // sandbox limits (mem/time), or NULL if the context is unlimited
};
// Shared shape for the two things a marshalled call can dispatch to: a CalogFnT
@ -44,6 +45,7 @@ typedef int32_t (*LuaDispatchFnT)(void *userData, CalogValueT *args, int32_t arg
static int luaCallDispatch(lua_State *L, int argBase, LuaDispatchFnT dispatch, void *userData, const char *noun, const char *failMessage);
static int32_t luaCallableInvoke(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void luaCallableRelease(CalogFnT *callable);
static void *luaCountingAlloc(void *ud, void *ptr, size_t osize, size_t nsize);
static CalogLuaT *luaContextOf(lua_State *L);
static int32_t luaExportAt(CalogLuaT *context, int idx, CalogFnT **out);
static int luaFunctionCall(lua_State *L);
@ -54,6 +56,7 @@ static int32_t luaPushValueDepth(lua_State *L, const CalogValueT *value, in
static int32_t luaToValueDepth(lua_State *L, int idx, CalogValueT *out, int32_t depth);
static int luaTrampoline(lua_State *L);
static int32_t luaTrampolineDispatch(void *userData, CalogValueT *args, int32_t argCount, CalogValueT *result);
static void luaTimeHook(lua_State *L, lua_Debug *ar);
// Shared marshal/dispatch/cleanup pipeline for both call sites that cross into Lua's
@ -156,7 +159,49 @@ static void luaCallableRelease(CalogFnT *callable) {
}
int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId) {
// The allocator for a memory-capped context: charges every allocation against the shared limit
// state and refuses a grow that would exceed the cap (Lua turns a NULL return into a clean OOM).
static void *luaCountingAlloc(void *ud, void *ptr, size_t osize, size_t nsize) {
CalogLimitStateT *limits;
int64_t oldReal;
int64_t delta;
void *grown;
limits = (CalogLimitStateT *)ud;
oldReal = ptr != NULL ? (int64_t)osize : 0; // for a new block Lua passes a type tag, not a size
if (nsize == 0) {
free(ptr);
limits->memUsed -= oldReal;
return NULL;
}
delta = (int64_t)nsize - oldReal;
if (delta > 0 && limits->memCap > 0 && limits->memUsed + delta > limits->memCap) {
return NULL;
}
grown = realloc(ptr, nsize);
if (grown == NULL) {
return NULL;
}
limits->memUsed += delta;
return grown;
}
// The instruction-count hook for a time-limited context: if the wall-clock deadline has passed, ask
// the actor layer to retire this context and raise an error out of the running script.
static void luaTimeHook(lua_State *L, lua_Debug *ar) {
CalogLuaT *context;
(void)ar;
context = luaContextOf(L);
if (context->limits != NULL && context->limits->deadlineMs != 0 && calogMonotonicMillis() >= context->limits->deadlineMs) {
calogCurrentRetire();
luaL_error(L, "context exceeded its time budget");
}
}
int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId, CalogLimitStateT *limits) {
CalogLuaT *context;
lua_State *L;
@ -165,7 +210,10 @@ int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId) {
if (context == NULL) {
return calogErrOomE;
}
L = luaL_newstate();
// A memory-capped context gets a counting allocator (charges the shared limit state and refuses
// to grow past the cap, which Lua turns into a clean "not enough memory"). The cap must exceed
// the base runtime (openlibs); a few MB is ample.
L = (limits != NULL && limits->memCap > 0) ? lua_newstate(luaCountingAlloc, limits) : luaL_newstate();
if (L == NULL) {
free(context);
return calogErrOomE;
@ -174,7 +222,12 @@ int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId) {
context->L = L;
context->broker = broker;
context->ctxId = ctxId;
context->limits = limits;
*(CalogLuaT **)lua_getextraspace(L) = context;
// A time-limited context gets an instruction-count hook that checks the wall-clock deadline.
if (limits != NULL && limits->deadlineMs > 0) {
lua_sethook(L, luaTimeHook, LUA_MASKCOUNT, 1000);
}
luaL_newmetatable(L, LUA_FUNCTION_META);
lua_pushcfunction(L, luaFunctionCall);

2
src/lua/luaAdapter.h
View file

@ -16,7 +16,7 @@
typedef struct CalogLuaT CalogLuaT;
int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId);
int32_t calogLuaCreate(CalogLuaT **out, CalogT *broker, uint64_t ctxId, CalogLimitStateT *limits);
void calogLuaDestroy(CalogLuaT *context);
int32_t calogLuaExport(CalogLuaT *context, const char *globalName, CalogFnT **out);
int32_t calogLuaExpose(CalogLuaT *context, const char *name);

2
src/lua/luaEngine.c
View file

@ -30,7 +30,7 @@ static int32_t luaEngineCreate(CalogContextT *context, void **interpOut) {
int32_t status;
*interpOut = NULL;
status = calogLuaCreate(&lc, calogContextBroker(context), calogContextId(context));
status = calogLuaCreate(&lc, calogContextBroker(context), calogContextId(context), calogContextLimitState(context));
if (status != calogOkE) {
return status;
}

6
src/value.c
View file

@ -598,12 +598,14 @@ int32_t calogMapSetStr(CalogAggT *map, const char *key, const char *bytes, int64
void calogRegistryRelease(pthread_mutex_t *initMutex, int32_t *refCount, void (*freeAll)(void)) {
pthread_mutex_lock(initMutex);
// freeAll runs exactly once, on the 1 -> 0 transition. A release at refcount 0 (e.g. an
// auto-shutdown after the caller already shut the library down by hand) is a safe no-op.
if (*refCount > 0) {
(*refCount)--;
}
if (*refCount <= 0) {
if (*refCount == 0) {
freeAll();
}
}
pthread_mutex_unlock(initMutex);
}

152
tests/testArchive.c Normal file
View file

@ -0,0 +1,152 @@
// testArchive.c -- exercises the compression/archive library (libs/calogArchive.c) from a Lua
// context: every filter round-trips (including embedded NULs), compression actually shrinks,
// level is honored, a tar.gz and a zip are built in memory and read back entry-by-entry, and a
// bad filter/format is reported as an error the script can catch.
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include "calogArchive.h"
#include <stdatomic.h>
#include <stdio.h>
#include <string.h>
#include <time.h>
#define PUMP_LIMIT 4000
static CalogT *calog = NULL;
static _Atomic int32_t checksRun = 0;
static _Atomic int32_t checksFailed = 0;
static _Atomic bool scriptDone = false;
static void checkFail(const char *message);
static int32_t nativeCheck(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData);
static void pumpUntilDone(void);
static const char *SCRIPT =
"local data = 'hi\\0there ' .. string.rep('compress me ', 300)\n"
"for _, f in ipairs({'gzip','bzip2','xz','zstd','lz4','none','compress'}) do\n"
" local c = compress(data, f)\n"
" check(decompress(c) == data, 'roundtrip ' .. f)\n"
" if f ~= 'none' then check(#c < #data, 'shrinks ' .. f) end\n"
"end\n"
"check(#compress(data,'gzip',9) <= #compress(data,'gzip',1), 'gzip level 9 <= level 1')\n"
"check(decompress(compress(data,'gzip')) == data, 'decompress auto-detects filter')\n"
"local w = archiveWriteOpen('tar','gzip')\n"
"archiveWriteEntry(w, 'a.txt', 'alpha')\n"
"archiveWriteEntry(w, 'b.txt', 'beta\\0gamma')\n"
"local tar = archiveWriteFinish(w)\n"
"local r = archiveReadOpen(tar)\n"
"local names, datas = {}, {}\n"
"local e = archiveReadNext(r)\n"
"while e do names[#names+1] = e.name; datas[e.name] = archiveReadData(r); e = archiveReadNext(r) end\n"
"archiveReadClose(r)\n"
"check(#names == 2, 'tar has two entries')\n"
"check(datas['a.txt'] == 'alpha', 'tar entry a data')\n"
"check(datas['b.txt'] == 'beta\\0gamma', 'tar entry b data is binary-safe')\n"
"local z = archiveWriteOpen('zip','none')\n"
"archiveWriteEntry(z, 'z.txt', 'zipped')\n"
"local zip = archiveWriteFinish(z)\n"
"local zr = archiveReadOpen(zip)\n"
"local ze = archiveReadNext(zr)\n"
"check(ze.name == 'z.txt' and archiveReadData(zr) == 'zipped', 'zip round-trip')\n"
"archiveReadClose(zr)\n"
"local x = archiveWriteOpen('xar','none')\n"
"archiveWriteEntry(x, 'x.txt', 'xarred\\0bin')\n"
"local xar = archiveWriteFinish(x)\n"
"local xr = archiveReadOpen(xar)\n"
"local xe = archiveReadNext(xr)\n"
"check(xe.name == 'x.txt' and archiveReadData(xr) == 'xarred\\0bin', 'xar round-trip (libxml2 + openssl)')\n"
"archiveReadClose(xr)\n"
"check(not pcall(function() compress('x','nosuchfilter') end), 'bad filter is an error')\n"
"check(not pcall(function() archiveWriteOpen('nosuchformat','none') end), 'bad format is an error')\n"
"done()\n";
static void checkFail(const char *message) {
atomic_fetch_add(&checksFailed, 1);
printf("FAIL %s\n", message);
}
static int32_t nativeCheck(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
atomic_fetch_add(&checksRun, 1);
if (argCount < 1 || args[0].type != calogBoolE) {
checkFail("check: expected a boolean");
return calogOkE;
}
if (!args[0].as.b) {
checkFail(argCount >= 2 && args[1].type == calogStringE ? args[1].as.s.bytes : "(unnamed)");
}
return calogOkE;
}
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&scriptDone, true);
calogValueNil(result);
return calogOkE;
}
static void onError(uint64_t contextId, const char *message, void *userData) {
(void)contextId;
(void)userData;
printf("FAIL script error: %s\n", message);
atomic_fetch_add(&checksFailed, 1);
atomic_store(&scriptDone, true);
}
static void pumpUntilDone(void) {
struct timespec ts = { 0, 500000 };
int32_t i;
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&scriptDone)) {
calogPump(calog);
return;
}
nanosleep(&ts, NULL);
}
}
int main(void) {
CalogContextT *ctx;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogSetErrorHandler(calog, onError, NULL);
if (calogArchiveRegister(calog) != calogOkE) {
printf("archive register failed\n");
return 1;
}
calogRegisterInline(calog, "check", nativeCheck, NULL);
calogRegisterInline(calog, "done", nativeDone, NULL);
ctx = calogContextOpen(calog, &calogLuaEngine);
if (ctx == NULL) {
printf("context open failed\n");
return 1;
}
calogContextEval(ctx, SCRIPT);
pumpUntilDone();
calogDestroy(calog);
printf("\n%d checks, %d failed\n", atomic_load(&checksRun), atomic_load(&checksFailed));
fflush(stdout);
return atomic_load(&checksFailed) == 0 ? 0 : 1;
}

1
tests/testCrypto.c
View file

@ -153,7 +153,6 @@ int main(void) {
CHECK(atomic_load(&results[14]) == 1, "cryptoBase64Decode trims trailing whitespace without spurious trailing NULs");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

1
tests/testDb.c
View file

@ -214,7 +214,6 @@ int main(void) {
testDbError();
calogDestroy(calog);
calogDbShutdown();
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);

4
tests/testDbMysql.c
View file

@ -148,9 +148,7 @@ int main(int argc, char **argv) {
if (atomic_load(&reportedCount) < 0) {
printf("SKIP: no MySQL server reachable (connection error path exercised, no crash)\n");
calogContextClose(ctx);
calogDestroy(calog);
calogDbShutdown();
return 0;
}
@ -161,9 +159,7 @@ int main(int argc, char **argv) {
CHECK(reportedScore == 7.25, "the double column read back as a real");
CHECK(atomic_load(&reportedOk) == 0, "the tinyint(bool) column read back as 0");
calogContextClose(ctx);
calogDestroy(calog);
calogDbShutdown();
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);

2
tests/testDbPg.c
View file

@ -180,9 +180,7 @@ int main(int argc, char **argv) {
CHECK(atomic_load(&reportedBlobLen) == 3, "a bytea param with an embedded NUL round-tripped at full length");
CHECK(reportedBlob[0] == 'x' && reportedBlob[1] == '\0' && reportedBlob[2] == 'y', "the embedded NUL survived the Postgres bound-param round-trip");
calogContextClose(ctx);
calogDestroy(calog);
calogDbShutdown();
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);

403
tests/testEngineJanet.c Normal file
View file

@ -0,0 +1,403 @@
// testEngineJanet.c -- Janet on a context thread, driven the host way: register natives, open a
// context, fire-and-forget a script, and calogPump on the host thread. Verifies host-thread
// dispatch, the inline escape hatch, the cross-thread callback, aggregates both ways (array/table),
// a foreign function value, the error handler, and THREE concurrent VMs (Janet's thread-local
// janet_vm is the make-or-break gate for the actor model).
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include <stdatomic.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
#define PUMP_LIMIT 4000
static CalogT *calog = NULL;
static _Atomic int64_t reportedValue = 0;
static _Atomic uint64_t reportedCtxId = 0xFFFFu;
static _Atomic uint64_t inlineCtxId = 0xFFFFu;
static _Atomic int32_t bumpCount = 0;
static _Atomic bool scriptDone = false;
static _Atomic int32_t errorCount = 0;
static _Atomic uint64_t errorCtxId = 0;
static CalogFnT *_Atomic storedCb = NULL;
static char storedName[32] = { 0 };
static _Atomic int32_t nameLen = -1;
static int32_t testsRun = 0;
static int32_t testsFailed = 0;
static void checkImpl(bool condition, const char *message, const char *file, int32_t line);
static int32_t nativeAdd(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeBump(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeGetAdder(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeMakeUser(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeMapAge(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReportInline(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReportName(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeSetCb(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData);
static void pumpUntilDone(void);
static void testConcurrentContexts(void);
static void testCrossThreadCallback(void);
static void testForeignFunction(void);
static void testHostAndInlineNatives(void);
static void testMapIngress(void);
static void testMaterializedRecord(void);
static void testScriptError(void);
static void checkImpl(bool condition, const char *message, const char *file, int32_t line) {
testsRun++;
if (!condition) {
testsFailed++;
printf("FAIL %s:%d %s\n", file, line, message);
}
}
static int32_t nativeAdd(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
if (argCount != 2 || args[0].type != calogIntE || args[1].type != calogIntE) {
return calogFail(result, calogErrArgE, "add expects two integers");
}
calogValueInt(result, args[0].as.i + args[1].as.i);
return calogOkE;
}
static int32_t nativeBump(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_fetch_add(&bumpCount, 1);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&scriptDone, true);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeGetAdder(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
CalogFnT *callable;
int32_t status;
(void)args;
(void)argCount;
(void)userData;
calogValueNil(result);
status = calogFnFromNative(&callable, calog, nativeAdd, NULL);
if (status != calogOkE) {
return calogFail(result, status, "getAdder could not allocate");
}
calogValueFn(result, callable);
return calogOkE;
}
static int32_t nativeMakeUser(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
CalogAggT *user;
CalogValueT key;
CalogValueT val;
int32_t status;
(void)args;
(void)argCount;
(void)userData;
calogValueNil(result);
status = calogAggCreate(&user, calogMapE);
if (status != calogOkE) {
return calogFail(result, status, "makeUser could not allocate");
}
calogValueString(&key, "name", 4);
calogValueString(&val, "ada", 3);
calogAggSet(user, &key, &val);
calogValueString(&key, "age", 3);
calogValueInt(&val, 36);
calogAggSet(user, &key, &val);
calogValueAgg(result, user);
return calogOkE;
}
static int32_t nativeMapAge(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
CalogValueT key;
CalogValueT *found;
(void)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogAggE) {
return calogFail(result, calogErrArgE, "mapAge expects a map");
}
calogValueString(&key, "age", 3);
found = calogAggGet(args[0].as.agg, &key);
calogValueFree(&key);
if (found != NULL && found->type == calogIntE) {
atomic_store(&reportedValue, found->as.i);
}
return calogOkE;
}
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogIntE) {
return calogFail(result, calogErrArgE, "report expects one integer");
}
atomic_store(&reportedCtxId, calogCurrentId());
atomic_store(&reportedValue, args[0].as.i);
return calogOkE;
}
static int32_t nativeReportInline(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&inlineCtxId, calogCurrentId());
calogValueNil(result);
return calogOkE;
}
static int32_t nativeReportName(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
size_t length;
(void)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogStringE) {
return calogFail(result, calogErrArgE, "reportName expects one string");
}
length = (size_t)args[0].as.s.length;
if (length >= sizeof(storedName)) {
length = sizeof(storedName) - 1;
}
memcpy(storedName, args[0].as.s.bytes, length);
storedName[length] = '\0';
atomic_store(&nameLen, (int32_t)length);
return calogOkE;
}
static int32_t nativeSetCb(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
if (argCount != 1 || args[0].type != calogFnE) {
return calogFail(result, calogErrArgE, "setCb expects one function");
}
calogFnRetain(args[0].as.fn);
atomic_store(&storedCb, args[0].as.fn);
return calogOkE;
}
static void onError(uint64_t contextId, const char *message, void *userData) {
(void)message;
(void)userData;
atomic_store(&errorCtxId, contextId);
atomic_fetch_add(&errorCount, 1);
}
static void pumpUntilDone(void) {
struct timespec ts = { 0, 500000 };
int errorsBefore;
int i;
errorsBefore = atomic_load(&errorCount);
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&scriptDone) || atomic_load(&errorCount) != errorsBefore) {
calogPump(calog);
return;
}
nanosleep(&ts, NULL);
}
}
static void testHostAndInlineNatives(void) {
CalogContextT *ctx;
ctx = calogContextOpen(calog, &calogJanetEngine);
CHECK(ctx != NULL, "opened a Janet context");
atomic_store(&scriptDone, false);
calogContextEval(ctx, "(report 42) (reportInline 1) (done)");
pumpUntilDone();
CHECK(atomic_load(&reportedValue) == 42, "host native received the argument");
CHECK(atomic_load(&reportedCtxId) == 0, "default native ran on the host thread (id 0)");
CHECK(atomic_load(&inlineCtxId) == calogContextId(ctx), "inline native ran on the script's own thread");
calogContextClose(ctx);
}
static void testMaterializedRecord(void) {
CalogContextT *ctx;
ctx = calogContextOpen(calog, &calogJanetEngine);
atomic_store(&scriptDone, false);
atomic_store(&nameLen, -1);
atomic_store(&reportedValue, 0);
calogContextEval(ctx, "(def u (makeUser)) (report (get u \"age\")) (reportName (get u \"name\")) (done)");
pumpUntilDone();
CHECK(atomic_load(&reportedValue) == 36, "read an int field from a materialized record table");
CHECK(atomic_load(&nameLen) == 3 && memcmp(storedName, "ada", 3) == 0, "read a string field from a materialized record table");
calogContextClose(ctx);
}
static void testMapIngress(void) {
CalogContextT *ctx;
ctx = calogContextOpen(calog, &calogJanetEngine);
atomic_store(&scriptDone, false);
atomic_store(&reportedValue, 0);
// The script builds a table (with a nested array, exercising array ingress too) and hands it
// to a native, which reads a field.
calogContextEval(ctx, "(def m @{\"age\" 9 \"tags\" @[1 2]}) (mapAge m) (done)");
pumpUntilDone();
CHECK(atomic_load(&reportedValue) == 9, "native read a field from a table the script built");
calogContextClose(ctx);
}
static void testForeignFunction(void) {
CalogContextT *ctx;
ctx = calogContextOpen(calog, &calogJanetEngine);
atomic_store(&scriptDone, false);
atomic_store(&reportedValue, 0);
// getAdder returns a function value (a callable abstract); the script calls it as (adder 2 3).
calogContextEval(ctx, "(def adder (getAdder)) (report (adder 2 3)) (done)");
pumpUntilDone();
CHECK(atomic_load(&reportedValue) == 5, "script called a foreign function value pushed in from the host");
calogContextClose(ctx);
}
static void testCrossThreadCallback(void) {
CalogContextT *ctx;
CalogFnT *callback;
CalogValueT arg;
CalogValueT result;
int32_t status;
ctx = calogContextOpen(calog, &calogJanetEngine);
atomic_store(&scriptDone, false);
atomic_store(&storedCb, NULL);
// A Janet function handed to setCb marshals to a CalogFnT (the function is gc-rooted).
calogContextEval(ctx, "(defn cb [x] (+ x 100)) (setCb cb) (done)");
pumpUntilDone();
callback = atomic_load(&storedCb);
CHECK(callback != NULL, "a Janet function was captured as a CalogFnT");
calogValueInt(&arg, 7);
status = calogFnInvoke(callback, &arg, 1, &result);
CHECK(status == calogOkE && result.type == calogIntE && result.as.i == 107, "cross-thread callable invoke routed to the owner");
calogValueFree(&result);
calogValueFree(&arg);
calogFnRelease(callback);
calogContextClose(ctx);
}
static void testScriptError(void) {
CalogContextT *ctx;
int32_t before;
ctx = calogContextOpen(calog, &calogJanetEngine);
before = atomic_load(&errorCount);
atomic_store(&scriptDone, false);
calogContextEval(ctx, "(error \"intentional test error\")");
pumpUntilDone();
CHECK(atomic_load(&errorCount) == before + 1, "a fire-and-forget script error reached the error handler");
CHECK(atomic_load(&errorCtxId) == calogContextId(ctx), "the error names the failing context");
calogContextClose(ctx);
}
static void testConcurrentContexts(void) {
CalogContextT *ctxs[3];
struct timespec ts = { 0, 500000 };
int32_t i;
atomic_store(&bumpCount, 0);
for (i = 0; i < 3; i++) {
ctxs[i] = calogContextOpen(calog, &calogJanetEngine);
calogContextEval(ctxs[i], "(bump) (bump) (bump)");
}
for (i = 0; i < PUMP_LIMIT && atomic_load(&bumpCount) < 9; i++) {
calogPump(calog);
nanosleep(&ts, NULL);
}
CHECK(atomic_load(&bumpCount) == 9, "three concurrent Janet VMs (thread-local janet_vm) all dispatched to the host");
for (i = 0; i < 3; i++) {
calogContextClose(ctxs[i]);
}
}
int main(void) {
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogSetErrorHandler(calog, onError, NULL);
calogRegister(calog, "report", nativeReport, NULL);
calogRegister(calog, "setCb", nativeSetCb, NULL);
calogRegister(calog, "done", nativeDone, NULL);
calogRegister(calog, "bump", nativeBump, NULL);
calogRegister(calog, "makeUser", nativeMakeUser, NULL);
calogRegister(calog, "reportName", nativeReportName, NULL);
calogRegister(calog, "getAdder", nativeGetAdder, NULL);
calogRegister(calog, "mapAge", nativeMapAge, NULL);
calogRegisterInline(calog, "reportInline", nativeReportInline, NULL);
testHostAndInlineNatives();
testMaterializedRecord();
testMapIngress();
testForeignFunction();
testCrossThreadCallback();
testScriptError();
testConcurrentContexts();
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);
if (testsFailed != 0) {
return 1;
}
return 0;
}

6
tests/testExport.c
View file

@ -249,12 +249,6 @@ int main(void) {
pumpUntilDone(8);
CHECK(atomic_load(&doneCount) >= 8, "resolving an unknown global after export shutdown is safe");
calogContextClose(luaCaller);
calogContextClose(jsCaller);
calogContextClose(s7Caller);
calogContextClose(squirrelCaller);
calogContextClose(mbCaller);
calogContextClose(exporter);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

1
tests/testFs.c
View file

@ -166,7 +166,6 @@ int main(void) {
CHECK(atomic_load(&results[11]) == 1, "fsStat of a missing path returns nil");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

124
tests/testHooks.c Normal file
View file

@ -0,0 +1,124 @@
// testHooks.c -- validates the library lifecycle hooks: calogAtDestroy (a runtime-shutdown hook run
// in one of two phases around context teardown) and calogAtContext (per-context init/shutdown run on
// each context's own thread). A monotonic sequence counter records the order of every hook event so
// the phase ordering can be asserted: before-hook < context shutdowns < after-hook.
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include <stdatomic.h>
#include <stdint.h>
#include <stdio.h>
#include <time.h>
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
#define PUMP_LIMIT 4000
static _Atomic int32_t seqCounter = 0; // hands out a global order to every hook event
static _Atomic int32_t destroyBeforeSeq = -1;
static _Atomic int32_t destroyAfterSeq = -1;
static _Atomic int32_t initCount = 0;
static _Atomic int32_t shutdownCount = 0;
static _Atomic int32_t firstShutdownSeq = -1;
static _Atomic int32_t lastShutdownSeq = -1;
static _Atomic int64_t seenUserData = 0;
static int32_t testsRun = 0;
static int32_t testsFailed = 0;
static int32_t userToken = 0x5A5A;
static void checkImpl(bool condition, const char *message, const char *file, int32_t line);
static void contextInitHook(CalogContextT *context, void *userData);
static void contextShutdownHook(CalogContextT *context, void *userData);
static void destroyAfterHook(void);
static void destroyBeforeHook(void);
static void checkImpl(bool condition, const char *message, const char *file, int32_t line) {
testsRun++;
if (!condition) {
testsFailed++;
printf("FAIL %s:%d %s\n", file, line, message);
}
}
static void contextInitHook(CalogContextT *context, void *userData) {
(void)context;
atomic_fetch_add(&seqCounter, 1);
atomic_fetch_add(&initCount, 1);
atomic_store(&seenUserData, (int64_t)*(int32_t *)userData);
}
static void contextShutdownHook(CalogContextT *context, void *userData) {
int32_t seq;
int32_t expected;
(void)context;
(void)userData;
seq = atomic_fetch_add(&seqCounter, 1);
atomic_fetch_add(&shutdownCount, 1);
expected = -1;
atomic_compare_exchange_strong(&firstShutdownSeq, &expected, seq);
atomic_store(&lastShutdownSeq, seq);
}
static void destroyAfterHook(void) {
atomic_store(&destroyAfterSeq, atomic_fetch_add(&seqCounter, 1));
}
static void destroyBeforeHook(void) {
atomic_store(&destroyBeforeSeq, atomic_fetch_add(&seqCounter, 1));
}
int main(void) {
CalogT *calog;
CalogContextT *ctxA;
CalogContextT *ctxB;
struct timespec ts = { 0, 500000 };
int32_t i;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogAtDestroy(calog, destroyBeforeHook, calogDestroyBeforeContextsE);
calogAtDestroy(calog, destroyAfterHook, calogDestroyAfterContextsE);
calogAtContext(calog, contextInitHook, contextShutdownHook, &userToken);
ctxA = calogContextOpen(calog, &calogLuaEngine);
ctxB = calogContextOpen(calog, &calogLuaEngine);
calogContextEval(ctxA, "local a = 1");
calogContextEval(ctxB, "local b = 2");
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&initCount) >= 2) {
break;
}
nanosleep(&ts, NULL);
}
CHECK(atomic_load(&initCount) == 2, "per-context init hook fired once per context");
CHECK(atomic_load(&seenUserData) == 0x5A5A, "context hook received its registered userData");
calogDestroy(calog); // before-hook -> tear down contexts (their shutdown hooks) -> after-hook
CHECK(atomic_load(&shutdownCount) == 2, "per-context shutdown hook fired once per context");
CHECK(atomic_load(&destroyBeforeSeq) >= 0, "before-context destroy hook fired");
CHECK(atomic_load(&destroyAfterSeq) >= 0, "after-context destroy hook fired");
CHECK(atomic_load(&destroyBeforeSeq) < atomic_load(&firstShutdownSeq), "before-hook ran before context teardown");
CHECK(atomic_load(&lastShutdownSeq) < atomic_load(&destroyAfterSeq), "after-hook ran after context teardown");
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);
if (testsFailed != 0) {
return 1;
}
return 0;
}

1
tests/testHttp.c
View file

@ -221,7 +221,6 @@ int main(void) {
CHECK(atomic_load(&results[6]) == 1, "a hostile oversized chunk size is rejected cleanly, not hung");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

171
tests/testHttpd.c Normal file
View file

@ -0,0 +1,171 @@
// testHttpd.c -- the polyglot HTTP server. A Lua context and a JavaScript context each stand up a
// server with route handlers written in their own language; the test drives real HTTP requests over
// a socket and checks the responses, a 404, a response map (custom status), and HOT RELOAD
// (re-registering a route on a live server swaps the handler).
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include "calogHttpd.h"
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdatomic.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <time.h>
#include <unistd.h>
#define LUA_PORT 38900
#define JS_PORT 38901
#define PUMP_LIMIT 4000
static CalogT *calog = NULL;
static _Atomic int32_t readyCount = 0;
static int32_t testsRun = 0;
static int32_t testsFailed = 0;
#define CHECK(cond, msg) checkImpl((cond), (msg), __LINE__)
static void checkImpl(bool condition, const char *message, int32_t line);
static bool httpGet(int port, const char *path, int *statusOut, char *body, size_t bodyCap);
static int32_t nativeReady(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void pumpForReady(int32_t target);
static void checkImpl(bool condition, const char *message, int32_t line) {
testsRun++;
if (!condition) {
testsFailed++;
printf("FAIL testHttpd.c:%d %s\n", line, message);
}
}
// Minimal HTTP/1.1 client: connect, GET path, parse status code + body.
static bool httpGet(int port, const char *path, int *statusOut, char *body, size_t bodyCap) {
struct sockaddr_in addr;
char request[256];
char response[8192];
char *bodyStart;
ssize_t total;
ssize_t got;
int fd;
int requestLen;
fd = socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
return false;
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons((uint16_t)port);
inet_pton(AF_INET, "127.0.0.1", &addr.sin_addr);
if (connect(fd, (struct sockaddr *)&addr, sizeof(addr)) != 0) {
close(fd);
return false;
}
requestLen = snprintf(request, sizeof(request), "GET %s HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n", path);
if (send(fd, request, (size_t)requestLen, 0) != requestLen) {
close(fd);
return false;
}
total = 0;
while ((got = recv(fd, response + total, sizeof(response) - 1 - (size_t)total, 0)) > 0) {
total += got;
if ((size_t)total >= sizeof(response) - 1) {
break;
}
}
close(fd);
if (total <= 0) {
return false;
}
response[total] = '\0';
*statusOut = 0;
if (strncmp(response, "HTTP/1.1 ", 9) == 0) {
*statusOut = atoi(response + 9);
}
bodyStart = strstr(response, "\r\n\r\n");
body[0] = '\0';
if (bodyStart != NULL) {
snprintf(body, bodyCap, "%s", bodyStart + 4);
}
return true;
}
static int32_t nativeReady(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_fetch_add(&readyCount, 1);
calogValueNil(result);
return calogOkE;
}
static void pumpForReady(int32_t target) {
struct timespec ts = { 0, 500000 };
int32_t i;
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&readyCount) >= target) {
return;
}
nanosleep(&ts, NULL);
}
}
int main(void) {
CalogContextT *lua;
CalogContextT *js;
char body[8192];
int status;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogHttpdRegister(calog);
calogRegister(calog, "ready", nativeReady, NULL);
// Lua server: a plain-string route, a response-map route (custom status), and a 404 for the rest.
lua = calogContextOpen(calog, &calogLuaEngine);
calogContextEval(lua,
"srv = httpdListen(38900)\n"
"httpdRoute(srv, 'GET', '/hi', function(req) return 'hello from lua path=' .. req.path end)\n"
"httpdRoute(srv, 'GET', '/made', function(req) return { status = 201, body = 'created' } end)\n"
"ready()");
pumpForReady(1);
// JavaScript server on another port -- a handler written in a different language.
js = calogContextOpen(calog, &calogJsEngine);
calogContextEval(js,
"var s = httpdListen(38901);\n"
"httpdRoute(s, 'GET', '/hi', function(req) { return 'hello from js'; });\n"
"ready();");
pumpForReady(2);
CHECK(httpGet(LUA_PORT, "/hi", &status, body, sizeof(body)) && status == 200 && strstr(body, "hello from lua") != NULL && strstr(body, "path=/hi") != NULL, "lua route serves + sees the request path");
CHECK(httpGet(JS_PORT, "/hi", &status, body, sizeof(body)) && status == 200 && strstr(body, "hello from js") != NULL, "js route serves on its own server");
CHECK(httpGet(LUA_PORT, "/made", &status, body, sizeof(body)) && status == 201 && strcmp(body, "created") == 0, "response map sets a custom status");
CHECK(httpGet(LUA_PORT, "/nope", &status, body, sizeof(body)) && status == 404, "an unmatched route is 404");
// Hot reload: re-register /hi on the LIVE lua server with a new handler.
atomic_store(&readyCount, 0);
calogContextEval(lua, "httpdRoute(srv, 'GET', '/hi', function(req) return 'reloaded' end); ready()");
pumpForReady(1);
CHECK(httpGet(LUA_PORT, "/hi", &status, body, sizeof(body)) && status == 200 && strcmp(body, "reloaded") == 0, "re-registering a route hot-swaps the handler");
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);
return testsFailed == 0 ? 0 : 1;
}

1
tests/testHttps.c
View file

@ -246,7 +246,6 @@ int main(void) {
CHECK(atomic_load(&results[3]) == 1, "the body is delivered over TLS when verification is skipped");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

2
tests/testJs.c
View file

@ -179,7 +179,7 @@ int main(void) {
printf("broker create failed\n");
return 1;
}
if (calogJsCreate(&jsctx, broker, 0) != calogOkE) {
if (calogJsCreate(&jsctx, broker, 0, NULL) != calogOkE) {
printf("js context create failed\n");
return 1;
}

1
tests/testJson.c
View file

@ -145,7 +145,6 @@ int main(void) {
CHECK(atomic_load(&results[9]) == 1, "an integer literal overflowing int64 parses as a real");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

3
tests/testKv.c
View file

@ -1,6 +1,5 @@
// testKv.c -- exercises the kv library: set/get scalars and tables, deep-copy independence,
// has/delete, key listing, and rejection of function values, driven from a Lua context.
// calogKvShutdown() runs BEFORE calogDestroy (like testExport).
#define _POSIX_C_SOURCE 200809L
@ -157,8 +156,6 @@ int main(void) {
CHECK(atomic_load(&results[10]) == 1, "kvSet rejects a function nested inside an aggregate");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogKvShutdown();
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

2
tests/testLua.c
View file

@ -209,7 +209,7 @@ int main(void) {
calogRegister(broker, "makeList", nativeMakeList, NULL);
calogRegister(broker, "getAdder", nativeGetAdder, NULL);
status = calogLuaCreate(&lua, broker, 1);
status = calogLuaCreate(&lua, broker, 1, NULL);
if (status != calogOkE) {
printf("lua context create failed\n");
return 1;

1
tests/testNet.c
View file

@ -356,7 +356,6 @@ int main(void) {
testEnetSafety();
calogDestroy(calog);
calogNetShutdown();
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);

2
tests/testPolyglot.c
View file

@ -130,7 +130,7 @@ int main(void) {
calogRegister(broker, "add", nativeAdd, NULL);
calogRegister(broker, "record", nativeRecord, NULL);
status = calogLuaCreate(&lua, broker, LUA_CTX_ID);
status = calogLuaCreate(&lua, broker, LUA_CTX_ID, NULL);
if (status != calogOkE) {
printf("lua context create failed\n");
return 1;

3
tests/testPubsub.c
View file

@ -187,9 +187,6 @@ int main(void) {
pumpUntilDone(8);
CHECK(atomic_load(&results[11]) == 0, "publishing after shutdown is safe and finds no subscribers");
calogContextClose(subA);
calogContextClose(subB);
calogContextClose(pub);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

186
tests/testSandbox.c Normal file
View file

@ -0,0 +1,186 @@
// testSandbox.c -- per-context resource limits (calogContextOpenLimited). Verifies the allow-list
// (a script may call only permitted natives, engine-agnostically), the Lua memory cap (an
// over-budget allocation fails cleanly instead of exhausting the host), and the wall-clock budget
// on Lua and JavaScript (a runaway loop is retired). The honest coverage is documented on
// CalogLimitsT: memory/time apply to Lua + JS only; the allow-list applies to every engine.
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include <stdatomic.h>
#include <stdio.h>
#include <string.h>
#include <time.h>
#define CHECK(cond, msg) checkImpl((cond), (msg), __LINE__)
#define PUMP_LIMIT 6000
static CalogT *calog = NULL;
static _Atomic bool reachedFlag = false;
static _Atomic bool forbiddenFlag = false;
static _Atomic int32_t reportValue = -1;
static _Atomic int32_t errorCount = 0;
static _Atomic bool doneFlag = false;
static int32_t testsRun = 0;
static int32_t testsFailed = 0;
static void checkImpl(bool condition, const char *message, int32_t line);
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeForbidden(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReached(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData);
static void resetFlags(void);
static void runLimited(const CalogEngineT *engine, const char *source, const CalogLimitsT *limits);
static void checkImpl(bool condition, const char *message, int32_t line) {
testsRun++;
if (!condition) {
testsFailed++;
printf("FAIL testSandbox.c:%d %s\n", line, message);
}
}
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&doneFlag, true);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeForbidden(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&forbiddenFlag, true);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeReached(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&reachedFlag, true);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
if (argCount == 1 && args[0].type == calogBoolE) {
atomic_store(&reportValue, args[0].as.b ? 1 : 0);
}
return calogOkE;
}
static void onError(uint64_t contextId, const char *message, void *userData) {
(void)contextId;
(void)message;
(void)userData;
atomic_fetch_add(&errorCount, 1);
atomic_store(&doneFlag, true);
}
static void resetFlags(void) {
atomic_store(&reachedFlag, false);
atomic_store(&forbiddenFlag, false);
atomic_store(&reportValue, -1);
atomic_store(&errorCount, 0);
atomic_store(&doneFlag, false);
}
// Open a limited context, run the script fire-and-forget, and pump until it signals done() or its
// error reaches the handler (or a bounded timeout).
static void runLimited(const CalogEngineT *engine, const char *source, const CalogLimitsT *limits) {
CalogContextT *ctx;
struct timespec ts = { 0, 500000 };
int32_t i;
ctx = calogContextOpenLimited(calog, engine, limits);
if (ctx == NULL) {
checkImpl(false, "context open failed", __LINE__);
return;
}
calogContextEval(ctx, source);
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&doneFlag)) {
calogPump(calog);
break;
}
nanosleep(&ts, NULL);
}
calogContextClose(ctx);
}
int main(void) {
static const char *const allowList[] = { "reached", "report", "done", NULL };
CalogLimitsT allowLimits;
CalogLimitsT memLimits;
CalogLimitsT timeLimits;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogSetErrorHandler(calog, onError, NULL);
calogRegister(calog, "reached", nativeReached, NULL);
calogRegister(calog, "forbidden", nativeForbidden, NULL);
calogRegister(calog, "report", nativeReport, NULL);
calogRegisterInline(calog, "done", nativeDone, NULL);
// 1. Allow-list: the script may call reached/report/done, but forbidden is denied (pcall catches
// the "not permitted" error), so its native body never runs.
memset(&allowLimits, 0, sizeof(allowLimits));
allowLimits.allowList = allowList;
resetFlags();
runLimited(&calogLuaEngine, "reached(); local ok = pcall(forbidden); report(ok); done()", &allowLimits);
CHECK(atomic_load(&reachedFlag), "allow-list: a permitted native runs");
CHECK(atomic_load(&reportValue) == 0, "allow-list: calling a forbidden native errors (pcall -> false)");
CHECK(!atomic_load(&forbiddenFlag), "allow-list: the forbidden native body never ran");
// 2. Memory cap (Lua): a 2 MiB context cannot allocate a 10 MiB string, so it errors before
// reached(); an unlimited context runs the same allocation fine.
memset(&memLimits, 0, sizeof(memLimits));
memLimits.memoryBytes = 2 * 1024 * 1024;
resetFlags();
runLimited(&calogLuaEngine, "local s = string.rep('x', 10 * 1024 * 1024); reached(); done()", &memLimits);
CHECK(atomic_load(&errorCount) >= 1, "memory cap: over-budget allocation errors");
CHECK(!atomic_load(&reachedFlag), "memory cap: the script did not continue past the failed allocation");
resetFlags();
runLimited(&calogLuaEngine, "local s = string.rep('x', 10 * 1024 * 1024); reached(); done()", NULL);
CHECK(atomic_load(&reachedFlag), "no memory cap: the same allocation succeeds");
// 3. Wall-clock budget: a runaway loop is retired (its error reaches the handler) within the
// bounded pump, on Lua and on JavaScript.
memset(&timeLimits, 0, sizeof(timeLimits));
timeLimits.wallClockMillis = 100;
resetFlags();
runLimited(&calogLuaEngine, "while true do end", &timeLimits);
CHECK(atomic_load(&errorCount) >= 1, "time budget (Lua): a runaway loop is retired");
resetFlags();
runLimited(&calogJsEngine, "while (true) {}", &timeLimits);
CHECK(atomic_load(&errorCount) >= 1, "time budget (JS): a runaway loop is retired");
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);
return testsFailed == 0 ? 0 : 1;
}

2
tests/testSsh.c
View file

@ -348,9 +348,7 @@ int main(void) {
CHECK(atomic_load(&results[8]) == 1, "sshExec drains a large multi-realloc output completely");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
calogSshShutdown();
kill(sshd, SIGTERM);
waitpid(sshd, NULL, 0);

1
tests/testTask.c
View file

@ -309,7 +309,6 @@ int main(void) {
testTaskError();
calogDestroy(calog);
calogTaskShutdown();
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);

1
tests/testTime.c
View file

@ -134,7 +134,6 @@ int main(void) {
CHECK(atomic_load(&results[5]) == 1, "timeSleep accepts a real millisecond count");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogContextClose(ctx);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

6
tests/testTimer.c
View file

@ -173,12 +173,6 @@ int main(void) {
CHECK(atomic_load(&doneCount) >= 3, "all three timer scripts completed");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
// Release pending callbacks + stop the thread while the contexts are still alive.
calogTimerShutdown();
calogContextClose(ctxAfter);
calogContextClose(ctxCancel);
calogContextClose(ctxEvery);
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);

135
tests/testTrace.c Normal file
View file

@ -0,0 +1,135 @@
// testTrace.c -- cross-boundary stack traces. A Lua context exports a function that errors; a
// JavaScript context calls it by name. The failure crosses the context/engine boundary, and the
// error delivered to the handler carries the boundary tag ("[lua ctx N] ...") -- so a cross-engine
// failure is readable without changing the error handler's signature.
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include "calogExport.h"
#include <stdatomic.h>
#include <stdio.h>
#include <string.h>
#include <time.h>
#define PUMP_LIMIT 4000
static CalogT *calog = NULL;
static _Atomic bool ready = false;
static _Atomic int32_t errorCount = 0;
static char lastMessage[512] = { 0 };
static int32_t testsRun = 0;
static int32_t testsFailed = 0;
static _Atomic bool reported = false;
static void checkImpl(bool condition, const char *message, int32_t line);
static int32_t nativeReady(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData);
static void pumpUntil(_Atomic bool *flag);
#define CHECK(cond, msg) checkImpl((cond), (msg), __LINE__)
static void checkImpl(bool condition, const char *message, int32_t line) {
testsRun++;
if (!condition) {
testsFailed++;
printf("FAIL testTrace.c:%d %s\n", line, message);
}
}
static int32_t nativeReady(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&ready, true);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
if (argCount == 1 && args[0].type == calogStringE) {
size_t len;
len = (size_t)args[0].as.s.length;
if (len >= sizeof(lastMessage)) {
len = sizeof(lastMessage) - 1;
}
memcpy(lastMessage, args[0].as.s.bytes, len);
lastMessage[len] = '\0';
atomic_store(&reported, true);
}
return calogOkE;
}
static void onError(uint64_t contextId, const char *message, void *userData) {
(void)contextId;
(void)message;
(void)userData;
atomic_fetch_add(&errorCount, 1);
}
static void pumpUntil(_Atomic bool *flag) {
struct timespec ts = { 0, 500000 };
int32_t i;
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(flag)) {
calogPump(calog);
return;
}
nanosleep(&ts, NULL);
}
}
int main(void) {
CalogContextT *lua;
CalogContextT *js;
_Atomic bool errored = false;
int32_t before;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogSetErrorHandler(calog, onError, NULL);
calogExportRegister(calog);
calogRegister(calog, "ready", nativeReady, NULL);
calogRegister(calog, "report", nativeReport, NULL);
// A Lua context exports a function that raises.
lua = calogContextOpen(calog, &calogLuaEngine);
calogContextEval(lua, "calogExport('boom', function() error('kaboom in lua') end); ready()");
pumpUntil(&ready);
CHECK(atomic_load(&ready), "lua context published the export");
// A JavaScript context calls the Lua export by name and catches the failure; the caught error
// is the cross-boundary-tagged message.
before = atomic_load(&errorCount);
js = calogContextOpen(calog, &calogJsEngine);
calogContextEval(js, "try { calogCall('boom'); } catch (e) { report(String(e)); }");
pumpUntil(&reported);
(void)errored;
(void)before;
CHECK(atomic_load(&reported), "the cross-engine call failed and JS caught it");
CHECK(strstr(lastMessage, "kaboom in lua") != NULL, "the original engine error survived the crossing");
CHECK(strstr(lastMessage, "lua ctx") != NULL, "the error is tagged with the boundary it unwound through");
calogDestroy(calog);
printf("last message: %s\n", lastMessage);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);
return testsFailed == 0 ? 0 : 1;
}

146
tests/testUtil.c Normal file
View file

@ -0,0 +1,146 @@
// testUtil.c -- exercises the utility libraries (calogRegex, calogCsv, calogProc) from a Lua
// context: regex match/search/replace/split with flags and groups, RFC 4180 CSV round-trips
// including quoting and embedded NULs, and subprocess run capturing stdout/stderr/exit + feeding
// stdin. A bad regex and a bad CSV delimiter are reported as errors the script can catch.
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include "calogCsv.h"
#include "calogProc.h"
#include "calogRegex.h"
#include <stdatomic.h>
#include <stdio.h>
#include <string.h>
#include <time.h>
#define PUMP_LIMIT 4000
static CalogT *calog = NULL;
static _Atomic int32_t checksRun = 0;
static _Atomic int32_t checksFailed = 0;
static _Atomic bool scriptDone = false;
static int32_t nativeCheck(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData);
static void pumpUntilDone(void);
static const char *SCRIPT =
// --- regex ---
"check(table.concat(regexMatch('c(a)(l)og', 'calog'), ',') == 'calog,a,l', 'regexMatch groups')\n"
"check(regexMatch('zzz', 'calog') == nil, 'regexMatch no match is nil')\n"
"check(regexMatch('A', 'abc', 'i') ~= nil, 'regexMatch i flag')\n"
"check(regexReplace('a', 'banana', 'X', 'g') == 'bXnXnX', 'regexReplace global')\n"
"check(regexReplace('a', 'banana', 'X') == 'bXnana', 'regexReplace first only')\n"
"check(regexReplace('([a-z]+)@([a-z]+)', 'user@host', '${2}.${1}') == 'host.user', 'regexReplace group refs')\n"
"check(table.concat(regexSplit(',', 'a,b,c'), '|') == 'a|b|c', 'regexSplit')\n"
"local s = regexSearch('([0-9]+)', 'abc123def')\n"
"check(s.match == '123' and s.start == 3 and s['end'] == 6, 'regexSearch offsets')\n"
"check(s.groups[1] == '123', 'regexSearch capture group')\n"
"check(not pcall(function() regexMatch('(', 'x') end), 'bad pattern is an error')\n"
// --- csv ---
"local rows = csvParse('a,b,c\\n\"x,y\",2,\"has\"\"q\"\\n')\n"
"check(#rows == 2, 'csvParse row count')\n"
"check(rows[1][3] == 'c', 'csvParse plain cell')\n"
"check(rows[2][1] == 'x,y', 'csvParse quoted cell with delimiter')\n"
"check(rows[2][3] == 'has\"q', 'csvParse escaped quote')\n"
"local nul = csvParse('a\\0b,c')\n"
"check(#nul[1][1] == 3, 'csvParse cell is binary-safe')\n"
"check(csvFormat({{'has,comma', 'x'}}) == '\"has,comma\",x', 'csvFormat quotes a delimiter')\n"
"check(csvFormat({{1, 2}, {3, 4}}) == '1,2\\r\\n3,4', 'csvFormat ints joined by CRLF')\n"
"check(not pcall(function() csvParse('a,b', ',,') end), 'bad delimiter is an error')\n"
// --- subprocess ---
"local r = procRun({'printf', 'hi there'})\n"
"check(r.exit == 0 and r.stdout == 'hi there', 'procRun captures stdout')\n"
"local r2 = procRun({'cat'}, {stdin = 'feed me'})\n"
"check(r2.stdout == 'feed me', 'procRun feeds stdin')\n"
"local r3 = procRun({'sh', '-c', 'exit 3'})\n"
"check(r3.exit == 3, 'procRun reports the exit code')\n"
"local r4 = procRun({'sh', '-c', 'echo out; echo err 1>&2'})\n"
"check(r4.exit == 0 and string.find(r4.stdout, 'out') ~= nil and string.find(r4.stderr, 'err') ~= nil, 'procRun splits stdout and stderr')\n"
"done()\n";
static int32_t nativeCheck(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)userData;
calogValueNil(result);
atomic_fetch_add(&checksRun, 1);
if (argCount < 1 || args[0].type != calogBoolE) {
atomic_fetch_add(&checksFailed, 1);
printf("FAIL check: expected a boolean\n");
return calogOkE;
}
if (!args[0].as.b) {
atomic_fetch_add(&checksFailed, 1);
printf("FAIL %s\n", argCount >= 2 && args[1].type == calogStringE ? args[1].as.s.bytes : "(unnamed)");
}
return calogOkE;
}
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_store(&scriptDone, true);
calogValueNil(result);
return calogOkE;
}
static void onError(uint64_t contextId, const char *message, void *userData) {
(void)contextId;
(void)userData;
printf("FAIL script error: %s\n", message);
atomic_fetch_add(&checksFailed, 1);
atomic_store(&scriptDone, true);
}
static void pumpUntilDone(void) {
struct timespec ts = { 0, 500000 };
int32_t i;
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&scriptDone)) {
calogPump(calog);
return;
}
nanosleep(&ts, NULL);
}
}
int main(void) {
CalogContextT *ctx;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogSetErrorHandler(calog, onError, NULL);
if (calogRegexRegister(calog) != calogOkE || calogCsvRegister(calog) != calogOkE || calogProcRegister(calog) != calogOkE) {
printf("utility register failed\n");
return 1;
}
calogRegisterInline(calog, "check", nativeCheck, NULL);
calogRegisterInline(calog, "done", nativeDone, NULL);
ctx = calogContextOpen(calog, &calogLuaEngine);
if (ctx == NULL) {
printf("context open failed\n");
return 1;
}
calogContextEval(ctx, SCRIPT);
pumpUntilDone();
calogDestroy(calog);
printf("\n%d checks, %d failed\n", atomic_load(&checksRun), atomic_load(&checksFailed));
fflush(stdout);
return atomic_load(&checksFailed) == 0 ? 0 : 1;
}

166
tests/testXml.c Normal file
View file

@ -0,0 +1,166 @@
// testXml.c -- exercises the XML library: parse tags/attributes/nested + mixed content, entity
// decode and re-escape, self-closing elements, round-trip, and error handling, driven from Lua.
#define _POSIX_C_SOURCE 200809L
#include "calog.h"
#include "calogXml.h"
#include <stdatomic.h>
#include <stdio.h>
#include <time.h>
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
#define PUMP_LIMIT 4000
#define RESULT_SLOTS 16
static CalogT *calog = NULL;
static _Atomic int64_t results[RESULT_SLOTS];
static _Atomic int32_t doneCount = 0;
static _Atomic int32_t errorCount = 0;
static int32_t testsRun = 0;
static int32_t testsFailed = 0;
static int64_t asInt(const CalogValueT *value);
static void checkImpl(bool condition, const char *message, const char *file, int32_t line);
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
static void onError(uint64_t contextId, const char *message, void *userData);
static void pumpUntilDone(int32_t target);
static int64_t asInt(const CalogValueT *value) {
if (value->type == calogIntE) {
return value->as.i;
}
return (int64_t)value->as.r;
}
static void checkImpl(bool condition, const char *message, const char *file, int32_t line) {
testsRun++;
if (!condition) {
testsFailed++;
printf("FAIL %s:%d %s\n", file, line, message);
}
}
static int32_t nativeDone(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
(void)args;
(void)argCount;
(void)userData;
atomic_fetch_add(&doneCount, 1);
calogValueNil(result);
return calogOkE;
}
static int32_t nativeReport(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
int64_t tag;
(void)userData;
calogValueNil(result);
if (argCount != 2 || (args[0].type != calogIntE && args[0].type != calogRealE)) {
return calogFail(result, calogErrArgE, "report expects (tag, value)");
}
tag = asInt(&args[0]);
if (tag < 0 || tag >= RESULT_SLOTS) {
return calogFail(result, calogErrArgE, "report: tag out of range");
}
atomic_store(&results[tag], asInt(&args[1]));
return calogOkE;
}
static void onError(uint64_t contextId, const char *message, void *userData) {
(void)contextId;
(void)userData;
fprintf(stderr, " [script error] %s\n", (message != NULL) ? message : "(null)");
atomic_fetch_add(&errorCount, 1);
}
static void pumpUntilDone(int32_t target) {
struct timespec ts = { 0, 500000 };
int i;
for (i = 0; i < PUMP_LIMIT; i++) {
calogPump(calog);
if (atomic_load(&doneCount) >= target) {
calogPump(calog);
return;
}
nanosleep(&ts, NULL);
}
}
int main(void) {
CalogContextT *ctx;
int32_t i;
calog = calogCreate();
if (calog == NULL) {
printf("calog create failed\n");
return 1;
}
calogSetErrorHandler(calog, onError, NULL);
calogRegister(calog, "report", nativeReport, NULL);
calogRegister(calog, "done", nativeDone, NULL);
calogXmlRegister(calog);
for (i = 0; i < RESULT_SLOTS; i++) {
atomic_store(&results[i], -1);
}
ctx = calogContextOpen(calog, &calogLuaEngine);
calogContextEval(ctx,
"local e = xmlParse('<greeting>hello</greeting>')\n"
"report(1, (e.tag == 'greeting' and e.children[1] == 'hello') and 1 or 0)\n" // 1, tag + text
"local it = xmlParse('<item id=\"42\" name=\"widget\"/>')\n"
"report(2, (it.attr.id == '42' and it.attr.name == 'widget' and #it.children == 0) and 1 or 0)\n" // 1, attrs + self-close
"local r = xmlParse('<root><a>1</a><b>2</b></root>')\n"
"report(3, tonumber(r.children[1].children[1]) + tonumber(r.children[2].children[1]))\n" // 3, nested navigation
"local x = '<doc><p k=\"v\">text</p></doc>'\n"
"report(4, xmlStringify(xmlParse(x)) == x and 1 or 0)\n" // 1, lossless round-trip
"local t = xmlParse('<t>a &amp; b &lt; c</t>')\n"
"report(5, t.children[1] == 'a & b < c' and 1 or 0)\n" // 1, entities decode
"report(6, xmlStringify(t) == '<t>a &amp; b &lt; c</t>' and 1 or 0)\n" // 1, text re-escaped
"local ok = pcall(function() xmlParse('<unclosed>') end)\n"
"report(7, ok and 0 or 1)\n" // 1, malformed is catchable
"local p = xmlParse('<p>before<b>bold</b>after</p>')\n"
"report(8, (p.children[1]=='before' and p.children[2].tag=='b' and p.children[2].children[1]=='bold' and p.children[3]=='after') and 1 or 0)\n" // 1, mixed content
"local a = xmlParse('<a t=\"x &amp; y\"/>')\n"
"report(9, a.attr.t == 'x & y' and 1 or 0)\n" // 1, attribute entity decode
"report(10, xmlStringify(a) == '<a t=\"x &amp; y\"/>' and 1 or 0)\n" // 1, attribute re-escaped
"local w = xmlParse('<a t=\"x&#9;y&#10;z&#13;w\">p&#13;q</a>')\n"
"local w2 = xmlParse(xmlStringify(w))\n"
"report(11, w2.attr.t == 'x\\ty\\nz\\rw' and 1 or 0)\n" // 1, TAB/LF/CR in attr survive round-trip
"report(12, w2.children[1] == 'p\\rq' and 1 or 0)\n" // 1, CR in text survives round-trip
"done()");
pumpUntilDone(1);
CHECK(atomic_load(&results[1]) == 1, "parse a tag with a text child");
CHECK(atomic_load(&results[2]) == 1, "parse attributes on a self-closing element");
CHECK(atomic_load(&results[3]) == 3, "navigate nested elements");
CHECK(atomic_load(&results[4]) == 1, "stringify + parse is lossless for compact XML");
CHECK(atomic_load(&results[5]) == 1, "predefined entities decode to text");
CHECK(atomic_load(&results[6]) == 1, "text is re-escaped on stringify");
CHECK(atomic_load(&results[7]) == 1, "malformed XML raises a catchable error");
CHECK(atomic_load(&results[8]) == 1, "mixed content preserves text and element order");
CHECK(atomic_load(&results[9]) == 1, "attribute entities decode");
CHECK(atomic_load(&results[10]) == 1, "attribute values are re-escaped on stringify");
CHECK(atomic_load(&results[11]) == 1, "TAB/LF/CR in an attribute survive a round-trip");
CHECK(atomic_load(&results[12]) == 1, "CR in text survives a round-trip");
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
calogDestroy(calog);
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
fflush(stdout);
if (testsFailed != 0) {
return 1;
}
return 0;
}

149
tools/crossArchive.sh Executable file
View file

@ -0,0 +1,149 @@
#!/usr/bin/env bash
#
# crossArchive.sh -- cross-compile calog's compression/archive stack (libarchive + its 5 codecs:
# zlib, bzip2, lz4, zstd, xz/liblzma) for musl (fully static Linux, zero runtime deps) and Windows
# x64, using a single zig toolchain as the cross-compiler. Companion to crossBuild.sh, which proves
# the core+engines+sockets; this proves the archive library, the one CMake-heavy vendored stack.
#
# The musl artifact is fully static and RUNS on the build host, so this script builds AND runs a
# static testArchive (real round-trip through every codec). The Windows artifact is a testArchive.exe
# verified as a valid PE (running it needs a Windows host or wine).
#
# Scope: this cross-builds libarchive + the five codecs only. The XAR format (native builds enable it
# via libxml2 + OpenSSL + iconv) is NOT included here -- cross XAR would need those three cross-built
# too. So XAR is a native-only feature; the cross artifacts cover every codec and the other formats.
#
# Requirements: zig (https://ziglang.org/download/) and cmake. Point ZIG at the binary or PATH it:
# ZIG=/path/to/zig ./tools/crossArchive.sh
#
# Why zig for the CMake libs: xz + libarchive build out of tree with CMake. zig is wrapped as a
# single-binary compiler (bin/zcc-*) so CMake sees a normal cc. The musl toolchain deliberately does
# NOT set CMAKE_SYSTEM_NAME -- musl-static binaries run on this Linux host, so the build stays
# "native" and CMake's try_run feature-detection works. The Windows toolchain DOES set it, so CMake
# enters cross mode and skips its run-only checks (libarchive guards its one CHECK_C_SOURCE_RUNS on
# CMAKE_CROSSCOMPILING; xz has none), so nothing needs pre-seeding.
set -u
cd "$(dirname "$0")/.."
R=$(pwd)
ZIG=${ZIG:-zig}
X=${OUT:-build/cross}
XABS="$R/$X"
if ! command -v "$ZIG" >/dev/null 2>&1; then
echo "error: zig not found. Set ZIG=/path/to/zig (see https://ziglang.org/download/)." >&2
exit 1
fi
command -v cmake >/dev/null 2>&1 || { echo "error: cmake not found." >&2; exit 1; }
ZIGABS=$(command -v "$ZIG"); ZIGABS=$(readlink -f "$ZIGABS" 2>/dev/null || echo "$ZIGABS")
echo "zig: $($ZIG version)"
echo "cmake: $(cmake --version | head -1)"
mkdir -p "$XABS/bin"
# --- single-binary compiler/archiver wrappers with the resolved zig path baked in (ephemeral) ---
gen() { printf '#!/bin/sh\nexec "%s" %s "$@"\n' "$ZIGABS" "$2" > "$XABS/bin/$1"; chmod +x "$XABS/bin/$1"; }
gen zcc-musl "cc -target x86_64-linux-musl"
gen zcc-win "cc -target x86_64-windows-gnu"
gen zar "ar"
gen zranlib "ranlib"
# musl toolchain: no CMAKE_SYSTEM_NAME -> native build, try_run works with static musl on the host.
cat > "$XABS/toolchain-musl.cmake" <<EOF
set(CMAKE_C_COMPILER "$XABS/bin/zcc-musl")
set(CMAKE_AR "$XABS/bin/zar")
set(CMAKE_RANLIB "$XABS/bin/zranlib")
set(CMAKE_EXE_LINKER_FLAGS_INIT "-static")
EOF
# windows toolchain: CMAKE_SYSTEM_NAME=Windows -> cross mode, run-only checks auto-skipped.
cat > "$XABS/toolchain-win.cmake" <<EOF
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR x86_64)
set(CMAKE_C_COMPILER "$XABS/bin/zcc-win")
set(CMAKE_AR "$XABS/bin/zar")
set(CMAKE_RANLIB "$XABS/bin/zranlib")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
EOF
pass=0; fail=0
CORE="src/broker.c src/value.c src/context.c"
LUASRC=$(ls vendor/lua/src/*.c | grep -vE '/(lua|luac)\.c$' | tr '\n' ' ')
# build_codecs <cc-wrapper> <outdir> -- the 4 object-rule codecs (zlib needs -DHAVE_UNISTD_H so it
# includes <unistd.h> for lseek; configure normally sets that, our object-rule build must pass it).
build_codecs() {
local CC=$1 O=$2 c
mkdir -p "$O/obj" "$O/lib"
for c in vendor/zlib/*.c; do "$CC" -std=c11 -D_GNU_SOURCE -DHAVE_UNISTD_H -w -O2 -Ivendor/zlib -c "$c" -o "$O/obj/z_$(basename "$c" .c).o" || return 1; done
"$XABS/bin/zar" rcs "$O/lib/libz.a" "$O"/obj/z_*.o
for c in vendor/bzip2/*.c; do "$CC" -std=c11 -D_GNU_SOURCE -w -O2 -Ivendor/bzip2 -c "$c" -o "$O/obj/bz_$(basename "$c" .c).o" || return 1; done
"$XABS/bin/zar" rcs "$O/lib/libbz2.a" "$O"/obj/bz_*.o
for c in vendor/lz4/*.c; do "$CC" -std=c11 -D_GNU_SOURCE -w -O2 -Ivendor/lz4 -c "$c" -o "$O/obj/l4_$(basename "$c" .c).o" || return 1; done
"$XABS/bin/zar" rcs "$O/lib/liblz4.a" "$O"/obj/l4_*.o
for c in vendor/zstd/common/*.c vendor/zstd/compress/*.c vendor/zstd/decompress/*.c; do "$CC" -std=c11 -D_GNU_SOURCE -w -O2 -DZSTD_DISABLE_ASM -Ivendor/zstd -c "$c" -o "$O/obj/zs_$(basename "$c" .c).o" || return 1; done
"$XABS/bin/zar" rcs "$O/lib/libzstd.a" "$O"/obj/zs_*.o
}
# build_cmake_libs <toolchain> <outdir> -- xz/liblzma then libarchive, each codec backend pinned to
# the just-built cross libs (headers are arch-independent -> include dirs stay the vendored source).
build_cmake_libs() {
local TC=$1 O=$2
cmake -S vendor/xz -B "$O/xz" -DCMAKE_TOOLCHAIN_FILE="$TC" \
-DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
-DENABLE_THREADS=OFF -DENABLE_NLS=OFF -DENABLE_DOXYGEN=OFF >"$O/xz-cfg.log" 2>&1 || { echo " xz configure FAILED (see $O/xz-cfg.log)"; return 1; }
cmake --build "$O/xz" --target liblzma -j >"$O/xz-build.log" 2>&1 || { echo " xz build FAILED (see $O/xz-build.log)"; return 1; }
cmake -S vendor/libarchive -B "$O/libarchive" -DCMAKE_TOOLCHAIN_FILE="$TC" \
-DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DENABLE_TEST=OFF -DENABLE_INSTALL=OFF \
-DENABLE_TAR=OFF -DENABLE_CPIO=OFF -DENABLE_CAT=OFF -DENABLE_UNZIP=OFF \
-DENABLE_ACL=OFF -DENABLE_XATTR=OFF -DENABLE_ICONV=OFF -DENABLE_LIBB2=OFF \
-DENABLE_LIBXML2=OFF -DENABLE_EXPAT=OFF -DENABLE_OPENSSL=OFF \
-DENABLE_LZO=OFF -DENABLE_NETTLE=OFF -DENABLE_MBEDTLS=OFF -DENABLE_PCREPOSIX=OFF -DENABLE_PCRE2POSIX=OFF \
-DENABLE_ZLIB=ON -DZLIB_INCLUDE_DIR="$R/vendor/zlib" -DZLIB_LIBRARY="$O/lib/libz.a" \
-DENABLE_BZip2=ON -DBZIP2_INCLUDE_DIR="$R/vendor/bzip2" -DBZIP2_LIBRARIES="$O/lib/libbz2.a" \
-DENABLE_LZMA=ON -DLIBLZMA_INCLUDE_DIR="$R/vendor/xz/src/liblzma/api" -DLIBLZMA_LIBRARY="$O/xz/liblzma.a" \
-DENABLE_ZSTD=ON -DZSTD_INCLUDE_DIR="$R/vendor/zstd" -DZSTD_LIBRARY="$O/lib/libzstd.a" \
-DENABLE_LZ4=ON -DLZ4_INCLUDE_DIR="$R/vendor/lz4" -DLZ4_LIBRARY="$O/lib/liblz4.a" \
>"$O/la-cfg.log" 2>&1 || { echo " libarchive configure FAILED (see $O/la-cfg.log)"; return 1; }
cmake --build "$O/libarchive" --target archive_static -j >"$O/la-build.log" 2>&1 || { echo " libarchive build FAILED (see $O/la-build.log)"; return 1; }
}
# the compression stack link line (libarchive FIRST for static link order), per target outdir.
archlibs() { echo "$1/libarchive/libarchive/libarchive.a $1/xz/liblzma.a $1/lib/libzstd.a $1/lib/liblz4.a $1/lib/libbz2.a $1/lib/libz.a"; }
ARCHINC="-Isrc -Isrc/lua -Ilibs -Ivendor/lua/src -Ivendor/libarchive/libarchive"
echo "== musl (fully static Linux, built AND run) =="
if build_codecs "$XABS/bin/zcc-musl" "$XABS/musl" && build_cmake_libs "$XABS/toolchain-musl.cmake" "$XABS/musl"; then
if "$XABS/bin/zcc-musl" -static -O2 -pthread -w $ARCHINC -DLUA_USE_POSIX \
$CORE src/lua/luaEngine.c src/lua/luaAdapter.c libs/calogArchive.c libs/calogHandle.c tests/testArchive.c \
$LUASRC $(archlibs "$XABS/musl") -lm -o "$XABS/musl/testArchive-musl" 2>"$XABS/musl/testArchive-link.err"; then
if "$XABS/musl/testArchive-musl" >/dev/null 2>&1; then
echo " [musl RUN ok] testArchive (libarchive + zlib/bzip2/lz4/zstd/xz, fully static)"; pass=$((pass+1))
else echo " [musl RAN, nonzero] testArchive"; fail=$((fail+1)); fi
else echo " [musl LINK FAIL] testArchive (see $XABS/musl/testArchive-link.err)"; fail=$((fail+1)); fi
else fail=$((fail+1)); fi
echo "== Windows x64 (.exe built against vendored winpthreads; run needs wine) =="
# testArchive uses calog's pthread actor model -> needs winpthreads (vendored), like crossBuild.sh.
rm -rf "$XABS/wpobj"; mkdir -p "$XABS/wpobj" "$XABS/win"
wpok=1
for c in vendor/winpthreads/src/*.c; do
"$XABS/bin/zcc-win" -c -O2 -w -Ivendor/winpthreads/include -Ivendor/winpthreads/src \
-DWINPTHREAD_STATIC=1 -DIN_WINPTHREAD=1 "$c" -o "$XABS/wpobj/$(basename "$c" .c).o" || { wpok=0; break; }
done
"$XABS/bin/zar" rcs "$XABS/win/libwinpthreads.a" "$XABS"/wpobj/*.o 2>/dev/null
# libarchive on Windows uses system libs the POSIX build does not: CNG crypto (bcrypt) for its AES/
# PBKDF2, and XmlLite + OLE (xmllite/ole32) for XAR, which auto-replaces the libxml2/expat we disable.
WINSYS="-lbcrypt -lxmllite -lole32 -ladvapi32 -lcrypt32 -lshlwapi"
if [ "$wpok" = 1 ] && build_codecs "$XABS/bin/zcc-win" "$XABS/win" && build_cmake_libs "$XABS/toolchain-win.cmake" "$XABS/win"; then
if "$XABS/bin/zcc-win" -O2 -w $ARCHINC -Ivendor/winpthreads/include -DWINPTHREAD_STATIC=1 \
$CORE src/lua/luaEngine.c src/lua/luaAdapter.c libs/calogArchive.c libs/calogHandle.c tests/testArchive.c \
$LUASRC $(archlibs "$XABS/win") "$XABS/win/libwinpthreads.a" $WINSYS -o "$XABS/win/testArchive.exe" 2>"$XABS/win/testArchive-link.err"; then
if file "$XABS/win/testArchive.exe" | grep -q 'PE32+ executable'; then
echo " [win BUILT] testArchive.exe (libarchive + 5 codecs, valid PE)"; pass=$((pass+1))
else echo " [win ??] testArchive.exe (not a PE?)"; fail=$((fail+1)); fi
else echo " [win LINK FAIL] testArchive.exe (see $XABS/win/testArchive-link.err)"; fail=$((fail+1)); fi
else fail=$((fail+1)); fi
echo
echo "== crossArchive: $pass ok, $fail failed (artifacts in $X/) =="
[ "$fail" -eq 0 ]

42
vendor/bzip2/LICENSE vendored Normal file
View file

@ -0,0 +1,42 @@
--------------------------------------------------------------------------
This program, "bzip2", the associated library "libbzip2", and all
documentation, are copyright (C) 1996-2019 Julian R Seward. All
rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, jseward@acm.org
bzip2/libbzip2 version 1.0.8 of 13 July 2019
--------------------------------------------------------------------------

20
vendor/bzip2/NOTICE.calog vendored Normal file
View file

@ -0,0 +1,20 @@
bzip2 / libbzip2 (vendored into calog)
======================================
bzip2, the block-sorting lossless compression library by Julian Seward
(https://sourceware.org/bzip2/). calog builds it from source into lib/libbz2.a
as the bzip2 filter backend for the archive/compression library (libarchive's
bzip2 filter) and as a single, reproducible source of truth for bzip2 across the
whole build.
Vendored from bzip2 1.0.8: the 7 library .c files (blocksort, huffman, crctable,
randtable, compress, decompress, bzlib) + the public header bzlib.h + the internal
header bzlib_private.h. No configure step is needed -- pure object-rules, compiled
straight to a static archive. The command-line tools (bzip2.c, bzip2recover.c),
tests, docs, and build cruft are NOT vendored.
The libbz2 core is single-threaded: it exposes no worker-thread encoder, so calog's
actor model and ThreadSanitizer targets stay clean.
License: the bzip2 license (permissive, BSD-style); see the LICENSE file in this
directory.

1094
vendor/bzip2/blocksort.c vendored Normal file
View file

File diff suppressed because it is too large Load diff

1572
vendor/bzip2/bzlib.c vendored Normal file
View file

File diff suppressed because it is too large Load diff

282
vendor/bzip2/bzlib.h vendored Normal file
View file

@ -0,0 +1,282 @@
/*-------------------------------------------------------------*/
/*--- Public header file for the library. ---*/
/*--- bzlib.h ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#ifndef _BZLIB_H
#define _BZLIB_H
#ifdef __cplusplus
extern "C" {
#endif
#define BZ_RUN 0
#define BZ_FLUSH 1
#define BZ_FINISH 2
#define BZ_OK 0
#define BZ_RUN_OK 1
#define BZ_FLUSH_OK 2
#define BZ_FINISH_OK 3
#define BZ_STREAM_END 4
#define BZ_SEQUENCE_ERROR (-1)
#define BZ_PARAM_ERROR (-2)
#define BZ_MEM_ERROR (-3)
#define BZ_DATA_ERROR (-4)
#define BZ_DATA_ERROR_MAGIC (-5)
#define BZ_IO_ERROR (-6)
#define BZ_UNEXPECTED_EOF (-7)
#define BZ_OUTBUFF_FULL (-8)
#define BZ_CONFIG_ERROR (-9)
typedef
struct {
char *next_in;
unsigned int avail_in;
unsigned int total_in_lo32;
unsigned int total_in_hi32;
char *next_out;
unsigned int avail_out;
unsigned int total_out_lo32;
unsigned int total_out_hi32;
void *state;
void *(*bzalloc)(void *,int,int);
void (*bzfree)(void *,void *);
void *opaque;
}
bz_stream;
#ifndef BZ_IMPORT
#define BZ_EXPORT
#endif
#ifndef BZ_NO_STDIO
/* Need a definitition for FILE */
#include <stdio.h>
#endif
#ifdef _WIN32
# include <windows.h>
# ifdef small
/* windows.h define small to char */
# undef small
# endif
# ifdef BZ_EXPORT
# define BZ_API(func) WINAPI func
# define BZ_EXTERN extern
# else
/* import windows dll dynamically */
# define BZ_API(func) (WINAPI * func)
# define BZ_EXTERN
# endif
#else
# define BZ_API(func) func
# define BZ_EXTERN extern
#endif
/*-- Core (low-level) library functions --*/
BZ_EXTERN int BZ_API(BZ2_bzCompressInit) (
bz_stream* strm,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN int BZ_API(BZ2_bzCompress) (
bz_stream* strm,
int action
);
BZ_EXTERN int BZ_API(BZ2_bzCompressEnd) (
bz_stream* strm
);
BZ_EXTERN int BZ_API(BZ2_bzDecompressInit) (
bz_stream *strm,
int verbosity,
int small
);
BZ_EXTERN int BZ_API(BZ2_bzDecompress) (
bz_stream* strm
);
BZ_EXTERN int BZ_API(BZ2_bzDecompressEnd) (
bz_stream *strm
);
/*-- High(er) level library functions --*/
#ifndef BZ_NO_STDIO
#define BZ_MAX_UNUSED 5000
typedef void BZFILE;
BZ_EXTERN BZFILE* BZ_API(BZ2_bzReadOpen) (
int* bzerror,
FILE* f,
int verbosity,
int small,
void* unused,
int nUnused
);
BZ_EXTERN void BZ_API(BZ2_bzReadClose) (
int* bzerror,
BZFILE* b
);
BZ_EXTERN void BZ_API(BZ2_bzReadGetUnused) (
int* bzerror,
BZFILE* b,
void** unused,
int* nUnused
);
BZ_EXTERN int BZ_API(BZ2_bzRead) (
int* bzerror,
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN BZFILE* BZ_API(BZ2_bzWriteOpen) (
int* bzerror,
FILE* f,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN void BZ_API(BZ2_bzWrite) (
int* bzerror,
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN void BZ_API(BZ2_bzWriteClose) (
int* bzerror,
BZFILE* b,
int abandon,
unsigned int* nbytes_in,
unsigned int* nbytes_out
);
BZ_EXTERN void BZ_API(BZ2_bzWriteClose64) (
int* bzerror,
BZFILE* b,
int abandon,
unsigned int* nbytes_in_lo32,
unsigned int* nbytes_in_hi32,
unsigned int* nbytes_out_lo32,
unsigned int* nbytes_out_hi32
);
#endif
/*-- Utility functions --*/
BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffCompress) (
char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffDecompress) (
char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int small,
int verbosity
);
/*--
Code contributed by Yoshioka Tsuneo (tsuneo@rr.iij4u.or.jp)
to support better zlib compatibility.
This code is not _officially_ part of libbzip2 (yet);
I haven't tested it, documented it, or considered the
threading-safeness of it.
If this code breaks, please contact both Yoshioka and me.
--*/
BZ_EXTERN const char * BZ_API(BZ2_bzlibVersion) (
void
);
#ifndef BZ_NO_STDIO
BZ_EXTERN BZFILE * BZ_API(BZ2_bzopen) (
const char *path,
const char *mode
);
BZ_EXTERN BZFILE * BZ_API(BZ2_bzdopen) (
int fd,
const char *mode
);
BZ_EXTERN int BZ_API(BZ2_bzread) (
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN int BZ_API(BZ2_bzwrite) (
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN int BZ_API(BZ2_bzflush) (
BZFILE* b
);
BZ_EXTERN void BZ_API(BZ2_bzclose) (
BZFILE* b
);
BZ_EXTERN const char * BZ_API(BZ2_bzerror) (
BZFILE *b,
int *errnum
);
#endif
#ifdef __cplusplus
}
#endif
#endif
/*-------------------------------------------------------------*/
/*--- end bzlib.h ---*/
/*-------------------------------------------------------------*/

509
vendor/bzip2/bzlib_private.h vendored Normal file
View file

@ -0,0 +1,509 @@
/*-------------------------------------------------------------*/
/*--- Private header file for the library. ---*/
/*--- bzlib_private.h ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#ifndef _BZLIB_PRIVATE_H
#define _BZLIB_PRIVATE_H
#include <stdlib.h>
#ifndef BZ_NO_STDIO
#include <stdio.h>
#include <ctype.h>
#include <string.h>
#endif
#include "bzlib.h"
/*-- General stuff. --*/
#define BZ_VERSION "1.0.8, 13-Jul-2019"
typedef char Char;
typedef unsigned char Bool;
typedef unsigned char UChar;
typedef int Int32;
typedef unsigned int UInt32;
typedef short Int16;
typedef unsigned short UInt16;
#define True ((Bool)1)
#define False ((Bool)0)
#ifndef __GNUC__
#define __inline__ /* */
#endif
#ifndef BZ_NO_STDIO
extern void BZ2_bz__AssertH__fail ( int errcode );
#define AssertH(cond,errcode) \
{ if (!(cond)) BZ2_bz__AssertH__fail ( errcode ); }
#if BZ_DEBUG
#define AssertD(cond,msg) \
{ if (!(cond)) { \
fprintf ( stderr, \
"\n\nlibbzip2(debug build): internal error\n\t%s\n", msg );\
exit(1); \
}}
#else
#define AssertD(cond,msg) /* */
#endif
#define VPrintf0(zf) \
fprintf(stderr,zf)
#define VPrintf1(zf,za1) \
fprintf(stderr,zf,za1)
#define VPrintf2(zf,za1,za2) \
fprintf(stderr,zf,za1,za2)
#define VPrintf3(zf,za1,za2,za3) \
fprintf(stderr,zf,za1,za2,za3)
#define VPrintf4(zf,za1,za2,za3,za4) \
fprintf(stderr,zf,za1,za2,za3,za4)
#define VPrintf5(zf,za1,za2,za3,za4,za5) \
fprintf(stderr,zf,za1,za2,za3,za4,za5)
#else
extern void bz_internal_error ( int errcode );
#define AssertH(cond,errcode) \
{ if (!(cond)) bz_internal_error ( errcode ); }
#define AssertD(cond,msg) do { } while (0)
#define VPrintf0(zf) do { } while (0)
#define VPrintf1(zf,za1) do { } while (0)
#define VPrintf2(zf,za1,za2) do { } while (0)
#define VPrintf3(zf,za1,za2,za3) do { } while (0)
#define VPrintf4(zf,za1,za2,za3,za4) do { } while (0)
#define VPrintf5(zf,za1,za2,za3,za4,za5) do { } while (0)
#endif
#define BZALLOC(nnn) (strm->bzalloc)(strm->opaque,(nnn),1)
#define BZFREE(ppp) (strm->bzfree)(strm->opaque,(ppp))
/*-- Header bytes. --*/
#define BZ_HDR_B 0x42 /* 'B' */
#define BZ_HDR_Z 0x5a /* 'Z' */
#define BZ_HDR_h 0x68 /* 'h' */
#define BZ_HDR_0 0x30 /* '0' */
/*-- Constants for the back end. --*/
#define BZ_MAX_ALPHA_SIZE 258
#define BZ_MAX_CODE_LEN 23
#define BZ_RUNA 0
#define BZ_RUNB 1
#define BZ_N_GROUPS 6
#define BZ_G_SIZE 50
#define BZ_N_ITERS 4
#define BZ_MAX_SELECTORS (2 + (900000 / BZ_G_SIZE))
/*-- Stuff for randomising repetitive blocks. --*/
extern Int32 BZ2_rNums[512];
#define BZ_RAND_DECLS \
Int32 rNToGo; \
Int32 rTPos \
#define BZ_RAND_INIT_MASK \
s->rNToGo = 0; \
s->rTPos = 0 \
#define BZ_RAND_MASK ((s->rNToGo == 1) ? 1 : 0)
#define BZ_RAND_UPD_MASK \
if (s->rNToGo == 0) { \
s->rNToGo = BZ2_rNums[s->rTPos]; \
s->rTPos++; \
if (s->rTPos == 512) s->rTPos = 0; \
} \
s->rNToGo--;
/*-- Stuff for doing CRCs. --*/
extern UInt32 BZ2_crc32Table[256];
#define BZ_INITIALISE_CRC(crcVar) \
{ \
crcVar = 0xffffffffL; \
}
#define BZ_FINALISE_CRC(crcVar) \
{ \
crcVar = ~(crcVar); \
}
#define BZ_UPDATE_CRC(crcVar,cha) \
{ \
crcVar = (crcVar << 8) ^ \
BZ2_crc32Table[(crcVar >> 24) ^ \
((UChar)cha)]; \
}
/*-- States and modes for compression. --*/
#define BZ_M_IDLE 1
#define BZ_M_RUNNING 2
#define BZ_M_FLUSHING 3
#define BZ_M_FINISHING 4
#define BZ_S_OUTPUT 1
#define BZ_S_INPUT 2
#define BZ_N_RADIX 2
#define BZ_N_QSORT 12
#define BZ_N_SHELL 18
#define BZ_N_OVERSHOOT (BZ_N_RADIX + BZ_N_QSORT + BZ_N_SHELL + 2)
/*-- Structure holding all the compression-side stuff. --*/
typedef
struct {
/* pointer back to the struct bz_stream */
bz_stream* strm;
/* mode this stream is in, and whether inputting */
/* or outputting data */
Int32 mode;
Int32 state;
/* remembers avail_in when flush/finish requested */
UInt32 avail_in_expect;
/* for doing the block sorting */
UInt32* arr1;
UInt32* arr2;
UInt32* ftab;
Int32 origPtr;
/* aliases for arr1 and arr2 */
UInt32* ptr;
UChar* block;
UInt16* mtfv;
UChar* zbits;
/* for deciding when to use the fallback sorting algorithm */
Int32 workFactor;
/* run-length-encoding of the input */
UInt32 state_in_ch;
Int32 state_in_len;
BZ_RAND_DECLS;
/* input and output limits and current posns */
Int32 nblock;
Int32 nblockMAX;
Int32 numZ;
Int32 state_out_pos;
/* map of bytes used in block */
Int32 nInUse;
Bool inUse[256];
UChar unseqToSeq[256];
/* the buffer for bit stream creation */
UInt32 bsBuff;
Int32 bsLive;
/* block and combined CRCs */
UInt32 blockCRC;
UInt32 combinedCRC;
/* misc administratium */
Int32 verbosity;
Int32 blockNo;
Int32 blockSize100k;
/* stuff for coding the MTF values */
Int32 nMTF;
Int32 mtfFreq [BZ_MAX_ALPHA_SIZE];
UChar selector [BZ_MAX_SELECTORS];
UChar selectorMtf[BZ_MAX_SELECTORS];
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 code [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 rfreq [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
/* second dimension: only 3 needed; 4 makes index calculations faster */
UInt32 len_pack[BZ_MAX_ALPHA_SIZE][4];
}
EState;
/*-- externs for compression. --*/
extern void
BZ2_blockSort ( EState* );
extern void
BZ2_compressBlock ( EState*, Bool );
extern void
BZ2_bsInitWrite ( EState* );
extern void
BZ2_hbAssignCodes ( Int32*, UChar*, Int32, Int32, Int32 );
extern void
BZ2_hbMakeCodeLengths ( UChar*, Int32*, Int32, Int32 );
/*-- states for decompression. --*/
#define BZ_X_IDLE 1
#define BZ_X_OUTPUT 2
#define BZ_X_MAGIC_1 10
#define BZ_X_MAGIC_2 11
#define BZ_X_MAGIC_3 12
#define BZ_X_MAGIC_4 13
#define BZ_X_BLKHDR_1 14
#define BZ_X_BLKHDR_2 15
#define BZ_X_BLKHDR_3 16
#define BZ_X_BLKHDR_4 17
#define BZ_X_BLKHDR_5 18
#define BZ_X_BLKHDR_6 19
#define BZ_X_BCRC_1 20
#define BZ_X_BCRC_2 21
#define BZ_X_BCRC_3 22
#define BZ_X_BCRC_4 23
#define BZ_X_RANDBIT 24
#define BZ_X_ORIGPTR_1 25
#define BZ_X_ORIGPTR_2 26
#define BZ_X_ORIGPTR_3 27
#define BZ_X_MAPPING_1 28
#define BZ_X_MAPPING_2 29
#define BZ_X_SELECTOR_1 30
#define BZ_X_SELECTOR_2 31
#define BZ_X_SELECTOR_3 32
#define BZ_X_CODING_1 33
#define BZ_X_CODING_2 34
#define BZ_X_CODING_3 35
#define BZ_X_MTF_1 36
#define BZ_X_MTF_2 37
#define BZ_X_MTF_3 38
#define BZ_X_MTF_4 39
#define BZ_X_MTF_5 40
#define BZ_X_MTF_6 41
#define BZ_X_ENDHDR_2 42
#define BZ_X_ENDHDR_3 43
#define BZ_X_ENDHDR_4 44
#define BZ_X_ENDHDR_5 45
#define BZ_X_ENDHDR_6 46
#define BZ_X_CCRC_1 47
#define BZ_X_CCRC_2 48
#define BZ_X_CCRC_3 49
#define BZ_X_CCRC_4 50
/*-- Constants for the fast MTF decoder. --*/
#define MTFA_SIZE 4096
#define MTFL_SIZE 16
/*-- Structure holding all the decompression-side stuff. --*/
typedef
struct {
/* pointer back to the struct bz_stream */
bz_stream* strm;
/* state indicator for this stream */
Int32 state;
/* for doing the final run-length decoding */
UChar state_out_ch;
Int32 state_out_len;
Bool blockRandomised;
BZ_RAND_DECLS;
/* the buffer for bit stream reading */
UInt32 bsBuff;
Int32 bsLive;
/* misc administratium */
Int32 blockSize100k;
Bool smallDecompress;
Int32 currBlockNo;
Int32 verbosity;
/* for undoing the Burrows-Wheeler transform */
Int32 origPtr;
UInt32 tPos;
Int32 k0;
Int32 unzftab[256];
Int32 nblock_used;
Int32 cftab[257];
Int32 cftabCopy[257];
/* for undoing the Burrows-Wheeler transform (FAST) */
UInt32 *tt;
/* for undoing the Burrows-Wheeler transform (SMALL) */
UInt16 *ll16;
UChar *ll4;
/* stored and calculated CRCs */
UInt32 storedBlockCRC;
UInt32 storedCombinedCRC;
UInt32 calculatedBlockCRC;
UInt32 calculatedCombinedCRC;
/* map of bytes used in block */
Int32 nInUse;
Bool inUse[256];
Bool inUse16[16];
UChar seqToUnseq[256];
/* for decoding the MTF values */
UChar mtfa [MTFA_SIZE];
Int32 mtfbase[256 / MTFL_SIZE];
UChar selector [BZ_MAX_SELECTORS];
UChar selectorMtf[BZ_MAX_SELECTORS];
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 limit [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 base [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 perm [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 minLens[BZ_N_GROUPS];
/* save area for scalars in the main decompress code */
Int32 save_i;
Int32 save_j;
Int32 save_t;
Int32 save_alphaSize;
Int32 save_nGroups;
Int32 save_nSelectors;
Int32 save_EOB;
Int32 save_groupNo;
Int32 save_groupPos;
Int32 save_nextSym;
Int32 save_nblockMAX;
Int32 save_nblock;
Int32 save_es;
Int32 save_N;
Int32 save_curr;
Int32 save_zt;
Int32 save_zn;
Int32 save_zvec;
Int32 save_zj;
Int32 save_gSel;
Int32 save_gMinlen;
Int32* save_gLimit;
Int32* save_gBase;
Int32* save_gPerm;
}
DState;
/*-- Macros for decompression. --*/
#define BZ_GET_FAST(cccc) \
/* c_tPos is unsigned, hence test < 0 is pointless. */ \
if (s->tPos >= (UInt32)100000 * (UInt32)s->blockSize100k) return True; \
s->tPos = s->tt[s->tPos]; \
cccc = (UChar)(s->tPos & 0xff); \
s->tPos >>= 8;
#define BZ_GET_FAST_C(cccc) \
/* c_tPos is unsigned, hence test < 0 is pointless. */ \
if (c_tPos >= (UInt32)100000 * (UInt32)ro_blockSize100k) return True; \
c_tPos = c_tt[c_tPos]; \
cccc = (UChar)(c_tPos & 0xff); \
c_tPos >>= 8;
#define SET_LL4(i,n) \
{ if (((i) & 0x1) == 0) \
s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0xf0) | (n); else \
s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0x0f) | ((n) << 4); \
}
#define GET_LL4(i) \
((((UInt32)(s->ll4[(i) >> 1])) >> (((i) << 2) & 0x4)) & 0xF)
#define SET_LL(i,n) \
{ s->ll16[i] = (UInt16)(n & 0x0000ffff); \
SET_LL4(i, n >> 16); \
}
#define GET_LL(i) \
(((UInt32)s->ll16[i]) | (GET_LL4(i) << 16))
#define BZ_GET_SMALL(cccc) \
/* c_tPos is unsigned, hence test < 0 is pointless. */ \
if (s->tPos >= (UInt32)100000 * (UInt32)s->blockSize100k) return True; \
cccc = BZ2_indexIntoF ( s->tPos, s->cftab ); \
s->tPos = GET_LL(s->tPos);
/*-- externs for decompression. --*/
extern Int32
BZ2_indexIntoF ( Int32, Int32* );
extern Int32
BZ2_decompress ( DState* );
extern void
BZ2_hbCreateDecodeTables ( Int32*, Int32*, Int32*, UChar*,
Int32, Int32, Int32 );
#endif
/*-- BZ_NO_STDIO seems to make NULL disappear on some platforms. --*/
#ifdef BZ_NO_STDIO
#ifndef NULL
#define NULL 0
#endif
#endif
/*-------------------------------------------------------------*/
/*--- end bzlib_private.h ---*/
/*-------------------------------------------------------------*/

672
vendor/bzip2/compress.c vendored Normal file
View file

@ -0,0 +1,672 @@
/*-------------------------------------------------------------*/
/*--- Compression machinery (not incl block sorting) ---*/
/*--- compress.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
/* CHANGES
0.9.0 -- original version.
0.9.0a/b -- no changes in this file.
0.9.0c -- changed setting of nGroups in sendMTFValues()
so as to do a bit better on small files
*/
#include "bzlib_private.h"
/*---------------------------------------------------*/
/*--- Bit stream I/O ---*/
/*---------------------------------------------------*/
/*---------------------------------------------------*/
void BZ2_bsInitWrite ( EState* s )
{
s->bsLive = 0;
s->bsBuff = 0;
}
/*---------------------------------------------------*/
static
void bsFinishWrite ( EState* s )
{
while (s->bsLive > 0) {
s->zbits[s->numZ] = (UChar)(s->bsBuff >> 24);
s->numZ++;
s->bsBuff <<= 8;
s->bsLive -= 8;
}
}
/*---------------------------------------------------*/
#define bsNEEDW(nz) \
{ \
while (s->bsLive >= 8) { \
s->zbits[s->numZ] \
= (UChar)(s->bsBuff >> 24); \
s->numZ++; \
s->bsBuff <<= 8; \
s->bsLive -= 8; \
} \
}
/*---------------------------------------------------*/
static
__inline__
void bsW ( EState* s, Int32 n, UInt32 v )
{
bsNEEDW ( n );
s->bsBuff |= (v << (32 - s->bsLive - n));
s->bsLive += n;
}
/*---------------------------------------------------*/
static
void bsPutUInt32 ( EState* s, UInt32 u )
{
bsW ( s, 8, (u >> 24) & 0xffL );
bsW ( s, 8, (u >> 16) & 0xffL );
bsW ( s, 8, (u >> 8) & 0xffL );
bsW ( s, 8, u & 0xffL );
}
/*---------------------------------------------------*/
static
void bsPutUChar ( EState* s, UChar c )
{
bsW( s, 8, (UInt32)c );
}
/*---------------------------------------------------*/
/*--- The back end proper ---*/
/*---------------------------------------------------*/
/*---------------------------------------------------*/
static
void makeMaps_e ( EState* s )
{
Int32 i;
s->nInUse = 0;
for (i = 0; i < 256; i++)
if (s->inUse[i]) {
s->unseqToSeq[i] = s->nInUse;
s->nInUse++;
}
}
/*---------------------------------------------------*/
static
void generateMTFValues ( EState* s )
{
UChar yy[256];
Int32 i, j;
Int32 zPend;
Int32 wr;
Int32 EOB;
/*
After sorting (eg, here),
s->arr1 [ 0 .. s->nblock-1 ] holds sorted order,
and
((UChar*)s->arr2) [ 0 .. s->nblock-1 ]
holds the original block data.
The first thing to do is generate the MTF values,
and put them in
((UInt16*)s->arr1) [ 0 .. s->nblock-1 ].
Because there are strictly fewer or equal MTF values
than block values, ptr values in this area are overwritten
with MTF values only when they are no longer needed.
The final compressed bitstream is generated into the
area starting at
(UChar*) (&((UChar*)s->arr2)[s->nblock])
These storage aliases are set up in bzCompressInit(),
except for the last one, which is arranged in
compressBlock().
*/
UInt32* ptr = s->ptr;
UChar* block = s->block;
UInt16* mtfv = s->mtfv;
makeMaps_e ( s );
EOB = s->nInUse+1;
for (i = 0; i <= EOB; i++) s->mtfFreq[i] = 0;
wr = 0;
zPend = 0;
for (i = 0; i < s->nInUse; i++) yy[i] = (UChar) i;
for (i = 0; i < s->nblock; i++) {
UChar ll_i;
AssertD ( wr <= i, "generateMTFValues(1)" );
j = ptr[i]-1; if (j < 0) j += s->nblock;
ll_i = s->unseqToSeq[block[j]];
AssertD ( ll_i < s->nInUse, "generateMTFValues(2a)" );
if (yy[0] == ll_i) {
zPend++;
} else {
if (zPend > 0) {
zPend--;
while (True) {
if (zPend & 1) {
mtfv[wr] = BZ_RUNB; wr++;
s->mtfFreq[BZ_RUNB]++;
} else {
mtfv[wr] = BZ_RUNA; wr++;
s->mtfFreq[BZ_RUNA]++;
}
if (zPend < 2) break;
zPend = (zPend - 2) / 2;
};
zPend = 0;
}
{
register UChar rtmp;
register UChar* ryy_j;
register UChar rll_i;
rtmp = yy[1];
yy[1] = yy[0];
ryy_j = &(yy[1]);
rll_i = ll_i;
while ( rll_i != rtmp ) {
register UChar rtmp2;
ryy_j++;
rtmp2 = rtmp;
rtmp = *ryy_j;
*ryy_j = rtmp2;
};
yy[0] = rtmp;
j = ryy_j - &(yy[0]);
mtfv[wr] = j+1; wr++; s->mtfFreq[j+1]++;
}
}
}
if (zPend > 0) {
zPend--;
while (True) {
if (zPend & 1) {
mtfv[wr] = BZ_RUNB; wr++;
s->mtfFreq[BZ_RUNB]++;
} else {
mtfv[wr] = BZ_RUNA; wr++;
s->mtfFreq[BZ_RUNA]++;
}
if (zPend < 2) break;
zPend = (zPend - 2) / 2;
};
zPend = 0;
}
mtfv[wr] = EOB; wr++; s->mtfFreq[EOB]++;
s->nMTF = wr;
}
/*---------------------------------------------------*/
#define BZ_LESSER_ICOST 0
#define BZ_GREATER_ICOST 15
static
void sendMTFValues ( EState* s )
{
Int32 v, t, i, j, gs, ge, totc, bt, bc, iter;
Int32 nSelectors, alphaSize, minLen, maxLen, selCtr;
Int32 nGroups, nBytes;
/*--
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
is a global since the decoder also needs it.
Int32 code[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 rfreq[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
are also globals only used in this proc.
Made global to keep stack frame size small.
--*/
UInt16 cost[BZ_N_GROUPS];
Int32 fave[BZ_N_GROUPS];
UInt16* mtfv = s->mtfv;
if (s->verbosity >= 3)
VPrintf3( " %d in block, %d after MTF & 1-2 coding, "
"%d+2 syms in use\n",
s->nblock, s->nMTF, s->nInUse );
alphaSize = s->nInUse+2;
for (t = 0; t < BZ_N_GROUPS; t++)
for (v = 0; v < alphaSize; v++)
s->len[t][v] = BZ_GREATER_ICOST;
/*--- Decide how many coding tables to use ---*/
AssertH ( s->nMTF > 0, 3001 );
if (s->nMTF < 200) nGroups = 2; else
if (s->nMTF < 600) nGroups = 3; else
if (s->nMTF < 1200) nGroups = 4; else
if (s->nMTF < 2400) nGroups = 5; else
nGroups = 6;
/*--- Generate an initial set of coding tables ---*/
{
Int32 nPart, remF, tFreq, aFreq;
nPart = nGroups;
remF = s->nMTF;
gs = 0;
while (nPart > 0) {
tFreq = remF / nPart;
ge = gs-1;
aFreq = 0;
while (aFreq < tFreq && ge < alphaSize-1) {
ge++;
aFreq += s->mtfFreq[ge];
}
if (ge > gs
&& nPart != nGroups && nPart != 1
&& ((nGroups-nPart) % 2 == 1)) {
aFreq -= s->mtfFreq[ge];
ge--;
}
if (s->verbosity >= 3)
VPrintf5( " initial group %d, [%d .. %d], "
"has %d syms (%4.1f%%)\n",
nPart, gs, ge, aFreq,
(100.0 * (float)aFreq) / (float)(s->nMTF) );
for (v = 0; v < alphaSize; v++)
if (v >= gs && v <= ge)
s->len[nPart-1][v] = BZ_LESSER_ICOST; else
s->len[nPart-1][v] = BZ_GREATER_ICOST;
nPart--;
gs = ge+1;
remF -= aFreq;
}
}
/*---
Iterate up to BZ_N_ITERS times to improve the tables.
---*/
for (iter = 0; iter < BZ_N_ITERS; iter++) {
for (t = 0; t < nGroups; t++) fave[t] = 0;
for (t = 0; t < nGroups; t++)
for (v = 0; v < alphaSize; v++)
s->rfreq[t][v] = 0;
/*---
Set up an auxiliary length table which is used to fast-track
the common case (nGroups == 6).
---*/
if (nGroups == 6) {
for (v = 0; v < alphaSize; v++) {
s->len_pack[v][0] = (s->len[1][v] << 16) | s->len[0][v];
s->len_pack[v][1] = (s->len[3][v] << 16) | s->len[2][v];
s->len_pack[v][2] = (s->len[5][v] << 16) | s->len[4][v];
}
}
nSelectors = 0;
totc = 0;
gs = 0;
while (True) {
/*--- Set group start & end marks. --*/
if (gs >= s->nMTF) break;
ge = gs + BZ_G_SIZE - 1;
if (ge >= s->nMTF) ge = s->nMTF-1;
/*--
Calculate the cost of this group as coded
by each of the coding tables.
--*/
for (t = 0; t < nGroups; t++) cost[t] = 0;
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
register UInt32 cost01, cost23, cost45;
register UInt16 icv;
cost01 = cost23 = cost45 = 0;
# define BZ_ITER(nn) \
icv = mtfv[gs+(nn)]; \
cost01 += s->len_pack[icv][0]; \
cost23 += s->len_pack[icv][1]; \
cost45 += s->len_pack[icv][2]; \
BZ_ITER(0); BZ_ITER(1); BZ_ITER(2); BZ_ITER(3); BZ_ITER(4);
BZ_ITER(5); BZ_ITER(6); BZ_ITER(7); BZ_ITER(8); BZ_ITER(9);
BZ_ITER(10); BZ_ITER(11); BZ_ITER(12); BZ_ITER(13); BZ_ITER(14);
BZ_ITER(15); BZ_ITER(16); BZ_ITER(17); BZ_ITER(18); BZ_ITER(19);
BZ_ITER(20); BZ_ITER(21); BZ_ITER(22); BZ_ITER(23); BZ_ITER(24);
BZ_ITER(25); BZ_ITER(26); BZ_ITER(27); BZ_ITER(28); BZ_ITER(29);
BZ_ITER(30); BZ_ITER(31); BZ_ITER(32); BZ_ITER(33); BZ_ITER(34);
BZ_ITER(35); BZ_ITER(36); BZ_ITER(37); BZ_ITER(38); BZ_ITER(39);
BZ_ITER(40); BZ_ITER(41); BZ_ITER(42); BZ_ITER(43); BZ_ITER(44);
BZ_ITER(45); BZ_ITER(46); BZ_ITER(47); BZ_ITER(48); BZ_ITER(49);
# undef BZ_ITER
cost[0] = cost01 & 0xffff; cost[1] = cost01 >> 16;
cost[2] = cost23 & 0xffff; cost[3] = cost23 >> 16;
cost[4] = cost45 & 0xffff; cost[5] = cost45 >> 16;
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++) {
UInt16 icv = mtfv[i];
for (t = 0; t < nGroups; t++) cost[t] += s->len[t][icv];
}
}
/*--
Find the coding table which is best for this group,
and record its identity in the selector table.
--*/
bc = 999999999; bt = -1;
for (t = 0; t < nGroups; t++)
if (cost[t] < bc) { bc = cost[t]; bt = t; };
totc += bc;
fave[bt]++;
s->selector[nSelectors] = bt;
nSelectors++;
/*--
Increment the symbol frequencies for the selected table.
--*/
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
# define BZ_ITUR(nn) s->rfreq[bt][ mtfv[gs+(nn)] ]++
BZ_ITUR(0); BZ_ITUR(1); BZ_ITUR(2); BZ_ITUR(3); BZ_ITUR(4);
BZ_ITUR(5); BZ_ITUR(6); BZ_ITUR(7); BZ_ITUR(8); BZ_ITUR(9);
BZ_ITUR(10); BZ_ITUR(11); BZ_ITUR(12); BZ_ITUR(13); BZ_ITUR(14);
BZ_ITUR(15); BZ_ITUR(16); BZ_ITUR(17); BZ_ITUR(18); BZ_ITUR(19);
BZ_ITUR(20); BZ_ITUR(21); BZ_ITUR(22); BZ_ITUR(23); BZ_ITUR(24);
BZ_ITUR(25); BZ_ITUR(26); BZ_ITUR(27); BZ_ITUR(28); BZ_ITUR(29);
BZ_ITUR(30); BZ_ITUR(31); BZ_ITUR(32); BZ_ITUR(33); BZ_ITUR(34);
BZ_ITUR(35); BZ_ITUR(36); BZ_ITUR(37); BZ_ITUR(38); BZ_ITUR(39);
BZ_ITUR(40); BZ_ITUR(41); BZ_ITUR(42); BZ_ITUR(43); BZ_ITUR(44);
BZ_ITUR(45); BZ_ITUR(46); BZ_ITUR(47); BZ_ITUR(48); BZ_ITUR(49);
# undef BZ_ITUR
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++)
s->rfreq[bt][ mtfv[i] ]++;
}
gs = ge+1;
}
if (s->verbosity >= 3) {
VPrintf2 ( " pass %d: size is %d, grp uses are ",
iter+1, totc/8 );
for (t = 0; t < nGroups; t++)
VPrintf1 ( "%d ", fave[t] );
VPrintf0 ( "\n" );
}
/*--
Recompute the tables based on the accumulated frequencies.
--*/
/* maxLen was changed from 20 to 17 in bzip2-1.0.3. See
comment in huffman.c for details. */
for (t = 0; t < nGroups; t++)
BZ2_hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]),
alphaSize, 17 /*20*/ );
}
AssertH( nGroups < 8, 3002 );
AssertH( nSelectors < 32768 &&
nSelectors <= BZ_MAX_SELECTORS,
3003 );
/*--- Compute MTF values for the selectors. ---*/
{
UChar pos[BZ_N_GROUPS], ll_i, tmp2, tmp;
for (i = 0; i < nGroups; i++) pos[i] = i;
for (i = 0; i < nSelectors; i++) {
ll_i = s->selector[i];
j = 0;
tmp = pos[j];
while ( ll_i != tmp ) {
j++;
tmp2 = tmp;
tmp = pos[j];
pos[j] = tmp2;
};
pos[0] = tmp;
s->selectorMtf[i] = j;
}
};
/*--- Assign actual codes for the tables. --*/
for (t = 0; t < nGroups; t++) {
minLen = 32;
maxLen = 0;
for (i = 0; i < alphaSize; i++) {
if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
if (s->len[t][i] < minLen) minLen = s->len[t][i];
}
AssertH ( !(maxLen > 17 /*20*/ ), 3004 );
AssertH ( !(minLen < 1), 3005 );
BZ2_hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]),
minLen, maxLen, alphaSize );
}
/*--- Transmit the mapping table. ---*/
{
Bool inUse16[16];
for (i = 0; i < 16; i++) {
inUse16[i] = False;
for (j = 0; j < 16; j++)
if (s->inUse[i * 16 + j]) inUse16[i] = True;
}
nBytes = s->numZ;
for (i = 0; i < 16; i++)
if (inUse16[i]) bsW(s,1,1); else bsW(s,1,0);
for (i = 0; i < 16; i++)
if (inUse16[i])
for (j = 0; j < 16; j++) {
if (s->inUse[i * 16 + j]) bsW(s,1,1); else bsW(s,1,0);
}
if (s->verbosity >= 3)
VPrintf1( " bytes: mapping %d, ", s->numZ-nBytes );
}
/*--- Now the selectors. ---*/
nBytes = s->numZ;
bsW ( s, 3, nGroups );
bsW ( s, 15, nSelectors );
for (i = 0; i < nSelectors; i++) {
for (j = 0; j < s->selectorMtf[i]; j++) bsW(s,1,1);
bsW(s,1,0);
}
if (s->verbosity >= 3)
VPrintf1( "selectors %d, ", s->numZ-nBytes );
/*--- Now the coding tables. ---*/
nBytes = s->numZ;
for (t = 0; t < nGroups; t++) {
Int32 curr = s->len[t][0];
bsW ( s, 5, curr );
for (i = 0; i < alphaSize; i++) {
while (curr < s->len[t][i]) { bsW(s,2,2); curr++; /* 10 */ };
while (curr > s->len[t][i]) { bsW(s,2,3); curr--; /* 11 */ };
bsW ( s, 1, 0 );
}
}
if (s->verbosity >= 3)
VPrintf1 ( "code lengths %d, ", s->numZ-nBytes );
/*--- And finally, the block data proper ---*/
nBytes = s->numZ;
selCtr = 0;
gs = 0;
while (True) {
if (gs >= s->nMTF) break;
ge = gs + BZ_G_SIZE - 1;
if (ge >= s->nMTF) ge = s->nMTF-1;
AssertH ( s->selector[selCtr] < nGroups, 3006 );
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
UInt16 mtfv_i;
UChar* s_len_sel_selCtr
= &(s->len[s->selector[selCtr]][0]);
Int32* s_code_sel_selCtr
= &(s->code[s->selector[selCtr]][0]);
# define BZ_ITAH(nn) \
mtfv_i = mtfv[gs+(nn)]; \
bsW ( s, \
s_len_sel_selCtr[mtfv_i], \
s_code_sel_selCtr[mtfv_i] )
BZ_ITAH(0); BZ_ITAH(1); BZ_ITAH(2); BZ_ITAH(3); BZ_ITAH(4);
BZ_ITAH(5); BZ_ITAH(6); BZ_ITAH(7); BZ_ITAH(8); BZ_ITAH(9);
BZ_ITAH(10); BZ_ITAH(11); BZ_ITAH(12); BZ_ITAH(13); BZ_ITAH(14);
BZ_ITAH(15); BZ_ITAH(16); BZ_ITAH(17); BZ_ITAH(18); BZ_ITAH(19);
BZ_ITAH(20); BZ_ITAH(21); BZ_ITAH(22); BZ_ITAH(23); BZ_ITAH(24);
BZ_ITAH(25); BZ_ITAH(26); BZ_ITAH(27); BZ_ITAH(28); BZ_ITAH(29);
BZ_ITAH(30); BZ_ITAH(31); BZ_ITAH(32); BZ_ITAH(33); BZ_ITAH(34);
BZ_ITAH(35); BZ_ITAH(36); BZ_ITAH(37); BZ_ITAH(38); BZ_ITAH(39);
BZ_ITAH(40); BZ_ITAH(41); BZ_ITAH(42); BZ_ITAH(43); BZ_ITAH(44);
BZ_ITAH(45); BZ_ITAH(46); BZ_ITAH(47); BZ_ITAH(48); BZ_ITAH(49);
# undef BZ_ITAH
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++) {
bsW ( s,
s->len [s->selector[selCtr]] [mtfv[i]],
s->code [s->selector[selCtr]] [mtfv[i]] );
}
}
gs = ge+1;
selCtr++;
}
AssertH( selCtr == nSelectors, 3007 );
if (s->verbosity >= 3)
VPrintf1( "codes %d\n", s->numZ-nBytes );
}
/*---------------------------------------------------*/
void BZ2_compressBlock ( EState* s, Bool is_last_block )
{
if (s->nblock > 0) {
BZ_FINALISE_CRC ( s->blockCRC );
s->combinedCRC = (s->combinedCRC << 1) | (s->combinedCRC >> 31);
s->combinedCRC ^= s->blockCRC;
if (s->blockNo > 1) s->numZ = 0;
if (s->verbosity >= 2)
VPrintf4( " block %d: crc = 0x%08x, "
"combined CRC = 0x%08x, size = %d\n",
s->blockNo, s->blockCRC, s->combinedCRC, s->nblock );
BZ2_blockSort ( s );
}
s->zbits = (UChar*) (&((UChar*)s->arr2)[s->nblock]);
/*-- If this is the first block, create the stream header. --*/
if (s->blockNo == 1) {
BZ2_bsInitWrite ( s );
bsPutUChar ( s, BZ_HDR_B );
bsPutUChar ( s, BZ_HDR_Z );
bsPutUChar ( s, BZ_HDR_h );
bsPutUChar ( s, (UChar)(BZ_HDR_0 + s->blockSize100k) );
}
if (s->nblock > 0) {
bsPutUChar ( s, 0x31 ); bsPutUChar ( s, 0x41 );
bsPutUChar ( s, 0x59 ); bsPutUChar ( s, 0x26 );
bsPutUChar ( s, 0x53 ); bsPutUChar ( s, 0x59 );
/*-- Now the block's CRC, so it is in a known place. --*/
bsPutUInt32 ( s, s->blockCRC );
/*--
Now a single bit indicating (non-)randomisation.
As of version 0.9.5, we use a better sorting algorithm
which makes randomisation unnecessary. So always set
the randomised bit to 'no'. Of course, the decoder
still needs to be able to handle randomised blocks
so as to maintain backwards compatibility with
older versions of bzip2.
--*/
bsW(s,1,0);
bsW ( s, 24, s->origPtr );
generateMTFValues ( s );
sendMTFValues ( s );
}
/*-- If this is the last block, add the stream trailer. --*/
if (is_last_block) {
bsPutUChar ( s, 0x17 ); bsPutUChar ( s, 0x72 );
bsPutUChar ( s, 0x45 ); bsPutUChar ( s, 0x38 );
bsPutUChar ( s, 0x50 ); bsPutUChar ( s, 0x90 );
bsPutUInt32 ( s, s->combinedCRC );
if (s->verbosity >= 2)
VPrintf1( " final combined CRC = 0x%08x\n ", s->combinedCRC );
bsFinishWrite ( s );
}
}
/*-------------------------------------------------------------*/
/*--- end compress.c ---*/
/*-------------------------------------------------------------*/

104
vendor/bzip2/crctable.c vendored Normal file
View file

@ -0,0 +1,104 @@
/*-------------------------------------------------------------*/
/*--- Table for doing CRCs ---*/
/*--- crctable.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*--
I think this is an implementation of the AUTODIN-II,
Ethernet & FDDI 32-bit CRC standard. Vaguely derived
from code by Rob Warnock, in Section 51 of the
comp.compression FAQ.
--*/
UInt32 BZ2_crc32Table[256] = {
/*-- Ugly, innit? --*/
0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L,
0x130476dcL, 0x17c56b6bL, 0x1a864db2L, 0x1e475005L,
0x2608edb8L, 0x22c9f00fL, 0x2f8ad6d6L, 0x2b4bcb61L,
0x350c9b64L, 0x31cd86d3L, 0x3c8ea00aL, 0x384fbdbdL,
0x4c11db70L, 0x48d0c6c7L, 0x4593e01eL, 0x4152fda9L,
0x5f15adacL, 0x5bd4b01bL, 0x569796c2L, 0x52568b75L,
0x6a1936c8L, 0x6ed82b7fL, 0x639b0da6L, 0x675a1011L,
0x791d4014L, 0x7ddc5da3L, 0x709f7b7aL, 0x745e66cdL,
0x9823b6e0L, 0x9ce2ab57L, 0x91a18d8eL, 0x95609039L,
0x8b27c03cL, 0x8fe6dd8bL, 0x82a5fb52L, 0x8664e6e5L,
0xbe2b5b58L, 0xbaea46efL, 0xb7a96036L, 0xb3687d81L,
0xad2f2d84L, 0xa9ee3033L, 0xa4ad16eaL, 0xa06c0b5dL,
0xd4326d90L, 0xd0f37027L, 0xddb056feL, 0xd9714b49L,
0xc7361b4cL, 0xc3f706fbL, 0xceb42022L, 0xca753d95L,
0xf23a8028L, 0xf6fb9d9fL, 0xfbb8bb46L, 0xff79a6f1L,
0xe13ef6f4L, 0xe5ffeb43L, 0xe8bccd9aL, 0xec7dd02dL,
0x34867077L, 0x30476dc0L, 0x3d044b19L, 0x39c556aeL,
0x278206abL, 0x23431b1cL, 0x2e003dc5L, 0x2ac12072L,
0x128e9dcfL, 0x164f8078L, 0x1b0ca6a1L, 0x1fcdbb16L,
0x018aeb13L, 0x054bf6a4L, 0x0808d07dL, 0x0cc9cdcaL,
0x7897ab07L, 0x7c56b6b0L, 0x71159069L, 0x75d48ddeL,
0x6b93dddbL, 0x6f52c06cL, 0x6211e6b5L, 0x66d0fb02L,
0x5e9f46bfL, 0x5a5e5b08L, 0x571d7dd1L, 0x53dc6066L,
0x4d9b3063L, 0x495a2dd4L, 0x44190b0dL, 0x40d816baL,
0xaca5c697L, 0xa864db20L, 0xa527fdf9L, 0xa1e6e04eL,
0xbfa1b04bL, 0xbb60adfcL, 0xb6238b25L, 0xb2e29692L,
0x8aad2b2fL, 0x8e6c3698L, 0x832f1041L, 0x87ee0df6L,
0x99a95df3L, 0x9d684044L, 0x902b669dL, 0x94ea7b2aL,
0xe0b41de7L, 0xe4750050L, 0xe9362689L, 0xedf73b3eL,
0xf3b06b3bL, 0xf771768cL, 0xfa325055L, 0xfef34de2L,
0xc6bcf05fL, 0xc27dede8L, 0xcf3ecb31L, 0xcbffd686L,
0xd5b88683L, 0xd1799b34L, 0xdc3abdedL, 0xd8fba05aL,
0x690ce0eeL, 0x6dcdfd59L, 0x608edb80L, 0x644fc637L,
0x7a089632L, 0x7ec98b85L, 0x738aad5cL, 0x774bb0ebL,
0x4f040d56L, 0x4bc510e1L, 0x46863638L, 0x42472b8fL,
0x5c007b8aL, 0x58c1663dL, 0x558240e4L, 0x51435d53L,
0x251d3b9eL, 0x21dc2629L, 0x2c9f00f0L, 0x285e1d47L,
0x36194d42L, 0x32d850f5L, 0x3f9b762cL, 0x3b5a6b9bL,
0x0315d626L, 0x07d4cb91L, 0x0a97ed48L, 0x0e56f0ffL,
0x1011a0faL, 0x14d0bd4dL, 0x19939b94L, 0x1d528623L,
0xf12f560eL, 0xf5ee4bb9L, 0xf8ad6d60L, 0xfc6c70d7L,
0xe22b20d2L, 0xe6ea3d65L, 0xeba91bbcL, 0xef68060bL,
0xd727bbb6L, 0xd3e6a601L, 0xdea580d8L, 0xda649d6fL,
0xc423cd6aL, 0xc0e2d0ddL, 0xcda1f604L, 0xc960ebb3L,
0xbd3e8d7eL, 0xb9ff90c9L, 0xb4bcb610L, 0xb07daba7L,
0xae3afba2L, 0xaafbe615L, 0xa7b8c0ccL, 0xa379dd7bL,
0x9b3660c6L, 0x9ff77d71L, 0x92b45ba8L, 0x9675461fL,
0x8832161aL, 0x8cf30badL, 0x81b02d74L, 0x857130c3L,
0x5d8a9099L, 0x594b8d2eL, 0x5408abf7L, 0x50c9b640L,
0x4e8ee645L, 0x4a4ffbf2L, 0x470cdd2bL, 0x43cdc09cL,
0x7b827d21L, 0x7f436096L, 0x7200464fL, 0x76c15bf8L,
0x68860bfdL, 0x6c47164aL, 0x61043093L, 0x65c52d24L,
0x119b4be9L, 0x155a565eL, 0x18197087L, 0x1cd86d30L,
0x029f3d35L, 0x065e2082L, 0x0b1d065bL, 0x0fdc1becL,
0x3793a651L, 0x3352bbe6L, 0x3e119d3fL, 0x3ad08088L,
0x2497d08dL, 0x2056cd3aL, 0x2d15ebe3L, 0x29d4f654L,
0xc5a92679L, 0xc1683bceL, 0xcc2b1d17L, 0xc8ea00a0L,
0xd6ad50a5L, 0xd26c4d12L, 0xdf2f6bcbL, 0xdbee767cL,
0xe3a1cbc1L, 0xe760d676L, 0xea23f0afL, 0xeee2ed18L,
0xf0a5bd1dL, 0xf464a0aaL, 0xf9278673L, 0xfde69bc4L,
0x89b8fd09L, 0x8d79e0beL, 0x803ac667L, 0x84fbdbd0L,
0x9abc8bd5L, 0x9e7d9662L, 0x933eb0bbL, 0x97ffad0cL,
0xafb010b1L, 0xab710d06L, 0xa6322bdfL, 0xa2f33668L,
0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L
};
/*-------------------------------------------------------------*/
/*--- end crctable.c ---*/
/*-------------------------------------------------------------*/

652
vendor/bzip2/decompress.c vendored Normal file
View file

@ -0,0 +1,652 @@
/*-------------------------------------------------------------*/
/*--- Decompression machinery ---*/
/*--- decompress.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*---------------------------------------------------*/
static
void makeMaps_d ( DState* s )
{
Int32 i;
s->nInUse = 0;
for (i = 0; i < 256; i++)
if (s->inUse[i]) {
s->seqToUnseq[s->nInUse] = i;
s->nInUse++;
}
}
/*---------------------------------------------------*/
#define RETURN(rrr) \
{ retVal = rrr; goto save_state_and_return; };
#define GET_BITS(lll,vvv,nnn) \
case lll: s->state = lll; \
while (True) { \
if (s->bsLive >= nnn) { \
UInt32 v; \
v = (s->bsBuff >> \
(s->bsLive-nnn)) & ((1 << nnn)-1); \
s->bsLive -= nnn; \
vvv = v; \
break; \
} \
if (s->strm->avail_in == 0) RETURN(BZ_OK); \
s->bsBuff \
= (s->bsBuff << 8) | \
((UInt32) \
(*((UChar*)(s->strm->next_in)))); \
s->bsLive += 8; \
s->strm->next_in++; \
s->strm->avail_in--; \
s->strm->total_in_lo32++; \
if (s->strm->total_in_lo32 == 0) \
s->strm->total_in_hi32++; \
}
#define GET_UCHAR(lll,uuu) \
GET_BITS(lll,uuu,8)
#define GET_BIT(lll,uuu) \
GET_BITS(lll,uuu,1)
/*---------------------------------------------------*/
#define GET_MTF_VAL(label1,label2,lval) \
{ \
if (groupPos == 0) { \
groupNo++; \
if (groupNo >= nSelectors) \
RETURN(BZ_DATA_ERROR); \
groupPos = BZ_G_SIZE; \
gSel = s->selector[groupNo]; \
gMinlen = s->minLens[gSel]; \
gLimit = &(s->limit[gSel][0]); \
gPerm = &(s->perm[gSel][0]); \
gBase = &(s->base[gSel][0]); \
} \
groupPos--; \
zn = gMinlen; \
GET_BITS(label1, zvec, zn); \
while (1) { \
if (zn > 20 /* the longest code */) \
RETURN(BZ_DATA_ERROR); \
if (zvec <= gLimit[zn]) break; \
zn++; \
GET_BIT(label2, zj); \
zvec = (zvec << 1) | zj; \
}; \
if (zvec - gBase[zn] < 0 \
|| zvec - gBase[zn] >= BZ_MAX_ALPHA_SIZE) \
RETURN(BZ_DATA_ERROR); \
lval = gPerm[zvec - gBase[zn]]; \
}
/*---------------------------------------------------*/
Int32 BZ2_decompress ( DState* s )
{
UChar uc;
Int32 retVal;
Int32 minLen, maxLen;
bz_stream* strm = s->strm;
/* stuff that needs to be saved/restored */
Int32 i;
Int32 j;
Int32 t;
Int32 alphaSize;
Int32 nGroups;
Int32 nSelectors;
Int32 EOB;
Int32 groupNo;
Int32 groupPos;
Int32 nextSym;
Int32 nblockMAX;
Int32 nblock;
Int32 es;
Int32 N;
Int32 curr;
Int32 zt;
Int32 zn;
Int32 zvec;
Int32 zj;
Int32 gSel;
Int32 gMinlen;
Int32* gLimit;
Int32* gBase;
Int32* gPerm;
if (s->state == BZ_X_MAGIC_1) {
/*initialise the save area*/
s->save_i = 0;
s->save_j = 0;
s->save_t = 0;
s->save_alphaSize = 0;
s->save_nGroups = 0;
s->save_nSelectors = 0;
s->save_EOB = 0;
s->save_groupNo = 0;
s->save_groupPos = 0;
s->save_nextSym = 0;
s->save_nblockMAX = 0;
s->save_nblock = 0;
s->save_es = 0;
s->save_N = 0;
s->save_curr = 0;
s->save_zt = 0;
s->save_zn = 0;
s->save_zvec = 0;
s->save_zj = 0;
s->save_gSel = 0;
s->save_gMinlen = 0;
s->save_gLimit = NULL;
s->save_gBase = NULL;
s->save_gPerm = NULL;
}
/*restore from the save area*/
i = s->save_i;
j = s->save_j;
t = s->save_t;
alphaSize = s->save_alphaSize;
nGroups = s->save_nGroups;
nSelectors = s->save_nSelectors;
EOB = s->save_EOB;
groupNo = s->save_groupNo;
groupPos = s->save_groupPos;
nextSym = s->save_nextSym;
nblockMAX = s->save_nblockMAX;
nblock = s->save_nblock;
es = s->save_es;
N = s->save_N;
curr = s->save_curr;
zt = s->save_zt;
zn = s->save_zn;
zvec = s->save_zvec;
zj = s->save_zj;
gSel = s->save_gSel;
gMinlen = s->save_gMinlen;
gLimit = s->save_gLimit;
gBase = s->save_gBase;
gPerm = s->save_gPerm;
retVal = BZ_OK;
switch (s->state) {
GET_UCHAR(BZ_X_MAGIC_1, uc);
if (uc != BZ_HDR_B) RETURN(BZ_DATA_ERROR_MAGIC);
GET_UCHAR(BZ_X_MAGIC_2, uc);
if (uc != BZ_HDR_Z) RETURN(BZ_DATA_ERROR_MAGIC);
GET_UCHAR(BZ_X_MAGIC_3, uc)
if (uc != BZ_HDR_h) RETURN(BZ_DATA_ERROR_MAGIC);
GET_BITS(BZ_X_MAGIC_4, s->blockSize100k, 8)
if (s->blockSize100k < (BZ_HDR_0 + 1) ||
s->blockSize100k > (BZ_HDR_0 + 9)) RETURN(BZ_DATA_ERROR_MAGIC);
s->blockSize100k -= BZ_HDR_0;
if (s->smallDecompress) {
s->ll16 = BZALLOC( s->blockSize100k * 100000 * sizeof(UInt16) );
s->ll4 = BZALLOC(
((1 + s->blockSize100k * 100000) >> 1) * sizeof(UChar)
);
if (s->ll16 == NULL || s->ll4 == NULL) RETURN(BZ_MEM_ERROR);
} else {
s->tt = BZALLOC( s->blockSize100k * 100000 * sizeof(Int32) );
if (s->tt == NULL) RETURN(BZ_MEM_ERROR);
}
GET_UCHAR(BZ_X_BLKHDR_1, uc);
if (uc == 0x17) goto endhdr_2;
if (uc != 0x31) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_2, uc);
if (uc != 0x41) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_3, uc);
if (uc != 0x59) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_4, uc);
if (uc != 0x26) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_5, uc);
if (uc != 0x53) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_6, uc);
if (uc != 0x59) RETURN(BZ_DATA_ERROR);
s->currBlockNo++;
if (s->verbosity >= 2)
VPrintf1 ( "\n [%d: huff+mtf ", s->currBlockNo );
s->storedBlockCRC = 0;
GET_UCHAR(BZ_X_BCRC_1, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_2, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_3, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_4, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_BITS(BZ_X_RANDBIT, s->blockRandomised, 1);
s->origPtr = 0;
GET_UCHAR(BZ_X_ORIGPTR_1, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
GET_UCHAR(BZ_X_ORIGPTR_2, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
GET_UCHAR(BZ_X_ORIGPTR_3, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
if (s->origPtr < 0)
RETURN(BZ_DATA_ERROR);
if (s->origPtr > 10 + 100000*s->blockSize100k)
RETURN(BZ_DATA_ERROR);
/*--- Receive the mapping table ---*/
for (i = 0; i < 16; i++) {
GET_BIT(BZ_X_MAPPING_1, uc);
if (uc == 1)
s->inUse16[i] = True; else
s->inUse16[i] = False;
}
for (i = 0; i < 256; i++) s->inUse[i] = False;
for (i = 0; i < 16; i++)
if (s->inUse16[i])
for (j = 0; j < 16; j++) {
GET_BIT(BZ_X_MAPPING_2, uc);
if (uc == 1) s->inUse[i * 16 + j] = True;
}
makeMaps_d ( s );
if (s->nInUse == 0) RETURN(BZ_DATA_ERROR);
alphaSize = s->nInUse+2;
/*--- Now the selectors ---*/
GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
if (nGroups < 2 || nGroups > BZ_N_GROUPS) RETURN(BZ_DATA_ERROR);
GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
for (i = 0; i < nSelectors; i++) {
j = 0;
while (True) {
GET_BIT(BZ_X_SELECTOR_3, uc);
if (uc == 0) break;
j++;
if (j >= nGroups) RETURN(BZ_DATA_ERROR);
}
/* Having more than BZ_MAX_SELECTORS doesn't make much sense
since they will never be used, but some implementations might
"round up" the number of selectors, so just ignore those. */
if (i < BZ_MAX_SELECTORS)
s->selectorMtf[i] = j;
}
if (nSelectors > BZ_MAX_SELECTORS)
nSelectors = BZ_MAX_SELECTORS;
/*--- Undo the MTF values for the selectors. ---*/
{
UChar pos[BZ_N_GROUPS], tmp, v;
for (v = 0; v < nGroups; v++) pos[v] = v;
for (i = 0; i < nSelectors; i++) {
v = s->selectorMtf[i];
tmp = pos[v];
while (v > 0) { pos[v] = pos[v-1]; v--; }
pos[0] = tmp;
s->selector[i] = tmp;
}
}
/*--- Now the coding tables ---*/
for (t = 0; t < nGroups; t++) {
GET_BITS(BZ_X_CODING_1, curr, 5);
for (i = 0; i < alphaSize; i++) {
while (True) {
if (curr < 1 || curr > 20) RETURN(BZ_DATA_ERROR);
GET_BIT(BZ_X_CODING_2, uc);
if (uc == 0) break;
GET_BIT(BZ_X_CODING_3, uc);
if (uc == 0) curr++; else curr--;
}
s->len[t][i] = curr;
}
}
/*--- Create the Huffman decoding tables ---*/
for (t = 0; t < nGroups; t++) {
minLen = 32;
maxLen = 0;
for (i = 0; i < alphaSize; i++) {
if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
if (s->len[t][i] < minLen) minLen = s->len[t][i];
}
BZ2_hbCreateDecodeTables (
&(s->limit[t][0]),
&(s->base[t][0]),
&(s->perm[t][0]),
&(s->len[t][0]),
minLen, maxLen, alphaSize
);
s->minLens[t] = minLen;
}
/*--- Now the MTF values ---*/
EOB = s->nInUse+1;
nblockMAX = 100000 * s->blockSize100k;
groupNo = -1;
groupPos = 0;
for (i = 0; i <= 255; i++) s->unzftab[i] = 0;
/*-- MTF init --*/
{
Int32 ii, jj, kk;
kk = MTFA_SIZE-1;
for (ii = 256 / MTFL_SIZE - 1; ii >= 0; ii--) {
for (jj = MTFL_SIZE-1; jj >= 0; jj--) {
s->mtfa[kk] = (UChar)(ii * MTFL_SIZE + jj);
kk--;
}
s->mtfbase[ii] = kk + 1;
}
}
/*-- end MTF init --*/
nblock = 0;
GET_MTF_VAL(BZ_X_MTF_1, BZ_X_MTF_2, nextSym);
while (True) {
if (nextSym == EOB) break;
if (nextSym == BZ_RUNA || nextSym == BZ_RUNB) {
es = -1;
N = 1;
do {
/* Check that N doesn't get too big, so that es doesn't
go negative. The maximum value that can be
RUNA/RUNB encoded is equal to the block size (post
the initial RLE), viz, 900k, so bounding N at 2
million should guard against overflow without
rejecting any legitimate inputs. */
if (N >= 2*1024*1024) RETURN(BZ_DATA_ERROR);
if (nextSym == BZ_RUNA) es = es + (0+1) * N; else
if (nextSym == BZ_RUNB) es = es + (1+1) * N;
N = N * 2;
GET_MTF_VAL(BZ_X_MTF_3, BZ_X_MTF_4, nextSym);
}
while (nextSym == BZ_RUNA || nextSym == BZ_RUNB);
es++;
uc = s->seqToUnseq[ s->mtfa[s->mtfbase[0]] ];
s->unzftab[uc] += es;
if (s->smallDecompress)
while (es > 0) {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
s->ll16[nblock] = (UInt16)uc;
nblock++;
es--;
}
else
while (es > 0) {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
s->tt[nblock] = (UInt32)uc;
nblock++;
es--;
};
continue;
} else {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
/*-- uc = MTF ( nextSym-1 ) --*/
{
Int32 ii, jj, kk, pp, lno, off;
UInt32 nn;
nn = (UInt32)(nextSym - 1);
if (nn < MTFL_SIZE) {
/* avoid general-case expense */
pp = s->mtfbase[0];
uc = s->mtfa[pp+nn];
while (nn > 3) {
Int32 z = pp+nn;
s->mtfa[(z) ] = s->mtfa[(z)-1];
s->mtfa[(z)-1] = s->mtfa[(z)-2];
s->mtfa[(z)-2] = s->mtfa[(z)-3];
s->mtfa[(z)-3] = s->mtfa[(z)-4];
nn -= 4;
}
while (nn > 0) {
s->mtfa[(pp+nn)] = s->mtfa[(pp+nn)-1]; nn--;
};
s->mtfa[pp] = uc;
} else {
/* general case */
lno = nn / MTFL_SIZE;
off = nn % MTFL_SIZE;
pp = s->mtfbase[lno] + off;
uc = s->mtfa[pp];
while (pp > s->mtfbase[lno]) {
s->mtfa[pp] = s->mtfa[pp-1]; pp--;
};
s->mtfbase[lno]++;
while (lno > 0) {
s->mtfbase[lno]--;
s->mtfa[s->mtfbase[lno]]
= s->mtfa[s->mtfbase[lno-1] + MTFL_SIZE - 1];
lno--;
}
s->mtfbase[0]--;
s->mtfa[s->mtfbase[0]] = uc;
if (s->mtfbase[0] == 0) {
kk = MTFA_SIZE-1;
for (ii = 256 / MTFL_SIZE-1; ii >= 0; ii--) {
for (jj = MTFL_SIZE-1; jj >= 0; jj--) {
s->mtfa[kk] = s->mtfa[s->mtfbase[ii] + jj];
kk--;
}
s->mtfbase[ii] = kk + 1;
}
}
}
}
/*-- end uc = MTF ( nextSym-1 ) --*/
s->unzftab[s->seqToUnseq[uc]]++;
if (s->smallDecompress)
s->ll16[nblock] = (UInt16)(s->seqToUnseq[uc]); else
s->tt[nblock] = (UInt32)(s->seqToUnseq[uc]);
nblock++;
GET_MTF_VAL(BZ_X_MTF_5, BZ_X_MTF_6, nextSym);
continue;
}
}
/* Now we know what nblock is, we can do a better sanity
check on s->origPtr.
*/
if (s->origPtr < 0 || s->origPtr >= nblock)
RETURN(BZ_DATA_ERROR);
/*-- Set up cftab to facilitate generation of T^(-1) --*/
/* Check: unzftab entries in range. */
for (i = 0; i <= 255; i++) {
if (s->unzftab[i] < 0 || s->unzftab[i] > nblock)
RETURN(BZ_DATA_ERROR);
}
/* Actually generate cftab. */
s->cftab[0] = 0;
for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1];
for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1];
/* Check: cftab entries in range. */
for (i = 0; i <= 256; i++) {
if (s->cftab[i] < 0 || s->cftab[i] > nblock) {
/* s->cftab[i] can legitimately be == nblock */
RETURN(BZ_DATA_ERROR);
}
}
/* Check: cftab entries non-descending. */
for (i = 1; i <= 256; i++) {
if (s->cftab[i-1] > s->cftab[i]) {
RETURN(BZ_DATA_ERROR);
}
}
s->state_out_len = 0;
s->state_out_ch = 0;
BZ_INITIALISE_CRC ( s->calculatedBlockCRC );
s->state = BZ_X_OUTPUT;
if (s->verbosity >= 2) VPrintf0 ( "rt+rld" );
if (s->smallDecompress) {
/*-- Make a copy of cftab, used in generation of T --*/
for (i = 0; i <= 256; i++) s->cftabCopy[i] = s->cftab[i];
/*-- compute the T vector --*/
for (i = 0; i < nblock; i++) {
uc = (UChar)(s->ll16[i]);
SET_LL(i, s->cftabCopy[uc]);
s->cftabCopy[uc]++;
}
/*-- Compute T^(-1) by pointer reversal on T --*/
i = s->origPtr;
j = GET_LL(i);
do {
Int32 tmp = GET_LL(j);
SET_LL(j, i);
i = j;
j = tmp;
}
while (i != s->origPtr);
s->tPos = s->origPtr;
s->nblock_used = 0;
if (s->blockRandomised) {
BZ_RAND_INIT_MASK;
BZ_GET_SMALL(s->k0); s->nblock_used++;
BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK;
} else {
BZ_GET_SMALL(s->k0); s->nblock_used++;
}
} else {
/*-- compute the T^(-1) vector --*/
for (i = 0; i < nblock; i++) {
uc = (UChar)(s->tt[i] & 0xff);
s->tt[s->cftab[uc]] |= (i << 8);
s->cftab[uc]++;
}
s->tPos = s->tt[s->origPtr] >> 8;
s->nblock_used = 0;
if (s->blockRandomised) {
BZ_RAND_INIT_MASK;
BZ_GET_FAST(s->k0); s->nblock_used++;
BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK;
} else {
BZ_GET_FAST(s->k0); s->nblock_used++;
}
}
RETURN(BZ_OK);
endhdr_2:
GET_UCHAR(BZ_X_ENDHDR_2, uc);
if (uc != 0x72) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_3, uc);
if (uc != 0x45) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_4, uc);
if (uc != 0x38) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_5, uc);
if (uc != 0x50) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_6, uc);
if (uc != 0x90) RETURN(BZ_DATA_ERROR);
s->storedCombinedCRC = 0;
GET_UCHAR(BZ_X_CCRC_1, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_2, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_3, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_4, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
s->state = BZ_X_IDLE;
RETURN(BZ_STREAM_END);
default: AssertH ( False, 4001 );
}
AssertH ( False, 4002 );
save_state_and_return:
s->save_i = i;
s->save_j = j;
s->save_t = t;
s->save_alphaSize = alphaSize;
s->save_nGroups = nGroups;
s->save_nSelectors = nSelectors;
s->save_EOB = EOB;
s->save_groupNo = groupNo;
s->save_groupPos = groupPos;
s->save_nextSym = nextSym;
s->save_nblockMAX = nblockMAX;
s->save_nblock = nblock;
s->save_es = es;
s->save_N = N;
s->save_curr = curr;
s->save_zt = zt;
s->save_zn = zn;
s->save_zvec = zvec;
s->save_zj = zj;
s->save_gSel = gSel;
s->save_gMinlen = gMinlen;
s->save_gLimit = gLimit;
s->save_gBase = gBase;
s->save_gPerm = gPerm;
return retVal;
}
/*-------------------------------------------------------------*/
/*--- end decompress.c ---*/
/*-------------------------------------------------------------*/

205
vendor/bzip2/huffman.c vendored Normal file
View file

@ -0,0 +1,205 @@
/*-------------------------------------------------------------*/
/*--- Huffman coding low-level stuff ---*/
/*--- huffman.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*---------------------------------------------------*/
#define WEIGHTOF(zz0) ((zz0) & 0xffffff00)
#define DEPTHOF(zz1) ((zz1) & 0x000000ff)
#define MYMAX(zz2,zz3) ((zz2) > (zz3) ? (zz2) : (zz3))
#define ADDWEIGHTS(zw1,zw2) \
(WEIGHTOF(zw1)+WEIGHTOF(zw2)) | \
(1 + MYMAX(DEPTHOF(zw1),DEPTHOF(zw2)))
#define UPHEAP(z) \
{ \
Int32 zz, tmp; \
zz = z; tmp = heap[zz]; \
while (weight[tmp] < weight[heap[zz >> 1]]) { \
heap[zz] = heap[zz >> 1]; \
zz >>= 1; \
} \
heap[zz] = tmp; \
}
#define DOWNHEAP(z) \
{ \
Int32 zz, yy, tmp; \
zz = z; tmp = heap[zz]; \
while (True) { \
yy = zz << 1; \
if (yy > nHeap) break; \
if (yy < nHeap && \
weight[heap[yy+1]] < weight[heap[yy]]) \
yy++; \
if (weight[tmp] < weight[heap[yy]]) break; \
heap[zz] = heap[yy]; \
zz = yy; \
} \
heap[zz] = tmp; \
}
/*---------------------------------------------------*/
void BZ2_hbMakeCodeLengths ( UChar *len,
Int32 *freq,
Int32 alphaSize,
Int32 maxLen )
{
/*--
Nodes and heap entries run from 1. Entry 0
for both the heap and nodes is a sentinel.
--*/
Int32 nNodes, nHeap, n1, n2, i, j, k;
Bool tooLong;
Int32 heap [ BZ_MAX_ALPHA_SIZE + 2 ];
Int32 weight [ BZ_MAX_ALPHA_SIZE * 2 ];
Int32 parent [ BZ_MAX_ALPHA_SIZE * 2 ];
for (i = 0; i < alphaSize; i++)
weight[i+1] = (freq[i] == 0 ? 1 : freq[i]) << 8;
while (True) {
nNodes = alphaSize;
nHeap = 0;
heap[0] = 0;
weight[0] = 0;
parent[0] = -2;
for (i = 1; i <= alphaSize; i++) {
parent[i] = -1;
nHeap++;
heap[nHeap] = i;
UPHEAP(nHeap);
}
AssertH( nHeap < (BZ_MAX_ALPHA_SIZE+2), 2001 );
while (nHeap > 1) {
n1 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1);
n2 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1);
nNodes++;
parent[n1] = parent[n2] = nNodes;
weight[nNodes] = ADDWEIGHTS(weight[n1], weight[n2]);
parent[nNodes] = -1;
nHeap++;
heap[nHeap] = nNodes;
UPHEAP(nHeap);
}
AssertH( nNodes < (BZ_MAX_ALPHA_SIZE * 2), 2002 );
tooLong = False;
for (i = 1; i <= alphaSize; i++) {
j = 0;
k = i;
while (parent[k] >= 0) { k = parent[k]; j++; }
len[i-1] = j;
if (j > maxLen) tooLong = True;
}
if (! tooLong) break;
/* 17 Oct 04: keep-going condition for the following loop used
to be 'i < alphaSize', which missed the last element,
theoretically leading to the possibility of the compressor
looping. However, this count-scaling step is only needed if
one of the generated Huffman code words is longer than
maxLen, which up to and including version 1.0.2 was 20 bits,
which is extremely unlikely. In version 1.0.3 maxLen was
changed to 17 bits, which has minimal effect on compression
ratio, but does mean this scaling step is used from time to
time, enough to verify that it works.
This means that bzip2-1.0.3 and later will only produce
Huffman codes with a maximum length of 17 bits. However, in
order to preserve backwards compatibility with bitstreams
produced by versions pre-1.0.3, the decompressor must still
handle lengths of up to 20. */
for (i = 1; i <= alphaSize; i++) {
j = weight[i] >> 8;
j = 1 + (j / 2);
weight[i] = j << 8;
}
}
}
/*---------------------------------------------------*/
void BZ2_hbAssignCodes ( Int32 *code,
UChar *length,
Int32 minLen,
Int32 maxLen,
Int32 alphaSize )
{
Int32 n, vec, i;
vec = 0;
for (n = minLen; n <= maxLen; n++) {
for (i = 0; i < alphaSize; i++)
if (length[i] == n) { code[i] = vec; vec++; };
vec <<= 1;
}
}
/*---------------------------------------------------*/
void BZ2_hbCreateDecodeTables ( Int32 *limit,
Int32 *base,
Int32 *perm,
UChar *length,
Int32 minLen,
Int32 maxLen,
Int32 alphaSize )
{
Int32 pp, i, j, vec;
pp = 0;
for (i = minLen; i <= maxLen; i++)
for (j = 0; j < alphaSize; j++)
if (length[j] == i) { perm[pp] = j; pp++; };
for (i = 0; i < BZ_MAX_CODE_LEN; i++) base[i] = 0;
for (i = 0; i < alphaSize; i++) base[length[i]+1]++;
for (i = 1; i < BZ_MAX_CODE_LEN; i++) base[i] += base[i-1];
for (i = 0; i < BZ_MAX_CODE_LEN; i++) limit[i] = 0;
vec = 0;
for (i = minLen; i <= maxLen; i++) {
vec += (base[i+1] - base[i]);
limit[i] = vec-1;
vec <<= 1;
}
for (i = minLen + 1; i <= maxLen; i++)
base[i] = ((limit[i-1] + 1) << 1) - base[i];
}
/*-------------------------------------------------------------*/
/*--- end huffman.c ---*/
/*-------------------------------------------------------------*/

84
vendor/bzip2/randtable.c vendored Normal file
View file

@ -0,0 +1,84 @@
/*-------------------------------------------------------------*/
/*--- Table for randomising repetitive blocks ---*/
/*--- randtable.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.0.8 of 13 July 2019
Copyright (C) 1996-2019 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*---------------------------------------------*/
Int32 BZ2_rNums[512] = {
619, 720, 127, 481, 931, 816, 813, 233, 566, 247,
985, 724, 205, 454, 863, 491, 741, 242, 949, 214,
733, 859, 335, 708, 621, 574, 73, 654, 730, 472,
419, 436, 278, 496, 867, 210, 399, 680, 480, 51,
878, 465, 811, 169, 869, 675, 611, 697, 867, 561,
862, 687, 507, 283, 482, 129, 807, 591, 733, 623,
150, 238, 59, 379, 684, 877, 625, 169, 643, 105,
170, 607, 520, 932, 727, 476, 693, 425, 174, 647,
73, 122, 335, 530, 442, 853, 695, 249, 445, 515,
909, 545, 703, 919, 874, 474, 882, 500, 594, 612,
641, 801, 220, 162, 819, 984, 589, 513, 495, 799,
161, 604, 958, 533, 221, 400, 386, 867, 600, 782,
382, 596, 414, 171, 516, 375, 682, 485, 911, 276,
98, 553, 163, 354, 666, 933, 424, 341, 533, 870,
227, 730, 475, 186, 263, 647, 537, 686, 600, 224,
469, 68, 770, 919, 190, 373, 294, 822, 808, 206,
184, 943, 795, 384, 383, 461, 404, 758, 839, 887,
715, 67, 618, 276, 204, 918, 873, 777, 604, 560,
951, 160, 578, 722, 79, 804, 96, 409, 713, 940,
652, 934, 970, 447, 318, 353, 859, 672, 112, 785,
645, 863, 803, 350, 139, 93, 354, 99, 820, 908,
609, 772, 154, 274, 580, 184, 79, 626, 630, 742,
653, 282, 762, 623, 680, 81, 927, 626, 789, 125,
411, 521, 938, 300, 821, 78, 343, 175, 128, 250,
170, 774, 972, 275, 999, 639, 495, 78, 352, 126,
857, 956, 358, 619, 580, 124, 737, 594, 701, 612,
669, 112, 134, 694, 363, 992, 809, 743, 168, 974,
944, 375, 748, 52, 600, 747, 642, 182, 862, 81,
344, 805, 988, 739, 511, 655, 814, 334, 249, 515,
897, 955, 664, 981, 649, 113, 974, 459, 893, 228,
433, 837, 553, 268, 926, 240, 102, 654, 459, 51,
686, 754, 806, 760, 493, 403, 415, 394, 687, 700,
946, 670, 656, 610, 738, 392, 760, 799, 887, 653,
978, 321, 576, 617, 626, 502, 894, 679, 243, 440,
680, 879, 194, 572, 640, 724, 926, 56, 204, 700,
707, 151, 457, 449, 797, 195, 791, 558, 945, 679,
297, 59, 87, 824, 713, 663, 412, 693, 342, 606,
134, 108, 571, 364, 631, 212, 174, 643, 304, 329,
343, 97, 430, 751, 497, 314, 983, 374, 822, 928,
140, 206, 73, 263, 980, 736, 876, 478, 430, 305,
170, 514, 364, 692, 829, 82, 855, 953, 676, 246,
369, 970, 294, 750, 807, 827, 150, 790, 288, 923,
804, 378, 215, 828, 592, 281, 565, 555, 710, 82,
896, 831, 547, 261, 524, 462, 293, 465, 502, 56,
661, 821, 976, 991, 658, 869, 905, 758, 745, 193,
768, 550, 608, 933, 378, 286, 215, 979, 792, 961,
61, 688, 793, 644, 986, 403, 106, 366, 905, 644,
372, 567, 466, 434, 645, 210, 389, 550, 919, 135,
780, 773, 635, 389, 707, 100, 626, 958, 165, 504,
920, 176, 193, 713, 857, 265, 203, 50, 668, 108,
645, 990, 626, 197, 510, 357, 358, 850, 858, 364,
936, 638
};
/*-------------------------------------------------------------*/
/*--- end randtable.c ---*/
/*-------------------------------------------------------------*/

19
vendor/janet/LICENSE vendored Normal file
View file

@ -0,0 +1,19 @@
Copyright (c) 2026 Calvin Rose and contributors
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

57303
vendor/janet/janet.c vendored Normal file
View file

File diff suppressed because it is too large Load diff

2455
vendor/janet/janet.h vendored Normal file
View file

File diff suppressed because it is too large Load diff

71
vendor/janet/janetconf.h vendored Normal file
View file

@ -0,0 +1,71 @@
/* This will be generated by the build system if this file is not used */
#ifndef JANETCONF_H
#define JANETCONF_H
#define JANET_VERSION_MAJOR 1
#define JANET_VERSION_MINOR 41
#define JANET_VERSION_PATCH 2
#define JANET_VERSION_EXTRA ""
#define JANET_VERSION "1.41.2"
/* #define JANET_BUILD "local" */
/* These settings all affect linking, so use cautiously. */
/* #define JANET_SINGLE_THREADED */
/* #define JANET_THREAD_LOCAL _Thread_local */
/* #define JANET_NO_DYNAMIC_MODULES */
/* #define JANET_NO_NANBOX */
/* #define JANET_API __attribute__((visibility ("default"))) */
/* These settings should be specified before amalgamation is
* built. Any build with these set should be considered non-standard, and
* certain Janet libraries should be expected not to work. */
/* #define JANET_NO_DOCSTRINGS */
/* #define JANET_NO_SOURCEMAPS */
/* #define JANET_REDUCED_OS */
/* #define JANET_NO_PROCESSES */
/* #define JANET_NO_ASSEMBLER */
/* #define JANET_NO_PEG */
/* #define JANET_NO_NET */
/* #define JANET_NO_INT_TYPES */
/* #define JANET_NO_EV */
/* #define JANET_NO_FILEWATCH */
/* #define JANET_NO_REALPATH */
/* #define JANET_NO_SYMLINKS */
/* #define JANET_NO_UMASK */
/* #define JANET_NO_THREADS */
/* #define JANET_NO_FFI */
/* #define JANET_NO_FFI_JIT */
/* Other settings */
/* #define JANET_DEBUG */
/* #define JANET_PRF */
/* #define JANET_NO_UTC_MKTIME */
/* #define JANET_OUT_OF_MEMORY do { printf("janet out of memory\n"); exit(1); } while (0) */
/* #define JANET_EXIT(msg) do { printf("C assert failed executing janet: %s\n", msg); exit(1); } while (0) */
/* #define JANET_TOP_LEVEL_SIGNAL(msg) call_my_function((msg), stderr) */
/* #define JANET_RECURSION_GUARD 1024 */
/* #define JANET_MAX_PROTO_DEPTH 200 */
/* #define JANET_MAX_MACRO_EXPAND 200 */
/* #define JANET_STACK_MAX 16384 */
/* #define JANET_OS_NAME my-custom-os */
/* #define JANET_ARCH_NAME pdp-8 */
/* #define JANET_EV_NO_EPOLL */
/* #define JANET_EV_NO_KQUEUE */
/* #define JANET_NO_INTERPRETER_INTERRUPT */
/* #define JANET_NO_IPV6 */
/* #define JANET_NO_CRYPTORAND */
/* #define JANET_USE_STDATOMIC */
/* Custom vm allocator support */
/* #include <mimalloc.h> */
/* #define janet_malloc(X) mi_malloc((X)) */
/* #define janet_realloc(X, Y) mi_realloc((X), (Y)) */
/* #define janet_calloc(X, Y) mi_calloc((X), (Y)) */
/* #define janet_free(X) mi_free((X)) */
/* Main client settings, does not affect library code */
/* #define JANET_SIMPLE_GETLINE */
#endif /* end of include guard: JANETCONF_H */

2240
vendor/libarchive/CMakeLists.txt vendored Normal file
View file

File diff suppressed because it is too large Load diff

65
vendor/libarchive/COPYING vendored Normal file
View file

@ -0,0 +1,65 @@
The libarchive distribution as a whole is Copyright by Tim Kientzle
and is subject to the copyright notice reproduced at the bottom of
this file.
Each individual file in this distribution should have a clear
copyright/licensing statement at the beginning of the file. If any do
not, please let me know and I will rectify it. The following is
intended to summarize the copyright status of the individual files;
the actual statements in the files are controlling.
* Except as listed below, all C sources (including .c and .h files)
and documentation files are subject to the copyright notice reproduced
at the bottom of this file.
* The following source files are also subject in whole or in part to
a 3-clause UC Regents copyright; please read the individual source
files for details:
libarchive/archive_read_support_filter_compress.c
libarchive/archive_write_add_filter_compress.c
libarchive/mtree.5
* The following source files are in the public domain:
libarchive/archive_parse_date.c
* The following source files are triple-licensed with the ability to choose
from CC0 1.0 Universal, OpenSSL or Apache 2.0 licenses:
libarchive/archive_blake2.h
libarchive/archive_blake2_impl.h
libarchive/archive_blake2s_ref.c
libarchive/archive_blake2sp_ref.c
* The build files---including Makefiles, configure scripts,
and auxiliary scripts used as part of the compile process---have
widely varying licensing terms. Please check individual files before
distributing them to see if those restrictions apply to you.
I intend for all new source code to use the license below and hope over
time to replace code with other licenses with new implementations that
do use the license below. The varying licensing of the build scripts
seems to be an unavoidable mess.
Copyright (c) 2003-2018 <author(s)>
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer
in this position and unchanged.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

35
vendor/libarchive/INSTALL vendored Normal file
View file

@ -0,0 +1,35 @@
More complete build documentation is available on the libarchive
Wiki: https://github.com/libarchive/libarchive/wiki
On most Unix-like systems, you should be able to install libarchive,
bsdtar, and bsdcpio using the following common steps:
./configure
make
make install
If you need to customize the target directories or otherwise adjust
the build setting, use
./configure --help
to list the configure options.
If you are developing libarchive and need to update the
configure script and other build files:
/bin/sh build/autogen.sh
To create a distribution, please use the 'distcheck' target:
/bin/sh build/autogen.sh && ./configure && make distcheck
On Unix-like and non-Unix-like systems, use the "cmake" utility (available from
http://cmake.org/) to generate suitable build files for your platform.
Cmake requires the name of the directory containing CmakeLists.txt and
the "generator" to use for your build environment. For example, to
build with Xcode on Mac OS, you can use the following command:
cmake -G "Xcode" ~/libarchive-download-dir/
The result will be appropriate makefiles, solution files, or project
files that can be used with the corresponding development tool.
The default on Unix-like systems is to generate Makefiles, so you
can also use cmake instead of the configure script:
cmake ~/libarchive-download-dir/
make
make install
See the libarchive Wiki or the cmake site for further documentation.

1746
vendor/libarchive/Makefile.am vendored Normal file
View file

File diff suppressed because it is too large Load diff

19472
vendor/libarchive/Makefile.in vendored Normal file
View file

File diff suppressed because it is too large Load diff

789
vendor/libarchive/NEWS vendored Normal file
View file

@ -0,0 +1,789 @@
Jun 23, 2026: libarchive 3.8.8 released
Apr 13, 2026: libarchive 3.8.7 released
Mar 10, 2026: libarchive 3.8.6 released
Jan 05, 2026: libarchive 3.8.5 released
Dec 01, 2025: libarchive 3.8.4 released
Nov 17, 2025: libarchive 3.8.3 released
Oct 15, 2025: libarchive 3.8.2 released
Jun 01, 2025: libarchive 3.8.1 released
May 20, 2025: libarchive 3.8.0 released
Mar 30, 2025: libarchive 3.7.9 released
Mar 20, 2025: libarchive 3.7.8 released
Oct 13, 2024: libarchive 3.7.7 released
Sep 23, 2024: libarchive 3.7.6 released
Sep 13, 2024: libarchive 3.7.5 released
Apr 26, 2024: libarchive 3.7.4 released
Apr 08, 2024: libarchive 3.7.3 released
Sep 12, 2023: libarchive 3.7.2 released
Jul 29, 2023: libarchive 3.7.1 released
Jul 18, 2023: libarchive 3.7.0 released
Jul 14, 2023: bsdunzip port from FreeBSD
Dec 07, 2022: libarchive 3.6.2 released
Apr 08, 2022: libarchive 3.6.1 released
Feb 09, 2022: libarchive 3.6.0 released
Feb 08, 2022: libarchive 3.5.3 released
Aug 22, 2021: libarchive 3.5.2 released
Dec 26, 2020: libarchive 3.5.1 released
Dec 01, 2020: libarchive 3.5.0 released
Oct 14, 2020: Support for system extended attributes
May 20, 2020: libarchive 3.4.3 released
Apr 30, 2020: Support for pzstd compressed files
Apr 16, 2020: Support for RHT.security.selinux tar extended attribute
Feb 11, 2020: libarchive 3.4.2 released
Jan 23, 2020: Important fixes for writing XAR archives
Jan 20, 2020: New tar option: --safe-writes (atomical file extraction)
Jan 03, 2020: Support mbed TLS (PolarSSL) as optional crypto provider
Dec 30, 2019: libarchive 3.4.1 released
Dec 11, 2019: New pax write option "xattrhdr"
Nov 17, 2019: Unicode filename support for reading lha/lzh archives
Jun 11, 2019: libarchive 3.4.0 released
May 18, 2019: Fixes for reading Android APK and JAR archives
Apr 16, 2019: Support for non-recursive list and extract
Apr 14, 2019: New tar option: --exclude-vcs
Mar 27, 2019: Support for file and directory symlinks on Windows
Mar 12, 2019: Important fixes for storing file attributes and flags
Jan 20, 2019: Support for xz, lzma, ppmd8 and bzip2 decompression in ZIP files
Oct 06, 2018: RAR 5.0 reader
Sep 03, 2018: libarchive 3.3.3 released
Jul 19, 2018: Avoid super-linear slowdown on malformed mtree files
Jan 27, 2018: Many fixes for building with Visual Studio
Oct 19, 2017: NO_OVERWRITE doesn't change existing directory attributes
Aug 12, 2017: New support for Zstandard read and write filters
Jul 09, 2017: libarchive 3.3.2 released
Mar 16, 2017: NFSv4 ACL support for Linux (librichacl)
Feb 26, 2017: libarchive 3.3.1 released
Security & Feature release
Feb 19, 2017: libarchive 3.3.0 released
Security & Feature release
Jan 29, 2017: Limited NFSv4 ACL support for Mac OS (Darwin)
Jan 10, 2017: POSIX.1e and NFSv4 ACL support for Solaris and derivates
Dec 27, 2016: NFSv4 ACL read and write support for pax
Deprecated functions: archive_entry_acl_text(), archive_entry_acl_text_w()
Nov, 2016: libarchive is now being tested by the OSS-Fuzz project
Oct 26, 2016: Remove liblzmadec support
Oct 23, 2016: libarchive 3.2.2 released
Security release
Jun 20, 2016: libarchive 3.2.1 released
This fixes a handful of security and other critical issues with 3.2.0
May 01, 2016: libarchive 3.2.0 released
Apr 09, 2016: libarchive 3.1.901a released
Another test release in preparation for 3.2.0
Feb 13, 2016: libarchive 3.1.900a released
This is a test release in preparation for 3.2.0
Oct 21, 2015: Preliminary port to OSF
Apr 11, 2015: libarchive's issue tracker is now hosted at GitHub.
https://github.com/libarchive/libarchive/issues
Early 2015: Many fixes to crash and overflow bugs thanks to Hanno Boeck
Oct 13, 2014: Zip encryption and decryption support
Aug 13, 2014: Add support for lz4 compression.
Jun 10, 2014: Add warc format support
May 3, 2014: Add experimental Zip streaming extension
Apr 6, 2014: Add bsdcat command-line tool
Jan 12, 2014: Add Zip64 support
Dec 1, 2013: Rewrite Zip write logic
Jul 1, 2013: Add ability to detect encrypted entries for many formats
(This does not add the ability to *decrypt* those entries, however)
Feb 23, 2013: "raw" write support added
Feb 09, 2013: libarchive 3.1.2 released
Jan 28, 2013: libarchive's new website moved to http://www.libarchive.org.
Jan 13, 2013: libarchive 3.1.1 released
Jan 13, 2013: libarchive 3.1.0 released
Dec 07, 2012: Implement functions to manually set the format and filters used.
Nov 11, 2012: Add support for __MACOSX directory in Zip archives, which resource
forks are stored in.
Oct 20, 2012: Add support for writing v7 tar format.
Oct 09, 2012: Add support for grzip compression.
Oct 07, 2012: Introduce b64encode filter.
Oct 07, 2012: Introduce uuencode filter.
Oct 06, 2012: Add support for lzop.
Sep 27, 2012: Implement function used to seek within data blocks.
(Currently only supported for uncompressed RAR archives).
Apr 22, 2012: Add basic archive read and write filter support for lrzip.
Mar 27, 2012: libarchive 3.0.4 released
Feb 05, 2012: libarchive development now hosted at GitHub.
http://libarchive.github.com/
Feb 05, 2012: libarchive's issue tracker remains at Google Code.
http://code.google.com/p/libarchive/issues/list
Feb 05, 2012: libarchive's mailing lists remain at Google Groups.
Dec 24, 2011: libarchive 3.0.2 released
Dec 23, 2011: Various fixes merged from FreeBSD
Dec 23, 2011: Symlink support in Zip reader and writer
Dec 23, 2011: Robustness fixes to 7Zip reader
Nov 27, 2011: libarchive 3.0.1b released
Nov 26, 2011: 7Zip reader
Nov 26, 2011: Small fixes to ISO and Zip to improve robustness with corrupted input
Nov 24, 2011: Improve streaming Zip reader's support for uncompressed entries
Nov 20, 2011: New seeking Zip reader supports SFX Zip archives
Nov 20, 2011: Build fixes on Windows
Nov 13, 2011: libarchive 3.0.0a released
Nov 06, 2011: Update shared-library version calculations for libarchive 3.x
Sep 04, 2011: Fix tar -s; follow GNU tar for controlling hardlink/symlink substitutions
Aug 18, 2011: Fix reading ISO images built by NetBSD's mkisofs
Aug 15, 2011: Old archive_read_support_compression_XXX functions are deprecated and
will disappear in libarchive 4.0.
Jun 26, 2011: RAR reader
Jun 16, 2011: Add tar:compat-2x option to emulate broken libarchive 2.x
handling of pax UTF-8 headers
Apr 25, 2011: Refactor read_open() into a collection of single-item setters;
support the old interfaces as wrappers
Apr 12, 2011: Split disk writer into separate POSIX and Windows implementations
Apr 10, 2011: Improvements to character translations on Windows.
Mar 30, 2011: More work to return errors instead of calling abort()
Mar 23, 2011: Add charset option to many writers to control MBCS filenames
Mar 17, 2011: Overhauled support for per-format extension options
Mar 17, 2011: Track character set used for mbcs strings, support
translating to/from user-specified locale
Mar 09, 2011: Recognize mtree files without requiring a signature
Mar 06, 2011: Use iconv to convert to/from Unicode instead of making bad
assumptions about the C90 character set translation functions
Feb 17, 2011: Fixes for AIX, TRU64, and other platforms
Dec 22, 2010: CAB reader
Dec 20, 2010: LHA/LZH reader
Jul 03, 2010: minitar example demonstrates archive_read_disk directory traversal
Jun 29, 2010: Many improvements to ISO reader compatibility
Jun 26, 2010: Use larger buffers when copy files into an archive
Jun 18, 2010: Reimplement Mac OS extensions in libarchive
Jun 09, 2010: archive_read_disk now supports traversals
May 28, 2010: XAR writer
May 16, 2010: Fix ^T handling; don't exit on interrupted reads and writes
May 09, 2010: Improved detection of platform-specific crypto support
May 04, 2010: lzip read and write filters
May 01, 2010: New options: tar --gid --gname --uid --uname
Apr 28, 2010: Use Red-black tree for ISO reader/writer to improve performance
Apr 17, 2010: Minimal writer for legacy GNU tar format
Mar 12, 2010: Don't dereference symlinks on Linux when reading ACLs.
Mar 06, 2010: Fix build when an older libarchive is already installed
Feb 28, 2010: Relax handling of state failures; misuse by clients now generally
results in a sticky ARCHIVE_FATAL rather than a visit to abort()
Feb 25, 2010: ISO writer
Feb 21, 2010: Split many man pages into smaller chunks.
Feb 21, 2010: Performance: Cheat on block sizes when reading archives from disk.
Feb 21, 2010: Use int64_t instead of off_t, dev_t, ino_t, uid_t, and gid_t
Feb 20, 2010: Document new ACL functions.
Feb 19, 2010: Support multiple write filters
Feb 07, 2010: Remove some legacy libarchive 1.x APIs
Feb 04, 2010: Read afio headers
Feb 02, 2010: Archive sparse files compatibly with GNU tar
Feb 01, 2010: Integrate Apple extensions for Mac OS extended attributes into bsdtar
Jan 31, 2010: Support cpio -V
Feb 04, 2010: libarchive 2.8.0 released
Jan 17, 2010: Fix error handling for 'echo nonexistent | cpio -o'
Jan 17, 2010: Don't use futimes() on Cygwin
Jan 02, 2010: libarchive 2.7.902a released (test release for 2.8)
Jan 02, 2010: Fix tar/test/test_windows on MinGW
Jan 02, 2010: Fix memory leaks in libarchive tests
Jan 01, 2010: Fix memory leak when filter startup fails
Dec 27, 2009: libarchive 2.7.901a released (test release for 2.8)
Aug 04, 2009: libarchive 2.7.1 released
Jul 20, 2009: Suppress bogus warning about unxz
Jul 19, 2009: Support Cygwin 1.7
Jun 11, 2009: Support lzma/xz files compressed with larger buffer sizes.
May 24, 2009: Handle gzip files signed with OpenBSD "gzsig" program.
May 07, 2009: Avoid false failures when reading from pipe.
Apr 16, 2009: libarchive 2.7.0 released
Apr 10, 2009: libarchive 2.6.992a released
Apr 09, 2009: Fix SIGPIPE issue building with MSVC.
Apr 09, 2009: Fix several minor memory leaks in libarchive and libarchive_test
Apr 08, 2009: libarchive 2.6.991a released
Apr 07, 2009: Additional tests added to bsdcpio_test
Apr 01, 2009: libarchive 2.6.990a released
Apr 01, 2009: Use command-line gunzip, bunzip2, unxz, unlzma for
decompression if the library is built without suitable
libraries. The setup functions return ARCHIVE_WARN
in this case so clients can adapt if necessary.
Apr 01, 2009: Use getpw*_r and getgr*_r functions for thread-safety.
Mar 24, 2009: Add archive_read_next_header2(), which is up to 25%
more efficient for some clients; from Brian Harring.
Mar 22, 2009: PDF versions of manpages are now included in the distribution.
Mar, 2009: Major work to improve Cygwin build by Charles Wilson.
Feb/Mar, 2009: Major work on cmake build support, mostly by Michihiro NAKAJIMA.
Feb/Mar, 2009: Major work on Visual Studio support by Michihiro NAKAJIMA.
All tests now pass.
Feb 25, 2009: Fix Debian Bug #516577
Feb 21, 2009: Yacc is no longer needed to build; date parser rewritten in C.
Jan/Feb, 2009: Mtree work by Michihiro.
Feb, 2009: Joliet support by Andreas Henriksson.
Jan/Feb, 2009: New options framework by Michihiro.
Feb, 2009: High-res timestamps on Tru64, AIX, and GNU Hurd, by Björn Jacke.
Jan 18, 2009: Extended attributes work on FreeBSD and Linux now with pax format.
Jan 07, 2009: New archive_read_disk_entry_from_file() knows about ACLs,
extended attributes, etc so that bsdtar and bsdcpio don't require
such system-specific knowledge.
Jan 03, 2009: Read filter system extensively refactored. In particular,
read filter pipelines are now built out automatically and individual
filters should be much easier to implement. Documentation on the
Googlecode Wiki explains how to implement new filters.
Dec 28, 2008: Many Windows/Visual Studio fixes from Michihiro NAKAJIMA.
Dec 28, 2008: Main libarchive development moved from FreeBSD Perforce
server to Google Code. This should make it easier for more
people to participate in libarchive development.
Dec 28, 2008: libarchive 2.6.0 released
Dec 25, 2008: libarchive 2.5.905a released
Dec 10, 2008: libarchive 2.5.904a released
Dec 04, 2008: libarchive 2.5.903a released
Nov 09, 2008: libarchive 2.5.902a released
Nov 08, 2008: libarchive 2.5.901a released
Nov 08, 2008: Start of pre-release testing for libarchive 2.6
Nov 07, 2008: Read filter refactor: The decompression routines just
consume and produce arbitrarily-sized blocks. The reblocking
from read_support_compression_none() has been pulled into the
read core. Also, the decompression bid now makes multiple
passes and stacks read filters.
Oct 21, 2008: bsdcpio: New command-line parser.
Oct 19, 2008: Internal read_ahead change: short reads are now an error
Oct 06, 2008: bsdtar: option parser no longer uses getopt_long(),
gives consistent option parsing on all platforms.
Sep 19, 2008: Jaakko Heinonen: shar utility built on libarchive
Sep 17, 2008: Pedro Giffuni: birthtime support
Sep 17, 2008: Miklos Vajna: lzma reader and test. Note: I still have
some concerns about the auto-detection (LZMA file format
doesn't support auto-detection well), so this is not yet
enabled under archive_read_support_compression_all(). For
now, you must call archive_read_support_compression_lzma() if
you want LZMA read support.
Sep 11, 2008: Ivailo Petrov: Many fixes to Windows build, new solution files
Jul 26, 2008: archive_entry now tracks which values have not been set.
This helps zip extraction (file size is often "unknown") and
time restores (tar usually doesn't know atime).
Jul 26, 2008: Joerg Sonnenberger: Performance improvements to shar writer
Jul 25, 2008: Joerg Sonnenberger: mtree write support
Jul 02, 2008: libarchive 2.5.5 released
Jul 02, 2008: libarchive 2.5.5b released
Jul 01, 2008: bsdcpio is being used by enough people, we can call it 1.0.0 now
Jun 20, 2008: bsdcpio: If a -l link fails with EXDEV, copy the file instead
Jun 19, 2008: bsdcpio: additional long options for better GNU cpio compat
Jun 15, 2008: Many small portability and bugfixes since 2.5.4b.
May 25, 2008: libarchive 2.5.4b released
May 21, 2008: Joerg Sonnenberger: fix bsdtar hardlink handling for newc format
May 21, 2008: More progress on Windows building. Thanks to "Scott"
for the Windows makefiles, thanks to Kees Zeelenberg for
code contributions.
May 21, 2008: Fix a number of non-exploitable integer and buffer overflows,
thanks to David Remahl at Apple for pointing these out.
May 21, 2008: Colin Percival: SIGINFO or SIGUSR1 to bsdtar prints progress info
May 16, 2008: bsdtar's test harness no longer depends on file ordering.
This was causing spurious test failures on a lot of systems.
Thanks to Bernhard R. Link for the diagnosis.
May 14, 2008: Joerg Sonnenberger: -s substitution support for bsdtar
May 13, 2008: Joerg Sonnenberger: Many mtree improvements
May 11, 2008: Joerg Sonnenberger: fix hardlink extraction when
hardlinks have different permissions from original file
April 30, 2008: Primary libarchive work has been moved into the FreeBSD
project's Perforce repository: http://perforce.freebsd.org/
The libarchive project can be browsed at
//depot/user/kientzle/libarchive-portable
Direct link: http://preview.tinyurl.com/46mdgr
May 04, 2008: libarchive 2.5.3b released
* libarchive: Several fixes to link resolver to address bsdcpio crashes
* bsdcpio: -p hardlink handling fixes
* tar/pax: Ensure ustar dirnames end in '/'; be more careful about
measuring filenames when deciding what pathname fields to use
* libarchive: Mark which entry strings are set; be accurate about
distinguishing empty strings ("") from unset ones (NULL)
* tar: Don't crash reading entries with empty filenames
* libarchive_test, bsdtar_test, bsdcpio_test: Better defaults:
run all tests, delete temp dirs, summarize repeated failures
* -no-undefined to libtool for Cygwin
* libarchive_test: Skip large file tests on systems with 32-bit off_t
* iso9660: Don't bother trying to find the body of an empty file;
this works around strange behavior from some ISO9660 writers
* tar: allow -r -T to be used together
* tar: allow --format with -r or -u
* libarchive: Don't build archive.h
May 04, 2008: Simplified building: archive.h is no longer constructed
This may require additional #if conditionals on some platforms.
Mar 30, 2008: libarchive 2.5.1b released
Mar 15, 2008: libarchive 2.5.0b released
Mar 15, 2008: bsdcpio now seems to correctly write hardlinks into newc,
ustar, and old cpio archives. Just a little more testing before
bsdcpio 1.0 becomes a reality.
Mar 15, 2008: I think the new linkify() interface is finally handling
all known hardlink strategies.
Mar 15, 2008: Mtree read fixes from Joerg Sonnenberger.
Mar 15, 2008: Many new bsdtar and bsdcpio options from Joerg Sonnenberger.
Mar 15, 2008: test harnesses no longer require uudecode; they
now have built-in decoding logic that decodes the reference
files as they are needed.
Mar 14, 2008: libarchive 2.4.14 released; identical to 2.4.13 except for
a point fix for gname/uname mixup in pax format that was introduced
with the UTF-8 fixes.
Feb 26, 2008: libarchive 2.4.13 released
Feb 25, 2008: Handle path, linkname, gname, or uname that can't be converted
to/from UTF-8. Implement "hdrcharset" attribute from SUS-2008.
Feb 25, 2008: Fix name clash on NetBSD.
Feb 18, 2008: Fix writing empty 'ar' archives, per Kai Wang
Feb 18, 2008: [bsdtar] Permit appending on block devices.
Feb 09, 2008: New "linkify" resolver to help with newc hardlink writing;
bsdcpio still needs to be converted to use this.
Feb 02, 2008: Windows compatibility fixes from Ivailo Petrov, Kees Zeelenberg
Jan 30, 2008: Ignore hardlink size for non-POSIX tar archives.
Jan 22, 2008: libarchive 2.4.12 released
Jan 22, 2008: Fix bad padding when writing symlinks to newc cpio archives.
Jan 22, 2008: Verify bsdcpio_test by getting it to work against GNU cpio 2.9.
bsdcpio_test complains about missing options (-y and -z), format
of informational messages (--version, --help), and a minor formatting
issue in odc format output. After this update, bsdcpio_test uncovered
several more cosmetic issues in bsdcpio, all now fixed.
Jan 22, 2008: Experimental support for self-extracting Zip archives.
Jan 22, 2008: Extend hardlink restore strategy to work correctly with
hardlinks extracted from newc cpio files. (Which store the body
only with the last occurrence of a link.)
Dec 30, 2007: libarchive 2.4.11 released
Dec 30, 2007: Fixed a compile error in bsdcpio on some systems.
Dec 29, 2007: libarchive 2.4.10 released
Dec 29, 2007: bsdcpio 0.9.0 is ready for wider use.
Dec 29, 2007: Completed initial test harness for bsdcpio.
Dec 22, 2007: libarchive 2.4.9 released
Dec 22, 2007: Implement the remaining options for bsdcpio: -a, -q, -L, -f,
pattern selection for -i and -it.
Dec 13, 2007: libarchive 2.4.8 released
Dec 13, 2007: gzip and bzip2 compression now handle zero-byte writes correctly,
Thanks to Damien Golding for bringing this to my attention.
Dec 12, 2007: libarchive 2.4.7 released
Dec 10, 2007: libarchive 2.4.6 released
Dec 09, 2007: tar/test/test_copy.c verifies "tar -c | tar -x" copy pipeline
Dec 07, 2007: Fix a couple of minor memory leaks.
Dec 04, 2007: libarchive 2.4.5 released
Dec 04, 2007: Fix cpio/test/test_write_odc by setting the umask first.
Dec 03, 2007: libarchive 2.4.4 released
Dec 03, 2007: New configure options --disable-xattr and --disable-acl,
thanks to Samuli Suominen.
Dec 03, 2007: libarchive 2.4.3 released
Dec 03, 2007: Thanks to Lapo Luchini for sending me a ZIP file that
libarchive couldn't handle. Fixed a bug in handling of
"length at end" flags in ZIP files.
Dec 03, 2007: Fixed bsdcpio -help, bsdtar -help tests.
Dec 02, 2007: First cut at real bsdtar test harness.
Dec 02, 2007: libarchive 2.4.2 released
Dec 02, 2007: libarchive 2.4.1 released
Dec 02, 2007: Minor fixes, rough cut of mdoc-to-man conversion for
man pages.
Oct 30, 2007: libarchive 2.4.0 released
Oct 30, 2007: Minor compile fix thanks to Joerg Schilling.
Oct 30, 2007: Only run the format auction once at the beginning of the
archive. This is simpler and supports better error recovery.
Oct 29, 2007: Test support for very large entries in tar archives:
libarchive_test now exercises entries from 2GB up to 1TB.
Oct 27, 2007: libarchive 2.3.5 released
Oct 27, 2007: Correct some unnecessary internal data copying in the
"compression none" reader and writer; this reduces user time
by up to 2/3 in some tests. (Thanks to Jan Psota for
publishing his performance test results to GNU tar's bug-tar
mailing list; those results pointed me towards this problem.)
Oct 27, 2007: Fix for skipping archive entries that are exactly
a multiple of 4G on 32-bit platforms.
Oct 25, 2007: Fix for reading very large (>8G) tar archives; this was
broken when I put in support for new GNU tar sparse formats.
Oct 20, 2007: Initial work on new pattern-matching code for cpio; I
hope this eventually replaces the code currently in bsdtar.
Oct 08, 2007: libarchive 2.3.4 released
Oct 05, 2007: Continuing work on bsdcpio test suite.
Oct 05, 2007: New cpio.5 manpage, updates to "History" of bsdcpio.1 and
bsdtar.1 manpages.
Oct 05, 2007: Fix zip reader to immediately return EOF if you try
to read body of non-regular file. In particular, this fixes
bsdtar extraction of zip archives.
Sep 30, 2007: libarchive 2.3.3 released
Sep 26, 2007: Rework Makefile.am so that the enable/disable options
actually do the right things.
Sep 26, 2007: cpio-odc and cpio-newc archives no longer write bodies
for non-regular files.
Sep 26, 2007: Test harness for bsdcpio is in place, needs more tests written.
This is much nicer than the ragtag collection of test scripts
that bsdtar has.
Sep 20, 2007: libarchive 2.3.2 released
Sep 20, 2007: libarchive 2.3.1 broke bsdtar because the archive_write_data()
fix was implemented incorrectly.
Sep 16, 2007: libarchive 2.3.1 released
Sep 16, 2007: Many fixes to bsdcpio 0.3: handle hardlinks with -p, recognize
block size on writing, fix a couple of segfaults.
Sep 16, 2007: Fixed return value from archive_write_data() when used
with archive_write_disk() to match the documentation and other
instances of this same function.
Sep 15, 2007: Add archive_entry_link_resolver, archive_entry_strmode
Sep 11, 2007: libarchive 2.2.8 released
Sep 09, 2007: bsdcpio 0.2 supports most (not yet all) of the old POSIX spec.
Sep 01, 2007: libarchive 2.2.7 released
Aug 31, 2007: Support for reading mtree files, including an mtree.5 manpage
(A little experimental still.)
Aug 18, 2007: Read gtar 1.17 --posix --sparse entries.
Aug 13, 2007: Refined suid/sgid restore handling; it is no longer
an error if suid/sgid bits are dropped when you request
perm restore but don't request owner restore.
Aug 06, 2007: Use --enable-bsdcpio if you want to try bsdcpio
Aug 05, 2007: libarchive 2.2.6 released
Aug 05, 2007: New configure option --disable-bsdtar, thanks to Joerg
Sonnenberger.
Aug 05, 2007: Several bug fixes from FreeBSD CVS repo.
Jul 13, 2007: libarchive 2.2.5 released
Jul 12, 2007: libarchive 2.2.4 released
Jul 12, 2007: Thanks to Colin Percival's help in diagnosing and
fixing several critical security bugs. Details available at
http://security.freebsd.org/advisories/FreeBSD-SA-07:05.libarchive.asc
May 26, 2007: libarchive 2.2.3 released
May 26, 2007: Fix memory leaks in ZIP reader and shar writer, add some
missing system headers to archive_entry.h, dead code cleanup
from Colin Percival, more tests for gzip/bzip2, fix an
EOF anomaly in bzip2 decompression.
May 12, 2007: libarchive 2.2.2 released
May 12, 2007: Fix archive_write_disk permission restore by cloning
entry passed into write_header so that permission info is
still available at finish_entry time. (archive_read_extract()
worked okay because it held onto the passed-in entry, but
direct consumers of archive_write_disk would break). This
required fixing archive_entry_clone(), which now works and has
a reasonably complete test case.
May 10, 2007: Skeletal cpio implementation.
May 06, 2007: libarchive 2.2.1 released
May 06, 2007: Flesh out a lot more of test_entry.c so as to catch
problems such as the device node breakage before releasing <sigh>.
May 05, 2007: Fix a bad bug introduced in 2.1.9 that broke device
node entries in tar archives.
May 03, 2007: Move 'struct stat' out of archive_entry core as well.
This removes some portability headaches and fixes a bunch
of corner cases that arise when manipulating archives on
dissimilar systems.
Apr 30, 2007: libarchive 2.1.10 released
Apr 31, 2007: Minor code cleanup.
Apr 24, 2007: libarchive 2.1.9 released
Apr 24, 2007: Fix some recently-introduced problems with libraries
(Just let automake handle it and it all works much better.)
Finish isolating major()/minor()/makedev() in archive_entry.c.
Apr 23, 2007: libarchive 2.1.8 released
Apr 23, 2007: Minor fixes found from building on MacOS X
Apr 22, 2007: libarchive 2.1.7 released
Apr 22, 2007: Eliminated all uses of 'struct stat' from the
format readers/writers. This should improve portability;
'struct stat' is now only used in archive_entry and in
code that actually touches the disk.
Apr 17, 2007: libarchive 2.1.6 released
Libarchive now compiles and passes all tests on Interix.
Apr 16, 2007: libarchive 2.1.5 released
Apr 15, 2007: libarchive 2.1b2 released
Apr 15, 2007: New libarchive_internals.3 documentation of internal APIs.
Not complete, but should prove helpful.
Apr 15, 2007: Experimental "read_compress_program" and "write_compress_program"
for using libarchive with external compression. Not yet
well tested, and likely has portability issues. Feedback
appreciated.
Apr 14, 2007: libarchive 2.0.31 released
Apr 14, 2007: More fixes for Interix, more 'ar' work
Apr 14, 2007: libarchive 2.0.30 released
Apr 13, 2007: libarchive now enforces trailing '/' on dirs
written to tar archives
Apr 11, 2007: libarchive 2.0.29 released
Apr 11, 2007: Make it easier to statically configure for different platforms.
Apr 11, 2007: Updated config.guess, config.sub, libtool
Apr 06, 2007: libarchive 2.0.28 released
Apr 06, 2007: 'ar' format read/write support thanks to Kai Wang.
Apr 01, 2007: libarchive 2.0.27 released
Mar 31, 2007: Several minor fixes from Colin Percival and Joerg Sonnenberger.
Mar 12, 2007: libarchive 2.0.25 released
Mar 12, 2007: Fix broken --unlink flag.
Mar 11, 2007: libarchive 2.0.24 released
Mar 10, 2007: Correct an ACL blunder that causes any ACL with an entry
that refers to a non-existent user or group to not be restored correctly.
The fix both makes the parser more tolerant (so that archives created
with the buggy ACLs can be read now) and corrects the ACL formatter.
Mar 10, 2007: More work on test portability to Linux.
Mar 10, 2007: libarchive 2.0.22 released
Mar 10, 2007: Header cleanups; added linux/fs.h, removed
some unnecessary headers, added #include guards in bsdtar.
If you see any obvious compile failures from this, let me know.
Mar 10, 2007: Work on bsdtar test scripts: not yet robust enough
to enable as part of "make check", but getting better.
Mar 10, 2007: libarchive now returns ARCHIVE_FAILED when
a header write fails in a way that only affects this item.
Less bad than ARCHIVE_FATAL, but worse than ARCHIVE_WARN.
Mar 07, 2007: libarchive 2.0.21 released
Mar 07, 2007: Add some ACL tests (only for the system-independent
portion of the ACL support for now).
Mar 07, 2007: tar's ability to read ACLs off disk got
turned off for FreeBSD; re-enable it. (ACL restores and
libarchive support for storing/reading ACLs from pax
archives was unaffected.)
Mar 02, 2007: libarchive 2.0.20 released
Mar 2, 2007: It's not perfect, but it's pretty good.
Libarchive 2.0 is officially out of beta.
Feb 28, 2007: libarchive 2.0b17 released
Feb 27, 2007: Make the GID restore checks more robust by checking
whether the current user has too few or too many privileges.
Feb 26, 2007: libarchive 2.0b15 released
Feb 26, 2007: Don't lose symlinks when extracting from ISOs.
Thanks to Diego "Flameeyes" Pettenò for telling me about the
broken testcase on Gentoo that (finally!) led me to the cause
of this long-standing bug.
Feb 26, 2007: libarchive 2.0b14 released
Feb 26, 2007: Fix a broken test on platforms that lack lchmod().
Feb 25, 2007: libarchive 2.0b13 released
Feb 25, 2007: Empty archives were being written as empty files,
without a proper end-of-archive marker. Fixed.
Feb 23, 2007: libarchive 2.0b12 released
Feb 22, 2007: Basic security checks added: _EXTRACT_SECURE_NODOTDOT
and _EXTRACT_SECURE_SYMLINK. These checks used to be in bsdtar,
but they belong down in libarchive where they can be used by
other tools and where they can be better optimized.
Feb 11, 2007: libarchive 2.0b11 released
Feb 10, 2007: Fixed a bunch of errors in libarchive's handling
of EXTRACT_PERM and EXTRACT_OWNER, especially relating
to SUID and SGID bits.
Jan 31, 2007: libarchive 2.0b9 released
Jan 31, 2007: Added read support for "empty" archives as a
distinct archive format. Bsdtar uses this to handle, e.g.,
"touch foo.tar; tar -rf foo.tar"
Jan 22, 2007: libarchive 2.0b6 released
Jan 22, 2007: archive_write_disk API is now in place. It provides
a finer-grained interface than archive_read_extract. In particular,
you can use it to create objects on disk without having an archive
around (just feed it archive_entry objects describing what you
want to create), you can override the uname/gname-to-uid/gid lookups
(minitar uses this to avoid getpwXXX() and getgrXXX() bloat).
Jan 09, 2007: libarchive 2.0a3 released
Jan 9, 2007: archive_extract is now much better; it handles the
most common cases with a minimal number of system calls.
Some features still need a lot of testing, especially corner
cases involving objects that already exist on disk. I expect
the next round of API overhaul will simplify building test cases.
Jan 9, 2007: a number of fixes thanks to Colin Percival, especially
corrections to the skip() framework and handling of large files.
Jan 9, 2007: Fixes for large ISOs. The code should correctly handle
very large ISOs with entries up to 4G. Thanks to Robert Sciuk
for pointing out these issues.
Sep 05, 2006: libarchive 1.3.1 released
Sep 5, 2006: Bump version to 1.3 for new I/O wrappers.
Sep 4, 2006: New memory and FILE read/write wrappers.
Sep 4, 2006: libarchive test harness is now minimally functional;
it's located a few minor bugs in error-handling logic
Aug 17, 2006: libarchive 1.2.54 released
Aug 17, 2006: Outline ABI changes for libarchive 2.0; these
are protected behind #ifdef's until I think I've found everything
that needs to change.
Aug 17, 2006: Fix error-handling in archive_read/write_close()
They weren't returning any errors before.
Aug 17, 2006: Fix recursive-add logic to not trigger if it's not set
Fixes a bug adding files when writing archive to pipe or when
using archive_write_open() directly.
Jul 2006: New "skip" handling improves performance extracting
single files from large uncompressed archives.
Mar 21, 2006: 1.2.52 released
Mar 21, 2006: Fix -p on platforms that don't have platform-specific
extended attribute code.
Mar 20, 2006: Add NEWS file; fill in some older history from other
files. I'll try to keep this file up-to-date from now on.
OLDER NEWS SUMMARIES
Mar 19, 2006: libarchive 1.2.51 released
Mar 18, 2006: Many fixes to extended attribute support, including a redesign
of the storage format to simplify debugging.
Mar 12, 2006: Remove 'tp' support; it was a fun idea, but not worth
spending much time on.
Mar 11, 2006: Incorporated Jaakko Heinonen's still-experimental support
for extended attributes (Currently Linux-only.).
Mar 11, 2006: Reorganized distribution package: There is now one tar.gz
file that builds both libarchive and bsdtar.
Feb 13, 2006: Minor bug fixes: correctly read cpio device entries, write
Pax attribute entry names.
Nov 7, 2005: Experimental 'tp' format support in libarchive. Feedback
appreciated; this is not enabled by archive_read_support_format_all()
yet as I'm not quite content with the format detection heuristics.
Nov 7, 2005: Some more portability improvements thanks to Darin Broady,
minor bugfixes.
Oct 12, 2005: Use GNU libtool to build shared libraries on many systems.
Aug 9, 2005: Correctly detect that MacOS X does not have POSIX ACLs.
Apr 17, 2005: Kees Zeelenberg has ported libarchive and bsdtar to Windows:
http://gnuwin32.sourceforge.net/
Apr 11, 2005: Extended Zip/Zip64 support thanks to Dan Nelson. -L/-h
fix from Jaakko Heinonen.
Mar 12, 2005: archive_read_extract can now handle very long
pathnames (I've tested with pathnames up to 1MB).
Mar 12, 2005: Marcus Geiger has written an article about libarchive
http://xsnil.antbear.org/2005/02/05/archive-mit-libarchive-verarbeiten/
including examples of using it from Objective-C. His MoinX
http://moinx.antbear.org/ desktop Wiki uses
libarchive for archiving and restoring Wiki pages.
Jan 22, 2005: Preliminary ZIP extraction support,
new directory-walking code for bsdtar.
Jan 16, 2005: ISO9660 extraction code added; manpage corrections.
May 22, 2004: Many gtar-compatible long options have been added; almost
all FreeBSD ports extract correctly with bsdtar.
May 18, 2004: bsdtar can read Solaris, HP-UX, Unixware, star, gtar,
and pdtar archives.

250
vendor/libarchive/README.md vendored Normal file
View file

@ -0,0 +1,250 @@
# Welcome to libarchive!
The libarchive project develops a portable, efficient C library that
can read and write streaming archives in a variety of formats. It
also includes implementations of the common `tar`, `cpio`, and `zcat`
command-line tools that use the libarchive library.
## Questions? Issues?
* https://www.libarchive.org is the home for ongoing
libarchive development, including documentation,
and links to the libarchive mailing lists.
* To report an issue, use the issue tracker at
https://github.com/libarchive/libarchive/issues
* To submit an enhancement to libarchive, please
submit a pull request via GitHub: https://github.com/libarchive/libarchive/pulls
## Contents of the Distribution
This distribution bundle includes the following major components:
* **libarchive**: a library for reading and writing streaming archives
* **tar**: the 'bsdtar' program is a full-featured 'tar' implementation built on libarchive
* **cpio**: the 'bsdcpio' program is a different interface to essentially the same functionality
* **cat**: the 'bsdcat' program is a simple replacement tool for zcat, bzcat, xzcat, and such
* **unzip**: the 'bsdunzip' program is a simple replacement tool for Info-ZIP's unzip
* **examples**: Some small example programs that you may find useful.
* **examples/minitar**: a compact sample demonstrating use of libarchive.
* **contrib**: Various items sent to me by third parties; please contact the authors with any questions.
The top-level directory contains the following information files:
* **NEWS** - highlights of recent changes
* **COPYING** - what you can do with this
* **INSTALL** - installation instructions
* **README** - this file
* **CMakeLists.txt** - input for "cmake" build tool, see INSTALL
* **configure** - configuration script, see INSTALL for details. If your copy of the source lacks a `configure` script, you can try to construct it by running the script in `build/autogen.sh` (or use `cmake`).
The following files in the top-level directory are related to the 'configure' script and are only needed by maintainers:
* `configure.ac` - used (by autoconf) to build the configure script and related files
* `Makefile.am` - used (by automake) to generate Makefile.in
* `aclocal.m4` - auto-generated file (created by aclocal) used to build the configure script
* `Makefile.in` - auto-generated template (created by automake) used by the configure script to create Makefile
* `config.h.in` - auto-generated template (created by autoheader) used by the configure script to create config.h
## Documentation
In addition to the informational articles and documentation
in the online [libarchive Wiki](https://github.com/libarchive/libarchive/wiki),
the distribution also includes a number of manual pages:
* bsdtar.1 explains the use of the bsdtar program
* bsdcpio.1 explains the use of the bsdcpio program
* bsdcat.1 explains the use of the bsdcat program
* libarchive.3 gives an overview of the library as a whole
* archive_read.3, archive_write.3, archive_write_disk.3, and
archive_read_disk.3 provide detailed calling sequences for the read
and write APIs
* archive_entry.3 details the "struct archive_entry" utility class
* archive_internals.3 provides some insight into libarchive's
internal structure and operation.
* libarchive-formats.5 documents the file formats supported by the library
* cpio.5, mtree.5, and tar.5 provide detailed information about these
popular archive formats, including hard-to-find details about
modern cpio and tar variants.
The manual pages above are provided in the 'doc' directory in
a number of different formats.
You should also read the copious comments in `archive.h` and the
source code for the sample programs for more details. Please let us
know about any errors or omissions you find.
## Supported Formats
Currently, the library automatically detects and reads the following formats:
* Old V7 tar archives
* POSIX ustar
* GNU tar format (including GNU long filenames, long link names, and sparse files)
* Solaris 9 extended tar format (including ACLs)
* POSIX pax interchange format
* POSIX octet-oriented cpio
* SVR4 ASCII cpio
* Binary cpio (big-endian or little-endian)
* PWB binary cpio
* ISO9660 CD-ROM images (with optional Rockridge or Joliet extensions)
* ZIP archives (with uncompressed or "deflate" compressed entries, including support for encrypted Zip archives)
* ZIPX archives (with support for bzip2, zstd, ppmd8, lzma and xz compressed entries)
* GNU and BSD 'ar' archives
* 'mtree' format
* 7-Zip archives (including archives that use zstandard compression)
* Microsoft CAB format
* LHA and LZH archives
* RAR and RAR 5.0 archives (with some limitations due to RAR's proprietary status)
* WARC archives
* XAR archives
The library also detects and handles any of the following before evaluating the archive:
* uuencoded files
* files with RPM wrapper
* gzip compression
* bzip2 compression
* compress/LZW compression
* lzma, lzip, and xz compression
* lz4 compression
* lzop compression
* zstandard compression
The library can create archives in any of the following formats:
* POSIX ustar
* POSIX pax interchange format
* "restricted" pax format, which will create ustar archives except for
entries that require pax extensions (for long filenames, ACLs, etc).
* Old GNU tar format
* Old V7 tar format
* POSIX octet-oriented cpio
* SVR4 "newc" cpio
* Binary cpio (little-endian)
* PWB binary cpio
* shar archives
* ZIP archives (with uncompressed or "deflate" compressed entries)
* ZIPX archives (with bzip2, zstd, lzma or xz compressed entries)
* GNU and BSD 'ar' archives
* 'mtree' format
* ISO9660 format
* 7-Zip archives (including archives that use zstandard compression)
* WARC archives
* XAR archives
When creating archives, the result can be filtered with any of the following:
* uuencode
* base64
* gzip compression
* bzip2 compression
* compress/LZW compression
* lzma, lzip, and xz compression
* lz4 compression
* lzop compression
* zstandard compression
## Notes about the Library Design
The following notes address many of the most common
questions we are asked about libarchive:
* This is a heavily stream-oriented system. That means that
it is optimized to read or write the archive in a single
pass from beginning to end. For example, this allows
libarchive to process archives too large to store on disk
by processing them on-the-fly as they are read from or
written to a network or tape drive. This also makes
libarchive useful for tools that need to produce
archives on-the-fly (such as webservers that provide
archived contents of a users account).
* In-place modification and random access to the contents
of an archive are not directly supported. For some formats,
this is not an issue: For example, tar.gz archives are not
designed for random access. In some other cases, libarchive
can re-open an archive and scan it from the beginning quickly
enough to provide the needed abilities even without true
random access. Of course, some applications do require true
random access; those applications should consider alternatives
to libarchive.
* The library is designed to be extended with new compression and
archive formats. The only requirement is that the format be
readable or writable as a stream and that each archive entry be
independent. There are articles on the libarchive Wiki explaining
how to extend libarchive.
* On read, compression and format are always detected automatically.
* The same API is used for all formats; it should be very
easy for software using libarchive to transparently handle
any of libarchive's archiving formats.
* Libarchive's automatic support for decompression can be used
without archiving by explicitly selecting the "raw" and "empty"
formats.
* I've attempted to minimize static link pollution. If you don't
explicitly invoke a particular feature (such as support for a
particular compression or format), it won't get pulled in to
statically-linked programs. In particular, if you don't explicitly
enable a particular compression or decompression support, you won't
need to link against the corresponding compression or decompression
libraries. This also reduces the size of statically-linked
binaries in environments where that matters.
* The library is generally _thread-safe_ depending on the platform:
it does not define any global variables of its own. However, some
platforms do not provide fully thread-safe versions of key C library
functions. On those platforms, libarchive will use the non-thread-safe
functions. Patches to improve this are of great interest to us.
* The function `archive_write_disk_header()` is _not_ thread safe on
POSIX machines and could lead to security issue resulting in world
writeable directories. Thus it must be mutexed by the calling code.
This is due to calling `umask(oldumask = umask(0))`, which sets the
umask for the whole process to 0 for a short time frame.
In case other thread calls the same function in parallel, it might
get interrupted by it and cause the executable to use umask=0 for the
remaining execution.
This will then lead to implicitly created directories to have 777
permissions without sticky bit.
* In particular, libarchive's modules to read or write a directory
tree do use `chdir()` to optimize the directory traversals. This
can cause problems for programs that expect to do disk access from
multiple threads. Of course, those modules are completely
optional and you can use the rest of libarchive without them.
* The library is _not_ thread-aware, however. It does no locking
or thread management of any kind. If you create a libarchive
object and need to access it from multiple threads, you will
need to provide your own locking.
* On read, the library accepts whatever blocks you hand it.
Your read callback is free to pass the library a byte at a time
or mmap the entire archive and give it to the library at once.
On write, the library always produces correctly-blocked output.
* The object-style approach allows you to have multiple archive streams
open at once. bsdtar uses this in its "@archive" extension.
* The archive itself is read/written using callback functions.
You can read an archive directly from an in-memory buffer or
write it to a socket, if you wish. There are some utility
functions to provide easy-to-use "open file," etc, capabilities.
* The read/write APIs are designed to allow individual entries
to be read or written to any data source: You can create
a block of data in memory and add it to a tar archive without
first writing a temporary file. You can also read an entry from
an archive and write the data directly to a socket. If you want
to read/write entries to disk, there are convenience functions to
make this especially easy.
* Note: The "pax interchange format" is a POSIX standard extended tar
format that should be used when the older _ustar_ format is not
appropriate. It has many advantages over other tar formats
(including the legacy GNU tar format) and is widely supported by
current tar implementations.

10904
vendor/libarchive/aclocal.m4 vendored Normal file
View file

File diff suppressed because it is too large Load diff

67
vendor/libarchive/build/autoconf/ax_append_compile_flags.m4 vendored Normal file
View file

@ -0,0 +1,67 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_append_compile_flags.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_APPEND_COMPILE_FLAGS([FLAG1 FLAG2 ...], [FLAGS-VARIABLE], [EXTRA-FLAGS], [INPUT])
#
# DESCRIPTION
#
# For every FLAG1, FLAG2 it is checked whether the compiler works with the
# flag. If it does, the flag is added FLAGS-VARIABLE
#
# If FLAGS-VARIABLE is not specified, the current language's flags (e.g.
# CFLAGS) is used. During the check the flag is always added to the
# current language's flags.
#
# If EXTRA-FLAGS is defined, it is added to the current language's default
# flags (e.g. CFLAGS) when the check is done. The check is thus made with
# the flags: "CFLAGS EXTRA-FLAGS FLAG". This can for example be used to
# force the compiler to issue an error when a bad flag is given.
#
# INPUT gives an alternative input source to AC_COMPILE_IFELSE.
#
# NOTE: This macro depends on the AX_APPEND_FLAG and
# AX_CHECK_COMPILE_FLAG. Please keep this macro in sync with
# AX_APPEND_LINK_FLAGS.
#
# LICENSE
#
# Copyright (c) 2011 Maarten Bosmans <mkbosmans@gmail.com>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
# scripts that are the output of Autoconf when processing the Macro. You
# need not follow the terms of the GNU General Public License when using
# or distributing such scripts, even though portions of the text of the
# Macro appear in them. The GNU General Public License (GPL) does govern
# all other use of the material that constitutes the Autoconf Macro.
#
# This special exception to the GPL applies to versions of the Autoconf
# Macro released by the Autoconf Archive. When you make and distribute a
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 5
AC_DEFUN([AX_APPEND_COMPILE_FLAGS],
[AX_REQUIRE_DEFINED([AX_CHECK_COMPILE_FLAG])
AX_REQUIRE_DEFINED([AX_APPEND_FLAG])
for flag in $1; do
AX_CHECK_COMPILE_FLAG([$flag], [AX_APPEND_FLAG([$flag], [$2])], [], [$3], [$4])
done
])dnl AX_APPEND_COMPILE_FLAGS

71
vendor/libarchive/build/autoconf/ax_append_flag.m4 vendored Normal file
View file

@ -0,0 +1,71 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_append_flag.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_APPEND_FLAG(FLAG, [FLAGS-VARIABLE])
#
# DESCRIPTION
#
# FLAG is appended to the FLAGS-VARIABLE shell variable, with a space
# added in between.
#
# If FLAGS-VARIABLE is not specified, the current language's flags (e.g.
# CFLAGS) is used. FLAGS-VARIABLE is not changed if it already contains
# FLAG. If FLAGS-VARIABLE is unset in the shell, it is set to exactly
# FLAG.
#
# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION.
#
# LICENSE
#
# Copyright (c) 2008 Guido U. Draheim <guidod@gmx.de>
# Copyright (c) 2011 Maarten Bosmans <mkbosmans@gmail.com>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
# scripts that are the output of Autoconf when processing the Macro. You
# need not follow the terms of the GNU General Public License when using
# or distributing such scripts, even though portions of the text of the
# Macro appear in them. The GNU General Public License (GPL) does govern
# all other use of the material that constitutes the Autoconf Macro.
#
# This special exception to the GPL applies to versions of the Autoconf
# Macro released by the Autoconf Archive. When you make and distribute a
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 6
AC_DEFUN([AX_APPEND_FLAG],
[dnl
AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_SET_IF
AS_VAR_PUSHDEF([FLAGS], [m4_default($2,_AC_LANG_PREFIX[FLAGS])])
AS_VAR_SET_IF(FLAGS,[
AS_CASE([" AS_VAR_GET(FLAGS) "],
[*" $1 "*], [AC_RUN_LOG([: FLAGS already contains $1])],
[
AS_VAR_APPEND(FLAGS,[" $1"])
AC_RUN_LOG([: FLAGS="$FLAGS"])
])
],
[
AS_VAR_SET(FLAGS,[$1])
AC_RUN_LOG([: FLAGS="$FLAGS"])
])
AS_VAR_POPDEF([FLAGS])dnl
])dnl AX_APPEND_FLAG

74
vendor/libarchive/build/autoconf/ax_check_compile_flag.m4 vendored Normal file
View file

@ -0,0 +1,74 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_check_compile_flag.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_CHECK_COMPILE_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS], [INPUT])
#
# DESCRIPTION
#
# Check whether the given FLAG works with the current language's compiler
# or gives an error. (Warnings, however, are ignored)
#
# ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on
# success/failure.
#
# If EXTRA-FLAGS is defined, it is added to the current language's default
# flags (e.g. CFLAGS) when the check is done. The check is thus made with
# the flags: "CFLAGS EXTRA-FLAGS FLAG". This can for example be used to
# force the compiler to issue an error when a bad flag is given.
#
# INPUT gives an alternative input source to AC_COMPILE_IFELSE.
#
# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this
# macro in sync with AX_CHECK_{PREPROC,LINK}_FLAG.
#
# LICENSE
#
# Copyright (c) 2008 Guido U. Draheim <guidod@gmx.de>
# Copyright (c) 2011 Maarten Bosmans <mkbosmans@gmail.com>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
# scripts that are the output of Autoconf when processing the Macro. You
# need not follow the terms of the GNU General Public License when using
# or distributing such scripts, even though portions of the text of the
# Macro appear in them. The GNU General Public License (GPL) does govern
# all other use of the material that constitutes the Autoconf Macro.
#
# This special exception to the GPL applies to versions of the Autoconf
# Macro released by the Autoconf Archive. When you make and distribute a
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 4
AC_DEFUN([AX_CHECK_COMPILE_FLAG],
[AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_IF
AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]flags_$4_$1])dnl
AC_CACHE_CHECK([whether _AC_LANG compiler accepts $1], CACHEVAR, [
ax_check_save_flags=$[]_AC_LANG_PREFIX[]FLAGS
_AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $4 $1"
AC_COMPILE_IFELSE([m4_default([$5],[AC_LANG_PROGRAM()])],
[AS_VAR_SET(CACHEVAR,[yes])],
[AS_VAR_SET(CACHEVAR,[no])])
_AC_LANG_PREFIX[]FLAGS=$ax_check_save_flags])
AS_VAR_IF(CACHEVAR,yes,
[m4_default([$2], :)],
[m4_default([$3], :)])
AS_VAR_POPDEF([CACHEVAR])dnl
])dnl AX_CHECK_COMPILE_FLAGS

37
vendor/libarchive/build/autoconf/ax_require_defined.m4 vendored Normal file
View file

@ -0,0 +1,37 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_require_defined.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_REQUIRE_DEFINED(MACRO)
#
# DESCRIPTION
#
# AX_REQUIRE_DEFINED is a simple helper for making sure other macros have
# been defined and thus are available for use. This avoids random issues
# where a macro isn't expanded. Instead the configure script emits a
# non-fatal:
#
# ./configure: line 1673: AX_CFLAGS_WARN_ALL: command not found
#
# It's like AC_REQUIRE except it doesn't expand the required macro.
#
# Here's an example:
#
# AX_REQUIRE_DEFINED([AX_CHECK_LINK_FLAG])
#
# LICENSE
#
# Copyright (c) 2014 Mike Frysinger <vapier@gentoo.org>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 1
AC_DEFUN([AX_REQUIRE_DEFINED], [dnl
m4_ifndef([$1], [m4_fatal([macro ]$1[ is not defined; is a m4 file missing?])])
])dnl AX_REQUIRE_DEFINED

Some files were not shown because too many files have changed in this diff Show more