A few fixes for security issues and some shutdown freezes.
This commit is contained in:
parent
2bdbe71eaf
commit
8dc0f82a25
29 changed files with 688 additions and 69 deletions
11
API.md
11
API.md
|
|
@ -53,7 +53,9 @@ etc. -- see the README and `src/calog.h`.)
|
||||||
native allow-list (every engine), a wall-clock time budget, and a memory cap (an over-budget or
|
native allow-list (every engine), a wall-clock time budget, and a memory cap (an over-budget or
|
||||||
runaway script is retired). The time budget and memory cap hold for all engines except s7 -- Lua,
|
runaway script is retired). The time budget and memory cap hold for all engines except s7 -- Lua,
|
||||||
JavaScript, my-basic, Berry, Tcl, mruby, Squirrel, Wren, and Janet -- each enforced by the cleanest
|
JavaScript, my-basic, Berry, Tcl, mruby, Squirrel, Wren, and Janet -- each enforced by the cleanest
|
||||||
mechanism its VM allows (see design.md sec 24); s7 is allow-list-only, a documented limit. Bound
|
mechanism its VM allows (see design.md sec 24); s7 is allow-list-only, a documented limit, and a
|
||||||
|
context asking for a time budget or a memory cap on s7 is refused (`calogContextOpenLimited` returns
|
||||||
|
NULL, and a capped script's `taskSpawn` onto s7 is an error) rather than opened without them. Bound
|
||||||
granularity varies: exact for Lua/JS/Berry/mruby, otherwise allocation-, loop-, or statement-granular
|
granularity varies: exact for Lua/JS/Berry/mruby, otherwise allocation-, loop-, or statement-granular
|
||||||
(a single operation may transiently overshoot the memory cap before the next check). For my-basic,
|
(a single operation may transiently overshoot the memory cap before the next check). For my-basic,
|
||||||
`INPUT` never reads host stdin: it yields an empty line, so a script takes input through natives like
|
`INPUT` never reads host stdin: it yields an empty line, so a script takes input through natives like
|
||||||
|
|
@ -101,7 +103,8 @@ refusal, a counting allocator, an instruction or heartbeat hook, the bytecode lo
|
||||||
or -- for Janet -- a watchdog thread); design.md sec 24 has the per-engine details and the two accepted
|
or -- for Janet -- a watchdog thread); design.md sec 24 has the per-engine details and the two accepted
|
||||||
limitations (a script that deliberately catches the sandbox error and loops can pin its thread; a
|
limitations (a script that deliberately catches the sandbox error and loops can pin its thread; a
|
||||||
single allocation of script-controlled size can transiently overshoot the cap). **s7 is allow-list
|
single allocation of script-controlled size can transiently overshoot the cap). **s7 is allow-list
|
||||||
only** -- its unchecked allocator and optimizer-collapsed loops leave no safe hook point.
|
only** -- its unchecked allocator and optimizer-collapsed loops leave no safe hook point -- so a
|
||||||
|
time budget or memory cap on s7 is refused, never silently dropped.
|
||||||
|
|
||||||
Value-model edges that follow from each language (not calog limits): Tcl and s7 have no true `nil` (it
|
Value-model edges that follow from each language (not calog limits): Tcl and s7 have no true `nil` (it
|
||||||
marshals out as `""` / `#<unspecified>`) and collapse `bool` to `0`/`1`; a hybrid list+map aggregate
|
marshals out as `""` / `#<unspecified>`) and collapse `bool` to `0`/`1`; a hybrid list+map aggregate
|
||||||
|
|
@ -205,6 +208,7 @@ be raised later without a migration. Compare the result with `cryptoEquals`.
|
||||||
| Function | Description |
|
| Function | Description |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `cryptoHashSha256(data: string) -> string` | SHA-256 as a 64-char lowercase hex digest. |
|
| `cryptoHashSha256(data: string) -> string` | SHA-256 as a 64-char lowercase hex digest. |
|
||||||
|
| `cryptoHashFileSha256(path: string) -> string` | SHA-256 of a file's contents, read a piece at a time, so a file of any size is hashed in constant memory. |
|
||||||
| `cryptoHashSha1(data: string) -> string` | SHA-1 as a 40-char lowercase hex digest. |
|
| `cryptoHashSha1(data: string) -> string` | SHA-1 as a 40-char lowercase hex digest. |
|
||||||
| `cryptoHmacSha256(key: string, data: string) -> string` | HMAC-SHA-256 as 64-char lowercase hex. |
|
| `cryptoHmacSha256(key: string, data: string) -> string` | HMAC-SHA-256 as 64-char lowercase hex. |
|
||||||
| `cryptoRandomBytes(count: int) -> string` | `count` cryptographically-random bytes. |
|
| `cryptoRandomBytes(count: int) -> string` | `count` cryptographically-random bytes. |
|
||||||
|
|
@ -256,11 +260,12 @@ POSIX filesystem access. A failed operation raises a catchable script error carr
|
||||||
|
|
||||||
| Function | Description |
|
| Function | Description |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `fsRead(path: string) -> string` | Read a whole file (binary-safe). |
|
| `fsRead(path: string [, offset: int [, length: int]]) -> string` | Read a whole file (binary-safe), or `length` bytes from `offset` (to the end without a length); short at the end of the file, empty past it. How a file too large to hold is read a piece at a time. |
|
||||||
| `fsWrite(path: string, data: string)` | Create/truncate and write `data`. |
|
| `fsWrite(path: string, data: string)` | Create/truncate and write `data`. |
|
||||||
| `fsAppend(path: string, data: string)` | Create if absent, append at the end. |
|
| `fsAppend(path: string, data: string)` | Create if absent, append at the end. |
|
||||||
| `fsExists(path: string) -> bool` | Whether the path exists. |
|
| `fsExists(path: string) -> bool` | Whether the path exists. |
|
||||||
| `fsRemove(path: string)` | Unlink a file. |
|
| `fsRemove(path: string)` | Unlink a file. |
|
||||||
|
| `fsRename(from: string, to: string)` | Move a file, replacing `to` if it exists. |
|
||||||
| `fsMkdir(path: string)` | Create one directory level (existing dir is OK). |
|
| `fsMkdir(path: string)` | Create one directory level (existing dir is OK). |
|
||||||
| `fsList(path: string) -> list` | Entry name strings, excluding `.` and `..`. |
|
| `fsList(path: string) -> list` | Entry name strings, excluding `.` and `..`. |
|
||||||
| `fsStat(path: string) -> map \| nil` | `{size, isDir, isFile, mtime}`, or nil if the path is absent. |
|
| `fsStat(path: string) -> map \| nil` | `{size, isDir, isFile, mtime}`, or nil if the path is absent. |
|
||||||
|
|
|
||||||
62
AUDIT.md
62
AUDIT.md
|
|
@ -2,12 +2,13 @@
|
||||||
|
|
||||||
Original audit: 15 scoped reviewers + 1 adversarial verifier per finding (118 raw, 4 refuted, 114 confirmed).
|
Original audit: 15 scoped reviewers + 1 adversarial verifier per finding (118 raw, 4 refuted, 114 confirmed).
|
||||||
|
|
||||||
A **follow-on audit** is appended at the end of this file: 36 findings, IDs prefixed `S`, from the
|
A **follow-on audit** is appended at the end of this file: 39 findings, IDs prefixed `S`, from the
|
||||||
2026-07-24 to 2026-08-04 session. Same format, different provenance, and it arrived in four passes:
|
2026-07-24 to 2026-08-04 session. Same format, different provenance, and it arrived in four passes:
|
||||||
one reported defect and everything pulling on it uncovered (S1-S17); three open items re-examined
|
one reported defect and everything pulling on it uncovered (S1-S17); three open items re-examined
|
||||||
after being written off as accepted limitations (S18-S20); a six-lens review sweep with adversarial
|
after being written off as accepted limitations (S18-S20); a six-lens review sweep with adversarial
|
||||||
verification (S21-S23); and the leads that sweep raised but never verified, plus the items its
|
verification (S21-S23); and the leads that sweep raised but never verified, plus the items its
|
||||||
verification confirmed (S24-S36). It is a separate body of work -- the counts in this section cover
|
verification confirmed (S24-S36); and a fifth pass on 2026-09-29 (S37-S39) that closed the one
|
||||||
|
open note left behind. It is a separate body of work -- the counts in this section cover
|
||||||
the original 114 only.
|
the original 114 only.
|
||||||
|
|
||||||
## Remediation status
|
## Remediation status
|
||||||
|
|
@ -2244,7 +2245,7 @@ every caller of a cross-context native already handles.
|
||||||
interrupted mid-chunk) stops cleanly in ~1.0 s across 8 consecutive runs, and all TSan targets stay at
|
interrupted mid-chunk) stops cleanly in ~1.0 s across 8 consecutive runs, and all TSan targets stay at
|
||||||
zero warnings. The broader structural claim behind that report -- that the before-contexts destroy
|
zero warnings. The broader structural claim behind that report -- that the before-contexts destroy
|
||||||
hooks join foreign threads while the host has stopped pumping -- is NOT closed by this and remains
|
hooks join foreign threads while the host has stopped pumping -- is NOT closed by this and remains
|
||||||
open; see the note in the remaining-work list.
|
open; see the note in the remaining-work list. (Closed 2026-09-29 by S37.)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -2321,3 +2322,58 @@ fix to prove a test has teeth, the disable must keep the build green -- gate it
|
||||||
the build output -- because a failed rebuild looks exactly like a test that passes.
|
the build output -- because a failed rebuild looks exactly like a test that passes.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Fifth pass -- the open note, and two found beside it (3), 2026-09-29
|
||||||
|
|
||||||
|
### S37. [bug] libs/calogTimer.c (calogTimerShutdown) --- FIXED
|
||||||
|
|
||||||
|
**`calogDestroy` hung whenever a timer callback re-armed itself during teardown: 10 runs in 10.**
|
||||||
|
|
||||||
|
The other leg of S34's structural note. `calogTimerShutdown` went through `calogRegistryRelease`,
|
||||||
|
which holds `gInitMutex` for the whole of `timerFreeAll` -- and that joins the timer thread. The
|
||||||
|
thread is inside `calogFnInvoke`, waiting on the context running its callback; the callback calls
|
||||||
|
`timerAfter`, an inline native, whose `timerSchedule` waits on `gInitMutex`. Host waits on thread,
|
||||||
|
thread on context, context on the host's lock. A callback that schedules another timer is ordinary.
|
||||||
|
|
||||||
|
Fixed by dropping the count under `gInitMutex` and tearing down after it is released
|
||||||
|
(`calogRegistryReleaseLast`, in value.c beside `calogRegistryRelease`): the waiting schedule then
|
||||||
|
takes the lock, sees zero, and fails with `calogErrDeadE`, the callback returns, and the join
|
||||||
|
completes. A new `gTeardownMutex`, held by a register and by the whole shutdown but never by a
|
||||||
|
schedule, keeps a runtime registering from racing the last one's teardown, which the old lock had
|
||||||
|
covered. Pubsub, export, and kv use the same release but join no thread, so they are unchanged.
|
||||||
|
|
||||||
|
*Verifier:* `testTeardown` gained `testDestroyJoinsWithRearmInFlight` (a Lua `timerEvery` whose
|
||||||
|
callback works a while and then calls `timerAfter`). With the old shutdown restored it hit the
|
||||||
|
watchdog 3 runs in 3; with the fix, 35 checks, 0 failed, 3 runs in 3, under ASan/UBSan.
|
||||||
|
|
||||||
|
### S38. [bug] src/context.c (contextOpenWithSandbox) --- FIXED
|
||||||
|
|
||||||
|
**A capped script could escape its memory cap and deadline by spawning an s7 task.**
|
||||||
|
|
||||||
|
s7 cannot enforce either limit (its loops bypass its begin-hook; its allocator is unchecked), which
|
||||||
|
was documented -- but `calogContextOpenLimited` still opened an s7 context with a cap it would ignore,
|
||||||
|
and `calogContextOpenInheriting` let a capped Lua script `taskSpawn('s7', ...)` a child that shared
|
||||||
|
the sandbox in name only. Sec 30 ("a task no longer escapes its parent's sandbox") did not cover it.
|
||||||
|
|
||||||
|
Fixed by an `ignoresLimits` field on `CalogEngineT` (true for s7, false for the nine engines that
|
||||||
|
enforce) and a refusal in `contextOpenWithSandbox` when the engine ignores limits and the sandbox is
|
||||||
|
`metered`: the limited open returns NULL and the spawn is an error. An allow-list alone still opens
|
||||||
|
on s7, since the broker enforces that for every engine.
|
||||||
|
|
||||||
|
*Verifier:* `testSandbox` gained five checks (a capped and a time-limited s7 open refused, an
|
||||||
|
allow-list-only one opening, a capped script's s7 spawn refused and never run, and a control spawning
|
||||||
|
s7 under an allow-list alone). With the flag cleared on s7, four of them fail; with it, 54 checks,
|
||||||
|
0 failed.
|
||||||
|
|
||||||
|
### S39. [build] tools/crossWinFull.sh, tools/crossMacFull.sh, Makefile (RELOBJ) --- FIXED
|
||||||
|
|
||||||
|
**`make test` could not run: `libs/calogTrust.c` never reached the cross scripts or the static build.**
|
||||||
|
|
||||||
|
`calogTrust.c` was split out of `calogHttp.c` (the platform root loaders, shared with the TLS
|
||||||
|
transport) and wired into the native build only. `cross-lint` failed first, which stopped `make
|
||||||
|
test` before any suite ran; behind it, `bin/calogStatic` failed to link (`calogNet` needs
|
||||||
|
`calogTrustLoad`). Added to both cross scripts -- the macOS Keychain define moves to the file that
|
||||||
|
now holds the Keychain code -- and to `RELOBJ`.
|
||||||
|
|
||||||
|
*Verifier:* `make cross-lint` ok; `make static` links and runs; `make test` exits 0.
|
||||||
|
|
||||||
|
|
|
||||||
4
Makefile
4
Makefile
|
|
@ -673,7 +673,7 @@ release:
|
||||||
# examples/staticDemo.c; the dlopen/getaddrinfo link warnings are expected and harmless
|
# examples/staticDemo.c; the dlopen/getaddrinfo link warnings are expected and harmless
|
||||||
# for a binary that uses neither.)
|
# for a binary that uses neither.)
|
||||||
RELFLAGS = -std=c11 $(WARN) $(DEP) -O2
|
RELFLAGS = -std=c11 $(WARN) $(DEP) -O2
|
||||||
RELOBJ = obj/rel/value.o obj/rel/broker.o obj/rel/context.o obj/rel/luaEngine.o obj/rel/luaAdapter.o obj/rel/calogHandle.o obj/rel/calogDb.o obj/rel/calogNet.o
|
RELOBJ = obj/rel/value.o obj/rel/broker.o obj/rel/context.o obj/rel/luaEngine.o obj/rel/luaAdapter.o obj/rel/calogHandle.o obj/rel/calogDb.o obj/rel/calogNet.o obj/rel/calogTrust.o
|
||||||
|
|
||||||
obj/rel:
|
obj/rel:
|
||||||
mkdir -p obj/rel
|
mkdir -p obj/rel
|
||||||
|
|
@ -761,7 +761,7 @@ bin/testTrace: obj/testTrace.o obj/calogExport.o lib/libcalog.a lib/liblua.a lib
|
||||||
|
|
||||||
# The httpd-as-a-script (examples/httpd.lua): protocol in Lua over the tcp* transport (calogNet, which
|
# The httpd-as-a-script (examples/httpd.lua): protocol in Lua over the tcp* transport (calogNet, which
|
||||||
# needs libenet) + the crypto natives (calogCrypto, which needs OpenSSL).
|
# needs libenet) + the crypto natives (calogCrypto, which needs OpenSSL).
|
||||||
bin/testHttpdLua: obj/testHttpdLua.o obj/calogNet.o obj/calogTrust.o obj/calogCrypto.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libenet.a $(SSLARCH) | bin
|
bin/testHttpdLua: obj/testHttpdLua.o obj/calogNet.o obj/calogTrust.o obj/calogCrypto.o obj/calogFs.o obj/calogHandle.o lib/libcalog.a lib/liblua.a lib/libenet.a $(SSLARCH) | bin
|
||||||
$(CC) $(LDFLAGS) -pthread -o $@ $(filter-out $(SSLARCH),$^) $(LUALIBS) $(SSLARCH)
|
$(CC) $(LDFLAGS) -pthread -o $@ $(filter-out $(SSLARCH),$^) $(LUALIBS) $(SSLARCH)
|
||||||
|
|
||||||
bin/testHttps: obj/testHttps.o obj/calogHttp.o obj/calogTrust.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
|
bin/testHttps: obj/testHttps.o obj/calogHttp.o obj/calogTrust.o lib/libcalog.a lib/liblua.a $(SSLARCH) | bin
|
||||||
|
|
|
||||||
|
|
@ -619,6 +619,8 @@ last drop -- are preserved). `testEngineLua` captures a Lua closure on its conte
|
||||||
thread, then invokes and releases it from the main thread; both marshal to the owner,
|
thread, then invokes and releases it from the main thread; both marshal to the owner,
|
||||||
ASan/TSan-clean. Limit: releasing a callable whose owner *context* has been destroyed is
|
ASan/TSan-clean. Limit: releasing a callable whose owner *context* has been destroyed is
|
||||||
the deferred non-quiescent-teardown case (sec 9) -- best-effort inline finalize for now.
|
the deferred non-quiescent-teardown case (sec 9) -- best-effort inline finalize for now.
|
||||||
|
(Superseded: secs 26 and 28 finalize orphaned releases in the drain and adopt drops landing in the
|
||||||
|
reclaim window; what remains is the deliberate leak on a failed post, which beats corrupting a VM.)
|
||||||
|
|
||||||
**JavaScript adapter (Duktape), the fourth engine.** Vendored Duktape 2.7.0 (the
|
**JavaScript adapter (Duktape), the fourth engine.** Vendored Duktape 2.7.0 (the
|
||||||
single amalgamated `duktape.c`/`duktape.h`/`duk_config.h`) in `vendor/duktape`, built
|
single amalgamated `duktape.c`/`duktape.h`/`duk_config.h`) in `vendor/duktape`, built
|
||||||
|
|
@ -820,7 +822,9 @@ returns the slot to the freelist; the next reuse bumps the generation. A stale i
|
||||||
recycler -- `testActor`'s generation test proves it. The registry lock is held
|
recycler -- `testActor`'s generation test proves it. The registry lock is held
|
||||||
across enqueue, so a foreign enqueue cannot race a destroy onto a freed queue mutex.
|
across enqueue, so a foreign enqueue cannot race a destroy onto a freed queue mutex.
|
||||||
Still quiescence-assuming (no call to the context in flight at teardown); in-flight
|
Still quiescence-assuming (no call to the context in flight at teardown); in-flight
|
||||||
reference draining is the remaining sec 9 hardening.
|
reference draining is the remaining sec 9 hardening. (Superseded: queued CALLs are answered
|
||||||
|
`calogErrDeadE` in the drain and enqueues onto a closed queue are refused -- AUDIT.md S34 and S36 --
|
||||||
|
and secs 26 and 28 drain the references.)
|
||||||
|
|
||||||
**Engine on a thread (the EngineT vtable).** `EngineT` gained `runSource`;
|
**Engine on a thread (the EngineT vtable).** `EngineT` gained `runSource`;
|
||||||
`contextEval(context, source, result)` marshals a script run onto the context's own
|
`contextEval(context, source, result)` marshals a script run onto the context's own
|
||||||
|
|
@ -1345,7 +1349,8 @@ Cross-cutting design points:
|
||||||
before the VM is torn down. Memory still needs the allocator (a 5 ms watchdog poll cannot bound an
|
before the VM is torn down. Memory still needs the allocator (a 5 ms watchdog poll cannot bound an
|
||||||
exponential loop), so the counting allocator interrupts on an over-cap allocation.
|
exponential loop), so the counting allocator interrupts on an over-cap allocation.
|
||||||
|
|
||||||
**s7 -- the documented remaining limit.** s7's `Malloc/Calloc/Realloc` are unchecked macros with no
|
**s7 -- the documented remaining limit** (and, since 2026-09-29, refused rather than ignored: a
|
||||||
|
limited open or a capped script's `taskSpawn` on s7 fails, AUDIT.md S38). s7's `Malloc/Calloc/Realloc` are unchecked macros with no
|
||||||
allocator hook (a single large allocation deref-crashes), its only heap control is a *cell-count*
|
allocator hook (a single large allocation deref-crashes), its only heap control is a *cell-count*
|
||||||
`(*s7* 'max-heap-size)` (not byte-accurate, and it does not stop a single big allocation), and its
|
`(*s7* 'max-heap-size)` (not byte-accurate, and it does not stop a single big allocation), and its
|
||||||
`begin_hook` does not fire inside s7's optimizer-collapsed loops -- the sandbox-critical case. Both
|
`begin_hook` does not fire inside s7's optimizer-collapsed loops -- the sandbox-critical case. Both
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,8 @@
|
||||||
-- local s = httpd.new()
|
-- local s = httpd.new()
|
||||||
-- s:route("GET", "/hi", function(req) return "hello " .. req.path end)
|
-- s:route("GET", "/hi", function(req) return "hello " .. req.path end)
|
||||||
-- s:route("GET", "/made", function(req) return { status = 201, body = "created" } end)
|
-- s:route("GET", "/made", function(req) return { status = 201, body = "created" } end)
|
||||||
|
-- s:route("POST", "/up", fn, { spool = { dir = "spool", max = 8e9 } }) -- body to a file: req.bodyFile, req.bodySize
|
||||||
|
-- return { file = { path = p, offset = 0, length = n } } -- a response streamed from a file
|
||||||
-- s:websocket("/ws", function(msg) return "echo: " .. msg.message end) -- returns a text reply or nil
|
-- s:websocket("/ws", function(msg) return "echo: " .. msg.message end) -- returns a text reply or nil
|
||||||
-- s:serve(8080) -- opts: { keep = fn, acceptTimeout = ms }
|
-- s:serve(8080) -- opts: { keep = fn, acceptTimeout = ms }
|
||||||
|
|
||||||
|
|
@ -23,32 +25,42 @@ httpd.__index = httpd
|
||||||
|
|
||||||
local WS_MAGIC = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
|
local WS_MAGIC = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
|
||||||
local HEADER_MAX = 64 * 1024
|
local HEADER_MAX = 64 * 1024
|
||||||
local BODY_MAX = 16 * 1024 * 1024
|
local BODY_MAX = 16 * 1024 * 1024 -- a body held in memory; a spooled route sets its own cap
|
||||||
local RECV_CHUNK = 8192
|
local RECV_CHUNK = 8192
|
||||||
|
local SPOOL_CHUNK = 65536 -- read this much at a time into a spooled body
|
||||||
|
local SPOOL_FLUSH = 1024 * 1024 -- and write it out once this much has gathered
|
||||||
|
local FILE_CHUNK = 65536 -- a file response is sent this much at a time
|
||||||
local ACCEPT_POLL = 200 -- ms; the accept loop wakes this often to re-check its keep predicate
|
local ACCEPT_POLL = 200 -- ms; the accept loop wakes this often to re-check its keep predicate
|
||||||
|
|
||||||
local REASON = {
|
local REASON = {
|
||||||
[101] = "Switching Protocols", [200] = "OK", [201] = "Created", [204] = "No Content",
|
[101] = "Switching Protocols", [200] = "OK", [201] = "Created", [202] = "Accepted", [204] = "No Content",
|
||||||
|
[206] = "Partial Content", [413] = "Content Too Large", [416] = "Range Not Satisfiable",
|
||||||
[301] = "Moved Permanently", [302] = "Found", [400] = "Bad Request", [401] = "Unauthorized",
|
[301] = "Moved Permanently", [302] = "Found", [400] = "Bad Request", [401] = "Unauthorized",
|
||||||
[403] = "Forbidden", [404] = "Not Found", [405] = "Method Not Allowed", [500] = "Internal Server Error",
|
[403] = "Forbidden", [404] = "Not Found", [405] = "Method Not Allowed", [500] = "Internal Server Error",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
function httpd.new()
|
function httpd.new()
|
||||||
return setmetatable({ routes = {}, prefixes = {}, wsRoutes = {} }, httpd)
|
return setmetatable({ routes = {}, routeOpts = {}, prefixes = {}, wsRoutes = {} }, httpd)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
||||||
function httpd:route(method, path, handler)
|
-- opts.spool = { dir = folder, max = bytes } writes the request body to a file in `dir` instead of
|
||||||
self.routes[method:upper() .. " " .. path] = handler
|
-- holding it in memory: the handler gets req.bodyFile and req.bodySize, the file is removed after
|
||||||
|
-- the handler returns unless it moved it, and a body over `max` is answered 413 unread.
|
||||||
|
-- opts.onRefused(req, status), if given, is told of a request answered that way without its handler,
|
||||||
|
-- so a server that logs every request can log that one too.
|
||||||
|
function httpd:route(method, path, handler, opts)
|
||||||
|
self.routes[method:upper() .. " " .. path] = handler
|
||||||
|
self.routeOpts[method:upper() .. " " .. path] = opts
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
||||||
-- A route for everything under a path. The handler gets req.prefix (what matched) and req.rest (the
|
-- A route for everything under a path. The handler gets req.prefix (what matched) and req.rest (the
|
||||||
-- segments after it), so "/v1/download/" can serve "/v1/download/dragon/3" without every id needing
|
-- segments after it), so "/v1/download/" can serve "/v1/download/dragon/3" without every id needing
|
||||||
-- its own registration. An exact route always wins, and between prefixes the longest one does.
|
-- its own registration. An exact route always wins, and between prefixes the longest one does.
|
||||||
function httpd:routePrefix(method, prefix, handler)
|
function httpd:routePrefix(method, prefix, handler, opts)
|
||||||
self.prefixes[#self.prefixes + 1] = { method = method:upper(), prefix = prefix, handler = handler }
|
self.prefixes[#self.prefixes + 1] = { method = method:upper(), prefix = prefix, handler = handler, opts = opts }
|
||||||
table.sort(self.prefixes, function(a, b) return #a.prefix > #b.prefix end)
|
table.sort(self.prefixes, function(a, b) return #a.prefix > #b.prefix end)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
@ -58,10 +70,12 @@ function httpd:websocket(path, handler)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
||||||
-- Read one request off the connection, starting from any bytes `buf` already holds from a previous
|
-- Read one request's head off the connection, starting from any bytes `buf` already holds from a
|
||||||
-- (pipelined) read. Returns (request, leftoverBuffer) or nil on close/malformed. request is
|
-- previous (pipelined) read. Returns (request, rest) -- request is { method, path, query, version,
|
||||||
-- { method, path, query, version, headers, body }; leftoverBuffer feeds the next keep-alive read.
|
-- headers, length } with the body still unread, rest the bytes already received past the head -- or
|
||||||
local function readRequest(conn, buf)
|
-- nil on close or a malformed head. The body is read by readBody or spoolBody once the route is
|
||||||
|
-- known, so a route can take it as a file and a body over its cap is refused before it is read.
|
||||||
|
local function readHead(conn, buf)
|
||||||
buf = buf or ""
|
buf = buf or ""
|
||||||
local headerEnd = nil
|
local headerEnd = nil
|
||||||
while true do
|
while true do
|
||||||
|
|
@ -91,25 +105,74 @@ local function readRequest(conn, buf)
|
||||||
-- the leftover-buffer keep-alive path (a TE.CL smuggle), so reject it rather than mis-frame it.
|
-- the leftover-buffer keep-alive path (a TE.CL smuggle), so reject it rather than mis-frame it.
|
||||||
if headers["transfer-encoding"] then return nil end
|
if headers["transfer-encoding"] then return nil end
|
||||||
-- Content-Length is DIGIT-only per RFC 7230; reject the hex/scientific/signed/float forms tonumber
|
-- Content-Length is DIGIT-only per RFC 7230; reject the hex/scientific/signed/float forms tonumber
|
||||||
-- would otherwise accept (a body-length desync / smuggling risk), and reject anything over the cap.
|
-- would otherwise accept (a body-length desync / smuggling risk). The cap is the route's.
|
||||||
local clen = 0
|
local clen = 0
|
||||||
local clRaw = headers["content-length"]
|
local clRaw = headers["content-length"]
|
||||||
if clRaw then
|
if clRaw then
|
||||||
if not clRaw:match("^%d+$") then return nil end
|
if not clRaw:match("^%d+$") or #clRaw > 18 then return nil end
|
||||||
local n = tonumber(clRaw)
|
clen = math.tointeger(tonumber(clRaw)) or 0
|
||||||
if not n or n > BODY_MAX then return nil end
|
|
||||||
clen = math.tointeger(n) or 0
|
|
||||||
end
|
end
|
||||||
while #rest < clen do
|
return { method = method:upper(), path = path, query = query, version = version, headers = headers, length = clen }, rest
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
-- The body into memory: gathered as pieces and joined once, since growing one string by each piece
|
||||||
|
-- copies all of it every time. Returns (body, leftover) or nil on close.
|
||||||
|
local function readBody(conn, rest, clen)
|
||||||
|
local parts, have = { rest }, #rest
|
||||||
|
while have < clen do
|
||||||
local chunk = tcpRecv(conn, RECV_CHUNK)
|
local chunk = tcpRecv(conn, RECV_CHUNK)
|
||||||
if not chunk then return nil end
|
if not chunk then return nil end
|
||||||
rest = rest .. chunk
|
parts[#parts + 1] = chunk
|
||||||
|
have = have + #chunk
|
||||||
end
|
end
|
||||||
local req = {
|
local all = table.concat(parts)
|
||||||
method = method:upper(), path = path, query = query, version = version,
|
return all:sub(1, clen), all:sub(clen + 1)
|
||||||
headers = headers, body = rest:sub(1, clen),
|
end
|
||||||
}
|
|
||||||
return req, rest:sub(clen + 1) -- leftover: bytes of the next pipelined request, if any
|
|
||||||
|
-- The body into a new file in `dir`, a piece at a time, written out in larger runs. Returns (path,
|
||||||
|
-- leftover) or nil on close, with any partial file removed.
|
||||||
|
local function spoolBody(conn, rest, clen, dir)
|
||||||
|
fsMkdir(dir)
|
||||||
|
local path = dir .. "/body-" .. cryptoHexEncode(cryptoRandomBytes(8)) .. ".part"
|
||||||
|
local parts, pending, have = {}, 0, 0
|
||||||
|
local leftover = ""
|
||||||
|
local function flush()
|
||||||
|
if pending > 0 then
|
||||||
|
fsAppend(path, table.concat(parts))
|
||||||
|
parts, pending = {}, 0
|
||||||
|
end
|
||||||
|
end
|
||||||
|
fsWrite(path, "")
|
||||||
|
local function take(chunk)
|
||||||
|
local need = clen - have
|
||||||
|
if #chunk > need then
|
||||||
|
leftover = chunk:sub(need + 1)
|
||||||
|
chunk = chunk:sub(1, need)
|
||||||
|
end
|
||||||
|
parts[#parts + 1] = chunk
|
||||||
|
pending = pending + #chunk
|
||||||
|
have = have + #chunk
|
||||||
|
if pending >= SPOOL_FLUSH then flush() end
|
||||||
|
end
|
||||||
|
-- A dropped connection raises out of tcpRecv: the partial file goes before the error does.
|
||||||
|
local ok, closed = pcall(function()
|
||||||
|
take(rest)
|
||||||
|
while have < clen do
|
||||||
|
local chunk = tcpRecv(conn, SPOOL_CHUNK)
|
||||||
|
if not chunk then return true end
|
||||||
|
take(chunk)
|
||||||
|
end
|
||||||
|
flush()
|
||||||
|
return false
|
||||||
|
end)
|
||||||
|
if not ok or closed then
|
||||||
|
fsRemove(path)
|
||||||
|
if not ok then error(closed, 0) end
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
return path, leftover
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -124,7 +187,8 @@ end
|
||||||
|
|
||||||
|
|
||||||
-- Turn a handler's return value into an HTTP response and send it. nil -> 204; a string -> 200 with
|
-- Turn a handler's return value into an HTTP response and send it. nil -> 204; a string -> 200 with
|
||||||
-- that body; a table -> { status = 200, headers = {}, body = "" }.
|
-- that body; a table -> { status = 200, headers = {}, body = "" }, or with file = { path, offset,
|
||||||
|
-- length } in place of a body, that piece of a file sent a chunk at a time rather than held whole.
|
||||||
local function writeResponse(conn, resp, keepAlive)
|
local function writeResponse(conn, resp, keepAlive)
|
||||||
local status, body, extra = 200, "", nil
|
local status, body, extra = 200, "", nil
|
||||||
if resp == nil then
|
if resp == nil then
|
||||||
|
|
@ -136,14 +200,27 @@ local function writeResponse(conn, resp, keepAlive)
|
||||||
body = resp.body or ""
|
body = resp.body or ""
|
||||||
extra = resp.headers
|
extra = resp.headers
|
||||||
end
|
end
|
||||||
|
local file = type(resp) == "table" and resp.file or nil
|
||||||
|
local length = file and file.length or #body
|
||||||
local out = { string.format("HTTP/1.1 %d %s\r\nContent-Length: %d\r\nConnection: %s\r\n",
|
local out = { string.format("HTTP/1.1 %d %s\r\nContent-Length: %d\r\nConnection: %s\r\n",
|
||||||
status, REASON[status] or "Status", #body, keepAlive and "keep-alive" or "close") }
|
status, REASON[status] or "Status", length, keepAlive and "keep-alive" or "close") }
|
||||||
if extra then
|
if extra then
|
||||||
for k, v in pairs(extra) do out[#out + 1] = k .. ": " .. v .. "\r\n" end
|
for k, v in pairs(extra) do out[#out + 1] = k .. ": " .. v .. "\r\n" end
|
||||||
end
|
end
|
||||||
out[#out + 1] = "\r\n"
|
out[#out + 1] = "\r\n"
|
||||||
out[#out + 1] = body
|
if not file then
|
||||||
tcpSend(conn, table.concat(out)) -- raises on a broken connection; httpd:handle's pcall catches it
|
out[#out + 1] = body
|
||||||
|
tcpSend(conn, table.concat(out)) -- raises on a broken connection; httpd:handle's pcall catches it
|
||||||
|
return
|
||||||
|
end
|
||||||
|
tcpSend(conn, table.concat(out))
|
||||||
|
local offset, sent = file.offset or 0, 0
|
||||||
|
while sent < length do
|
||||||
|
local piece = fsRead(file.path, offset + sent, math.min(FILE_CHUNK, length - sent))
|
||||||
|
if #piece == 0 then error("httpd: the file ended before its length", 0) end
|
||||||
|
tcpSend(conn, piece)
|
||||||
|
sent = sent + #piece
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -266,7 +343,7 @@ function httpd:handle(conn)
|
||||||
local buf = ""
|
local buf = ""
|
||||||
while true do
|
while true do
|
||||||
local req
|
local req
|
||||||
req, buf = readRequest(conn, buf)
|
req, buf = readHead(conn, buf)
|
||||||
if not req then break end
|
if not req then break end
|
||||||
local upgrade = (req.headers["upgrade"] or ""):lower():find("websocket", 1, true)
|
local upgrade = (req.headers["upgrade"] or ""):lower():find("websocket", 1, true)
|
||||||
local connhdr = (req.headers["connection"] or ""):lower():find("upgrade", 1, true)
|
local connhdr = (req.headers["connection"] or ""):lower():find("upgrade", 1, true)
|
||||||
|
|
@ -278,25 +355,51 @@ function httpd:handle(conn)
|
||||||
return -- the connection is now a (closed) WebSocket, not keep-alive HTTP
|
return -- the connection is now a (closed) WebSocket, not keep-alive HTTP
|
||||||
end
|
end
|
||||||
local keepAlive = wantsKeepAlive(req)
|
local keepAlive = wantsKeepAlive(req)
|
||||||
local handler = self.routes[req.method .. " " .. req.path] or self.routes["* " .. req.path]
|
local key = req.method .. " " .. req.path
|
||||||
|
local handler = self.routes[key] or self.routes["* " .. req.path]
|
||||||
|
local opts = self.routeOpts[key] or self.routeOpts["* " .. req.path]
|
||||||
if not handler then
|
if not handler then
|
||||||
for _, entry in ipairs(self.prefixes) do
|
for _, entry in ipairs(self.prefixes) do
|
||||||
if (entry.method == req.method or entry.method == "*") and req.path:sub(1, #entry.prefix) == entry.prefix then
|
if (entry.method == req.method or entry.method == "*") and req.path:sub(1, #entry.prefix) == entry.prefix then
|
||||||
req.prefix = entry.prefix
|
req.prefix = entry.prefix
|
||||||
req.rest = req.path:sub(#entry.prefix + 1)
|
req.rest = req.path:sub(#entry.prefix + 1)
|
||||||
handler = entry.handler
|
handler = entry.handler
|
||||||
|
opts = entry.opts
|
||||||
break
|
break
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
-- The body, now that the route says how to take it. One over its cap is answered 413
|
||||||
|
-- unread, and the connection closed, since the unread body is still on the wire.
|
||||||
|
local spool = opts and opts.spool
|
||||||
|
local cap = spool and spool.max or BODY_MAX
|
||||||
|
if req.length > cap then
|
||||||
|
if opts and opts.onRefused then
|
||||||
|
pcall(opts.onRefused, req, 413)
|
||||||
|
end
|
||||||
|
writeResponse(conn, { status = 413, body = "Content Too Large" }, false)
|
||||||
|
break
|
||||||
|
end
|
||||||
|
if spool then
|
||||||
|
req.bodyFile, buf = spoolBody(conn, buf, req.length, spool.dir)
|
||||||
|
if not req.bodyFile then break end
|
||||||
|
req.bodySize, req.body = req.length, ""
|
||||||
|
else
|
||||||
|
req.body, buf = readBody(conn, buf, req.length)
|
||||||
|
if not req.body then break end
|
||||||
|
end
|
||||||
if handler then
|
if handler then
|
||||||
local ok, resp = pcall(handler, req)
|
local ok, resp = pcall(handler, req)
|
||||||
|
if req.bodyFile and fsExists(req.bodyFile) then
|
||||||
|
fsRemove(req.bodyFile)
|
||||||
|
end
|
||||||
if ok then
|
if ok then
|
||||||
writeResponse(conn, resp, keepAlive)
|
writeResponse(conn, resp, keepAlive)
|
||||||
else
|
else
|
||||||
writeResponse(conn, { status = 500, body = "Internal Server Error" }, keepAlive)
|
writeResponse(conn, { status = 500, body = "Internal Server Error" }, keepAlive)
|
||||||
end
|
end
|
||||||
else
|
else
|
||||||
|
if req.bodyFile then fsRemove(req.bodyFile) end
|
||||||
writeResponse(conn, { status = 404, body = "Not Found" }, keepAlive)
|
writeResponse(conn, { status = 404, body = "Not Found" }, keepAlive)
|
||||||
end
|
end
|
||||||
if not keepAlive then break end
|
if not keepAlive then break end
|
||||||
|
|
|
||||||
|
|
@ -7,9 +7,12 @@
|
||||||
#include "calogCrypto.h"
|
#include "calogCrypto.h"
|
||||||
#include "calogInternal.h"
|
#include "calogInternal.h"
|
||||||
|
|
||||||
|
#include <errno.h>
|
||||||
#include <limits.h>
|
#include <limits.h>
|
||||||
#include <stdint.h>
|
#include <stdint.h>
|
||||||
|
#include <stdio.h>
|
||||||
#include <stdlib.h>
|
#include <stdlib.h>
|
||||||
|
#include <string.h>
|
||||||
|
|
||||||
#include <openssl/crypto.h>
|
#include <openssl/crypto.h>
|
||||||
#include <openssl/evp.h>
|
#include <openssl/evp.h>
|
||||||
|
|
@ -21,6 +24,7 @@ static int32_t cryptoBase64EncodeNative(CalogValueT *args, int32_t argCount, Cal
|
||||||
static int32_t cryptoBytesToHex(CalogValueT *result, const unsigned char *bytes, size_t length);
|
static int32_t cryptoBytesToHex(CalogValueT *result, const unsigned char *bytes, size_t length);
|
||||||
static int32_t cryptoDigestHex(CalogValueT *args, int32_t argCount, CalogValueT *result, const EVP_MD *md, const char *usage);
|
static int32_t cryptoDigestHex(CalogValueT *args, int32_t argCount, CalogValueT *result, const EVP_MD *md, const char *usage);
|
||||||
static int32_t cryptoEqualsNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t cryptoEqualsNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
|
static int32_t cryptoHashFileSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t cryptoHashSha1Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t cryptoHashSha1Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t cryptoHashSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t cryptoHashSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t cryptoHexDecodeNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t cryptoHexDecodeNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
|
|
@ -45,10 +49,14 @@ static const char cryptoHexDigits[] = "0123456789abcdef";
|
||||||
#define CRYPTO_PBKDF2_ITERATIONS_MAX 10000000
|
#define CRYPTO_PBKDF2_ITERATIONS_MAX 10000000
|
||||||
#define CRYPTO_PBKDF2_LENGTH_MIN 16
|
#define CRYPTO_PBKDF2_LENGTH_MIN 16
|
||||||
#define CRYPTO_PBKDF2_LENGTH_MAX 1024
|
#define CRYPTO_PBKDF2_LENGTH_MAX 1024
|
||||||
|
// cryptoHashFileSha256 reads the file this much at a time, so a file of any size is hashed in
|
||||||
|
// constant memory.
|
||||||
|
#define CRYPTO_FILE_CHUNK 65536
|
||||||
// Every inline native this library exposes. One table so registration is a single
|
// Every inline native this library exposes. One table so registration is a single
|
||||||
// checked call (calogRegisterBatch) rather than a run of calls whose status was dropped.
|
// checked call (calogRegisterBatch) rather than a run of calls whose status was dropped.
|
||||||
static const CalogNativeEntryT gCryptoNatives[] = {
|
static const CalogNativeEntryT gCryptoNatives[] = {
|
||||||
{ "cryptoHashSha256", cryptoHashSha256Native },
|
{ "cryptoHashSha256", cryptoHashSha256Native },
|
||||||
|
{ "cryptoHashFileSha256", cryptoHashFileSha256Native },
|
||||||
{ "cryptoHashSha1", cryptoHashSha1Native },
|
{ "cryptoHashSha1", cryptoHashSha1Native },
|
||||||
{ "cryptoHmacSha256", cryptoHmacSha256Native },
|
{ "cryptoHmacSha256", cryptoHmacSha256Native },
|
||||||
{ "cryptoRandomBytes", cryptoRandomBytesNative },
|
{ "cryptoRandomBytes", cryptoRandomBytesNative },
|
||||||
|
|
@ -248,6 +256,55 @@ static int32_t cryptoEqualsNative(CalogValueT *args, int32_t argCount, CalogValu
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// The SHA-256 of a file's contents, as hex, read a piece at a time: the digest of a file too large
|
||||||
|
// to hold as a string, which cryptoHashSha256 would need whole.
|
||||||
|
static int32_t cryptoHashFileSha256Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
||||||
|
unsigned char digest[EVP_MAX_MD_SIZE];
|
||||||
|
unsigned char *chunk;
|
||||||
|
EVP_MD_CTX *context;
|
||||||
|
FILE *file;
|
||||||
|
unsigned int digestLen;
|
||||||
|
size_t got;
|
||||||
|
bool good;
|
||||||
|
|
||||||
|
(void)userData;
|
||||||
|
calogValueNil(result);
|
||||||
|
if (argCount != 1 || args[0].type != calogStringE) {
|
||||||
|
return calogFail(result, calogErrArgE, "cryptoHashFileSha256 expects (path)");
|
||||||
|
}
|
||||||
|
file = fopen(args[0].as.s.bytes, "rb");
|
||||||
|
if (file == NULL) {
|
||||||
|
return calogFail(result, calogErrNotFoundE, strerror(errno));
|
||||||
|
}
|
||||||
|
chunk = (unsigned char *)malloc(CRYPTO_FILE_CHUNK);
|
||||||
|
context = EVP_MD_CTX_new();
|
||||||
|
if (chunk == NULL || context == NULL) {
|
||||||
|
free(chunk);
|
||||||
|
EVP_MD_CTX_free(context);
|
||||||
|
fclose(file);
|
||||||
|
return calogFail(result, calogErrOomE, "cryptoHashFileSha256: out of memory");
|
||||||
|
}
|
||||||
|
good = (EVP_DigestInit_ex(context, EVP_sha256(), NULL) == 1);
|
||||||
|
while (good && (got = fread(chunk, 1, CRYPTO_FILE_CHUNK, file)) > 0) {
|
||||||
|
good = (EVP_DigestUpdate(context, chunk, got) == 1);
|
||||||
|
}
|
||||||
|
if (good && ferror(file)) {
|
||||||
|
good = false;
|
||||||
|
}
|
||||||
|
digestLen = 0;
|
||||||
|
if (good) {
|
||||||
|
good = (EVP_DigestFinal_ex(context, digest, &digestLen) == 1);
|
||||||
|
}
|
||||||
|
free(chunk);
|
||||||
|
EVP_MD_CTX_free(context);
|
||||||
|
fclose(file);
|
||||||
|
if (!good) {
|
||||||
|
return calogFail(result, calogErrNotFoundE, "cryptoHashFileSha256: reading or hashing the file failed");
|
||||||
|
}
|
||||||
|
return cryptoBytesToHex(result, digest, (size_t)digestLen);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
static int32_t cryptoHashSha1Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
static int32_t cryptoHashSha1Native(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
||||||
(void)userData;
|
(void)userData;
|
||||||
return cryptoDigestHex(args, argCount, result, EVP_sha1(), "cryptoHashSha1 expects (data)");
|
return cryptoDigestHex(args, argCount, result, EVP_sha1(), "cryptoHashSha1 expects (data)");
|
||||||
|
|
|
||||||
105
libs/calogFs.c
105
libs/calogFs.c
|
|
@ -13,6 +13,7 @@
|
||||||
#include <errno.h>
|
#include <errno.h>
|
||||||
#include <fcntl.h>
|
#include <fcntl.h>
|
||||||
#include <stdint.h>
|
#include <stdint.h>
|
||||||
|
#include <stdio.h>
|
||||||
#include <stdlib.h>
|
#include <stdlib.h>
|
||||||
#include <string.h>
|
#include <string.h>
|
||||||
#include <sys/stat.h>
|
#include <sys/stat.h>
|
||||||
|
|
@ -35,9 +36,13 @@
|
||||||
#ifdef _WIN32
|
#ifdef _WIN32
|
||||||
typedef struct _stat64 CalogStatT;
|
typedef struct _stat64 CalogStatT;
|
||||||
#define calogStat _stat64
|
#define calogStat _stat64
|
||||||
|
#define calogFstat _fstat64
|
||||||
|
#define calogSeek _lseeki64
|
||||||
#else
|
#else
|
||||||
typedef struct stat CalogStatT;
|
typedef struct stat CalogStatT;
|
||||||
#define calogStat stat
|
#define calogStat stat
|
||||||
|
#define calogFstat fstat
|
||||||
|
#define calogSeek lseek
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
// fsRead's starting buffer capacity when fstat's st_size reports 0 (procfs/sysfs/FIFO-style
|
// fsRead's starting buffer capacity when fstat's st_size reports 0 (procfs/sysfs/FIFO-style
|
||||||
|
|
@ -52,7 +57,9 @@ static int32_t fsMapSet(CalogAggT *agg, const char *keyName, CalogValueT *value)
|
||||||
static int32_t fsMkdirNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t fsMkdirNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t fsPutFile(const char *path, const char *bytes, int64_t length, bool append, CalogValueT *result);
|
static int32_t fsPutFile(const char *path, const char *bytes, int64_t length, bool append, CalogValueT *result);
|
||||||
static int32_t fsReadNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t fsReadNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
|
static int32_t fsReadRange(const char *path, int64_t start, int64_t length, CalogValueT *result);
|
||||||
static int32_t fsRemoveNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t fsRemoveNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
|
static int32_t fsRenameNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t fsStatNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t fsStatNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t fsWriteNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t fsWriteNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
// Every inline native this library exposes. One table so registration is a single
|
// Every inline native this library exposes. One table so registration is a single
|
||||||
|
|
@ -63,6 +70,7 @@ static const CalogNativeEntryT gFsNatives[] = {
|
||||||
{ "fsAppend", fsAppendNative },
|
{ "fsAppend", fsAppendNative },
|
||||||
{ "fsExists", fsExistsNative },
|
{ "fsExists", fsExistsNative },
|
||||||
{ "fsRemove", fsRemoveNative },
|
{ "fsRemove", fsRemoveNative },
|
||||||
|
{ "fsRename", fsRenameNative },
|
||||||
{ "fsMkdir", fsMkdirNative },
|
{ "fsMkdir", fsMkdirNative },
|
||||||
{ "fsList", fsListNative },
|
{ "fsList", fsListNative },
|
||||||
{ "fsStat", fsStatNative },
|
{ "fsStat", fsStatNative },
|
||||||
|
|
@ -264,10 +272,18 @@ static int32_t fsReadNative(CalogValueT *args, int32_t argCount, CalogValueT *re
|
||||||
|
|
||||||
(void)userData;
|
(void)userData;
|
||||||
calogValueNil(result);
|
calogValueNil(result);
|
||||||
if (argCount != 1 || args[0].type != calogStringE) {
|
if (argCount < 1 || argCount > 3 || args[0].type != calogStringE || (argCount >= 2 && args[1].type != calogIntE) || (argCount == 3 && args[2].type != calogIntE)) {
|
||||||
return calogFail(result, calogErrArgE, "fsRead expects (path)");
|
return calogFail(result, calogErrArgE, "fsRead expects (path [, offset [, length]])");
|
||||||
}
|
}
|
||||||
path = args[0].as.s.bytes;
|
path = args[0].as.s.bytes;
|
||||||
|
// A piece of the file: from offset, length bytes or to the end. A file too large to hold is
|
||||||
|
// read this way a piece at a time.
|
||||||
|
if (argCount >= 2) {
|
||||||
|
if (args[1].as.i < 0 || (argCount == 3 && args[2].as.i < 0)) {
|
||||||
|
return calogFail(result, calogErrRangeE, "fsRead: offset and length must not be negative");
|
||||||
|
}
|
||||||
|
return fsReadRange(path, args[1].as.i, (argCount == 3) ? args[2].as.i : -1, result);
|
||||||
|
}
|
||||||
fd = open(path, O_RDONLY | O_BINARY);
|
fd = open(path, O_RDONLY | O_BINARY);
|
||||||
if (fd < 0) {
|
if (fd < 0) {
|
||||||
return fsFail(result, errno);
|
return fsFail(result, errno);
|
||||||
|
|
@ -323,6 +339,70 @@ static int32_t fsReadNative(CalogValueT *args, int32_t argCount, CalogValueT *re
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// length bytes of path from start, or to the end when length is negative; fewer when the file ends
|
||||||
|
// first, and "" from at or past the end. The buffer is the size of what will be read, never of
|
||||||
|
// what was asked for, so a bogus length cannot ask for memory the file does not justify.
|
||||||
|
static int32_t fsReadRange(const char *path, int64_t start, int64_t length, CalogValueT *result) {
|
||||||
|
CalogStatT st;
|
||||||
|
char *data;
|
||||||
|
int64_t available;
|
||||||
|
int64_t have;
|
||||||
|
ssize_t got;
|
||||||
|
int fd;
|
||||||
|
int saved;
|
||||||
|
int32_t status;
|
||||||
|
|
||||||
|
fd = open(path, O_RDONLY | O_BINARY);
|
||||||
|
if (fd < 0) {
|
||||||
|
return fsFail(result, errno);
|
||||||
|
}
|
||||||
|
if (calogFstat(fd, &st) != 0) {
|
||||||
|
saved = errno;
|
||||||
|
close(fd);
|
||||||
|
return fsFail(result, saved);
|
||||||
|
}
|
||||||
|
available = ((int64_t)st.st_size > start) ? (int64_t)st.st_size - start : 0;
|
||||||
|
if (length < 0 || length > available) {
|
||||||
|
length = available;
|
||||||
|
}
|
||||||
|
if (length == 0) {
|
||||||
|
close(fd);
|
||||||
|
return calogValueString(result, "", 0);
|
||||||
|
}
|
||||||
|
if (calogSeek(fd, start, SEEK_SET) < 0) {
|
||||||
|
saved = errno;
|
||||||
|
close(fd);
|
||||||
|
return fsFail(result, saved);
|
||||||
|
}
|
||||||
|
data = (char *)malloc((size_t)length);
|
||||||
|
if (data == NULL) {
|
||||||
|
close(fd);
|
||||||
|
return calogFail(result, calogErrOomE, "fsRead: out of memory");
|
||||||
|
}
|
||||||
|
have = 0;
|
||||||
|
while (have < length) {
|
||||||
|
got = read(fd, data + have, (size_t)(length - have));
|
||||||
|
if (got < 0) {
|
||||||
|
if (errno == EINTR) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
saved = errno;
|
||||||
|
free(data);
|
||||||
|
close(fd);
|
||||||
|
return fsFail(result, saved);
|
||||||
|
}
|
||||||
|
if (got == 0) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
have += (int64_t)got;
|
||||||
|
}
|
||||||
|
close(fd);
|
||||||
|
status = calogValueString(result, data, have);
|
||||||
|
free(data);
|
||||||
|
return status;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
static int32_t fsRemoveNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
static int32_t fsRemoveNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
||||||
(void)userData;
|
(void)userData;
|
||||||
calogValueNil(result);
|
calogValueNil(result);
|
||||||
|
|
@ -348,6 +428,27 @@ static int32_t fsRemoveNative(CalogValueT *args, int32_t argCount, CalogValueT *
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// Moves from to to, replacing to when it exists (Windows refuses to rename over a file, so there it
|
||||||
|
// is removed first). Within one filesystem this is how a finished file takes its place whole.
|
||||||
|
static int32_t fsRenameNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
||||||
|
(void)userData;
|
||||||
|
calogValueNil(result);
|
||||||
|
if (argCount != 2 || args[0].type != calogStringE || args[1].type != calogStringE) {
|
||||||
|
return calogFail(result, calogErrArgE, "fsRename expects (from, to)");
|
||||||
|
}
|
||||||
|
#ifdef _WIN32
|
||||||
|
// Windows rename refuses an existing target, and _unlink refuses a read-only one.
|
||||||
|
if (_chmod(args[1].as.s.bytes, _S_IWRITE) == 0) {
|
||||||
|
unlink(args[1].as.s.bytes);
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
if (rename(args[0].as.s.bytes, args[1].as.s.bytes) != 0) {
|
||||||
|
return fsFail(result, errno);
|
||||||
|
}
|
||||||
|
return calogOkE;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
static int32_t fsStatNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
static int32_t fsStatNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData) {
|
||||||
CalogStatT st;
|
CalogStatT st;
|
||||||
CalogValueT value;
|
CalogValueT value;
|
||||||
|
|
|
||||||
|
|
@ -65,6 +65,10 @@ static bool gThreadStarted = false;
|
||||||
static bool gShutdown = false;
|
static bool gShutdown = false;
|
||||||
static pthread_mutex_t gInitMutex = PTHREAD_MUTEX_INITIALIZER;
|
static pthread_mutex_t gInitMutex = PTHREAD_MUTEX_INITIALIZER;
|
||||||
static int32_t gRefCount = 0;
|
static int32_t gRefCount = 0;
|
||||||
|
// Held by a register and by the whole of a shutdown, never by a schedule: it keeps a runtime
|
||||||
|
// registering from racing the last one's teardown, without the teardown's join holding the lock a
|
||||||
|
// timer callback's own schedule is waiting on.
|
||||||
|
static pthread_mutex_t gTeardownMutex = PTHREAD_MUTEX_INITIALIZER;
|
||||||
|
|
||||||
static int32_t timerAfterNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t timerAfterNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t timerCancelNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t timerCancelNative(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
|
|
@ -90,7 +94,9 @@ static const CalogNativeEntryT gTimerNatives[] = {
|
||||||
int32_t calogTimerRegister(CalogT *calog) {
|
int32_t calogTimerRegister(CalogT *calog) {
|
||||||
int32_t status;
|
int32_t status;
|
||||||
|
|
||||||
|
pthread_mutex_lock(&gTeardownMutex);
|
||||||
calogRegistryRetain(&gInitMutex, &gRefCount);
|
calogRegistryRetain(&gInitMutex, &gRefCount);
|
||||||
|
pthread_mutex_unlock(&gTeardownMutex);
|
||||||
status = calogRegisterBatch(calog, gTimerNatives, (int64_t)(sizeof(gTimerNatives) / sizeof(gTimerNatives[0])), NULL);
|
status = calogRegisterBatch(calog, gTimerNatives, (int64_t)(sizeof(gTimerNatives) / sizeof(gTimerNatives[0])), NULL);
|
||||||
if (status != calogOkE) {
|
if (status != calogOkE) {
|
||||||
return status;
|
return status;
|
||||||
|
|
@ -100,10 +106,16 @@ int32_t calogTimerRegister(CalogT *calog) {
|
||||||
|
|
||||||
|
|
||||||
void calogTimerShutdown(void) {
|
void calogTimerShutdown(void) {
|
||||||
// calogRegistryRelease holds gInitMutex for the whole call, including timerFreeAll below,
|
// The count drops under gInitMutex, so a timerSchedule (which checks it under the same lock)
|
||||||
// so a timerSchedule that checks gRefCount under gInitMutex (see timerSchedule) can never
|
// runs entirely before this or sees zero and fails. The teardown -- which joins the timer
|
||||||
// observe a mid-teardown state: it either runs entirely before this or entirely after.
|
// thread -- runs after gInitMutex is dropped: the thread may be inside a callback whose own
|
||||||
calogRegistryRelease(&gInitMutex, &gRefCount, timerFreeAll);
|
// timerAfter is waiting for gInitMutex, and holding it across the join deadlocked the three
|
||||||
|
// (AUDIT.md S34). gTeardownMutex keeps a new registration out until the teardown is done.
|
||||||
|
pthread_mutex_lock(&gTeardownMutex);
|
||||||
|
if (calogRegistryReleaseLast(&gInitMutex, &gRefCount)) {
|
||||||
|
timerFreeAll();
|
||||||
|
}
|
||||||
|
pthread_mutex_unlock(&gTeardownMutex);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -185,8 +197,8 @@ static int32_t timerFindByIdLocked(int64_t id) {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
// Invoked by calogRegistryRelease while gInitMutex is held, exactly once, when the last
|
// Invoked by calogTimerShutdown, under gTeardownMutex but not gInitMutex, exactly once, when the
|
||||||
// registered runtime releases the timer library. Stops the thread (if it ever started), then
|
// last registered runtime releases the timer library. Stops the thread (if it ever started), then
|
||||||
// releases every remaining callback and frees the list. Contexts are still alive here, so a
|
// releases every remaining callback and frees the list. Contexts are still alive here, so a
|
||||||
// callback release routes a finalize to its owner thread just like calogExport.
|
// callback release routes a finalize to its owner thread just like calogExport.
|
||||||
static void timerFreeAll(void) {
|
static void timerFreeAll(void) {
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogBerryEngine = {
|
||||||
berryExtensions,
|
berryExtensions,
|
||||||
berryEngineCreate,
|
berryEngineCreate,
|
||||||
berryEngineDestroy,
|
berryEngineDestroy,
|
||||||
berryEngineRun
|
berryEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
18
src/calog.h
18
src/calog.h
|
|
@ -137,6 +137,10 @@ typedef struct CalogEngineT {
|
||||||
int32_t (*createInterpreter)(CalogContextT *context, void **interpOut);
|
int32_t (*createInterpreter)(CalogContextT *context, void **interpOut);
|
||||||
void (*destroyInterpreter)(void *interp);
|
void (*destroyInterpreter)(void *interp);
|
||||||
int32_t (*runSource)(void *interp, const char *source, CalogValueT *result);
|
int32_t (*runSource)(void *interp, const char *source, CalogValueT *result);
|
||||||
|
// True for an engine that cannot enforce a memory cap or a wall-clock limit (s7: its loops
|
||||||
|
// bypass its begin-hook). A context with either limit is refused on it rather than opened
|
||||||
|
// unlimited. An embedder's own engine sets it false unless it has the same gap.
|
||||||
|
bool ignoresLimits;
|
||||||
} CalogEngineT;
|
} CalogEngineT;
|
||||||
|
|
||||||
// ---- runtime ----
|
// ---- runtime ----
|
||||||
|
|
@ -220,18 +224,20 @@ CalogContextT *calogContextOpen(CalogT *calog, const CalogEngineT *engine); //
|
||||||
// wallClockMillis -- enforced for ALL engines EXCEPT s7: Lua/JS/my-basic/mruby via a periodic VM
|
// wallClockMillis -- enforced for ALL engines EXCEPT s7: Lua/JS/my-basic/mruby via a periodic VM
|
||||||
// hook, Berry via its instruction heartbeat, Squirrel/Wren from the bytecode loop,
|
// hook, Berry via its instruction heartbeat, Squirrel/Wren from the bytecode loop,
|
||||||
// Tcl via its built-in time limit, Janet via an out-of-band watchdog thread. A
|
// Tcl via its built-in time limit, Janet via an out-of-band watchdog thread. A
|
||||||
// script blocked in a long-running C native is not preempted (inherent). s7 ignores
|
// script blocked in a long-running C native is not preempted (inherent). s7 cannot
|
||||||
// it -- its begin-hook does not fire inside its optimized loops (design.md sec 24).
|
// enforce it -- its begin-hook does not fire inside its optimized loops (design.md
|
||||||
|
// sec 24) -- so a limited open or a capped script's taskSpawn on s7 is REFUSED
|
||||||
|
// (NULL / an error) rather than handed a context that ignores the limit.
|
||||||
// memoryBytes -- enforced for the same nine engines: a per-VM allocator (Lua/JS), or a counting
|
// memoryBytes -- enforced for the same nine engines: a per-VM allocator (Lua/JS), or a counting
|
||||||
// allocator / VM-loop check charged to the running context (the rest). s7 ignores
|
// allocator / VM-loop check charged to the running context (the rest). Refused on s7,
|
||||||
// it. Bound granularity varies (exact for Lua/JS/Berry/mruby, else allocation- or
|
// as above. Bound granularity varies (exact for Lua/JS/Berry/mruby, else allocation- or
|
||||||
// loop- or statement-granular: a single operation may transiently overshoot).
|
// loop- or statement-granular: a single operation may transiently overshoot).
|
||||||
// Cooperative model: retirement is serviced after the eval returns, so a script that DELIBERATELY
|
// Cooperative model: retirement is serviced after the eval returns, so a script that DELIBERATELY
|
||||||
// catches the sandbox error and loops can pin its context thread (every engine, incl. Lua/JS -- see
|
// catches the sandbox error and loops can pin its context thread (every engine, incl. Lua/JS -- see
|
||||||
// design.md sec 24). A merely runaway script (no catch) is always retired.
|
// design.md sec 24). A merely runaway script (no catch) is always retired.
|
||||||
typedef struct CalogLimitsT {
|
typedef struct CalogLimitsT {
|
||||||
int64_t memoryBytes; // 0 = unlimited (all engines but s7)
|
int64_t memoryBytes; // 0 = unlimited; non-zero is refused on s7
|
||||||
int64_t wallClockMillis; // 0 = unlimited (all engines but s7)
|
int64_t wallClockMillis; // 0 = unlimited; non-zero is refused on s7
|
||||||
const char *const *allowList; // NULL = all natives permitted; else a NULL-terminated list of allowed names
|
const char *const *allowList; // NULL = all natives permitted; else a NULL-terminated list of allowed names
|
||||||
int32_t maxContexts; // 0 = unbounded; else the most contexts this sandbox may contain, counting the first
|
int32_t maxContexts; // 0 = unbounded; else the most contexts this sandbox may contain, counting the first
|
||||||
} CalogLimitsT;
|
} CalogLimitsT;
|
||||||
|
|
|
||||||
|
|
@ -315,6 +315,7 @@ int32_t calogMapSetBool(CalogAggT *map, const char *key, bool flag);
|
||||||
// reference drops.
|
// reference drops.
|
||||||
void calogRegistryRetain(pthread_mutex_t *initMutex, int32_t *refCount);
|
void calogRegistryRetain(pthread_mutex_t *initMutex, int32_t *refCount);
|
||||||
void calogRegistryRelease(pthread_mutex_t *initMutex, int32_t *refCount, void (*freeAll)(void));
|
void calogRegistryRelease(pthread_mutex_t *initMutex, int32_t *refCount, void (*freeAll)(void));
|
||||||
|
bool calogRegistryReleaseLast(pthread_mutex_t *initMutex, int32_t *refCount);
|
||||||
|
|
||||||
// ---- library shutdown hooks ----
|
// ---- library shutdown hooks ----
|
||||||
// Each is defined in its libs/calog<Name>.c and registered with the runtime via calogAtDestroy, so
|
// Each is defined in its libs/calog<Name>.c and registered with the runtime via calogAtDestroy, so
|
||||||
|
|
|
||||||
|
|
@ -726,6 +726,12 @@ static CalogContextT *contextOpenWithSandbox(CalogT *broker, const CalogEngineT
|
||||||
int64_t index;
|
int64_t index;
|
||||||
uint32_t generation;
|
uint32_t generation;
|
||||||
|
|
||||||
|
// A limit the engine would silently ignore is refused instead: a limited open, or a sandboxed
|
||||||
|
// script's taskSpawn, would otherwise hand back a context free of the caps it was asked for.
|
||||||
|
if (engine != NULL && engine->ignoresLimits && sandbox != NULL && sandbox->metered) {
|
||||||
|
sandboxRelease(sandbox);
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
context = (CalogContextT *)calloc(1, sizeof(*context));
|
context = (CalogContextT *)calloc(1, sizeof(*context));
|
||||||
if (context == NULL) {
|
if (context == NULL) {
|
||||||
sandboxRelease(sandbox);
|
sandboxRelease(sandbox);
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogJanetEngine = {
|
||||||
janetExtensions,
|
janetExtensions,
|
||||||
janetEngineCreate,
|
janetEngineCreate,
|
||||||
janetEngineDestroy,
|
janetEngineDestroy,
|
||||||
janetEngineRun
|
janetEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogJsEngine = {
|
||||||
jsExtensions,
|
jsExtensions,
|
||||||
jsEngineCreate,
|
jsEngineCreate,
|
||||||
jsEngineDestroy,
|
jsEngineDestroy,
|
||||||
jsEngineRun
|
jsEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,8 @@ const CalogEngineT calogLuaEngine = {
|
||||||
luaExtensions,
|
luaExtensions,
|
||||||
luaEngineCreate,
|
luaEngineCreate,
|
||||||
luaEngineDestroy,
|
luaEngineDestroy,
|
||||||
luaEngineRun
|
luaEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogMrubyEngine = {
|
||||||
mrubyExtensions,
|
mrubyExtensions,
|
||||||
mrubyEngineCreate,
|
mrubyEngineCreate,
|
||||||
mrubyEngineDestroy,
|
mrubyEngineDestroy,
|
||||||
mrubyEngineRun
|
mrubyEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -31,7 +31,8 @@ const CalogEngineT calogMyBasicEngine = {
|
||||||
mybasicExtensions,
|
mybasicExtensions,
|
||||||
mybasicEngineCreate,
|
mybasicEngineCreate,
|
||||||
mybasicEngineDestroy,
|
mybasicEngineDestroy,
|
||||||
mybasicEngineRun
|
mybasicEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogS7Engine = {
|
||||||
s7Extensions,
|
s7Extensions,
|
||||||
s7EngineCreate,
|
s7EngineCreate,
|
||||||
s7EngineDestroy,
|
s7EngineDestroy,
|
||||||
s7EngineRun
|
s7EngineRun,
|
||||||
|
true
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogSquirrelEngine = {
|
||||||
squirrelExtensions,
|
squirrelExtensions,
|
||||||
squirrelEngineCreate,
|
squirrelEngineCreate,
|
||||||
squirrelEngineDestroy,
|
squirrelEngineDestroy,
|
||||||
squirrelEngineRun
|
squirrelEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,8 @@ const CalogEngineT calogTclEngine = {
|
||||||
tclExtensions,
|
tclExtensions,
|
||||||
tclEngineCreate,
|
tclEngineCreate,
|
||||||
tclEngineDestroy,
|
tclEngineDestroy,
|
||||||
tclEngineRun
|
tclEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
18
src/value.c
18
src/value.c
|
|
@ -771,6 +771,24 @@ void calogRegistryRelease(pthread_mutex_t *initMutex, int32_t *refCount, void (*
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// The release for a library whose teardown must not run under initMutex: its freeAll joins a
|
||||||
|
// thread that may itself be waiting for initMutex (a timer callback scheduling another timer).
|
||||||
|
// Answers whether this was the 1 -> 0 transition, so the caller tears down after the lock is
|
||||||
|
// dropped; a schedule that then takes the lock sees refcount 0 and fails instead of blocking.
|
||||||
|
bool calogRegistryReleaseLast(pthread_mutex_t *initMutex, int32_t *refCount) {
|
||||||
|
bool last;
|
||||||
|
|
||||||
|
last = false;
|
||||||
|
pthread_mutex_lock(initMutex);
|
||||||
|
if (*refCount > 0) {
|
||||||
|
(*refCount)--;
|
||||||
|
last = (*refCount == 0);
|
||||||
|
}
|
||||||
|
pthread_mutex_unlock(initMutex);
|
||||||
|
return last;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
void calogRegistryRetain(pthread_mutex_t *initMutex, int32_t *refCount) {
|
void calogRegistryRetain(pthread_mutex_t *initMutex, int32_t *refCount) {
|
||||||
pthread_mutex_lock(initMutex);
|
pthread_mutex_lock(initMutex);
|
||||||
(*refCount)++;
|
(*refCount)++;
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,8 @@ const CalogEngineT calogWrenEngine = {
|
||||||
wrenExtensions,
|
wrenExtensions,
|
||||||
wrenEngineCreate,
|
wrenEngineCreate,
|
||||||
wrenEngineDestroy,
|
wrenEngineDestroy,
|
||||||
wrenEngineRun
|
wrenEngineRun,
|
||||||
|
false
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,13 @@
|
||||||
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
|
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
|
||||||
|
|
||||||
#define PUMP_LIMIT 4000
|
#define PUMP_LIMIT 4000
|
||||||
#define RESULT_SLOTS 24
|
#define RESULT_SLOTS 27
|
||||||
|
// Files for cryptoHashFileSha256; the large one spans several of its 64 KiB reads.
|
||||||
|
#define CRYPTO_TEST_SMALL "/tmp/calogCryptoSmall.bin"
|
||||||
|
#define CRYPTO_TEST_LARGE "/tmp/calogCryptoLarge.bin"
|
||||||
|
#define CRYPTO_TEST_LARGE_BYTES 200000
|
||||||
|
#define CRYPTO_TEST_TEXT_(x) #x
|
||||||
|
#define CRYPTO_TEST_TEXT(x) CRYPTO_TEST_TEXT_(x)
|
||||||
|
|
||||||
static CalogT *calog = NULL;
|
static CalogT *calog = NULL;
|
||||||
static _Atomic int64_t results[RESULT_SLOTS];
|
static _Atomic int64_t results[RESULT_SLOTS];
|
||||||
|
|
@ -115,6 +121,25 @@ int main(void) {
|
||||||
atomic_store(&results[i], -1);
|
atomic_store(&results[i], -1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Two files for cryptoHashFileSha256: a known vector, and one larger than its read chunk.
|
||||||
|
{
|
||||||
|
FILE *file;
|
||||||
|
int32_t n;
|
||||||
|
|
||||||
|
file = fopen(CRYPTO_TEST_SMALL, "wb");
|
||||||
|
if (file != NULL) {
|
||||||
|
fputs("abc", file);
|
||||||
|
fclose(file);
|
||||||
|
}
|
||||||
|
file = fopen(CRYPTO_TEST_LARGE, "wb");
|
||||||
|
if (file != NULL) {
|
||||||
|
for (n = 0; n < CRYPTO_TEST_LARGE_BYTES; n++) {
|
||||||
|
fputc('x', file);
|
||||||
|
}
|
||||||
|
fclose(file);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
ctx = calogContextOpen(calog, &calogLuaEngine);
|
ctx = calogContextOpen(calog, &calogLuaEngine);
|
||||||
calogContextEval(ctx,
|
calogContextEval(ctx,
|
||||||
"report(1, #cryptoHashSha256('abc'))\n" // 64 hex chars
|
"report(1, #cryptoHashSha256('abc'))\n" // 64 hex chars
|
||||||
|
|
@ -145,6 +170,10 @@ int main(void) {
|
||||||
"report(21, cryptoEquals('abc', 'abd') and 0 or 1)\n"
|
"report(21, cryptoEquals('abc', 'abd') and 0 or 1)\n"
|
||||||
"report(22, cryptoEquals('abc', 'abcd') and 0 or 1)\n" // differing lengths are unequal, not an error
|
"report(22, cryptoEquals('abc', 'abcd') and 0 or 1)\n" // differing lengths are unequal, not an error
|
||||||
"report(23, cryptoEquals('a\\0b', 'a\\0b') and 1 or 0)\n" // binary-safe over embedded NULs
|
"report(23, cryptoEquals('a\\0b', 'a\\0b') and 1 or 0)\n" // binary-safe over embedded NULs
|
||||||
|
"report(24, cryptoHashFileSha256('" CRYPTO_TEST_SMALL "') == cryptoHashSha256('abc') and 1 or 0)\n"
|
||||||
|
"report(25, cryptoHashFileSha256('" CRYPTO_TEST_LARGE "') == cryptoHashSha256(string.rep('x', " CRYPTO_TEST_TEXT(CRYPTO_TEST_LARGE_BYTES) ")) and 1 or 0)\n"
|
||||||
|
"local missing = pcall(function() cryptoHashFileSha256('" CRYPTO_TEST_SMALL ".none') end)\n"
|
||||||
|
"report(26, missing and 0 or 1)\n"
|
||||||
"done()");
|
"done()");
|
||||||
pumpUntilDone(1);
|
pumpUntilDone(1);
|
||||||
|
|
||||||
|
|
@ -171,6 +200,11 @@ int main(void) {
|
||||||
CHECK(atomic_load(&results[21]) == 1, "cryptoEquals is false for strings of equal length that differ");
|
CHECK(atomic_load(&results[21]) == 1, "cryptoEquals is false for strings of equal length that differ");
|
||||||
CHECK(atomic_load(&results[22]) == 1, "cryptoEquals is false for strings of different lengths");
|
CHECK(atomic_load(&results[22]) == 1, "cryptoEquals is false for strings of different lengths");
|
||||||
CHECK(atomic_load(&results[23]) == 1, "cryptoEquals is binary-safe over embedded NULs");
|
CHECK(atomic_load(&results[23]) == 1, "cryptoEquals is binary-safe over embedded NULs");
|
||||||
|
CHECK(atomic_load(&results[24]) == 1, "cryptoHashFileSha256 matches the known vector");
|
||||||
|
CHECK(atomic_load(&results[25]) == 1, "cryptoHashFileSha256 hashes a file larger than its chunk as the whole string");
|
||||||
|
CHECK(atomic_load(&results[26]) == 1, "cryptoHashFileSha256 of a missing file raises a catchable error");
|
||||||
|
remove(CRYPTO_TEST_SMALL);
|
||||||
|
remove(CRYPTO_TEST_LARGE);
|
||||||
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
|
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
|
||||||
|
|
||||||
calogDestroy(calog);
|
calogDestroy(calog);
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@
|
||||||
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
|
#define CHECK(cond, msg) checkImpl((cond), (msg), __FILE__, __LINE__)
|
||||||
|
|
||||||
#define PUMP_LIMIT 4000
|
#define PUMP_LIMIT 4000
|
||||||
#define RESULT_SLOTS 12
|
#define RESULT_SLOTS 19
|
||||||
|
|
||||||
static CalogT *calog = NULL;
|
static CalogT *calog = NULL;
|
||||||
static _Atomic int64_t results[RESULT_SLOTS];
|
static _Atomic int64_t results[RESULT_SLOTS];
|
||||||
|
|
@ -146,6 +146,19 @@ int main(void) {
|
||||||
"fsRemove(path)\n"
|
"fsRemove(path)\n"
|
||||||
"report(10, fsExists(path) and 0 or 1)\n" // 1, exists went false
|
"report(10, fsExists(path) and 0 or 1)\n" // 1, exists went false
|
||||||
"report(11, fsStat(path) == nil and 1 or 0)\n" // 1, stat of a missing path is nil
|
"report(11, fsStat(path) == nil and 1 or 0)\n" // 1, stat of a missing path is nil
|
||||||
|
"fsWrite(path, '0123456789')\n"
|
||||||
|
"report(12, fsRead(path, 0, 4) == '0123' and 1 or 0)\n" // 1, a piece from the start
|
||||||
|
"report(13, fsRead(path, 6) == '6789' and 1 or 0)\n" // 1, from an offset to the end
|
||||||
|
"report(14, (fsRead(path, 8, 100) == '89' and fsRead(path, 20, 4) == '') and 1 or 0)\n" // 1, short at the end, empty past it
|
||||||
|
"local neg = pcall(function() fsRead(path, -1, 2) end)\n"
|
||||||
|
"report(15, neg and 0 or 1)\n" // 1, a negative offset is refused
|
||||||
|
"local other = dir .. '/other.bin'\n"
|
||||||
|
"fsWrite(other, 'old')\n"
|
||||||
|
"fsRename(path, other)\n"
|
||||||
|
"report(16, (fsExists(path) == false and fsRead(other) == '0123456789') and 1 or 0)\n" // 1, renamed over the old file
|
||||||
|
"local gone = pcall(function() fsRename(path, other) end)\n"
|
||||||
|
"report(17, gone and 0 or 1)\n" // 1, renaming a missing file raises
|
||||||
|
"fsRemove(other)\n"
|
||||||
"fsRemove(dir)\n"
|
"fsRemove(dir)\n"
|
||||||
"done()", dir);
|
"done()", dir);
|
||||||
|
|
||||||
|
|
@ -164,6 +177,12 @@ int main(void) {
|
||||||
CHECK(atomic_load(&results[9]) == 1, "reading a missing file raises a catchable error");
|
CHECK(atomic_load(&results[9]) == 1, "reading a missing file raises a catchable error");
|
||||||
CHECK(atomic_load(&results[10]) == 1, "fsRemove deletes the file (fsExists transitions to false)");
|
CHECK(atomic_load(&results[10]) == 1, "fsRemove deletes the file (fsExists transitions to false)");
|
||||||
CHECK(atomic_load(&results[11]) == 1, "fsStat of a missing path returns nil");
|
CHECK(atomic_load(&results[11]) == 1, "fsStat of a missing path returns nil");
|
||||||
|
CHECK(atomic_load(&results[12]) == 1, "fsRead with an offset and a length reads that piece");
|
||||||
|
CHECK(atomic_load(&results[13]) == 1, "fsRead with an offset alone reads to the end");
|
||||||
|
CHECK(atomic_load(&results[14]) == 1, "fsRead past the end is short, then empty");
|
||||||
|
CHECK(atomic_load(&results[15]) == 1, "fsRead refuses a negative offset");
|
||||||
|
CHECK(atomic_load(&results[16]) == 1, "fsRename moves a file over an existing one");
|
||||||
|
CHECK(atomic_load(&results[17]) == 1, "fsRename of a missing file raises a catchable error");
|
||||||
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
|
CHECK(atomic_load(&errorCount) == 0, "no uncaught errors");
|
||||||
|
|
||||||
calogDestroy(calog);
|
calogDestroy(calog);
|
||||||
|
|
|
||||||
|
|
@ -9,6 +9,7 @@
|
||||||
|
|
||||||
#include "calog.h"
|
#include "calog.h"
|
||||||
#include "calogCrypto.h"
|
#include "calogCrypto.h"
|
||||||
|
#include "calogFs.h"
|
||||||
#include "calogNet.h"
|
#include "calogNet.h"
|
||||||
|
|
||||||
#include <openssl/ssl.h>
|
#include <openssl/ssl.h>
|
||||||
|
|
@ -17,6 +18,7 @@
|
||||||
#include <netinet/in.h>
|
#include <netinet/in.h>
|
||||||
#include <stdatomic.h>
|
#include <stdatomic.h>
|
||||||
#include <stdio.h>
|
#include <stdio.h>
|
||||||
|
#include <dirent.h>
|
||||||
#include <stdlib.h>
|
#include <stdlib.h>
|
||||||
#include <string.h>
|
#include <string.h>
|
||||||
#include <sys/socket.h>
|
#include <sys/socket.h>
|
||||||
|
|
@ -26,6 +28,14 @@
|
||||||
#define PORT 38910
|
#define PORT 38910
|
||||||
#define HTTPS_PORT 38911
|
#define HTTPS_PORT 38911
|
||||||
#define PUMP_LIMIT 4000
|
#define PUMP_LIMIT 4000
|
||||||
|
// The spooled-body and file-response cases: a body larger than the server's read chunks, a cap just
|
||||||
|
// above it, and where the files go.
|
||||||
|
#define SPOOL_DIR "/tmp/calogHttpdSpool"
|
||||||
|
#define SPOOL_BYTES 200000
|
||||||
|
#define SPOOL_CAP 300000
|
||||||
|
#define FILE_PATH "/tmp/calogHttpdFile.bin"
|
||||||
|
#define HTTPD_TEST_TEXT_(x) #x
|
||||||
|
#define HTTPD_TEST_TEXT(x) HTTPD_TEST_TEXT_(x)
|
||||||
|
|
||||||
static CalogT *calog = NULL;
|
static CalogT *calog = NULL;
|
||||||
static _Atomic bool serving = true;
|
static _Atomic bool serving = true;
|
||||||
|
|
@ -43,6 +53,7 @@ static char *loadScript(const char *path);
|
||||||
static int32_t nativeKeepServing(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t nativeKeepServing(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static int32_t nativeReady(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
static int32_t nativeReady(CalogValueT *args, int32_t argCount, CalogValueT *result, void *userData);
|
||||||
static bool readResponse(int fd, int *statusOut, char *body, size_t bodyCap);
|
static bool readResponse(int fd, int *statusOut, char *body, size_t bodyCap);
|
||||||
|
static bool spoolDirEmpty(void);
|
||||||
|
|
||||||
|
|
||||||
static void checkImpl(bool condition, const char *message, int32_t line) {
|
static void checkImpl(bool condition, const char *message, int32_t line) {
|
||||||
|
|
@ -129,6 +140,27 @@ static bool readResponse(int fd, int *statusOut, char *body, size_t bodyCap) {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// Whether the spool folder holds no files (a spooled body's file is removed after its handler).
|
||||||
|
static bool spoolDirEmpty(void) {
|
||||||
|
DIR *dir;
|
||||||
|
struct dirent *entry;
|
||||||
|
bool empty;
|
||||||
|
|
||||||
|
dir = opendir(SPOOL_DIR);
|
||||||
|
if (dir == NULL) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
empty = true;
|
||||||
|
while ((entry = readdir(dir)) != NULL) {
|
||||||
|
if (strcmp(entry->d_name, ".") != 0 && strcmp(entry->d_name, "..") != 0) {
|
||||||
|
empty = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
closedir(dir);
|
||||||
|
return empty;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
static char *loadScript(const char *path) {
|
static char *loadScript(const char *path) {
|
||||||
FILE *f;
|
FILE *f;
|
||||||
char *buffer;
|
char *buffer;
|
||||||
|
|
@ -273,6 +305,7 @@ int main(void) {
|
||||||
}
|
}
|
||||||
calogNetRegister(calog);
|
calogNetRegister(calog);
|
||||||
calogCryptoRegister(calog);
|
calogCryptoRegister(calog);
|
||||||
|
calogFsRegister(calog);
|
||||||
calogRegisterInline(calog, "keepServing", nativeKeepServing, NULL);
|
calogRegisterInline(calog, "keepServing", nativeKeepServing, NULL);
|
||||||
calogRegisterInline(calog, "ready", nativeReady, NULL);
|
calogRegisterInline(calog, "ready", nativeReady, NULL);
|
||||||
|
|
||||||
|
|
@ -288,6 +321,9 @@ int main(void) {
|
||||||
"s:route('GET', '/hi', function(req) return 'hello ' .. req.path end)\n"
|
"s:route('GET', '/hi', function(req) return 'hello ' .. req.path end)\n"
|
||||||
"s:route('GET', '/made', function(req) return { status = 201, body = 'created' } end)\n"
|
"s:route('GET', '/made', function(req) return { status = 201, body = 'created' } end)\n"
|
||||||
"s:route('GET', '/boom', function(req) return nil .. 'x' end)\n" /* runtime error -> 500 */
|
"s:route('GET', '/boom', function(req) return nil .. 'x' end)\n" /* runtime error -> 500 */
|
||||||
|
"s:route('POST', '/up', function(req) return req.bodySize .. ' ' .. ((cryptoHashFileSha256(req.bodyFile) == cryptoHashSha256(string.rep('x', " HTTPD_TEST_TEXT(SPOOL_BYTES) "))) and 'ok' or 'bad') end, { spool = { dir = '" SPOOL_DIR "', max = " HTTPD_TEST_TEXT(SPOOL_CAP) " } })\n"
|
||||||
|
"s:route('POST', '/small', function(req) return tostring(#req.body) end)\n"
|
||||||
|
"s:route('GET', '/file', function(req) return { file = { path = '" FILE_PATH "', offset = 2, length = 5 } } end)\n"
|
||||||
"s:websocket('/ws', function(msg) return 'echo: ' .. msg.message end)\n"
|
"s:websocket('/ws', function(msg) return 'echo: ' .. msg.message end)\n"
|
||||||
"s:serve(38910, { keep = keepServing, onReady = ready })\n";
|
"s:serve(38910, { keep = keepServing, onReady = ready })\n";
|
||||||
source = (char *)malloc(strlen(script) + strlen(setup) + 64);
|
source = (char *)malloc(strlen(script) + strlen(setup) + 64);
|
||||||
|
|
@ -297,6 +333,17 @@ int main(void) {
|
||||||
sprintf(source, "httpd = (function()\n%s\nend)()\n%s", script, setup);
|
sprintf(source, "httpd = (function()\n%s\nend)()\n%s", script, setup);
|
||||||
free(script);
|
free(script);
|
||||||
|
|
||||||
|
// The file the file-response route serves a piece of.
|
||||||
|
{
|
||||||
|
FILE *file;
|
||||||
|
|
||||||
|
file = fopen(FILE_PATH, "wb");
|
||||||
|
if (file != NULL) {
|
||||||
|
fputs("0123456789", file);
|
||||||
|
fclose(file);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
lua = calogContextOpen(calog, &calogLuaEngine);
|
lua = calogContextOpen(calog, &calogLuaEngine);
|
||||||
calogContextEval(lua, source);
|
calogContextEval(lua, source);
|
||||||
free(source);
|
free(source);
|
||||||
|
|
@ -336,6 +383,59 @@ int main(void) {
|
||||||
close(fd);
|
close(fd);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- A spooled body: written to a file the handler reads, then removed; one over the route's cap
|
||||||
|
// answered 413 unread; an ordinary body in memory; a response streamed from a piece of a file ---
|
||||||
|
fd = clientConnect(PORT);
|
||||||
|
if (fd >= 0) {
|
||||||
|
char head[128];
|
||||||
|
char *payload;
|
||||||
|
size_t sent;
|
||||||
|
ssize_t n;
|
||||||
|
|
||||||
|
snprintf(head, sizeof(head), "POST /up HTTP/1.1\r\nHost: x\r\nContent-Length: %d\r\nConnection: close\r\n\r\n", SPOOL_BYTES);
|
||||||
|
payload = (char *)malloc(SPOOL_BYTES);
|
||||||
|
if (payload != NULL) {
|
||||||
|
memset(payload, 'x', SPOOL_BYTES);
|
||||||
|
send(fd, head, strlen(head), 0);
|
||||||
|
sent = 0;
|
||||||
|
while (sent < SPOOL_BYTES) {
|
||||||
|
n = send(fd, payload + sent, SPOOL_BYTES - sent, 0);
|
||||||
|
if (n <= 0) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
sent += (size_t)n;
|
||||||
|
}
|
||||||
|
free(payload);
|
||||||
|
CHECK(readResponse(fd, &status, body, sizeof(body)) && status == 200 && strcmp(body, HTTPD_TEST_TEXT(SPOOL_BYTES) " ok") == 0,
|
||||||
|
"a spooled body reaches the handler as a file of the right size and contents");
|
||||||
|
}
|
||||||
|
close(fd);
|
||||||
|
CHECK(spoolDirEmpty(), "the spooled body's file is removed after the handler");
|
||||||
|
}
|
||||||
|
fd = clientConnect(PORT);
|
||||||
|
if (fd >= 0) {
|
||||||
|
char head[128];
|
||||||
|
|
||||||
|
snprintf(head, sizeof(head), "POST /up HTTP/1.1\r\nHost: x\r\nContent-Length: %d\r\nConnection: close\r\n\r\n", SPOOL_CAP + 1);
|
||||||
|
send(fd, head, strlen(head), 0);
|
||||||
|
CHECK(readResponse(fd, &status, body, sizeof(body)) && status == 413, "a body over the route's cap is answered 413 before it is sent");
|
||||||
|
close(fd);
|
||||||
|
}
|
||||||
|
fd = clientConnect(PORT);
|
||||||
|
if (fd >= 0) {
|
||||||
|
const char *req = "POST /small HTTP/1.1\r\nHost: x\r\nContent-Length: 10\r\nConnection: close\r\n\r\n0123456789";
|
||||||
|
send(fd, req, strlen(req), 0);
|
||||||
|
CHECK(readResponse(fd, &status, body, sizeof(body)) && status == 200 && strcmp(body, "10") == 0, "an ordinary body arrives whole in memory");
|
||||||
|
close(fd);
|
||||||
|
}
|
||||||
|
fd = clientConnect(PORT);
|
||||||
|
if (fd >= 0) {
|
||||||
|
const char *req = "GET /file HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n";
|
||||||
|
send(fd, req, strlen(req), 0);
|
||||||
|
CHECK(readResponse(fd, &status, body, sizeof(body)) && status == 200 && strcmp(body, "23456") == 0, "a file response sends the piece it names");
|
||||||
|
close(fd);
|
||||||
|
}
|
||||||
|
|
||||||
// --- HTTP/1.1 keep-alive: two requests on ONE connection ---
|
// --- HTTP/1.1 keep-alive: two requests on ONE connection ---
|
||||||
fd = clientConnect(PORT);
|
fd = clientConnect(PORT);
|
||||||
if (fd >= 0) {
|
if (fd >= 0) {
|
||||||
|
|
@ -554,6 +654,8 @@ int main(void) {
|
||||||
}
|
}
|
||||||
calogDestroy(calog);
|
calogDestroy(calog);
|
||||||
|
|
||||||
|
remove(FILE_PATH);
|
||||||
|
rmdir(SPOOL_DIR);
|
||||||
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
|
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
|
||||||
fflush(stdout);
|
fflush(stdout);
|
||||||
return testsFailed == 0 ? 0 : 1;
|
return testsFailed == 0 ? 0 : 1;
|
||||||
|
|
|
||||||
|
|
@ -38,6 +38,15 @@ static void runLimited(const CalogEngineT *engine, const char *source, const
|
||||||
static void runLimitedSettle(const CalogEngineT *engine, const char *source, const CalogLimitsT *limits);
|
static void runLimitedSettle(const CalogEngineT *engine, const char *source, const CalogLimitsT *limits);
|
||||||
|
|
||||||
|
|
||||||
|
// s7 allocates permanent strings once per process and never frees them (an s7 design
|
||||||
|
// characteristic, not a calog defect); the s7 checks below start its interpreter. Suppress exactly
|
||||||
|
// that allocation site so the leak check stays meaningful. LSan calls this weak hook automatically.
|
||||||
|
const char *__lsan_default_suppressions(void);
|
||||||
|
const char *__lsan_default_suppressions(void) {
|
||||||
|
return "leak:make_permanent_string\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
static void checkImpl(bool condition, const char *message, int32_t line) {
|
static void checkImpl(bool condition, const char *message, int32_t line) {
|
||||||
testsRun++;
|
testsRun++;
|
||||||
if (!condition) {
|
if (!condition) {
|
||||||
|
|
@ -380,6 +389,33 @@ int main(void) {
|
||||||
"local b = pcall(taskSpawn, 'lua', 'reached()') "
|
"local b = pcall(taskSpawn, 'lua', 'reached()') "
|
||||||
"report(a and not b)", &spawnLimits);
|
"report(a and not b)", &spawnLimits);
|
||||||
CHECK(atomic_load(&reportValue) == 1, "spawn: maxContexts refuses the spawn past the bound");
|
CHECK(atomic_load(&reportValue) == 1, "spawn: maxContexts refuses the spawn past the bound");
|
||||||
|
|
||||||
|
// (e) An engine that cannot enforce a cap (s7) is refused a capped context rather than
|
||||||
|
// handed one that silently ignores it, whether the host asks directly or a capped script
|
||||||
|
// spawns a task on it. An allow-list alone is still fine: s7 enforces that.
|
||||||
|
{
|
||||||
|
CalogContextT *s7Ctx;
|
||||||
|
|
||||||
|
s7Ctx = calogContextOpenLimited(calog, &calogS7Engine, &memLimits);
|
||||||
|
CHECK(s7Ctx == NULL, "s7: a memory-capped context is refused");
|
||||||
|
s7Ctx = calogContextOpenLimited(calog, &calogS7Engine, &timeLimits);
|
||||||
|
CHECK(s7Ctx == NULL, "s7: a time-limited context is refused");
|
||||||
|
s7Ctx = calogContextOpenLimited(calog, &calogS7Engine, &allowLimits);
|
||||||
|
CHECK(s7Ctx != NULL, "s7: an allow-list-only context still opens");
|
||||||
|
if (s7Ctx != NULL) {
|
||||||
|
calogContextClose(s7Ctx);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
memset(&spawnLimits, 0, sizeof(spawnLimits));
|
||||||
|
spawnLimits.allowList = spawnAllow;
|
||||||
|
resetFlags();
|
||||||
|
runLimitedSettle(&calogLuaEngine, "local ok = pcall(taskSpawn, 's7', '(reached)') report(ok)", &spawnLimits);
|
||||||
|
CHECK(atomic_load(&reportValue) == 1 && atomic_load(&reachedFlag), "spawn: an allow-list-only script spawns an s7 child");
|
||||||
|
spawnLimits.memoryBytes = 8 * 1024 * 1024;
|
||||||
|
resetFlags();
|
||||||
|
runLimitedSettle(&calogLuaEngine, "local ok = pcall(taskSpawn, 's7', '(reached)') report(ok)", &spawnLimits);
|
||||||
|
CHECK(atomic_load(&reportValue) == 0, "spawn: a capped script cannot spawn an s7 child");
|
||||||
|
CHECK(!atomic_load(&reachedFlag), "spawn: the refused s7 child never ran");
|
||||||
}
|
}
|
||||||
|
|
||||||
calogDestroy(calog);
|
calogDestroy(calog);
|
||||||
|
|
|
||||||
|
|
@ -70,6 +70,7 @@ static void testCrossEngineValueOutlivesOwner(void);
|
||||||
static void testReclaimUnderConcurrentDrops(void);
|
static void testReclaimUnderConcurrentDrops(void);
|
||||||
static void testDropInsideTheReclaimWindow(void);
|
static void testDropInsideTheReclaimWindow(void);
|
||||||
static void testDestroyJoinsWithCallInFlight(void);
|
static void testDestroyJoinsWithCallInFlight(void);
|
||||||
|
static void testDestroyJoinsWithRearmInFlight(void);
|
||||||
static void testGuardsAfterShutdown(void);
|
static void testGuardsAfterShutdown(void);
|
||||||
static void testHeldCallableSurvivesTeardown(const char *what, const char *source);
|
static void testHeldCallableSurvivesTeardown(const char *what, const char *source);
|
||||||
static void testOwnerDiesBeforeTheRuntime(const char *what, const char *source, bool expectError);
|
static void testOwnerDiesBeforeTheRuntime(const char *what, const char *source, bool expectError);
|
||||||
|
|
@ -363,6 +364,50 @@ static void testDestroyJoinsWithCallInFlight(void) {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
// calogDestroy must not hang when a timer callback re-arms itself while the timer library shuts down.
|
||||||
|
//
|
||||||
|
// The other leg of the cycle above, through the timer library's own lock rather than the host: the
|
||||||
|
// shutdown held gInitMutex while it joined the timer thread, the timer thread waited on the context
|
||||||
|
// running its callback, and that callback's timerAfter waited on gInitMutex (AUDIT.md S34). A
|
||||||
|
// callback that schedules another timer is ordinary, and it hung 10 runs in 10 before the fix. The
|
||||||
|
// busy loop widens the window the callback spends in flight; without it the race is rarely hit.
|
||||||
|
static void testDestroyJoinsWithRearmInFlight(void) {
|
||||||
|
CalogContextT *ctx;
|
||||||
|
pthread_t watchdog;
|
||||||
|
struct timespec tick = { 0, PUMP_INTERVAL_NS };
|
||||||
|
int32_t index;
|
||||||
|
|
||||||
|
calog = calogCreate();
|
||||||
|
if (calog == NULL) {
|
||||||
|
CHECK(false, "destroy-with-rearm-in-flight: runtime create failed");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
calogSetErrorHandler(calog, onError, NULL);
|
||||||
|
calogTimerRegister(calog);
|
||||||
|
|
||||||
|
ctx = calogContextOpen(calog, &calogLuaEngine);
|
||||||
|
if (ctx == NULL) {
|
||||||
|
CHECK(false, "destroy-with-rearm-in-flight: context open failed");
|
||||||
|
calogDestroy(calog);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
calogContextEval(ctx, "timerEvery(1, function() local x = 0 for i = 1, 300000 do x = x + i end timerAfter(100000, function() end) end)");
|
||||||
|
for (index = 0; index < PUMP_LIMIT; index++) {
|
||||||
|
calogPump(calog);
|
||||||
|
nanosleep(&tick, NULL);
|
||||||
|
}
|
||||||
|
atomic_store(&destroyDone, false);
|
||||||
|
if (pthread_create(&watchdog, NULL, destroyWatchdogThread, NULL) != 0) {
|
||||||
|
CHECK(false, "destroy-with-rearm-in-flight: could not start the watchdog");
|
||||||
|
}
|
||||||
|
calogDestroy(calog);
|
||||||
|
atomic_store(&destroyDone, true);
|
||||||
|
pthread_join(watchdog, NULL);
|
||||||
|
CHECK(true, "destroy-with-rearm-in-flight: calogDestroy returned while a callback was scheduling");
|
||||||
|
printf(" destroy with a timer callback re-arming itself\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
static void testGuardsAfterShutdown(void) {
|
static void testGuardsAfterShutdown(void) {
|
||||||
CalogContextT *ctx;
|
CalogContextT *ctx;
|
||||||
|
|
||||||
|
|
@ -609,6 +654,7 @@ int main(void) {
|
||||||
testReclaimUnderConcurrentDrops();
|
testReclaimUnderConcurrentDrops();
|
||||||
testDropInsideTheReclaimWindow();
|
testDropInsideTheReclaimWindow();
|
||||||
testDestroyJoinsWithCallInFlight();
|
testDestroyJoinsWithCallInFlight();
|
||||||
|
testDestroyJoinsWithRearmInFlight();
|
||||||
testGuardsAfterShutdown();
|
testGuardsAfterShutdown();
|
||||||
|
|
||||||
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
|
printf("\n%d checks, %d failed\n", testsRun, testsFailed);
|
||||||
|
|
|
||||||
|
|
@ -27,9 +27,9 @@ ZIG=${ZIG:-${CALOG_ZIG:-/home/scott/zig/current/zig}}
|
||||||
export CALOG_ZIG="$ZIG" # the build/cross/bin wrappers resolve zig via $CALOG_ZIG
|
export CALOG_ZIG="$ZIG" # the build/cross/bin wrappers resolve zig via $CALOG_ZIG
|
||||||
AR="$R/build/cross/bin/zar"
|
AR="$R/build/cross/bin/zar"
|
||||||
|
|
||||||
# macOS Keychain trust for the HTTPS client (calogHttp httpLoadMacRoots). It needs the real Apple SDK
|
# macOS Keychain trust for the HTTPS client and the TLS transport (calogTrust, shared by calogHttp and
|
||||||
# framework headers (Security, CoreFoundation, libDER), which zig does NOT ship. When an SDK is
|
# calogNet). It needs the real Apple SDK framework headers (Security, CoreFoundation, libDER), which
|
||||||
# present, calogHttp is compiled with -DCALOG_MAC_KEYCHAIN_TRUST against those headers and linked
|
# zig does NOT ship. When an SDK is present, calogTrust is compiled with -DCALOG_MAC_KEYCHAIN_TRUST against those headers and linked
|
||||||
# against the minimal tools/macStubs/*.tbd (zig's Mach-O linker segfaults on the real multi-target
|
# against the minimal tools/macStubs/*.tbd (zig's Mach-O linker segfaults on the real multi-target
|
||||||
# SDK .tbd, so we link tiny hand-written stubs that export only the few symbols we call; the binary
|
# SDK .tbd, so we link tiny hand-written stubs that export only the few symbols we call; the binary
|
||||||
# still imports the real system frameworks by install-name at runtime). Without an SDK the build
|
# still imports the real system frameworks by install-name at runtime). Without an SDK the build
|
||||||
|
|
@ -167,7 +167,8 @@ build_arch(){
|
||||||
cc calogDbFull libs/calogDb.c -Ivendor/sqlite -Ivendor/postgres/src/interfaces/libpq -Ivendor/postgres/src/include -I"$M/postgres-build/src/include" -Ivendor/mariadb/include -I"$M/mariadb/include" -DCALOG_WITH_SQLITE -DCALOG_WITH_PG -DCALOG_WITH_MYSQL
|
cc calogDbFull libs/calogDb.c -Ivendor/sqlite -Ivendor/postgres/src/interfaces/libpq -Ivendor/postgres/src/include -I"$M/postgres-build/src/include" -Ivendor/mariadb/include -I"$M/mariadb/include" -DCALOG_WITH_SQLITE -DCALOG_WITH_PG -DCALOG_WITH_MYSQL
|
||||||
cc calogExport libs/calogExport.c
|
cc calogExport libs/calogExport.c
|
||||||
cc calogFs libs/calogFs.c
|
cc calogFs libs/calogFs.c
|
||||||
cc calogHttp libs/calogHttp.c -I"$M/openssl-src/include" $MACTRUST
|
cc calogHttp libs/calogHttp.c -I"$M/openssl-src/include"
|
||||||
|
cc calogTrust libs/calogTrust.c -I"$M/openssl-src/include" $MACTRUST
|
||||||
cc calogJson libs/calogJson.c
|
cc calogJson libs/calogJson.c
|
||||||
cc calogKv libs/calogKv.c
|
cc calogKv libs/calogKv.c
|
||||||
cc calogNet libs/calogNet.c -Ivendor/enet/include -I"$M/openssl-src/include"
|
cc calogNet libs/calogNet.c -Ivendor/enet/include -I"$M/openssl-src/include"
|
||||||
|
|
|
||||||
|
|
@ -89,6 +89,7 @@ cc calogDbFull libs/calogDb.c $BASE -Ivendor/sqlite -I"$W/postgres/include" -Ive
|
||||||
cc calogExport libs/calogExport.c $BASE
|
cc calogExport libs/calogExport.c $BASE
|
||||||
cc calogFs libs/calogFs.c $BASE
|
cc calogFs libs/calogFs.c $BASE
|
||||||
cc calogHttp libs/calogHttp.c $BASE -I"$W/openssl-src/include"
|
cc calogHttp libs/calogHttp.c $BASE -I"$W/openssl-src/include"
|
||||||
|
cc calogTrust libs/calogTrust.c $BASE -I"$W/openssl-src/include"
|
||||||
cc calogJson libs/calogJson.c $BASE
|
cc calogJson libs/calogJson.c $BASE
|
||||||
cc calogKv libs/calogKv.c $BASE
|
cc calogKv libs/calogKv.c $BASE
|
||||||
cc calogNet libs/calogNet.c $BASE -Ivendor/enet/include -I"$W/openssl-src/include"
|
cc calogNet libs/calogNet.c $BASE -Ivendor/enet/include -I"$W/openssl-src/include"
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue