diff --git a/docs/operations.md b/docs/operations.md index 90ebaaa..f1ff3f4 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -43,10 +43,13 @@ writes `sessions.sqlite` at the process cwd, per `startServer.sh` sources `.dev-state/env.sh`, runs `saltcorn install-plugin -d ./dev-deploy` (non-fatal on failure -- the previously installed version still -loads), then `cd .dev-state` and `exec saltcorn serve "$@"`. MAIN is the only -instance that sets `SALTCORN_IDP_LDAP_PORT=1636`, so saltcorn-idp opens its -LDAPS listener there. The other instances leave that port unset (PG sets 1637; -see below), so the three do not collide. saltcorn-idp is installed via +loads), then `cd .dev-state` and `exec saltcorn serve "$@"`. saltcorn-idp's +LDAPS listener is enabled and configured (host/port) from the public-site admin +panel; the `SALTCORN_IDP_LDAP_PORT` / `SALTCORN_IDP_LDAP_HOST` env vars are +optional per-setting overrides that win when set (a set port also forces the +listener on). MAIN is the only instance that pins `SALTCORN_IDP_LDAP_PORT=1636` +in its `env.sh`; the others leave that port unset (PG pins 1637; see below), so +the three do not collide on the same bound port. saltcorn-idp is installed via `reinstallIdp.sh` rather than in the start script (the shared plugins_folder makes a per-boot install race on concurrent starts); it loads at serve time from that install. @@ -56,8 +59,9 @@ that install. `startServerTest.sh` sources `.dev-state-test/env.sh` (which sets `SALTCORN_PORT=3001`), installs the plugin the same way, then `cd .dev-state-test` and `exec saltcorn serve -p "$SALTCORN_PORT" "$@"`. TEST is -the promote/pull peer used by the e2e suite. It does not set -`SALTCORN_IDP_LDAP_PORT`, so no LDAP listener runs there. +the promote/pull peer used by the e2e suite. Its `env.sh` does not pin +`SALTCORN_IDP_LDAP_PORT`, and the public-site LDAP config is left disabled, so no +LDAP listener runs there. ### PG (:3002, Postgres multi-tenant) @@ -69,9 +73,10 @@ the SQLite start scripts, `startServerPg.sh` does NOT run `install-plugin` at boot -- the plugin is installed into the public schema by `reinstallDevDeploy.sh` and activated per tenant by `installDevDeployTenant.sh` (see below). After per-tenant activation, each tenant's `onLoad` re-runs automatically on every -boot via `init_multi_tenant` -> `loadAllPlugins`. The PG env also sets -`SALTCORN_IDP_LDAP_PORT=1637` (distinct from MAIN's 1636) so the multi-tenant -LDAP gate can exercise tenant-in-DN binds against this instance. +boot via `init_multi_tenant` -> `loadAllPlugins`. The PG `env.sh` also pins +`SALTCORN_IDP_LDAP_PORT=1637` (distinct from MAIN's 1636), which overrides the +public-site config and forces the listener on, so the multi-tenant LDAP gate can +exercise tenant-in-DN binds against this instance. ## Environment files @@ -95,6 +100,17 @@ set (`connect.ts` `getConnectObject`). Authentication is via the Unix socket with peer auth, which ignores the password, so `PGPASSWORD=peer` is a dummy that just satisfies that check. +`SALTCORN_SESSION_SECRET` is set in each dev `env.sh`, but it is NOT required for +the plugin. dev-deploy resolves the at-rest KEK secret from the Saltcorn +installation: it falls back to `db.connectObj.session_secret` (which already +merges any env var and the `~/.config/.saltcorn` config file), then to the config +file directly, then to the `session_secret` DB config. `SALTCORN_SESSION_SECRET` +is just an explicit override that wins when present (it is checked first). A plain +Saltcorn install configured via the config file needs no extra env var; the dev +`env.sh` sets it only to pin a known value across the dev instances. dev-deploy +throws `dev-deploy: session_secret not available ...` only if all four sources +miss. Rotating the resolved secret still invalidates all sealed data. + ## Starting the instances Run from the project root: diff --git a/docs/peering.md b/docs/peering.md index 0fdefae..1bdade1 100644 --- a/docs/peering.md +++ b/docs/peering.md @@ -47,12 +47,27 @@ columns as hex. The hex-text storage is deliberate: Saltcorn's SQLite insert layer JSON-stringifies object values, which would mangle a raw `Buffer` column (`lib/schema.js`). -The 32-byte key-encryption key (KEK) used by `seal`/`open` is derived once per -process via HKDF-SHA256 from `SALTCORN_SESSION_SECRET` (`getKek()`, -`lib/crypto.js`; falls back to the Saltcorn `session_secret` config). Because -the KEK is tied to the session secret, rotating `SALTCORN_SESSION_SECRET` -invalidates every stored pairing -- existing ciphertexts no longer decrypt -(documented in `lib/crypto.js`). +The 32-byte key-encryption key (KEK) used by `seal`/`open` is derived via +HKDF-SHA256 from the Saltcorn session secret (`getKek()`, `lib/crypto.js`). +That secret is resolved from the Saltcorn installation, not a required env var. +`getSessionSecret` (`lib/crypto.js`) tries four sources in order and throws +`dev-deploy: session_secret not available ...` only if all four miss: + +1. `process.env.SALTCORN_SESSION_SECRET` -- an explicit override, checked first; + it wins when present but is not required. +2. `require("@saltcorn/data/db").connectObj.session_secret` -- Saltcorn's own + resolved value, which already merges the env var and the `~/.config/.saltcorn` + config file. +3. `getConfigFile()` from `@saltcorn/data/db/connect` -- reads + `~/.config/.saltcorn` directly, covering the case where the db module is not + yet initialised. +4. `getState().getConfig("session_secret")` -- the DB config value. + +A plain Saltcorn install configured via the config file needs no extra env var. +The derived KEK is cached, keyed by a hash of the resolved secret, so it +re-derives automatically if the secret changes. Because the KEK is tied to that +secret, rotating it invalidates every stored pairing -- existing ciphertexts no +longer decrypt (documented in `lib/crypto.js`). Plaintext only crosses the process boundary at two moments: @@ -297,7 +312,7 @@ All four machine-API routes are registered with `noCsrf: true` ### Machine API (HMAC peer auth) -| Method | Path | Handler | File:line | Purpose | +| Method | Path | Handler | File | Purpose | | --- | --- | --- | --- | --- | | GET | `/dev-deploy/api/journal?since=op_id` | `apiJournal` | `lib/routes.js` | Return local env ops after `since`, oldest first, max 1000. Returns `{ source_env_id, ops }`. | | POST | `/dev-deploy/api/ingest` | `apiIngest` | `lib/routes.js` | Apply `{ ops }` from a peer; advance that peer's inbound anchor. Returns `{ received, results }`. | @@ -306,7 +321,7 @@ All four machine-API routes are registered with `noCsrf: true` ### Admin peer and sync routes (session + admin role) -| Method | Path | Handler | File:line | Purpose | +| Method | Path | Handler | File | Purpose | | --- | --- | --- | --- | --- | | GET | `/admin/dev-deploy/peers` | `peersView` | `lib/routes.js` | List peers, show this env's `env_id`, add-peer form. | | POST | `/admin/dev-deploy/peers/add` | `peersAdd` | `lib/routes.js` | Pair a peer; generate or accept a 64-hex secret. | diff --git a/lib/crypto.js b/lib/crypto.js index 92bab1c..2850097 100644 --- a/lib/crypto.js +++ b/lib/crypto.js @@ -2,7 +2,9 @@ // // seal/open — AES-256-GCM for at-rest encryption of peer secrets. // The 32-byte KEK is derived once per process via -// HKDF-SHA256 from SALTCORN_SESSION_SECRET. +// HKDF-SHA256 from Saltcorn's session secret (resolved +// from the installation: env var, ~/.config/.saltcorn, +// or DB config -- see getSessionSecret). // sign/verify — HMAC-SHA256 for signed peer-to-peer requests. // buildCanonical — canonical string format every signed request agrees on. // randomSecret — 32 random bytes (peer_secret) at pairing time. @@ -23,34 +25,71 @@ const NONCE_BYTES = 16; const SKEW_TOLERANCE_MS = 5 * 60 * 1000; -let cachedKek = null; +let cachedKek = null; +let cachedSecretHash = null; +// Resolve the session secret from the Saltcorn installation, mirroring the +// precedence Saltcorn itself uses in db/connect.ts (env var -> ~/.config/.saltcorn +// -> generated). Leaning on Saltcorn's own resolution rather than requiring +// SALTCORN_SESSION_SECRET in the plugin's process environment lets dev-deploy +// install cleanly on an instance configured purely through the config file. const getSessionSecret = () => { + // 1. Explicit environment override (also what Saltcorn checks first). const fromEnv = process.env.SALTCORN_SESSION_SECRET; if (fromEnv && fromEnv.length > 0) { return fromEnv; } - // Fallback to Saltcorn state config if available + // 2. Saltcorn's resolved connection object. session_secret here has already + // been merged from the env var and the .saltcorn config file (and is the + // exact value Saltcorn signs its own session cookies with), so this is the + // single source of truth whenever the db module is initialised. + try { + const db = require("@saltcorn/data/db"); + if (db.connectObj && db.connectObj.session_secret) { + return db.connectObj.session_secret; + } + } catch (e) { + // db module not loadable in this context; fall through + } + // 3. Read the Saltcorn config file (~/.config/.saltcorn) directly, in case + // the db module is not yet initialised when we are called. + try { + const { getConfigFile } = require("@saltcorn/data/db/connect"); + const cfg = getConfigFile(); + if (cfg && cfg.session_secret) { + return cfg.session_secret; + } + } catch (e) { + // connect module not available; fall through + } + // 4. DB-stored config value, if any. try { const { getState } = require("@saltcorn/data/db/state"); const v = getState().getConfig("session_secret"); - if (v) return v; + if (v) { + return v; + } } catch (e) { - // ignore + // getState not available (e.g. outside a request context); fall through } - throw new Error("dev-deploy: SALTCORN_SESSION_SECRET not available; cannot derive KEK"); + throw new Error("dev-deploy: session_secret not available from environment, the Saltcorn config file, or DB config; cannot derive KEK"); }; const getKek = () => { - if (cachedKek) { + // Re-derive if the session secret changes so a rotated secret never keeps + // serving a stale KEK. session_secret is global, so this cache is shared + // safely across tenants. + const sessionSecret = getSessionSecret(); + const secretHash = crypto.createHash(HMAC_ALGORITHM).update(sessionSecret, "utf8").digest("hex"); + if (cachedKek && cachedSecretHash === secretHash) { return cachedKek; } - const sessionSecret = getSessionSecret(); const salt = Buffer.from(KEK_INFO, "utf8"); const ikm = Buffer.from(sessionSecret, "utf8"); - cachedKek = crypto.hkdfSync(HMAC_ALGORITHM, ikm, salt, Buffer.from(KEK_INFO, "utf8"), 32); + cachedKek = crypto.hkdfSync(HMAC_ALGORITHM, ikm, salt, Buffer.from(KEK_INFO, "utf8"), 32); + cachedSecretHash = secretHash; return cachedKek; }; diff --git a/scripts/installDevDeployTenant.js b/scripts/installDevDeployTenant.js index f3337c4..0803e8e 100644 --- a/scripts/installDevDeployTenant.js +++ b/scripts/installDevDeployTenant.js @@ -38,16 +38,23 @@ const installInto = async (tenant) => { const plugin = new Plugin({ name: PLUGIN_NAME, source: "local", location: DEV_DEPLOY_DIR, configuration: {} }); await Plugin.loadAndSaveNewPlugin(plugin, true, false); }); - // Verify against dev-deploy's own table so a stale row can't pass. + // Verify against dev-deploy's own artifacts so a stale row can't pass. + // onLoad both creates the raw journal table (_dd_ops) and bootstraps the + // environment identity into _sc_config (key "dev_deploy_env"). The env is + // no longer a _dd_env table -- it was migrated into _sc_config (see + // lib/env.js + the legacy-table drop in lib/schema.js), so checking for + // _dd_env here used to fail for every freshly-installed tenant. const row = await db.selectMaybeOne("_sc_plugins", { name: PLUGIN_NAME }); - const svc = await db.query( - "SELECT 1 FROM information_schema.tables WHERE table_schema = $1 AND table_name = '_dd_env'", + const tbl = await db.query( + "SELECT 1 FROM information_schema.tables WHERE table_schema = $1 AND table_name = '_dd_ops'", [db.getTenantSchema()] ); - const hasSvc = svc && svc.rows && svc.rows.length > 0; + const hasTable = tbl && tbl.rows && tbl.rows.length > 0; + const env = await db.selectMaybeOne("_sc_config", { key: "dev_deploy_env" }); + const hasEnv = !!env; // eslint-disable-next-line no-console - console.log(`[installDevDeployTenant] ${tenant}: _sc_plugins=${row ? "yes" : "NO"} _dd_env=${hasSvc ? "yes" : "NO"}`); - if (!row || !hasSvc) { + console.log(`[installDevDeployTenant] ${tenant}: _sc_plugins=${row ? "yes" : "NO"} _dd_ops=${hasTable ? "yes" : "NO"} dev_deploy_env=${hasEnv ? "yes" : "NO"}`); + if (!row || !hasTable || !hasEnv) { throw new Error("install verification failed for tenant " + tenant + " (onLoad did not run)"); } }); diff --git a/test/e2e.js b/test/e2e.js index d853bc8..0dceba8 100644 --- a/test/e2e.js +++ b/test/e2e.js @@ -933,16 +933,26 @@ const main = async () => { assert.ok(parseInt(c, 10) >= 1); }); - scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name","REV_BEFORE");`); - scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name","REV_AFTER");`); - const cfgOp = sql(MAIN.db, "SELECT op_id FROM _dd_ops WHERE op_type='set_config' AND payload LIKE '%REV_AFTER%' ORDER BY created_at DESC LIMIT 1"); + // Markers are UNIQUE per run. The two setConfig calls run in separate scExec + // processes that write the DB but NOT the running server's in-memory config + // cache. Saltcorn's setConfig skips the DB write when the worker's cached value + // already equals the target (state.ts), so a fixed "REV_BEFORE" can collide with + // a stale server-cached value and turn the revert into a silent no-op (the value + // never persists). A never-before-used marker can never equal the server cache, + // so the revert's setConfig always differs from cache and always persists. + const cfgStamp = `${Date.now()}`; + const cfgBefore = `REV_BEFORE_${cfgStamp}`; + const cfgAfter = `REV_AFTER_${cfgStamp}`; + scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name", ${JSON.stringify(cfgBefore)});`); + scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name", ${JSON.stringify(cfgAfter)});`); + const cfgOp = sql(MAIN.db, `SELECT op_id FROM _dd_ops WHERE op_type='set_config' AND payload LIKE '%${cfgAfter}%' ORDER BY created_at DESC LIMIT 1`); const rcfg = adminPost(MAIN, MAIN_COOKIES, "/admin/dev-deploy/revert", { op_id: cfgOp }); await test("revert set_config returns 302", async () => { assert.equal(rcfg.status, 302); }); - await test("set_config revert restored site_name to REV_BEFORE", async () => { + await test("set_config revert restored site_name to the prior value", async () => { const v = sql(MAIN.db, "SELECT value FROM _sc_config WHERE key='site_name'"); - assert.match(v, /REV_BEFORE/); + assert.match(v, new RegExp(cfgBefore)); }); scExec(MAIN.env, `const t = await Table.create("revrows"); await Field.create({table_id:t.id,name:"label",label:"Label",type:"String"});`);