Fixes.

docs/operations.md

@@ -43,10 +43,13 @@ writes `sessions.sqlite` at the process cwd, per
 
 `startServer.sh` sources `.dev-state/env.sh`, runs `saltcorn install-plugin -d
 ./dev-deploy` (non-fatal on failure -- the previously installed version still
-loads), then `cd .dev-state` and `exec saltcorn serve "$@"`. MAIN is the only
+loads), then `cd .dev-state` and `exec saltcorn serve "$@"`. saltcorn-idp's
-instance that sets `SALTCORN_IDP_LDAP_PORT=1636`, so saltcorn-idp opens its
+LDAPS listener is enabled and configured (host/port) from the public-site admin
-LDAPS listener there. The other instances leave that port unset (PG sets 1637;
+panel; the `SALTCORN_IDP_LDAP_PORT` / `SALTCORN_IDP_LDAP_HOST` env vars are
-see below), so the three do not collide. saltcorn-idp is installed via
+optional per-setting overrides that win when set (a set port also forces the
+listener on). MAIN is the only instance that pins `SALTCORN_IDP_LDAP_PORT=1636`
+in its `env.sh`; the others leave that port unset (PG pins 1637; see below), so
+the three do not collide on the same bound port. saltcorn-idp is installed via
 `reinstallIdp.sh` rather than in the start script (the shared plugins_folder
 makes a per-boot install race on concurrent starts); it loads at serve time from
 that install.
@@ -56,8 +59,9 @@ that install.
 `startServerTest.sh` sources `.dev-state-test/env.sh` (which sets
 `SALTCORN_PORT=3001`), installs the plugin the same way, then
 `cd .dev-state-test` and `exec saltcorn serve -p "$SALTCORN_PORT" "$@"`. TEST is
-the promote/pull peer used by the e2e suite. It does not set
+the promote/pull peer used by the e2e suite. Its `env.sh` does not pin
-`SALTCORN_IDP_LDAP_PORT`, so no LDAP listener runs there.
+`SALTCORN_IDP_LDAP_PORT`, and the public-site LDAP config is left disabled, so no
+LDAP listener runs there.
 
 ### PG (:3002, Postgres multi-tenant)
 
@@ -69,9 +73,10 @@ the SQLite start scripts, `startServerPg.sh` does NOT run `install-plugin` at
 boot -- the plugin is installed into the public schema by `reinstallDevDeploy.sh`
 and activated per tenant by `installDevDeployTenant.sh` (see below). After
 per-tenant activation, each tenant's `onLoad` re-runs automatically on every
-boot via `init_multi_tenant` -> `loadAllPlugins`. The PG env also sets
+boot via `init_multi_tenant` -> `loadAllPlugins`. The PG `env.sh` also pins
-`SALTCORN_IDP_LDAP_PORT=1637` (distinct from MAIN's 1636) so the multi-tenant
+`SALTCORN_IDP_LDAP_PORT=1637` (distinct from MAIN's 1636), which overrides the
-LDAP gate can exercise tenant-in-DN binds against this instance.
+public-site config and forces the listener on, so the multi-tenant LDAP gate can
+exercise tenant-in-DN binds against this instance.
 
 ## Environment files
 
@@ -95,6 +100,17 @@ set (`connect.ts` `getConnectObject`). Authentication is via the Unix socket
 with peer auth, which ignores the password, so `PGPASSWORD=peer` is a dummy that
 just satisfies that check.
 
+`SALTCORN_SESSION_SECRET` is set in each dev `env.sh`, but it is NOT required for
+the plugin. dev-deploy resolves the at-rest KEK secret from the Saltcorn
+installation: it falls back to `db.connectObj.session_secret` (which already
+merges any env var and the `~/.config/.saltcorn` config file), then to the config
+file directly, then to the `session_secret` DB config. `SALTCORN_SESSION_SECRET`
+is just an explicit override that wins when present (it is checked first). A plain
+Saltcorn install configured via the config file needs no extra env var; the dev
+`env.sh` sets it only to pin a known value across the dev instances. dev-deploy
+throws `dev-deploy: session_secret not available ...` only if all four sources
+miss. Rotating the resolved secret still invalidates all sealed data.
+
 ## Starting the instances
 
 Run from the project root:

docs/peering.md

@@ -47,12 +47,27 @@ columns as hex. The hex-text storage is deliberate: Saltcorn's SQLite insert
 layer JSON-stringifies object values, which would mangle a raw `Buffer` column
 (`lib/schema.js`).
 
-The 32-byte key-encryption key (KEK) used by `seal`/`open` is derived once per
+The 32-byte key-encryption key (KEK) used by `seal`/`open` is derived via
-process via HKDF-SHA256 from `SALTCORN_SESSION_SECRET` (`getKek()`,
+HKDF-SHA256 from the Saltcorn session secret (`getKek()`, `lib/crypto.js`).
-`lib/crypto.js`; falls back to the Saltcorn `session_secret` config). Because
+That secret is resolved from the Saltcorn installation, not a required env var.
-the KEK is tied to the session secret, rotating `SALTCORN_SESSION_SECRET`
+`getSessionSecret` (`lib/crypto.js`) tries four sources in order and throws
-invalidates every stored pairing -- existing ciphertexts no longer decrypt
+`dev-deploy: session_secret not available ...` only if all four miss:
-(documented in `lib/crypto.js`).
+
+1. `process.env.SALTCORN_SESSION_SECRET` -- an explicit override, checked first;
+   it wins when present but is not required.
+2. `require("@saltcorn/data/db").connectObj.session_secret` -- Saltcorn's own
+   resolved value, which already merges the env var and the `~/.config/.saltcorn`
+   config file.
+3. `getConfigFile()` from `@saltcorn/data/db/connect` -- reads
+   `~/.config/.saltcorn` directly, covering the case where the db module is not
+   yet initialised.
+4. `getState().getConfig("session_secret")` -- the DB config value.
+
+A plain Saltcorn install configured via the config file needs no extra env var.
+The derived KEK is cached, keyed by a hash of the resolved secret, so it
+re-derives automatically if the secret changes. Because the KEK is tied to that
+secret, rotating it invalidates every stored pairing -- existing ciphertexts no
+longer decrypt (documented in `lib/crypto.js`).
 
 Plaintext only crosses the process boundary at two moments:
 
@@ -297,7 +312,7 @@ All four machine-API routes are registered with `noCsrf: true`
 
 ### Machine API (HMAC peer auth)
 
-| Method | Path | Handler | File:line | Purpose |
+| Method | Path | Handler | File | Purpose |
 | --- | --- | --- | --- | --- |
 | GET | `/dev-deploy/api/journal?since=op_id` | `apiJournal` | `lib/routes.js` | Return local env ops after `since`, oldest first, max 1000. Returns `{ source_env_id, ops }`. |
 | POST | `/dev-deploy/api/ingest` | `apiIngest` | `lib/routes.js` | Apply `{ ops }` from a peer; advance that peer's inbound anchor. Returns `{ received, results }`. |
@@ -306,7 +321,7 @@ All four machine-API routes are registered with `noCsrf: true`
 
 ### Admin peer and sync routes (session + admin role)
 
-| Method | Path | Handler | File:line | Purpose |
+| Method | Path | Handler | File | Purpose |
 | --- | --- | --- | --- | --- |
 | GET | `/admin/dev-deploy/peers` | `peersView` | `lib/routes.js` | List peers, show this env's `env_id`, add-peer form. |
 | POST | `/admin/dev-deploy/peers/add` | `peersAdd` | `lib/routes.js` | Pair a peer; generate or accept a 64-hex secret. |

lib/crypto.js

@@ -2,7 +2,9 @@
 //
 //   seal/open           — AES-256-GCM for at-rest encryption of peer secrets.
 //                         The 32-byte KEK is derived once per process via
-//                         HKDF-SHA256 from SALTCORN_SESSION_SECRET.
+//                         HKDF-SHA256 from Saltcorn's session secret (resolved
+//                         from the installation: env var, ~/.config/.saltcorn,
+//                         or DB config -- see getSessionSecret).
 //   sign/verify         — HMAC-SHA256 for signed peer-to-peer requests.
 //   buildCanonical      — canonical string format every signed request agrees on.
 //   randomSecret        — 32 random bytes (peer_secret) at pairing time.
@@ -23,34 +25,71 @@ const NONCE_BYTES    = 16;
 const SKEW_TOLERANCE_MS = 5 * 60 * 1000;
 
 
-let cachedKek = null;
+let cachedKek        = null;
+let cachedSecretHash = null;
 
 
+// Resolve the session secret from the Saltcorn installation, mirroring the
+// precedence Saltcorn itself uses in db/connect.ts (env var -> ~/.config/.saltcorn
+// -> generated). Leaning on Saltcorn's own resolution rather than requiring
+// SALTCORN_SESSION_SECRET in the plugin's process environment lets dev-deploy
+// install cleanly on an instance configured purely through the config file.
 const getSessionSecret = () => {
+    // 1. Explicit environment override (also what Saltcorn checks first).
     const fromEnv = process.env.SALTCORN_SESSION_SECRET;
     if (fromEnv && fromEnv.length > 0) {
         return fromEnv;
     }
-    // Fallback to Saltcorn state config if available
+    // 2. Saltcorn's resolved connection object. session_secret here has already
+    //    been merged from the env var and the .saltcorn config file (and is the
+    //    exact value Saltcorn signs its own session cookies with), so this is the
+    //    single source of truth whenever the db module is initialised.
+    try {
+        const db = require("@saltcorn/data/db");
+        if (db.connectObj && db.connectObj.session_secret) {
+            return db.connectObj.session_secret;
+        }
+    } catch (e) {
+        // db module not loadable in this context; fall through
+    }
+    // 3. Read the Saltcorn config file (~/.config/.saltcorn) directly, in case
+    //    the db module is not yet initialised when we are called.
+    try {
+        const { getConfigFile } = require("@saltcorn/data/db/connect");
+        const cfg = getConfigFile();
+        if (cfg && cfg.session_secret) {
+            return cfg.session_secret;
+        }
+    } catch (e) {
+        // connect module not available; fall through
+    }
+    // 4. DB-stored config value, if any.
     try {
         const { getState } = require("@saltcorn/data/db/state");
         const v = getState().getConfig("session_secret");
-        if (v) return v;
+        if (v) {
+            return v;
+        }
     } catch (e) {
-        // ignore
+        // getState not available (e.g. outside a request context); fall through
     }
-    throw new Error("dev-deploy: SALTCORN_SESSION_SECRET not available; cannot derive KEK");
+    throw new Error("dev-deploy: session_secret not available from environment, the Saltcorn config file, or DB config; cannot derive KEK");
 };
 
 
 const getKek = () => {
-    if (cachedKek) {
+    // Re-derive if the session secret changes so a rotated secret never keeps
+    // serving a stale KEK. session_secret is global, so this cache is shared
+    // safely across tenants.
+    const sessionSecret = getSessionSecret();
+    const secretHash    = crypto.createHash(HMAC_ALGORITHM).update(sessionSecret, "utf8").digest("hex");
+    if (cachedKek && cachedSecretHash === secretHash) {
         return cachedKek;
     }
-    const sessionSecret = getSessionSecret();
     const salt = Buffer.from(KEK_INFO, "utf8");
     const ikm  = Buffer.from(sessionSecret, "utf8");
-    cachedKek  = crypto.hkdfSync(HMAC_ALGORITHM, ikm, salt, Buffer.from(KEK_INFO, "utf8"), 32);
+    cachedKek        = crypto.hkdfSync(HMAC_ALGORITHM, ikm, salt, Buffer.from(KEK_INFO, "utf8"), 32);
+    cachedSecretHash = secretHash;
     return cachedKek;
 };
 

scripts/installDevDeployTenant.js

@@ -38,16 +38,23 @@ const installInto = async (tenant) => {
             const plugin = new Plugin({ name: PLUGIN_NAME, source: "local", location: DEV_DEPLOY_DIR, configuration: {} });
             await Plugin.loadAndSaveNewPlugin(plugin, true, false);
         });
-        // Verify against dev-deploy's own table so a stale row can't pass.
+        // Verify against dev-deploy's own artifacts so a stale row can't pass.
+        // onLoad both creates the raw journal table (_dd_ops) and bootstraps the
+        // environment identity into _sc_config (key "dev_deploy_env"). The env is
+        // no longer a _dd_env table -- it was migrated into _sc_config (see
+        // lib/env.js + the legacy-table drop in lib/schema.js), so checking for
+        // _dd_env here used to fail for every freshly-installed tenant.
         const row = await db.selectMaybeOne("_sc_plugins", { name: PLUGIN_NAME });
-        const svc = await db.query(
+        const tbl = await db.query(
-            "SELECT 1 FROM information_schema.tables WHERE table_schema = $1 AND table_name = '_dd_env'",
+            "SELECT 1 FROM information_schema.tables WHERE table_schema = $1 AND table_name = '_dd_ops'",
             [db.getTenantSchema()]
         );
-        const hasSvc = svc && svc.rows && svc.rows.length > 0;
+        const hasTable = tbl && tbl.rows && tbl.rows.length > 0;
+        const env      = await db.selectMaybeOne("_sc_config", { key: "dev_deploy_env" });
+        const hasEnv   = !!env;
         // eslint-disable-next-line no-console
-        console.log(`[installDevDeployTenant] ${tenant}: _sc_plugins=${row ? "yes" : "NO"} _dd_env=${hasSvc ? "yes" : "NO"}`);
+        console.log(`[installDevDeployTenant] ${tenant}: _sc_plugins=${row ? "yes" : "NO"} _dd_ops=${hasTable ? "yes" : "NO"} dev_deploy_env=${hasEnv ? "yes" : "NO"}`);
-        if (!row || !hasSvc) {
+        if (!row || !hasTable || !hasEnv) {
             throw new Error("install verification failed for tenant " + tenant + " (onLoad did not run)");
         }
     });

test/e2e.js

@@ -933,16 +933,26 @@ const main = async () => {
         assert.ok(parseInt(c, 10) >= 1);
     });
 
-    scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name","REV_BEFORE");`);
+    // Markers are UNIQUE per run. The two setConfig calls run in separate scExec
-    scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name","REV_AFTER");`);
+    // processes that write the DB but NOT the running server's in-memory config
-    const cfgOp = sql(MAIN.db, "SELECT op_id FROM _dd_ops WHERE op_type='set_config' AND payload LIKE '%REV_AFTER%' ORDER BY created_at DESC LIMIT 1");
+    // cache. Saltcorn's setConfig skips the DB write when the worker's cached value
+    // already equals the target (state.ts), so a fixed "REV_BEFORE" can collide with
+    // a stale server-cached value and turn the revert into a silent no-op (the value
+    // never persists). A never-before-used marker can never equal the server cache,
+    // so the revert's setConfig always differs from cache and always persists.
+    const cfgStamp  = `${Date.now()}`;
+    const cfgBefore = `REV_BEFORE_${cfgStamp}`;
+    const cfgAfter  = `REV_AFTER_${cfgStamp}`;
+    scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name", ${JSON.stringify(cfgBefore)});`);
+    scExec(MAIN.env, `const {getState}=require("@saltcorn/data/db/state"); await getState().setConfig("site_name", ${JSON.stringify(cfgAfter)});`);
+    const cfgOp = sql(MAIN.db, `SELECT op_id FROM _dd_ops WHERE op_type='set_config' AND payload LIKE '%${cfgAfter}%' ORDER BY created_at DESC LIMIT 1`);
     const rcfg = adminPost(MAIN, MAIN_COOKIES, "/admin/dev-deploy/revert", { op_id: cfgOp });
     await test("revert set_config returns 302", async () => {
         assert.equal(rcfg.status, 302);
     });
-    await test("set_config revert restored site_name to REV_BEFORE", async () => {
+    await test("set_config revert restored site_name to the prior value", async () => {
         const v = sql(MAIN.db, "SELECT value FROM _sc_config WHERE key='site_name'");
-        assert.match(v, /REV_BEFORE/);
+        assert.match(v, new RegExp(cfgBefore));
     });
 
     scExec(MAIN.env, `const t = await Table.create("revrows"); await Field.create({table_id:t.id,name:"label",label:"Label",type:"String"});`);