sc-idp/docs/operations.md

Operations & Deployment

How to run, install, and operate the saltcorn-idp plugin in the local development setup. For protocol/internal details see the sibling docs: OIDC, LDAP, SAML, architecture, configuration.

All commands below assume the project root:

/home/scott/claude/saltcorn

The upstream Saltcorn checkout lives in saltcorn/ under that root; the plugin source lives in idp/. Each env.sh prepends saltcorn/packages/saltcorn-cli/bin to PATH so saltcorn ... resolves to the in-tree CLI.

The three dev instances

There are three parallel Saltcorn dev instances, each with its own start script, state directory, and env.sh. They are intentionally isolated (distinct ports, session stores, session secrets) so they can run at the same time.

Instance	Script	HTTP port	Backend	State dir	LDAPS port
MAIN	startServer.sh	3000 (default)	SQLite	.dev-state/	1636
TEST	startServerTest.sh	3001 (SALTCORN_PORT)	SQLite	.dev-state-test/	none
PG	startServerPg.sh	3002 (-p 3002)	Postgres, multi-tenant	.dev-state-pg/	1637

Each start script cds into its state directory before running saltcorn serve. This is deliberate: on SQLite, Saltcorn's session store writes sessions.sqlite at the process current working directory (packages/server/routes/utils.js, the db.isSQLite branch), so running from inside the state dir gives each SQLite instance its own sessions.sqlite alongside its saltcorn.sqlite. (PG uses a Postgres-backed session store; see below.)

What each env.sh sets

Common to all three: NVM_DIR (sources nvm to pick up Node), and the PATH prepend for the in-tree CLI.

MAIN -- .dev-state/env.sh

Variable	Value	Purpose
SQLITE_FILEPATH	.dev-state/saltcorn.sqlite	Forces the SQLite backend; DB file location
SALTCORN_FILE_STORE	.dev-state/files	Uploaded-file directory
SALTCORN_SESSION_SECRET	32552b95...410151	Session/cookie key; also the IdP KEK + oidc cookie-key source
SALTCORN_IDP_LDAP_PORT	1636	Enables the LDAPS listener on 1636. Only MAIN sets this

TEST -- .dev-state-test/env.sh

Variable	Value	Purpose
SQLITE_FILEPATH	.dev-state-test/saltcorn.sqlite	TEST's own SQLite DB
SALTCORN_FILE_STORE	.dev-state-test/files	TEST's own file store
SALTCORN_SESSION_SECRET	cde4d5ce...86265	Different secret from MAIN
SALTCORN_PORT	3001	HTTP listen port (passed via saltcorn serve -p "$SALTCORN_PORT")

TEST does not set SALTCORN_IDP_LDAP_PORT, so it runs no LDAP listener.

PG -- .dev-state-pg/env.sh

Variable	Value	Purpose
PGHOST	/var/run/postgresql	Unix socket; peer auth (no real password)
PGUSER	scott	OS user matched by peer auth
PGDATABASE	saltcorn_idp	DB name (must already exist)
PGPASSWORD	peer	Dummy value; peer auth ignores it, but Saltcorn only selects Postgres when user+password+database are all set (connect.ts getConnectObject)
SALTCORN_MULTI_TENANT	true	Enables schema-per-tenant (Postgres only)
SALTCORN_IDP_LDAP_PORT	1637	LDAPS listener on 1637 (distinct from MAIN's 1636)
SALTCORN_FILE_STORE	.dev-state-pg/files	File store
SALTCORN_SESSION_SECRET	3ca4fab8...41a	Different secret from MAIN/TEST
SALTCORN_JWT_SECRET	d379db38...f158f	JWT signing secret

PG deliberately does not set SQLITE_FILEPATH, so Saltcorn's getConnectObject() selects Postgres.

Note: on Postgres the session store is Postgres-backed, not SQLite. Saltcorn only uses the connect-sqlite3 sessions.sqlite store when db.isSQLite (packages/server/routes/utils.js); on Postgres it uses connect-pg-simple with a _sc_session table, so no sessions.sqlite is written for the PG instance.

Starting and stopping

Start each instance from the project root:

./startServer.sh        # MAIN  -> http://localhost:3000  (LDAPS :1636)
./startServerTest.sh    # TEST  -> http://localhost:3001  (no LDAP)
./startServerPg.sh      # PG    -> http://localhost:3002  (LDAPS :1637)

On boot, MAIN and TEST run saltcorn install-plugin -d ./dev-deploy (the separate metadata-migration plugin) before serving; that install is per-instance safe because each uses its own source dir. Failures there are non-fatal -- the previously installed version still loads. startServerPg.sh does not run any install on boot.

saltcorn-idp is not installed during boot of any instance -- see the next section for why. Each instance loads it from a prior install.

Stop an instance with Ctrl-C (the scripts exec saltcorn serve, so the foreground process is the server). Stop all instances before re-installing the plugin (see below).

Installing the plugin

MAIN + TEST: reinstallIdp.sh

After editing idp/ source, reinstall into both SQLite instances with the dedicated script, then restart the servers:

./reinstallIdp.sh
./startServer.sh &
./startServerTest.sh &

reinstallIdp.sh sources each instance's env.sh in a subshell and runs saltcorn install-plugin -d "$IDP_DIR" once per instance, where IDP_DIR="$PWD/idp".

Why a separate script (not in startServer*.sh)

Three reasons, all in the script's header comment and code:

1. Absolute -d path required. saltcorn install-plugin path.join()s the -d argument and then require()s it. A leading ./ gets collapsed and is resolved as a node module instead of a filesystem path, so it fails. The script uses the absolute IDP_DIR="$PWD/idp" instead of ./idp.

2. EEXIST on existing node_modules symlinks. install-plugin aborts with EEXIST if the per-plugin-dir node_modules symlinks already exist (from a prior install). clearSymlinks() removes them before each install so they can be recreated cleanly:

   clearSymlinks() {
       find "$PLUGINS_ROOT" -maxdepth 2 -name node_modules -type l -delete 2>/dev/null || true
   }
   

   PLUGINS_ROOT is $HOME/.local/share/saltcorn-plugins.

3. Shared plugins_folder race. That plugins_folder is shared by both MAIN and TEST. Doing the install in each start script would race on symlink creation when both instances boot concurrently, so installs are centralized here. Run it with the servers stopped.

The additive-copy-does-not-prune gotcha

saltcorn install-plugin copies the idp/ source into the shared plugins_folder copy additively: new and modified files are copied, but files you have deleted from idp/ are not pruned from the copy. A stale file can linger and still be loaded. reinstallIdp.sh clears the node_modules symlinks but does not prune stale source files either. For a guaranteed clean slate, delete the plugin's directory under ~/.local/share/saltcorn-plugins/ before re-running the install.

PG (multi-tenant): per-tenant install

The Postgres instance installs the plugin per tenant schema. Two steps:

1. One-time, into the public schema (so the shared plugins_folder copy exists):

   source .dev-state-pg/env.sh
   saltcorn install-plugin -d ./idp
   

2. Per tenant (registers + enables the plugin in each tenant schema and runs its onLoad):

   ./idp/scripts/installIdpTenant.sh t1 t2     # named tenants
   ./idp/scripts/installIdpTenant.sh '*'       # all tenants
   

Prerequisites (from the script header): the tenants must already exist (saltcorn create-tenant <name>), and the public-schema install above must have run first.

installIdpTenant.sh cds to the project root, sources .dev-state-pg/env.sh, and runs idp/scripts/installIdpTenant.js with the tenant arguments.

What installIdpTenant.js does:

• Plugin.loadAllPlugins(), then resolves the target tenant list from process.argv (or all tenants via getAllTenants() when given * or no args), mapping each to its subdomain.
• init_multi_tenant(Plugin.loadAllPlugins, true, tenants) -- initializes per-tenant State (so getState() resolves inside runWithTenant) without running migrations; this also re-runs existing plugins' idempotent onLoad.
• getRootState().setConfig("tenants_unsafe_plugins", true) -- a root-only config that permits installing this local (-d) plugin into tenant schemas. In this Saltcorn build, loadAndSaveNewPlugin skips any non-npm plugin on a non-root tenant before its allowUnsafe argument is consulted, so this config is the supported lever; the CLI's install-plugin -t <tenant> -d <dir> cannot do it. It is intended for a multi-tenant deployment that offers the IdP plugin to its tenants.
• For each tenant, installInto(tenant) runs inside db.runWithTenant(tenant, ...) within a transaction:
  • db.deleteWhere("_sc_plugins", { name: "saltcorn-idp" }) first -- a delete-then-insert that removes any prior rows (including the old manual _sc_plugins SQL hack and earlier installs) so it converges on exactly one _sc_plugins row (one source of truth).
  • Creates a new Plugin({ name: "saltcorn-idp", source: "local", location: IDP_DIR, configuration: {} }) and calls Plugin.loadAndSaveNewPlugin(plugin, true, false).
  • Verifies by checking the _sc_plugins row exists and that _idp_ldap_service exists in the tenant's schema (information_schema.tables); throws if either is missing (meaning onLoad did not run).

On every subsequent boot of startServerPg.sh, per-tenant onLoad re-runs automatically via init_multi_tenant -> loadAllPlugins; this is idempotent (tables already exist, signing key already sealed, etc.), so no per-tenant reinstall is needed on a normal restart -- only after editing the plugin source.

Multi-tenant host routing

The PG instance is multi-tenant. The OIDC issuer is derived per request by issuerForReq() (lib/oidc/discovery.js): it prefers the tenant's configured base_url and falls back to req.protocol + "://" + req.get("host") when base_url is unset. The SAML entity ID is that issuer plus /saml (lib/saml/idp.js). See architecture and OIDC. In the dev/test setup the host conventions are:

• Subdomain selects the tenant. Saltcorn's multi-tenant mode uses a subdomain offset of 1 (packages/server/app.js), so the leading label of a host like t1.localhost.localdomain:3002 selects the tenant schema t1.
• Issuer comes from base_url (or the request host). There is no automatic host transform: the issuer is exactly base_url + "/idp" when base_url is set, otherwise <scheme>://<request-host>/idp. Whatever value results must match exactly what a relying party used to fetch /.well-known/openid-configuration, so set base_url per tenant.

For LDAP, the tenant is encoded in the bind/search DN as an extra dc=<tenant> component immediately before the base DN, e.g. uid=admin@t1.local,ou=people,dc=t1,dc=saltcorn,dc=local. Single-tenant (SQLite) uses the base DN with no tenant component. See LDAP.

Known issues

Intermittent PG LDAP bind flake on :1637

On the Postgres multi-tenant instance, the cluster primary process occasionally does not bind the LDAPS listener on :1637 on a fresh boot. The listener is simply absent; LDAP authentication is unavailable for that run.

This is distinct from the handled EADDRINUSE/EACCES retry case. The LDAP server start path (lib/ldap/server.js, listenWithRetry) already retries transient bind failures up to LDAP_BIND_MAX_ATTEMPTS (5) with a linear backoff of LDAP_BIND_RETRY_BASE_MS (500ms) x attempt, binding only in the cluster primary (isPrimary()), and logs a loud LDAP ... UNAVAILABLE warning on final failure. The flake observed here is not that path -- it is the primary not binding :1637 at all on a fresh boot.

Workaround: restart the PG instance; the listener comes up on the next boot. Root cause is unresolved.

MAIN's LDAPS on :1636 (SQLite, single-tenant) is not known to exhibit this.