sc-idp/docs/operations.md

Operations & Deployment

How to run, install, and operate the saltcorn-idp plugin in the local development setup. For protocol/internal details see the sibling docs: OIDC, LDAP, SAML, architecture, configuration.

All commands below assume the project root:

/home/scott/claude/saltcorn

The upstream Saltcorn checkout lives in saltcorn/ under that root; the plugin source lives in idp/. Each env.sh prepends saltcorn/packages/saltcorn-cli/bin to PATH so saltcorn ... resolves to the in-tree CLI.

The three dev instances

There are three parallel Saltcorn dev instances, each with its own start script, state directory, and env.sh. They are intentionally isolated (distinct ports, session stores, session secrets) so they can run at the same time.

Instance	Script	HTTP port	Backend	State dir	LDAPS port
MAIN	startServer.sh	3000 (default)	SQLite	.dev-state/	1636
TEST	startServerTest.sh	3001 (SALTCORN_PORT)	SQLite	.dev-state-test/	none
PG	startServerPg.sh	3002 (-p 3002)	Postgres, multi-tenant	.dev-state-pg/	1637

Each start script cds into its state directory before running saltcorn serve. This is deliberate: on SQLite, Saltcorn's session store writes sessions.sqlite at the process current working directory (packages/server/routes/utils.js, the db.isSQLite branch), so running from inside the state dir gives each SQLite instance its own sessions.sqlite alongside its saltcorn.sqlite. (PG uses a Postgres-backed session store; see below.)

What each env.sh sets

Common to all three: NVM_DIR (sources nvm to pick up Node), and the PATH prepend for the in-tree CLI.

MAIN -- .dev-state/env.sh

Variable	Value	Purpose
SQLITE_FILEPATH	.dev-state/saltcorn.sqlite	Forces the SQLite backend; DB file location
SALTCORN_FILE_STORE	.dev-state/files	Uploaded-file directory
SALTCORN_SESSION_SECRET	32552b95...410151	Session/cookie key; also the IdP KEK + oidc cookie-key source
SALTCORN_IDP_LDAP_PORT	1636	Override for the LDAPS listener port (and, by its presence, enables LDAP on 1636). Only MAIN sets this

LDAP is now configured from the public-site admin panel (see Configuring the LDAP listener below); these SALTCORN_IDP_LDAP_PORT / SALTCORN_IDP_LDAP_HOST env vars are now optional per-setting overrides, not the only way to enable LDAP.

TEST -- .dev-state-test/env.sh

Variable	Value	Purpose
SQLITE_FILEPATH	.dev-state-test/saltcorn.sqlite	TEST's own SQLite DB
SALTCORN_FILE_STORE	.dev-state-test/files	TEST's own file store
SALTCORN_SESSION_SECRET	cde4d5ce...86265	Different secret from MAIN
SALTCORN_PORT	3001	HTTP listen port (passed via saltcorn serve -p "$SALTCORN_PORT")

TEST does not set the SALTCORN_IDP_LDAP_PORT override and does not enable LDAP from the admin panel, so it runs no LDAP listener.

PG -- .dev-state-pg/env.sh

Variable	Value	Purpose
PGHOST	/var/run/postgresql	Unix socket; peer auth (no real password)
PGUSER	scott	OS user matched by peer auth
PGDATABASE	saltcorn_idp	DB name (must already exist)
PGPASSWORD	peer	Dummy value; peer auth ignores it, but Saltcorn only selects Postgres when user+password+database are all set (connect.ts getConnectObject)
SALTCORN_MULTI_TENANT	true	Enables schema-per-tenant (Postgres only)
SALTCORN_IDP_LDAP_PORT	1637	Override for the LDAPS listener port, distinct from MAIN's 1636 (and, by its presence, enables LDAP on 1637)
SALTCORN_FILE_STORE	.dev-state-pg/files	File store
SALTCORN_SESSION_SECRET	3ca4fab8...41a	Different secret from MAIN/TEST
SALTCORN_JWT_SECRET	d379db38...f158f	JWT signing secret

PG deliberately does not set SQLITE_FILEPATH, so Saltcorn's getConnectObject() selects Postgres.

Note: on Postgres the session store is Postgres-backed, not SQLite. Saltcorn only uses the connect-sqlite3 sessions.sqlite store when db.isSQLite (packages/server/routes/utils.js); on Postgres it uses connect-pg-simple with a _sc_session table, so no sessions.sqlite is written for the PG instance.

Starting and stopping

Start each instance from the project root:

./startServer.sh        # MAIN  -> http://localhost:3000  (LDAPS :1636)
./startServerTest.sh    # TEST  -> http://localhost:3001  (no LDAP)
./startServerPg.sh      # PG    -> http://localhost:3002  (LDAPS :1637)

On boot, MAIN and TEST run saltcorn install-plugin -d ./dev-deploy (the separate metadata-migration plugin) before serving; that install is per-instance safe because each uses its own source dir. Failures there are non-fatal -- the previously installed version still loads. startServerPg.sh does not run any install on boot.

saltcorn-idp is not installed during boot of any instance -- see the next section for why. Each instance loads it from a prior install.

Stop an instance with Ctrl-C (the scripts exec saltcorn serve, so the foreground process is the server). Stop all instances before re-installing the plugin (see below).

Installing the plugin

MAIN + TEST: reinstallIdp.sh

After editing idp/ source, reinstall into both SQLite instances with the dedicated script, then restart the servers:

./reinstallIdp.sh
./startServer.sh &
./startServerTest.sh &

reinstallIdp.sh sources each instance's env.sh in a subshell and runs saltcorn install-plugin -d "$IDP_DIR" once per instance, where IDP_DIR="$PWD/idp".

Why a separate script (not in startServer*.sh)

Three reasons, all in the script's header comment and code:

1. Absolute -d path required. saltcorn install-plugin path.join()s the -d argument and then require()s it. A leading ./ gets collapsed and is resolved as a node module instead of a filesystem path, so it fails. The script uses the absolute IDP_DIR="$PWD/idp" instead of ./idp.

2. EEXIST on existing node_modules symlinks. install-plugin aborts with EEXIST if the per-plugin-dir node_modules symlinks already exist (from a prior install). clearSymlinks() removes them before each install so they can be recreated cleanly:

   clearSymlinks() {
       find "$PLUGINS_ROOT" -maxdepth 2 -name node_modules -type l -delete 2>/dev/null || true
   }
   

   PLUGINS_ROOT is $HOME/.local/share/saltcorn-plugins.

3. Shared plugins_folder race. That plugins_folder is shared by both MAIN and TEST. Doing the install in each start script would race on symlink creation when both instances boot concurrently, so installs are centralized here. Run it with the servers stopped.

The additive-copy-does-not-prune gotcha

saltcorn install-plugin copies the idp/ source into the shared plugins_folder copy additively: new and modified files are copied, but files you have deleted from idp/ are not pruned from the copy. A stale file can linger and still be loaded. reinstallIdp.sh clears the node_modules symlinks but does not prune stale source files either. For a guaranteed clean slate, delete the plugin's directory under ~/.local/share/saltcorn-plugins/ before re-running the install.

PG (multi-tenant): per-tenant install

The Postgres instance installs the plugin per tenant schema. Two steps:

1. One-time, into the public schema (so the shared plugins_folder copy exists):

   source .dev-state-pg/env.sh
   saltcorn install-plugin -d ./idp
   

2. Per tenant (registers + enables the plugin in each tenant schema and runs its onLoad):

   ./idp/scripts/installIdpTenant.sh t1 t2     # named tenants
   ./idp/scripts/installIdpTenant.sh '*'       # all tenants
   

Prerequisites (from the script header): the tenants must already exist (saltcorn create-tenant <name>), and the public-schema install above must have run first.

installIdpTenant.sh cds to the project root, sources .dev-state-pg/env.sh, and runs idp/scripts/installIdpTenant.js with the tenant arguments.

What installIdpTenant.js does:

• Plugin.loadAllPlugins(), then resolves the target tenant list from process.argv (or all tenants via getAllTenants() when given * or no args), mapping each to its subdomain.
• init_multi_tenant(Plugin.loadAllPlugins, true, tenants) -- initializes per-tenant State (so getState() resolves inside runWithTenant) without running migrations; this also re-runs existing plugins' idempotent onLoad.
• getRootState().setConfig("tenants_unsafe_plugins", true) -- a root-only config that permits installing this local (-d) plugin into tenant schemas. In this Saltcorn build, loadAndSaveNewPlugin skips any non-npm plugin on a non-root tenant before its allowUnsafe argument is consulted, so this config is the supported lever; the CLI's install-plugin -t <tenant> -d <dir> cannot do it. It is intended for a multi-tenant deployment that offers the IdP plugin to its tenants.
• For each tenant, installInto(tenant) runs inside db.runWithTenant(tenant, ...) within a transaction:
  • db.deleteWhere("_sc_plugins", { name: "saltcorn-idp" }) first -- a delete-then-insert that removes any prior rows (including the old manual _sc_plugins SQL hack and earlier installs) so it converges on exactly one _sc_plugins row (one source of truth).
  • Creates a new Plugin({ name: "saltcorn-idp", source: "local", location: IDP_DIR, configuration: {} }) and calls Plugin.loadAndSaveNewPlugin(plugin, true, false).
  • Verifies by checking the _sc_plugins row exists and that _idp_ldap_service exists in the tenant's schema (information_schema.tables); throws if either is missing (meaning onLoad did not run).

On every subsequent boot of startServerPg.sh, per-tenant onLoad re-runs automatically via init_multi_tenant -> loadAllPlugins; this is idempotent (tables already exist, signing key already sealed, etc.), so no per-tenant reinstall is needed on a normal restart -- only after editing the plugin source.

Multi-tenant host routing

The PG instance is multi-tenant. The OIDC issuer is derived per request by issuerForReq() (lib/oidc/discovery.js): it prefers the tenant's configured base_url and falls back to req.protocol + "://" + req.get("host") when base_url is unset. The SAML entity ID is that issuer plus /saml (lib/saml/idp.js). See architecture and OIDC. In the dev/test setup the host conventions are:

• Subdomain selects the tenant. Saltcorn's multi-tenant mode uses a subdomain offset of 1 (packages/server/app.js), so the leading label of a host like t1.localhost.localdomain:3002 selects the tenant schema t1.
• Issuer comes from base_url (or the request host). There is no automatic host transform: the issuer is exactly base_url + "/idp" when base_url is set, otherwise <scheme>://<request-host>/idp. Whatever value results must match exactly what a relying party used to fetch /.well-known/openid-configuration, so set base_url per tenant.

For LDAP, the tenant is encoded in the bind/search DN as an extra dc=<tenant> component immediately before the base DN, e.g. uid=admin@t1.local,ou=people,dc=t1,dc=saltcorn,dc=local. Single-tenant (SQLite) uses the base DN with no tenant component. See LDAP.

Configuring the LDAP listener

The in-process LDAPS listener's enable flag, host, and port are configured from the admin panel on the public (root) site at /admin/idp/ldap. There is exactly one listener per Saltcorn instance, shared by all tenants, so these settings are instance-global: they live in the root/public tenant's _sc_config under the keys idp_ldap_enabled, idp_ldap_host, and idp_ldap_port. Tenant admins see them read-only; only a root-tenant admin can change them.

SALTCORN_IDP_LDAP_PORT and SALTCORN_IDP_LDAP_HOST are optional per-setting overrides -- when set, the env value wins for that setting, and the presence of the env port also still enables LDAP. They are never required. The dev env.sh files (MAIN 1636, PG 1637) set the port env var, so it acts as the override there; the admin panel is the supported way to enable/configure LDAP otherwise.

resolveRuntime() (lib/ldap/settings.js) computes the effective runtime settings with this precedence: env over config, defaulting the host to loopback (127.0.0.1), and treating LDAP as disabled unless it is both enabled and has a valid port. startLdap (lib/ldap/server.js) and the self-heal watchdog (index.js) consult resolveRuntime() instead of reading process.env directly.

Changes apply on restart. The binding process records the settings it actually applied in the root config as idp_ldap_applied ({ enabled, host, port }). The panel compares that against the resolved desired settings and shows a "restart Saltcorn to apply" reminder banner when they differ; the banner self-clears after a restart re-applies the settings.

Security: the panel is root-tenant + admin gated and its save is CSRF-protected; the port is validated to the 1-65535 range; the host defaults to loopback (127.0.0.1), and a network-exposure warning is shown when the host is set to a non-loopback address.

Known issues

Intermittent PG LDAP bind flake on :1637

On the Postgres multi-tenant instance, the cluster primary process occasionally does not bind the LDAPS listener on :1637 on a fresh boot. The listener is simply absent; LDAP authentication is unavailable for that run.

This is distinct from the handled EADDRINUSE/EACCES retry case. The LDAP server start path (lib/ldap/server.js, listenWithRetry) already retries transient bind failures up to LDAP_BIND_MAX_ATTEMPTS (5) with a linear backoff of LDAP_BIND_RETRY_BASE_MS (500ms) x attempt, binding only in the cluster primary (isPrimary()), and logs a loud LDAP ... UNAVAILABLE warning on final failure. The flake observed here is not that path -- it is the primary not binding :1637 at all on a fresh boot.

Workaround: restart the PG instance; the listener comes up on the next boot. Root cause is unresolved.

MAIN's LDAPS on :1636 (SQLite, single-tenant) is not known to exhibit this.