sc-idp/docs/testing.md

# Testing

The `saltcorn-idp` test suite is a set of nine end-to-end "gates", most of them
one per protocol surface. Each gate is a standalone Node script under `test/` that
drives a live, already-running Saltcorn instance over the wire (HTTP, LDAPS,
or SAML) and asserts on real responses. There is no mocking: a gate is a black
box client, so a passing gate means the deployed plugin actually behaves.

The gates assume the local dev instances described in
[operations.md](./operations.md) (MAIN on `:3000`, TEST on `:3001`, PG on
`:3002`). See the [README](../README.md) for the plugin overview.

## Conventions shared by all gates

- Each gate prints `PASS`/`FAIL` per check and ends with a
  `RESULT: <n> passed, <m> failed` line.
- Exit code: `0` when all checks pass (or a gate self-skips), `1` when any
  check fails, `2` on an uncaught error (a thrown exception, e.g. a server that
  is down and not handled by a self-skip).
- The admin credentials used throughout are `admin@local` /
  `AdminP@ss1` (single-tenant gates) or `admin@<tenant>.local` / `AdminP@ss1`
  (multi-tenant gates). The relying-party/SP redirect and callback targets are
  fixed test values (`http://localhost:9099/...`) and do not need a real
  listener there -- the gates only inspect the redirect/form, they never follow
  the callback.
- Several gates register their own client/SP/group via the admin UI
  (`/admin/idp/...`). Each does a delete-then-create at the start, so re-running
  a gate is safe even if a prior run left an artifact behind. `e2e.js` and
  `mtGate.js` also delete their clients again at the end; `samlGate.js` leaves
  its registered SP in place (the next run's delete-then-create absorbs it).

## The nine gates

| Gate file | Covers | Prerequisites (instance / port) | Backend | Self-skip when port unreachable | Run command |
|-----------|--------|---------------------------------|---------|---------------------------------|-------------|
| `test/e2e.js` | OIDC: discovery + JWKS on both instances (Phase 0 asserts `>= 1` RSA key, tolerating a retiring key during the grace window); client registration via admin UI; authorization-code + PKCE + consent; token + `id_token` RS256 verify; userinfo; groups claim (`role:admin` + custom `group:` claim); confidential-client one-time secret | MAIN `:3000` and TEST `:3001` both up | SQLite | No -- throws (exit 2) if an instance is down | `node test/e2e.js` (or `npm test`) |
| `test/jwksConsistencyGate.js` | Cross-worker JWKS consistency after rotation: rotate the signing key via the admin UI, then fetch `/idp/jwks` over many fresh connections and assert the new active kid appears in ALL responses and every cluster worker serves an identical key set (regression guard for a worker serving a stale JWKS keyed only by issuer; each worker's `oidc-provider` cache is now keyed on `(issuer, keys.signingSetFingerprint())` and rebuilds on mismatch) | MAIN `:3000` up with multiple cluster workers | SQLite | No -- throws (exit 2) if MAIN is down | `node test/jwksConsistencyGate.js` |
| `test/ldapGate.js` | LDAP (Phase 4): LDAPS simple bind vs bcrypt hash; authenticated user search (`mail`, `memberOf`); wrong-password + anonymous-search rejection; case-insensitive attribute selection; `groupOfNames` entries; filter-depth rejection; per-message inbound byte cap (DoS); configurable service account (search-then-bind); RFC 4514 DN escaping + `uidFromDn` unescape (unit) | MAIN `:3000` up AND its LDAPS listener on `:1636` | SQLite | No -- if LDAPS is down the bind checks fail (exit 1) rather than self-skip | `node test/ldapGate.js` |
| `test/samlGate.js` | SAML (Phase 5): IdP metadata; SP-initiated SSO (signed assertion + real `AuthnStatement`); SP registry + ACS allow-list (unregistered SP and out-of-allow-list ACS rejected, 403); DTD/ENTITY (XXE) rejection (400); decompression-bomb rejection (400); empty-ACS SP refused at registration; signed-SP AuthnRequest accepted + unsigned rejected (403) for a cert-registered SP; IdP-initiated SSO (no `InResponseTo`); Single Logout incl. no-session and wrong-NameID rejection (403) (LAST -- destroys the session) | MAIN `:3000` up | SQLite | No -- throws (exit 2) if MAIN is down | `node test/samlGate.js` |
| `test/keyRotationGate.js` | Key-rotation lifecycle invariants: an admin rotation mints a NEW active kid and KEEPS the prior key in JWKS (status `retiring`) for the grace window; the rotate endpoint is admin-gated; assertions are invariant-based (not absolute key counts) and all requests are pinned to one cluster worker via a keep-alive agent | MAIN `:3000` up | SQLite | No -- throws (exit 2) if MAIN is down | `node test/keyRotationGate.js` |
| `test/adminGate.js` | Admin defense-in-depth: `javascript:`/`data:` `redirect_uri` and out-of-scheme ACS rejected; an expired SP signing cert rejected at registration; per-session admin-mutation rate-limit returns HTTP 429 over the cap (this check runs LAST as it throttles the session) | MAIN `:3000` up | SQLite | No -- throws (exit 2) if MAIN is down | `node test/adminGate.js` |
| `test/mtGate.js` | Multi-tenant OIDC: distinct per-tenant issuers + signing keys; full auth-code + PKCE flow on `t1`; cross-tenant isolation -- a `t1` code, access token, and `id_token` are all REJECTED at `t2` (per-tenant store / Provider / key); positive control that the `t1` token still works at `t1` | PG `:3002` up, tenants `t1` + `t2` present with the plugin installed | Postgres | No explicit skip -- throws (exit 2) if PG/tenants are unavailable | `node test/mtGate.js` (or `npm run test:mt`) |
| `test/ldapMultiTenantGate.js` | Multi-tenant LDAP: tenant-in-DN binding (`dc=<tenant>,dc=saltcorn,dc=local`); correct-tenant bind + search; same uid under another tenant fails; cross-tenant search denied; unknown tenant denied; tenant-scoped `groupOfNames` member DNs. Bootstraps each tenant admin over HTTP first | PG `:3002` up AND its LDAPS listener on `:1637` | Postgres | Yes -- if `127.0.0.1:1637` is not reachable it prints `SKIP` and exits 0 (no-op on SQLite-only setups) | `node test/ldapMultiTenantGate.js` |
| `test/samlMultiTenantGate.js` | Multi-tenant SAML isolation: `t1` and `t2` serve DISTINCT signing certs + issuers in their IdP metadata; a `t1` SP-initiated Response embeds `t1`'s cert and validates against `t1`'s metadata-derived IdP but is REJECTED against `t2`'s | PG `:3002` up, tenants `t1` + `t2` with the plugin installed per tenant | Postgres | Yes -- prints `SKIP` and exits 0 if `:3002` is unreachable | `node test/samlMultiTenantGate.js` |

### Expected pass counts

When run against a correctly configured instance, each gate currently reports:

| Gate | Expected passes |
|------|-----------------|
| `test/e2e.js` | 28 |
| `test/jwksConsistencyGate.js` | 7 |
| `test/ldapGate.js` | 22 |
| `test/samlGate.js` | 28 |
| `test/keyRotationGate.js` | 8 |
| `test/adminGate.js` | 7 |
| `test/mtGate.js` | 15 |
| `test/ldapMultiTenantGate.js` | 8 |
| `test/samlMultiTenantGate.js` | 10 |
| **Total** | **133** |

A gate that self-skips (`ldapMultiTenantGate.js` and `samlMultiTenantGate.js` do
this) reports `0 passed, 0 failed (skipped)` and exits 0; that is not a failure,
but it also does not contribute its passes to the total.

### Ordering and shared state

Checks inside a gate run sequentially and share login/session state -- they are
not independent unit tests:

- `e2e.js` is organized as Phase 0..3 and reuses one admin session and cookie
  jar across phases. The Phase 1 client registration is a prerequisite for the
  Phase 1/2 auth-code flows.
- `samlGate.js` logs in once, registers the test SP, then runs SSO checks; the
  Single Logout check is intentionally LAST because it destroys the session.
- `mtGate.js` keeps a separate cookie jar per tenant and depends on the
  positive `t1` flow having produced a token before the cross-tenant rejection
  checks run.
- `ldapMultiTenantGate.js` bootstraps both tenant admins over HTTP before any
  LDAP bind.

Do not reorder checks within a gate.

## npm scripts

From `package.json`:

| Script | Command | Gate |
|--------|---------|------|
| `npm test` | `node test/e2e.js` | OIDC e2e gate (MAIN + TEST, SQLite) |
| `npm run test:mt` | `node test/mtGate.js` | multi-tenant OIDC gate (PG) |

The remaining seven gates (`ldapGate.js`, `samlGate.js`, `keyRotationGate.js`,
`jwksConsistencyGate.js`, `adminGate.js`, `ldapMultiTenantGate.js`,
`samlMultiTenantGate.js`) have no npm script; run them directly with `node` as
shown in the table above.

## Running the full suite

The PG gates self-skip or fail fast when their backend is unreachable, so which
servers you start determines which gates can pass. See
[operations.md](./operations.md) for how to start each instance and install the
plugin.

1. **SQLite gates (`e2e`, `ldapGate`, `samlGate`, `keyRotationGate`,
   `jwksConsistencyGate`, `adminGate`).** Start MAIN and TEST:

   ```bash
   ./startServer.sh        # MAIN -> :3000  (LDAPS :1636)
   ./startServerTest.sh    # TEST -> :3001  (no LDAP)
   ```

   - `e2e.js` needs both MAIN and TEST (Phase 0 checks discovery/JWKS on each).
   - `ldapGate.js` needs MAIN plus its LDAPS listener on `:1636`. The listener
     host/port/enable are configured from the admin panel (root tenant) at
     `/admin/idp/ldap`; MAIN's `env.sh` sets `SALTCORN_IDP_LDAP_PORT=1636` as a
     dev override (env wins per-setting, and the env port also enables LDAP), so
     the dev instance comes up on `:1636` without touching the panel.
   - `samlGate.js` needs MAIN only.
   - `keyRotationGate.js` needs MAIN only.
   - `jwksConsistencyGate.js` needs MAIN only, and is meaningful only when MAIN
     runs more than one cluster worker (it rotates the signing key and asserts
     every worker converges on the new JWKS).
   - `adminGate.js` needs MAIN only; run it LAST, because its rate-limit check
     deliberately throttles the admin session (a 429), which would interfere with
     other admin-driven gates run after it in the same session.

   Then:

   ```bash
   npm test                          # e2e
   node test/ldapGate.js
   node test/samlGate.js
   node test/keyRotationGate.js
   node test/jwksConsistencyGate.js
   node test/adminGate.js            # LAST -- throttles the admin session
   ```

2. **Postgres gates (`mtGate`, `ldapMultiTenantGate`, `samlMultiTenantGate`).**
   Start PG and make sure tenants `t1` and `t2` exist with the plugin installed
   per tenant (see operations.md, "PG (multi-tenant): per-tenant install"):

   ```bash
   ./startServerPg.sh      # PG -> :3002  (LDAPS :1637)
   ```

   Then:

   ```bash
   npm run test:mt                       # mtGate (HTTP :3002)
   node test/ldapMultiTenantGate.js      # PG LDAPS :1637
   node test/samlMultiTenantGate.js      # PG SAML isolation (HTTP :3002)
   ```

   `ldapMultiTenantGate.js` probes `127.0.0.1:1637` first and self-skips
   (exit 0) if the LDAPS listener is not up -- which is also the workaround
   surface for the intermittent `:1637` bind flake documented in
   operations.md. `mtGate.js` has no self-skip and will exit 2 if PG or the
   tenants are not available, so only run it once PG is confirmed up.

If you do not need the multi-tenant coverage, running only the SQLite gates is
a valid quick pass.