204 lines
9.1 KiB
Markdown
204 lines
9.1 KiB
Markdown
# dev-deploy operations
|
|
|
|
Running and maintaining the local development instances for the dev-deploy
|
|
plugin. Three Saltcorn instances back development and testing: a SQLite MAIN
|
|
instance on :3000 (which also hosts the saltcorn-idp LDAPS listener), a SQLite
|
|
TEST instance on :3001, and a Postgres multi-tenant instance on :3002.
|
|
|
|
See also: [architecture.md](architecture.md), [peering.md](peering.md),
|
|
[multitenancy.md](multitenancy.md), [testing.md](testing.md).
|
|
|
|
## Contents
|
|
|
|
- [Layout](#layout)
|
|
- [The three dev instances](#the-three-dev-instances)
|
|
- [Environment files](#environment-files)
|
|
- [Starting the instances](#starting-the-instances)
|
|
- [Installing the plugin](#installing-the-plugin)
|
|
- [Per-tenant install on Postgres](#per-tenant-install-on-postgres)
|
|
- [Known issues](#known-issues)
|
|
|
|
## Layout
|
|
|
|
The project root is `~/claude/saltcorn`. The dev-deploy plugin source lives in
|
|
the `dev-deploy/` subfolder; the saltcorn-idp plugin source is a sibling in
|
|
`idp/`; upstream Saltcorn is checked out under `saltcorn/`. Each instance keeps
|
|
its database, file store, and session store under its own state directory in the
|
|
project root:
|
|
|
|
| Instance | Port | State dir | Database | LDAPS port |
|
|
| --- | --- | --- | --- | --- |
|
|
| MAIN | 3000 | `.dev-state/` | SQLite (`saltcorn.sqlite`) | 1636 |
|
|
| TEST | 3001 | `.dev-state-test/` | SQLite (`saltcorn.sqlite`) | none |
|
|
| PG (multi-tenant) | 3002 | `.dev-state-pg/` | Postgres `saltcorn_idp` | 1637 |
|
|
|
|
The start scripts `cd` into the state dir before running `saltcorn serve` so
|
|
each instance writes its own `sessions.sqlite` (Saltcorn's SQLite session store
|
|
writes `sessions.sqlite` at the process cwd, per
|
|
`packages/server/routes/utils.js`).
|
|
|
|
## The three dev instances
|
|
|
|
### MAIN (:3000, SQLite)
|
|
|
|
`startServer.sh` sources `.dev-state/env.sh`, runs `saltcorn install-plugin -d
|
|
./dev-deploy` (non-fatal on failure -- the previously installed version still
|
|
loads), then `cd .dev-state` and `exec saltcorn serve "$@"`. MAIN is the only
|
|
instance that sets `SALTCORN_IDP_LDAP_PORT=1636`, so saltcorn-idp opens its
|
|
LDAPS listener there. The other instances leave that port unset (PG sets 1637;
|
|
see below), so the three do not collide. saltcorn-idp is installed via
|
|
`reinstallIdp.sh` rather than in the start script (the shared plugins_folder
|
|
makes a per-boot install race on concurrent starts); it loads at serve time from
|
|
that install.
|
|
|
|
### TEST (:3001, SQLite)
|
|
|
|
`startServerTest.sh` sources `.dev-state-test/env.sh` (which sets
|
|
`SALTCORN_PORT=3001`), installs the plugin the same way, then
|
|
`cd .dev-state-test` and `exec saltcorn serve -p "$SALTCORN_PORT" "$@"`. TEST is
|
|
the promote/pull peer used by the e2e suite. It does not set
|
|
`SALTCORN_IDP_LDAP_PORT`, so no LDAP listener runs there.
|
|
|
|
### PG (:3002, Postgres multi-tenant)
|
|
|
|
`startServerPg.sh` sources `.dev-state-pg/env.sh`, then `cd .dev-state-pg` and
|
|
`exec saltcorn serve -p 3002 "$@"`. The PG env deliberately does NOT set
|
|
`SQLITE_FILEPATH`, so `getConnectObject()` selects Postgres; it sets
|
|
`SALTCORN_MULTI_TENANT=true` to enable schema-per-tenant (Postgres only). Unlike
|
|
the SQLite start scripts, `startServerPg.sh` does NOT run `install-plugin` at
|
|
boot -- the plugin is installed into the public schema by `reinstallDevDeploy.sh`
|
|
and activated per tenant by `installDevDeployTenant.sh` (see below). After
|
|
per-tenant activation, each tenant's `onLoad` re-runs automatically on every
|
|
boot via `init_multi_tenant` -> `loadAllPlugins`. The PG env also sets
|
|
`SALTCORN_IDP_LDAP_PORT=1637` (distinct from MAIN's 1636) so the multi-tenant
|
|
LDAP gate can exercise tenant-in-DN binds against this instance.
|
|
|
|
## Environment files
|
|
|
|
Each state dir has an `env.sh` that is sourced before running `saltcorn`. They
|
|
all prepend the in-tree CLI (`saltcorn/packages/saltcorn-cli/bin`) to `PATH` so
|
|
`saltcorn ...` resolves to this checkout, and load nvm.
|
|
|
|
| Variable | MAIN | TEST | PG |
|
|
| --- | --- | --- | --- |
|
|
| `SQLITE_FILEPATH` | `.dev-state/saltcorn.sqlite` | `.dev-state-test/saltcorn.sqlite` | unset (selects Postgres) |
|
|
| `SALTCORN_FILE_STORE` | `.dev-state/files` | `.dev-state-test/files` | `.dev-state-pg/files` |
|
|
| `SALTCORN_SESSION_SECRET` | set | set | set |
|
|
| `SALTCORN_PORT` | unset (defaults to 3000) | `3001` | unset (passed `-p 3002`) |
|
|
| `SALTCORN_IDP_LDAP_PORT` | `1636` | unset | `1637` |
|
|
| `SALTCORN_MULTI_TENANT` | unset | unset | `true` |
|
|
| `SALTCORN_JWT_SECRET` | unset | unset | set |
|
|
| `PGHOST` / `PGUSER` / `PGDATABASE` / `PGPASSWORD` | unset | unset | `/var/run/postgresql` / `scott` / `saltcorn_idp` / `peer` |
|
|
|
|
On PG, Saltcorn only selects Postgres when user, password, and database are all
|
|
set (`connect.ts` `getConnectObject`). Authentication is via the Unix socket
|
|
with peer auth, which ignores the password, so `PGPASSWORD=peer` is a dummy that
|
|
just satisfies that check.
|
|
|
|
## Starting the instances
|
|
|
|
Run from the project root:
|
|
|
|
```
|
|
cd ~/claude/saltcorn
|
|
./startServer.sh & # MAIN :3000 (+ LDAPS :1636)
|
|
./startServerTest.sh & # TEST :3001
|
|
./startServerPg.sh & # PG multi-tenant :3002 (+ LDAPS :1637)
|
|
```
|
|
|
|
The two SQLite start scripts each attempt `install-plugin -d ./dev-deploy` at
|
|
boot (failures are non-fatal). The PG instance does not; install it explicitly
|
|
as described below.
|
|
|
|
## Installing the plugin
|
|
|
|
After editing the plugin source, reinstall it into all three instances with the
|
|
servers stopped:
|
|
|
|
```
|
|
cd ~/claude/saltcorn
|
|
./reinstallDevDeploy.sh
|
|
```
|
|
|
|
`reinstallDevDeploy.sh` installs dev-deploy into MAIN (`.dev-state`), TEST
|
|
(`.dev-state-test`), and the PG public schema (`.dev-state-pg`). It is a separate
|
|
script (not folded into the start scripts) because the saltcorn plugins_folder
|
|
(`~/.local/share/saltcorn-plugins`) is shared by all instances, and `saltcorn
|
|
install-plugin`:
|
|
|
|
- needs an ABSOLUTE `-d` path (it `path.join()`s then `require()`s, so a leading
|
|
`./` is collapsed and resolved as a node module), and
|
|
- aborts (EEXIST) if the per-plugin-dir `node_modules` symlinks already exist.
|
|
|
|
Doing this in each start script would race when instances boot concurrently, so
|
|
reinstalls are centralized here. Before each install the script clears the
|
|
`node_modules` symlinks under the plugins root so `install-plugin` can recreate
|
|
them cleanly.
|
|
|
|
For the PG instance, installing into the public schema is only step one; you must
|
|
then activate the plugin per tenant (next section).
|
|
|
|
## Per-tenant install on Postgres
|
|
|
|
The public-schema install does not create the `_dd_*` tables in each tenant
|
|
schema. To register + enable dev-deploy in a tenant schema and run its `onLoad`
|
|
(creating the `_dd_*` tables and bootstrapping the env row), use:
|
|
|
|
```
|
|
./dev-deploy/scripts/installDevDeployTenant.sh t1 t2 # named tenants
|
|
./dev-deploy/scripts/installDevDeployTenant.sh '*' # all tenants
|
|
```
|
|
|
|
The shell wrapper resolves the project root, sources `.dev-state-pg/env.sh`, and
|
|
runs `installDevDeployTenant.js`. The JS uses Saltcorn's supported
|
|
`Plugin.loadAndSaveNewPlugin` inside `runWithTenant` (replacing the old manual
|
|
`INSERT INTO <tenant>._sc_plugins` SQL hack). It sets the root-only config
|
|
`tenants_unsafe_plugins=true` so a LOCAL plugin can be installed into tenant
|
|
schemas, converges each tenant to exactly one `_sc_plugins` row, and verifies the
|
|
install by confirming both the `_sc_plugins` row and the tenant-schema `_dd_env`
|
|
table exist. With no args or a single `*`, it installs into every tenant from the
|
|
public `_sc_tenants` list.
|
|
|
|
Prerequisites:
|
|
|
|
- the tenants must already exist (`saltcorn create-tenant <name>`), and
|
|
- the plugin must be installed into the PG public schema once (the
|
|
`reinstallDevDeploy.sh` path) so the shared plugins_folder copy exists.
|
|
|
|
After per-tenant activation, each tenant's `onLoad` re-runs automatically on
|
|
every boot of `startServerPg.sh`. The SQLite MAIN/:3000 and TEST/:3001 instances
|
|
are unaffected (they set `SQLITE_FILEPATH`). See
|
|
[multitenancy.md](multitenancy.md) for the schema-qualification details.
|
|
|
|
## Known issues
|
|
|
|
### Shared node_modules symlinks couple the two plugins
|
|
|
|
The `node_modules` symlinks under the plugins root
|
|
(`~/.local/share/saltcorn-plugins`) are SHARED across plugins. The
|
|
`clearSymlinks()` step in `reinstallDevDeploy.sh` therefore also clears
|
|
saltcorn-idp's symlinks. After running `reinstallDevDeploy.sh`, you MUST also
|
|
re-run `reinstallIdp.sh` (and vice versa) to keep both plugins' installs
|
|
consistent, then restart the servers. `reinstallIdp.sh` installs saltcorn-idp
|
|
into MAIN and TEST only (it does not touch the PG instance).
|
|
|
|
Recommended sequence after editing plugin source:
|
|
|
|
```
|
|
# stop the servers first
|
|
./reinstallDevDeploy.sh
|
|
./dev-deploy/scripts/installDevDeployTenant.sh '*' # PG only, if using tenants
|
|
./reinstallIdp.sh
|
|
# restart the servers
|
|
```
|
|
|
|
### Themeless freshly-created tenant returns 500 from sendWrap
|
|
|
|
A freshly created Postgres tenant has no theme/layout configured, so Saltcorn's
|
|
own admin pages that call `res.sendWrap` (for example `GET /table/new/`) return
|
|
HTTP 500. dev-deploy's admin pages self-render and are unaffected. When you need
|
|
a CSRF token for a Saltcorn admin POST against a themeless tenant, grab the token
|
|
from a dev-deploy page (for example `GET /admin/dev-deploy/peers`) instead -- the
|
|
mutate-then-redirect POST (such as `POST /table`) does not call `sendWrap` and so
|
|
succeeds. This is exploited throughout the PG gates; see
|
|
[testing.md](testing.md).
|