# saltcorn — dev environment for the dev-deploy plugin

This project root holds the `dev-deploy` Saltcorn plugin and the launcher
scripts + per-instance state needed to develop and test it against two local
Saltcorn instances. Upstream Saltcorn lives in `saltcorn/` (a sibling git
checkout managed independently from this project).

## Layout

```
saltcorn/                    project root (this git repo)
├── dev-deploy/              the plugin (Saltcorn plugin, sc_plugin_api_version 1)
├── startServer.sh           launch MAIN instance on :3000
├── startServerTest.sh       launch TEST instance on :3001
├── devServer.sh             launch MAIN under `saltcorn dev:serve` (tsc watch)
├── installSaltcorn.sh       reproduce this dev environment from scratch
├── .dev-state/              MAIN instance: env.sh, saltcorn.sqlite, files/, sessions
├── .dev-state-test/         TEST instance: same shape, separate port + secret
└── saltcorn/                upstream Saltcorn checkout (its own .git, gitignored here)
```

The upstream subfolder, both `.dev-state*` directories, and `node_modules/`
are all gitignored — only the plugin + scripts + this README are tracked.

## Prerequisites

- `git`, `curl`
- `nvm` (auto-installed at `~/.nvm` if missing) → Node 20
- `git-lfs` for the `.gitattributes` LFS filters (`git lfs install` once per clone)

`installSaltcorn.sh` handles nvm + Node 20 install for you.

## Fresh install

```bash
./installSaltcorn.sh [destination]    # default: ./saltcorn
```

Clones upstream Saltcorn into `<dest>/saltcorn/`, runs `npm install` + `npm
run tsc`, generates `<dest>/.dev-state/env.sh` with a fresh session secret,
initializes the SQLite schema, creates the admin user, and writes
`<dest>/startServer.sh`.

The script sets up the **MAIN** instance only. To add the **TEST** instance,
duplicate `.dev-state/` as `.dev-state-test/`, edit its `env.sh` to use a new
`SALTCORN_SESSION_SECRET` and add `export SALTCORN_PORT="3001"`, then:

```bash
source .dev-state-test/env.sh
saltcorn reset-schema -f
saltcorn create-user -a -e admin@local -p AdminP@ss1
```

The reference `.dev-state-test/env.sh` already in this repo shows the exact
pattern.

## Running

```bash
./startServer.sh        # MAIN  → http://localhost:3000/
./startServerTest.sh    # TEST  → http://localhost:3001/
./devServer.sh          # MAIN under `saltcorn dev:serve` (nodemon + tsc watch)
```

Login on either instance: `admin@local` / `AdminP@ss1`.

Each `startServer*.sh` runs `saltcorn install-plugin -d ./dev-deploy` on every
boot, so edits to `dev-deploy/` go live on the next restart.

## The dev-deploy plugin

`dev-deploy/` migrates Saltcorn metadata (tables, fields, views, pages,
triggers, roles, library, tags, constraints, files, page groups, workflow
steps, plus selected config keys and plugin configuration) across Dev/Test/Prod
environments via an append-only ops journal with stable cross-environment
UUIDs and HMAC-authenticated peer endpoints. Concurrent edits surface as
conflicts in the admin UI with theirs/mine/per-field-merge resolution. User
row data is left alone unless an admin explicitly marks a table as `managed`
or `starter`.

Admin UI: `/admin/dev-deploy/` (logged in as an admin). Machine API:
`/dev-deploy/api/{journal,ingest,file/:uuid,health}` (HMAC-signed peer requests).

### Pairing two instances

Do this once per pair (e.g. MAIN ↔ TEST). Each side needs the other's
`env_id` (a UUID shown on its `/admin/dev-deploy/` dashboard) and base URL.

1. **On instance A** — go to `/admin/dev-deploy/peers`. Under "Add peer", fill
   in B's `env_id`, a label, B's base URL, and leave "Existing secret" blank.
   Click **Pair**. A's response shows a 64-hex shared secret **once** — copy it.
2. **On instance B** — `/admin/dev-deploy/peers` → "Add peer". Fill in A's
   `env_id` + base URL, paste the secret from step 1 into "Existing secret",
   click **Pair**.

Both sides now have a peer row with a sealed copy of the shared secret. The
plain secret is never persisted on either side — only the AES-256-GCM
ciphertext (KEK derived from `SALTCORN_SESSION_SECRET`). Use **Rotate** on
either side to roll the secret (the peer page surfaces the new one once;
paste it on the other side via Rotate).

### Promoting changes (source → target)

1. Make metadata edits on the source (e.g. create a table on MAIN).
2. Go to `/admin/dev-deploy/plan`, pick the target peer, click **Show plan**.
   Lists the ops that would be sent (everything since the last outbound
   anchor for that peer).
3. Click **Promote N ops**. The target applies them, advances the outbound
   anchor, and returns per-op results (applied / error / conflict counts in
   the redirect flash).

Plugin-list mismatches between source and target are surfaced as warnings in
the same flash message (e.g. "peer missing plugin X", "plugin version mismatch
on Y"). They don't block the promote — they're advisory.

### Pulling changes (target ← source)

On the target, `/admin/dev-deploy/peers` → **Pull** on the source's peer row.
Fetches everything since the last inbound anchor and applies it. If any op
conflicts with a local op on the same entity since the last sync, the incoming
op is journaled with `status='conflict'` and **not** applied; the flash
redirects to `/admin/dev-deploy/conflicts`.

### Resolving conflicts

A conflict means both sides edited the same entity since the last sync. On
the target side at `/admin/dev-deploy/conflicts`, each pending conflict shows
the incoming op (theirs) next to the local op (mine):

- **Use theirs** — apply the incoming op now, overwriting the local change.
  The local op stays in the journal but its effect is gone.
- **Use mine** — mark the incoming op `rejected`. Local state stands. The
  peer keeps trying to ship the op on future pulls; we skip it by op_id.
- **Merge per field** (only for `update_X` vs `update_X` conflicts) — opens
  a form with each diverging field. Pick **keep current**, **take incoming**,
  or type a custom value per field. Submitting marks the conflict `merged`.

### Per-table data mode

By default, table **rows** are local-only (`user` mode) and dev-deploy only
migrates the table's schema. Change this per-table at `/admin/dev-deploy/tables`:

- **user** — rows belong to each environment; never synced. The Saltcorn
  `users` table is hard-locked to this mode.
- **starter** — on the first promote, ships current rows once. The target
  then owns them; future row changes on the source don't propagate.
- **managed** — rows always sync from source. Source is canonical; target's
  row edits get overwritten or surface as row conflicts.

Switching to `managed` or `starter` adds a hidden `_dd_row_uuid` column to
the table via raw `ALTER` (not registered in `_sc_fields`, so Saltcorn's
table builder doesn't show it) and ships current rows in the next promote.
Switching back to `user` drops the column (best-effort; older SQLite without
DROP COLUMN support will error). FKs to user-mode tables are NULL'd on the
target with a warning attached to the op.

### Reverting an op

The journal at `/admin/dev-deploy/ops` shows every op with a **Revert** button.
Revert appends a compensating op (`create_X` → drop; `drop_X` → recreate from
the captured before-snapshot; `update_X` → re-apply before-snapshot as a
patch) rather than rewriting history. The inverse op promotes to peers like
any other op on the next push. Reverting a `drop` produces a new entity with
a fresh UUID — content is restored, identity is not.

## Tests

```bash
# Both servers must be running first.
./startServer.sh & ./startServerTest.sh &
cd dev-deploy && npm test
```

Runs `dev-deploy/test/e2e.js` — ~50+ end-to-end tests covering pairing,
promote, pull, conflict resolution, constraints, files, page groups,
workflow steps, config propagation, managed/starter row data, revert, and
machine-endpoint security. Tests run in order and share state; don't reorder.

`test/sc-exec.js` is a shim used by the tests to run JS against Saltcorn's
models with full `require()` access (saltcorn's built-in `run-js` uses a vm
sandbox that hides Field/TableConstraint/File/etc.).

## Notes

- **Per-instance `sessions.sqlite`.** Saltcorn's SQLite session store writes
  `sessions.sqlite` at process cwd (`packages/server/routes/utils.js`).
  `startServer.sh` and `startServerTest.sh` `cd` into their respective state
  directory before `exec saltcorn serve`, so each instance gets its own
  sessions DB. `devServer.sh` cannot do this (its `dev:serve` runs `npm run
  tsc` which needs cwd=upstream root), so its sessions land in
  `saltcorn/sessions.sqlite` — don't run `devServer.sh` and `startServer.sh`
  against the MAIN instance simultaneously.

- **`SALTCORN_SESSION_SECRET` is the pairing identity.** dev-deploy derives
  its at-rest encryption key (KEK) for peer secrets via HKDF from this value.
  Rotating it invalidates every peer pairing on the instance. The secret is
  generated once per instance and persisted in that instance's `env.sh`.

- **Upstream Saltcorn updates.** Pull from upstream via
  `git -C saltcorn pull` — the project root is a separate repo and doesn't
  see those commits. After a pull, you may need `npm install && npm run tsc`
  inside `saltcorn/`.