AI_Slop/dev-deploy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
dev-deploy/README.md
Scott Duensing 60409b9014 Docs updated.
2026-05-17 17:38:33 -05:00

9.1 KiB
Raw Permalink Blame History

saltcorn — dev environment for the dev-deploy plugin

This project root holds the dev-deploy Saltcorn plugin and the launcher scripts + per-instance state needed to develop and test it against two local Saltcorn instances. Upstream Saltcorn lives in saltcorn/ (a sibling git checkout managed independently from this project).

Layout

saltcorn/                    project root (this git repo)
├── dev-deploy/              the plugin (Saltcorn plugin, sc_plugin_api_version 1)
├── startServer.sh           launch MAIN instance on :3000
├── startServerTest.sh       launch TEST instance on :3001
├── devServer.sh             launch MAIN under `saltcorn dev:serve` (tsc watch)
├── installSaltcorn.sh       reproduce this dev environment from scratch
├── .dev-state/              MAIN instance: env.sh, saltcorn.sqlite, files/, sessions
├── .dev-state-test/         TEST instance: same shape, separate port + secret
└── saltcorn/                upstream Saltcorn checkout (its own .git, gitignored here)

The upstream subfolder, both .dev-state* directories, and node_modules/ are all gitignored — only the plugin + scripts + this README are tracked.

Prerequisites

  • git, curl
  • nvm (auto-installed at ~/.nvm if missing) → Node 20
  • git-lfs for the .gitattributes LFS filters (git lfs install once per clone)

installSaltcorn.sh handles nvm + Node 20 install for you.

Fresh install

./installSaltcorn.sh [destination]    # default: ./saltcorn

Clones upstream Saltcorn into <dest>/saltcorn/, runs npm install + npm run tsc, generates <dest>/.dev-state/env.sh with a fresh session secret, initializes the SQLite schema, creates the admin user, and writes <dest>/startServer.sh.

The script sets up the MAIN instance only. To add the TEST instance, duplicate .dev-state/ as .dev-state-test/, edit its env.sh to use a new SALTCORN_SESSION_SECRET and add export SALTCORN_PORT="3001", then:

source .dev-state-test/env.sh
saltcorn reset-schema -f
saltcorn create-user -a -e admin@local -p AdminP@ss1

The reference .dev-state-test/env.sh already in this repo shows the exact pattern.

Running

./startServer.sh        # MAIN  → http://localhost:3000/
./startServerTest.sh    # TEST  → http://localhost:3001/
./devServer.sh          # MAIN under `saltcorn dev:serve` (nodemon + tsc watch)

Login on either instance: admin@local / AdminP@ss1.

Each startServer*.sh runs saltcorn install-plugin -d ./dev-deploy on every boot, so edits to dev-deploy/ go live on the next restart.

The dev-deploy plugin

dev-deploy/ migrates Saltcorn metadata (tables, fields, views, pages, triggers, roles, library, tags, constraints, files, page groups, workflow steps, plus selected config keys and plugin configuration) across Dev/Test/Prod environments via an append-only ops journal with stable cross-environment UUIDs and HMAC-authenticated peer endpoints. Concurrent edits surface as conflicts in the admin UI with theirs/mine/per-field-merge resolution. User row data is left alone unless an admin explicitly marks a table as managed or starter.

Admin UI: /admin/dev-deploy/ (logged in as an admin). Machine API: /dev-deploy/api/{journal,ingest,file/:uuid,health} (HMAC-signed peer requests).

Pairing two instances

Do this once per pair (e.g. MAIN ↔ TEST). Each side needs the other's env_id (a UUID shown on its /admin/dev-deploy/ dashboard) and base URL.

  1. On instance A — go to /admin/dev-deploy/peers. Under "Add peer", fill in B's env_id, a label, B's base URL, and leave "Existing secret" blank. Click Pair. A's response shows a 64-hex shared secret once — copy it.
  2. On instance B — /admin/dev-deploy/peers → "Add peer". Fill in A's env_id + base URL, paste the secret from step 1 into "Existing secret", click Pair.

Both sides now have a peer row with a sealed copy of the shared secret. The plain secret is never persisted on either side — only the AES-256-GCM ciphertext (KEK derived from SALTCORN_SESSION_SECRET). Use Rotate on either side to roll the secret (the peer page surfaces the new one once; paste it on the other side via Rotate).

Promoting changes (source → target)

  1. Make metadata edits on the source (e.g. create a table on MAIN).
  2. Go to /admin/dev-deploy/plan, pick the target peer, click Show plan. Lists the ops that would be sent (everything since the last outbound anchor for that peer).
  3. Click Promote N ops. The target applies them, advances the outbound anchor, and returns per-op results (applied / error / conflict counts in the redirect flash).

Plugin-list mismatches between source and target are surfaced as warnings in the same flash message (e.g. "peer missing plugin X", "plugin version mismatch on Y"). They don't block the promote — they're advisory.

Pulling changes (target ← source)

On the target, /admin/dev-deploy/peers → Pull on the source's peer row. Fetches everything since the last inbound anchor and applies it. If any op conflicts with a local op on the same entity since the last sync, the incoming op is journaled with status='conflict' and not applied; the flash redirects to /admin/dev-deploy/conflicts.

Resolving conflicts

A conflict means both sides edited the same entity since the last sync. On the target side at /admin/dev-deploy/conflicts, each pending conflict shows the incoming op (theirs) next to the local op (mine):

  • Use theirs — apply the incoming op now, overwriting the local change. The local op stays in the journal but its effect is gone.
  • Use mine — mark the incoming op rejected. Local state stands. The peer keeps trying to ship the op on future pulls; we skip it by op_id.
  • Merge per field (only for update_X vs update_X conflicts) — opens a form with each diverging field. Pick keep current, take incoming, or type a custom value per field. Submitting marks the conflict merged.

Per-table data mode

By default, table rows are local-only (user mode) and dev-deploy only migrates the table's schema. Change this per-table at /admin/dev-deploy/tables:

  • user — rows belong to each environment; never synced. The Saltcorn users table is hard-locked to this mode.
  • starter — on the first promote, ships current rows once. The target then owns them; future row changes on the source don't propagate.
  • managed — rows always sync from source. Source is canonical; target's row edits get overwritten or surface as row conflicts.

Switching to managed or starter adds a hidden _dd_row_uuid column to the table via raw ALTER (not registered in _sc_fields, so Saltcorn's table builder doesn't show it) and ships current rows in the next promote. Switching back to user drops the column (best-effort; older SQLite without DROP COLUMN support will error). FKs to user-mode tables are NULL'd on the target with a warning attached to the op.

Reverting an op

The journal at /admin/dev-deploy/ops shows every op with a Revert button. Revert appends a compensating op (create_X → drop; drop_X → recreate from the captured before-snapshot; update_X → re-apply before-snapshot as a patch) rather than rewriting history. The inverse op promotes to peers like any other op on the next push. Reverting a drop produces a new entity with a fresh UUID — content is restored, identity is not.

Tests

# Both servers must be running first.
./startServer.sh & ./startServerTest.sh &
cd dev-deploy && npm test

Runs dev-deploy/test/e2e.js — ~50+ end-to-end tests covering pairing, promote, pull, conflict resolution, constraints, files, page groups, workflow steps, config propagation, managed/starter row data, revert, and machine-endpoint security. Tests run in order and share state; don't reorder.

test/sc-exec.js is a shim used by the tests to run JS against Saltcorn's models with full require() access (saltcorn's built-in run-js uses a vm sandbox that hides Field/TableConstraint/File/etc.).

Notes

  • Per-instance sessions.sqlite. Saltcorn's SQLite session store writes sessions.sqlite at process cwd (packages/server/routes/utils.js). startServer.sh and startServerTest.sh cd into their respective state directory before exec saltcorn serve, so each instance gets its own sessions DB. devServer.sh cannot do this (its dev:serve runs npm run tsc which needs cwd=upstream root), so its sessions land in saltcorn/sessions.sqlite — don't run devServer.sh and startServer.sh against the MAIN instance simultaneously.

  • SALTCORN_SESSION_SECRET is the pairing identity. dev-deploy derives its at-rest encryption key (KEK) for peer secrets via HKDF from this value. Rotating it invalidates every peer pairing on the instance. The secret is generated once per instance and persisted in that instance's env.sh.

  • Upstream Saltcorn updates. Pull from upstream via git -C saltcorn pull — the project root is a separate repo and doesn't see those commits. After a pull, you may need npm install && npm run tsc inside saltcorn/.