AI_Slop/DVX_GUI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
DVX_GUI/server/README.md

9.7 KiB
Raw Blame History

DVX package server

The catalogue of add-on packages (.DPK files) that DVX's Software program browses, installs, and updates from, and where publishers upload them. It is a calog application over SQLite, built on singemaster's shared modules: accounts, sessions, the log, mail, keys, the blob store, and lib/service.lua's route wrapper, account routes, and operator checks. The two services can run in one calog process.

cp dvx.conf.example dvx.conf           # and edit it
calog dvxserver.lua -- dvx.conf        # in this folder

The server finds singemaster at ../../singemaster and calog at ../../calog from its own folder, wherever calog is started; singe=DIR and calog=DIR arguments say where they are when they are elsewhere.

Both services in one process

calog ../../singemaster/master.lua dvxserver.lua -- singe=../../singemaster master=master.conf dvx=dvx.conf

calogArgs() is one list for every script in a run, so each service takes its own configuration by name (master=, dvx=). Each listens on its own bindPort with its own database, and the shared modules are loaded once per service. test/run.sh runs both this way.

How it is deployed

As singemaster is: plain HTTP bound to loopback, behind a reverse proxy that answers for the public name. One difference matters: DVX's web library speaks plain HTTP only (there is no TLS on a DOS machine), so the proxy must answer DVX's requests over plain HTTP as well as HTTPS. That costs no trust, because no package is trusted for the way it arrived: DVX installs a package only when the repository's signature of it checks against the repository key DVX was installed with.

A client that opens a connection and then stalls holds one serving context until it gives up, so the proxy should time out slow requests (client_header_timeout and client_body_timeout in nginx) and buffer responses, which also frees a context as soon as a slow download is handed to the proxy.

Trust

  • The repository key stays offline. The service holds only the repository's public key (repoKey). The operator signs an approved package on their own computer (tools/review.lua), and the service checks that signature with the package tool before it publishes the version. A stolen server can withhold or withdraw packages, but it cannot make one DVX will install.
  • Every upload is checked by the package tool (dvxpkg check, the same code DVX installs with): the manifest, every file against it, and the rules for where each kind of package may put its files. Nothing a manifest says is taken on the uploader's word.
  • Publishers may sign their own packages with their own keys (dvxpkg keygen, or the Package Builder). The catalogue shows each version's publisher key, if it has one, so a user can tell two publishers apart.
  • An id belongs to the account that first uploaded it. Another account's upload of the same id is refused, as is a second upload of a version already taken. Ids of the vendor kpunch are DVX's own and only the operator uploads them.

The API

All JSON except the downloads. Every reply carries X-Request-Id, which is also in the log.

Method Path What it does
GET /v1/health Liveness and the schema version.
GET /v1/catalogue Every published package at its newest approved version: id, version, name, kind, summary, publisher, author, licence, homepage, dvx, abi, size, sha256, signer, signed, and depends ([{id, min}]). No session.
GET /v1/package/<id> One package's approved versions, newest first.
GET /v1/download/<id>/<version> The bytes, with X-Content-Sha256. Resumable: Range is answered.
GET /v1/signature/<id>/<version> The repository's signature of that version (a NAME.SIG file).
POST /v1/updates {dvx, abi, installed: [{id, version}]}, the client's own list: answers updates (the newer versions) and unknown (ids the catalogue does not have). Nothing of it is kept.
POST /v1/upload The .DPK as the body, from a signed-in account. Taken in as a pending version for review. DVX's Software sends it (File > Publish a Package) with the account its settings give.
GET /v1/mine The account's own packages, every version with its review state and note.
GET /v1/admin/pending Operator. Versions waiting for a review, oldest first.
GET /v1/admin/download/<id>/<version> Operator. Any version's bytes, to review and sign.
POST /v1/admin/version/state Operator. {package, version, state: approved or rejected, signature, note}. Approving needs the repository's signature of the version.
POST /v1/admin/package/state Operator. {package, state: published or withdrawn}.
GET /v1/admin/log Operator. The log, searched by any field (as singemaster's).
POST /v1/admin/key/grant, /v1/admin/key/revoke Operator. {account, key}; the key admin makes an administrator.

The account routes (/v1/account/register, verify, login, logout, recover, reset, and GET /v1/account) are singemaster's, with mail that names DVX.

The operator is the console token (adminToken) or the session of an account holding the admin key. Withdrawing a package stops it being listed and downloaded; the copies installed keep working and simply stop being offered updates.

Serving contexts

A DOS machine downloads slowly. The main context listens and does the housekeeping (queued mail, expired accounts); serveContexts serving contexts, each with its own database connection and routes, accept on its listener, so one slow download holds up only the context sending it. Each connection goes to one of them; with every one busy, new connections wait in the system's queue until one is free.

Reviewing uploads

tools/review.lua runs on the operator's own computer, the one place the repository's secret key is kept. It needs singemaster's lib/config.lua (singe=DIR when singemaster is not beside this repository) and the package tool from a DVX build or the SDK.

calog tools/review.lua -- review.conf pending
calog tools/review.lua -- review.conf fetch acme.cards 1.0.0
calog tools/review.lua -- review.conf approve acme.cards 1.0.0 "looks fine"
calog tools/review.lua -- review.conf reject acme.cards 1.1.0 "the help file is missing"

review.conf holds server (the service's public address), token (the console token or an administrator's session), dvxpkg, secret (the repository's secret key), and work (where fetched packages and their signatures are kept). fetch checks the download against the digest the service sent and shows the manifest and files as the package tool reads them. approve signs the fetched file itself, so what is approved is exactly what was reviewed; if that file has changed since, the service refuses the signature. withdraw and publish take a package off the catalogue and put it back.

Publishing DVX itself

make in the DVX tree writes dist/DVXUPD.DPK: DVX itself as the package kpunch.dvx (kind system), every file of the release but the optional fonts, at this DVX's version and ABI. Packages of the vendor kpunch are the operator's, so it is uploaded with the console token and approved like any other:

curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/octet-stream" \
     --data-binary @dist/DVXUPD.DPK https://packages.example.com/v1/upload
calog tools/review.lua -- review.conf approve kpunch.dvx 1.0.1

Software then lists it first on its Updates page. DVX installs it only when the repository's signature checks against the key in its own SYSTEM\KEYS, and names the add-ons a new ABI would leave unable to load before anything is installed.

The same build writes DVX's own add-ons to dist/addons/ (src/addons/README.md): instrument banks for MIDI music, kept out of the base install. They are uploaded and approved the same way:

for dpk in dist/addons/*.DPK; do
    curl -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/octet-stream" \
         --data-binary @"$dpk" https://packages.example.com/v1/upload
done
calog tools/review.lua -- review.conf approve kpunch.dmxopl 1.11.0
calog tools/review.lua -- review.conf approve kpunch.dmxopl3 2.11.0

Configuration

dvx.conf.example lists every key with a comment. Beyond singemaster's (bindHost, bindPort, publicOrigin, dataDir, database, pbkdf2Iterations, mailFrom, the relay settings, and adminToken), the server needs dvxpkg (the package tool), repoKey (the repository's public key), serveContexts, and uploadMax (the largest upload in bytes).

Tests

test/run.sh

Needs bin/host/dvxpkg from a DVX build, calog, and singemaster beside this repository. It makes its own keys and packages with the package tool, starts the service on a scratch database, and runs two suites: testPackages (uploads checked and refused for the right reasons, review with the repository's signature, the catalogue, resumed downloads that DVX would trust, updates, withdrawing, an account's own packages, and a slow client not holding up the others) and testReview (the review tool end to end, including a changed file whose signature is refused). Then it starts singemaster and this service in one calog process and checks that both answer.

test/serve.sh DATA PORT FIXTURES DVXPKG

The package server for DVX's DOS tests: the DOS test harness runs it for a script that says # REQUIRES: PKGSERVER. It makes a test repository key, gives the test DVX that key, a record of DVX itself, and a theme to update (installed with the package tool), uploads and approves the fixture packages and the next version of DVX with tools/review.lua, and serves them on 127.0.0.1:PORT, which the emulator's guest reaches as 10.0.2.2.