calog/PORTING.md

# Porting calog

calog targets **Linux, macOS, and Windows**. Its core -- values, broker, the actor/threading
model, and every engine adapter -- is portable C11. The only operating-system-specific surface is
small and centralized:

- **Threads**: POSIX `pthreads`. Native on Linux and macOS; on Windows via mingw-w64's `winpthreads`.
- **Sockets + DNS**: BSD sockets + `getaddrinfo`, used only by the net/ssh/http libraries. BSD
  sockets on Linux/macOS, Winsock on Windows. All of this is behind `src/calogPlatform.h`.

There is no `epoll`/`eventfd`/`kqueue`, no `fork`/`exec`, and no `dlopen` in calog's own code (the
actor model uses blocking sockets on per-context threads), so a port is a thin shim, not a rewrite.

## The platform abstraction: `src/calogPlatform.h`

The net/ssh/http libraries use these instead of raw socket calls, so the same source compiles on all
three platforms:

| Abstraction | POSIX (Linux/macOS) | Windows |
|---|---|---|
| `CalogSocketT` | `int` | `SOCKET` |
| `CALOG_INVALID_SOCKET` | `-1` | `INVALID_SOCKET` |
| `calogSockClose(fd)` | `close` | `closesocket` |
| `calogSockErrno()` / `calogSockErrStr()` | `errno` / `strerror` | `WSAGetLastError` / `FormatMessage` |
| `calogSockSetNonblock(fd, on)` | `fcntl(O_NONBLOCK)` | `ioctlsocket(FIONBIO)` |
| `calogSockInProgress(err)` | `EINPROGRESS` | `WSAEWOULDBLOCK`/`WSAEINPROGRESS` |
| `calogPoll(...)` | `poll` | `WSAPoll` |
| `calogPlatformNetInit()` / `Shutdown()` | no-op | `WSAStartup` / `WSACleanup` |
| `CALOG_MSG_NOSIGNAL` | `MSG_NOSIGNAL` | `0` (Windows has no `SIGPIPE`) |

A socket `fd` is unsigned on Windows, so the code compares against `CALOG_INVALID_SOCKET` (never
`fd < 0`). The abstraction is behavior-identical on POSIX (it maps to the same names), so the Linux
and macOS builds are unchanged by it.

## Build matrix

| Target | Command | Runtime dependencies | DNS |
|---|---|---|---|
| Linux dev | `make` | glibc, libstdc++, libgcc, ASan/UBSan (dev only) | yes |
| Linux release | `make release` | **glibc only** (libstdc++/libgcc linked static) | yes |
| Linux static | `make static` on Alpine/musl | **none** (fully static) | yes (musl resolver) |
| Linux static | `make static` on glibc | none, but see caveat | **no** (glibc NSS needs dlopen) |
| macOS | `make` (Darwin auto-detected) | libSystem (always present) | yes |
| Windows | mingw-w64 build | core OS DLLs (`kernel32`, `ws2_32`) | yes |

The Makefile auto-detects the platform (`PLATFORM` = `linux`/`darwin`/`windows`/`posix`) and adjusts
the link line: `DLLIB` is `-ldl` on Linux (empty on macOS, where `dlopen` is in libSystem) and
`SOCKETLIBS` is `-lws2_32` on Windows.

## Linux

- **`make`** -- the sanitized development build (ASan + UBSan + strict warnings).
- **`make release`** -- the distribution build: sanitizers off, `-static-libstdc++ -static-libgcc`,
  glibc dynamic. `ldd` collapses to just `libc`/`libm` + the loader. Because glibc stays dynamic,
  `getaddrinfo` can still `dlopen` its NSS modules, so **DNS works**. Runs on any mainstream
  glibc distro with nothing to install.
- **`make static` on Alpine (musl)** -- a fully static binary with **zero** runtime dependencies and
  **working DNS** (musl has a built-in resolver, no NSS/dlopen). This is the minimal-dependency
  artifact. Because calog builds every dependency from source, and Alpine's default toolchain is
  musl, no per-dependency changes are needed -- build it in an Alpine container.
- **`make static` on glibc** -- fully static, but name resolution and `dlopen`-based Lua C modules
  do not work (glibc NSS requires runtime `dlopen`). Connect by IP, or use the musl route above.

## macOS (Darwin)

The C is already POSIX-compatible (pthreads, BSD sockets, `getaddrinfo` are all native), so the port
is build configuration, not code -- confirmed by `tools/crossBuild.sh`, which links valid Mach-O
executables for **both x86_64 (Intel) and arm64 (Apple Silicon)** using zig's bundled `libSystem`
stub (calog only needs `libSystem` -- libc/pthreads/sockets -- so no Apple SDK/frameworks are
required for the core). Apple does not support statically linking libc, but `libSystem` is part of
the OS and always present, so "static everything except libSystem" is the natural minimal-deps build.
Each vendored dependency builds its own way and needs a macOS configuration:

- OpenSSL: `./Configure darwin64-<arch>-cc`
- Tcl: its `configure` detects Darwin
- mruby / Lua / SQLite / ENet / QuickJS / Squirrel / Berry / s7 / Wren: compile as-is with clang
- libssh2 / MariaDB: CMake with the macOS toolchain

Link against dynamic `libSystem` (the default); `-ldl` is dropped automatically (`DLLIB` is empty on
Darwin) and the C++ runtime is `libc++` (`CXXLIB = -lc++`). SIGPIPE is ignored at CLI startup, since
macOS `send()` has no `MSG_NOSIGNAL`. ENet's `unix.c` backend is correct on macOS (BSD sockets).

## Windows (mingw-w64)

calog's pthread code compiles unchanged on Windows against **winpthreads** (POSIX threads over the
Win32 API), which is **vendored** at `vendor/winpthreads/` and built from source like every other
dependency (see its `NOTICE.calog`). `src/calogPlatform.h` handles the Winsock differences. Building
uses a **mingw-w64** toolchain (not MSVC) for its POSIX-ish environment; `tools/crossBuild.sh` cross-
compiles from Linux with a single zig toolchain and is the tested path. Remaining work is build
configuration:

- Build each vendored dependency for mingw (OpenSSL `mingw64`, Tcl's `win/` makefile or configure,
  libssh2/MariaDB via CMake with the mingw toolchain, mruby with a mingw build_config).
- ENet selects its `win32.c` backend automatically (`ENETBACKEND`); the link adds `-lws2_32 -lwinmm`
  (handled by `SOCKETLIBS`).
- Static-link the runtime (`-static -static-libgcc -static-libstdc++`) so the binary depends only on
  core Windows DLLs, which are always present. DNS works through Winsock's `getaddrinfo`.

Remaining code review for Windows: `calogFs.c` and `calogHttp.c` use POSIX headers (`dirent.h`,
`sys/stat.h`, `sys/time.h`) that mingw-w64 mostly provides, but path handling (`/` vs `\`) and a few
calls may need adjustment; `calogSsh.c` already guards `sys/select.h` (Winsock supplies
`fd_set`/`select`), and `httpSetTimeouts` already branches on the Winsock `SO_RCVTIMEO` DWORD form.

## Cross-compiling from Linux with zig (`tools/crossBuild.sh`)

A single [zig](https://ziglang.org/download/) toolchain cross-compiles C and C++ to musl-static and
Windows without installing per-target toolchains, so the ports can be exercised on a plain Linux box:

```
ZIG=/path/to/zig ./tools/crossBuild.sh
```

It builds a representative set -- core + one engine of each kind (Lua, QuickJS, Squirrel/C++,
my-basic) + the socket library (`calogNet`) -- because the crypto/db/http/ssh libraries pull in
OpenSSL/MariaDB/PostgreSQL/libssh2, each needing its own per-target build. What it does:

- **musl**: fully static Linux binaries. These **run on the build host**, so the script builds AND
  runs them -- verifying the core, all four engine kinds, C++, and the socket abstraction on musl.
- **Windows**: builds `vendor/winpthreads` into `libwinpthreads.a`, then links real `.exe`s
  (including the Winsock + ENet-win32 socket path). Verified as valid PE executables; running them
  needs wine.

Verified results (this repo): musl -- Lua, QuickJS, my-basic, Squirrel, and `testNet` all build and
**pass** (fully static); Windows -- `testEngineLua.exe` and `testNet.exe` build against vendored
winpthreads. Since the Windows binaries are the *same source* that passes on musl, the build is strong
evidence; full run-verification needs a Windows host or wine.

## Cross-compiling the archive stack with zig (`tools/crossArchive.sh`)

The compression/archive library (libarchive + its five codecs: zlib, bzip2, lz4, zstd, xz/liblzma)
is the one vendored stack that leans on CMake, so it gets its own cross-build script. Same idea as
`crossBuild.sh`, same single zig toolchain:

```
ZIG=/path/to/zig ./tools/crossArchive.sh
```

It builds all five codecs plus libarchive for **musl** (fully static, RUN on the host) and **Windows
x64** (a `testArchive.exe` verified as a PE), driving each through calog's real `testArchive`
(compress/decompress round-trips through every codec). CMake cross-compilation with zig works by
wrapping `zig cc` as a single-binary compiler (`build/cross/bin/zcc-*`) plus a toolchain file. The
key asymmetry:

- **musl toolchain does NOT set `CMAKE_SYSTEM_NAME`.** A musl-static binary runs on this Linux host,
  so the build stays "native" (`CMAKE_CROSSCOMPILING` false) and CMake's `try_run` feature-detection
  executes normally. zig supplies its own musl sysroot, so no host headers leak in.
- **Windows toolchain sets `CMAKE_SYSTEM_NAME Windows`** -> cross mode. libarchive's single
  `CHECK_C_SOURCE_RUNS` is already guarded on `CMAKE_CROSSCOMPILING` (it degrades to a warning) and
  xz has no run-checks, so nothing needs pre-seeding.

Two target-specific gotchas the script handles:

- **zlib on musl** needs `-DHAVE_UNISTD_H` so it includes `<unistd.h>` for `lseek`. zlib's
  `./configure` normally sets this; the object-rule build (no configure) must pass it, and musl --
  unlike glibc -- does not declare `lseek` without the include. Only zlib is affected.
- **libarchive on Windows** pulls in system libraries the POSIX build does not: CNG crypto
  (`-lbcrypt`) for its AES/PBKDF2, and XmlLite + OLE (`-lxmllite -lole32`) for XAR, which libarchive
  auto-substitutes for the libxml2/expat we disable. The link adds
  `-lbcrypt -lxmllite -lole32 -ladvapi32 -lcrypt32 -lshlwapi`.

Everything lands under `build/cross/` (gitignored); the vendored sources and the native
`vendor/*/build` / `_cmk` trees the normal `make` uses are never touched.

## Status

- **Platform abstraction** (`calogPlatform.h`) and the net/ssh/http conversion: **done**, verified
  behavior-identical on Linux (`make test` 30/30, socket + HTTP loopback smoke, ASan- and TSan-clean).
- **Linux release** target: **done and verified** -- `ldd bin/calog` shows only `libc`/`libm` + loader.
- **Linux musl-static**: **run-verified** via `tools/crossBuild.sh` (zig) -- core + Lua + QuickJS +
  my-basic + Squirrel + `testNet` all build fully static and pass on the host. Also works via
  `make static` in an Alpine container (musl default toolchain).
- **Windows**: **build-verified** via `tools/crossBuild.sh` -- `testEngineLua.exe` and `testNet.exe`
  (core + engine + Winsock socket abstraction) link against vendored `winpthreads` into real PE
  executables. Run-verification needs a Windows host or wine; the binaries are the same source that
  passes on musl.
- **Build system**: OS-aware (`PLATFORM`, `DLLIB`, `CXXLIB`, `SOCKETLIBS`, `ENETBACKEND`, vendored
  `winpthreads`); the Linux values are identical to before, so the dev build is unchanged.
- **macOS**: **build-verified** via `tools/crossBuild.sh` -- core + Lua + `testNet` link into valid
  Mach-O executables for both **x86_64 and arm64** (zig's libSystem stub; no Apple SDK needed for
  calog's libc surface). Run-verification needs a Mac. The C code and Makefile hooks are all in place
  (`libc++`, no `-ldl`, SIGPIPE, native pthreads + BSD sockets).
- **Archive stack** (libarchive + zlib/bzip2/lz4/zstd/xz): **musl run-verified, Windows
  build-verified** via `tools/crossArchive.sh` (zig). The musl `testArchive` is fully static and
  passes all 21 checks on the host (real compress/decompress through every codec); the Windows build
  is a valid `testArchive.exe` (PE32+) linked against vendored winpthreads + the CNG/XmlLite system
  libs. This was the one CMake-heavy vendored stack, so it is the strongest cross-build evidence
  after the core. The XAR format (native builds enable it via libxml2 + OpenSSL + iconv) is **not**
  in the cross build -- it is a native-only feature until those three are cross-built too.
- The remaining heavy vendored libraries (OpenSSL, Tcl, mruby, libssh2, MariaDB, PostgreSQL) still
  need a per-target build to reach a *full-featured* calog on macOS/Windows; the cross-build proves
  the core broker + engines + socket + archive layers, which is where all the portability risk lived.
  No code rewrite is expected -- the OS surface is only threads + sockets + DNS, all abstracted.