diff --git a/README.md b/README.md index 8439d3f..064be19 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,31 @@ bash scripts/runInMame.sh hello.bin --check 0x025000=0037 See [docs/USAGE.md](docs/USAGE.md) for a full walkthrough including multi-segment builds and the Apple IIgs Toolbox. +### Register-heavy code and the regalloc fallback + +The 65816 has a single accumulator, so a few unusually register-heavy +functions exceed what LLVM's default `greedy` allocator can place and +abort with `ran out of registers during register allocation`. This is a +hardware constraint, not a miscompile; LLVM's `basic` allocator handles +those functions correctly (greedy stays the default everywhere else +because its codegen is smaller and faster). + +The project build scripts handle this automatically via +`scripts/ccRegallocFallback.sh`, which compiles with greedy and silently +retries the affected translation unit with `-mllvm -regalloc=basic` only +when it hits that error. `tests/lua/build.sh`, `tests/coremark/build.sh`, +and `runtime/build.sh` route their compiles through it. + +If you invoke `clang` directly and hit the error on your own +register-heavy code, either compile that file through the wrapper: + +```bash +./scripts/ccRegallocFallback.sh ./tools/llvm-mos-build/bin/clang \ + --target=w65816 -O2 -c heavy.c -o heavy.o +``` + +or just add `-mllvm -regalloc=basic` to that one compile. + ## Project layout ``` diff --git a/SESSION_STATE.md b/SESSION_STATE.md index 9af65c4..aedc895 100644 --- a/SESSION_STATE.md +++ b/SESSION_STATE.md @@ -377,7 +377,7 @@ All under `/home/scott/claude/llvm816/tools/`: | MAME 0.264 | `/usr/games/mame` (apt) | supports `-console` (Lua) | | Apple IIgs ROMs | `tools/mame/roms/apple2gs.zip`, `apple2gsr1.zip` | from archive.org | | Calypsi 5.16 | `tools/calypsi/` | extracted .deb | -| ORCA/C source | `tools/orca-c/` | reference only | +| Toolbox catalogue | `tools/nlist/NList.Data.txt` | Dave Lyons' NiftyList data; source for `scripts/genToolbox.py` | `./setup.sh --verify-only` passed all checks as of the prior session. @@ -390,7 +390,7 @@ llvm816/ # git repo, branch main ├── setup.sh # tracked ├── scripts/ # tracked │ ├── common.sh -│ ├── installDeps.sh installCalypsi.sh installOrcaC.sh +│ ├── installDeps.sh installCalypsi.sh installGno.sh │ ├── installLlvmMos.sh # non-destructive (see §8) │ ├── installMame.sh verify.sh │ ├── applyBackend.sh # src/ + patches/ -> tools/llvm-mos/ diff --git a/STATUS.md b/STATUS.md index e1cd587..30973d5 100644 --- a/STATUS.md +++ b/STATUS.md @@ -332,8 +332,14 @@ which runs correctly under MAME (apple2gs). | memcmp | 682 | 716 | 0.95× | | fib | 11594 | 10912 | 1.06× | - **Geomean: 0.62× Calypsi.** 9 of 10 below 1.0×; only fib trails - (recursive call overhead, structural). Speed is the optimization + **Geomean: 0.62× Calypsi.** 9 of 10 below 1.0×; only fib trails. + The ~6% gap is the per-call defensive `REP #$30` mode-set our + FrameLowering emits at every function entry (Calypsi omits it via a + global M/X=16 invariant) plus a marginally larger stack frame. Both + are deliberate correctness choices; eliding the entry REP safely needs + the deferred per-region REP/SEP scheduling pass (design §3.3) -- a + naive global elision risks the silent i16-immediate-in-M=1 miscompile + documented in W65816FrameLowering.cpp. Speed is the optimization priority, not size. - `compare/` holds three side-by-side C tests with our asm and @@ -395,11 +401,12 @@ which runs correctly under MAME (apple2gs). Resource Manager, MIDI, Video Overlay, TextEdit, Media Control, Print Manager, Scheduler, Desk Manager, …). Names match Apple's IIgs Toolbox Reference exactly (TLStartUp, - MMStartUp, NewWindow, SysBeep, …). 417 simple wrappers + MMStartUp, NewWindow, SysBeep, …). 495 simple wrappers (zero/single-arg, i16-or-void return) inline in the header; - 890 multi-arg ones live in `runtime/src/iigsToolbox.s`. - Generated by `scripts/genToolbox.py` from ORCA-C's - `ORCACDefs/` (re-runnable when ORCA-C updates). + 896 multi-arg ones live in `runtime/src/iigsToolbox.s`. + Generated by `scripts/genToolbox.py` from Dave Lyons' + NiftyList database (`tools/nlist/NList.Data.txt`, + re-runnable when the catalogue updates). ## What's next @@ -592,10 +599,11 @@ for the common-case C / minimal-C++ workload. Priority is speed - **Lua 5.1.5 compiles cleanly** (2026-05-20). Reference C implementation (17K lines, 24 source files) builds + links into a multi-segment binary. Loads in MAME. Lives under `tests/lua/`. - Three large functions (luaV_execute, symbexec, auxsort) hit - greedy regalloc's complexity budget and need `-mllvm - -regalloc=basic` (still at -O2 — basic-regalloc -O2 is ~3.5× - smaller than fast-regalloc -O0). Largest "real-world C" test + A few large functions (luaV_execute, symbexec, auxsort, and + lstrlib under --layer2) exceed what greedy regalloc can place + with the single accumulator; `build.sh` auto-falls-back to + `-mllvm -regalloc=basic` for just those TUs via + `scripts/ccRegallocFallback.sh` (still -O2). Largest "real-world C" test in the project. **Open limitations:** @@ -615,17 +623,28 @@ for the common-case C / minimal-C++ workload. Priority is speed bank N needs the program to set DBR=N or use `sta long` via inline asm. -- **C++ exceptions absent from CI smoke.** The SJLJ runtime - round-trip is in smoke; the full clang++ → backend → MAME - execution path runs reliably interactively but is excluded - from automated smoke due to MAME-side I/O flakiness. +- **C++ exceptions now run in CI smoke** (2026-06-30). Both the + pure-C SJLJ runtime round-trip AND the full clang++ + `-fsjlj-exceptions` → backend → MAME execution path are in smoke: + the latter throws an int across a recursive call boundary and + verifies the catch fired and the value propagated (407/0x0197). + The earlier -O2 call_site-store isel mis-route and the + "intermittent MAME crash" that previously kept this compile+link + only are resolved (verified 20/20 across -O0/-O2). -- **GS/OS validation uses a stub dispatcher.** The wrapper - contract (PHA + PEA 0 + LDX + JSL $E100A8 + post-call SP - fixup) is verified end-to-end in MAME against a stub - (`scripts/runInMameWithGsosStub.sh`). Validation against a - real bootable GS/OS volume is left out of CI as it needs a - smartport hard-disk image and live Tool Locator init. +- **GS/OS validated against BOTH a stub and a real bootable + volume.** The wrapper contract (PHA + PEA 0 + LDX + JSL $E100A8 + + post-call SP fixup) is verified in MAME two ways: a fast isolated + stub round-trip (`scripts/runInMameWithGsosStub.sh`), and -- since + the `runViaFinder` path landed -- end-to-end against a real booted + GS/OS. `scripts/runViaFinder.sh` boots real GS/OS 6.0.2 + (`tools/gsos/sys602.po`) with live Tool Locator init and mounts a + cadius-built smartport data volume; the smoke check "GS/OS + fopen/fread reads /DATA/TESTFILE" drives `fopen`/`fread`/`fclose` + through the wrapper to real GS/OS on that volume and verifies the + content round-trips. ~15 further smoke checks run real GS/OS + 6.0.2/6.0.4 (Resource Manager, StandardFile, Cursor Mgr, the + Loader's cINTERSEG + indirect dispatch, etc.). - **VLAs work end-to-end** (2026-05-09). Backend Custom-lowers `ISD::DYNAMIC_STACKALLOC` for both i16 and i32 result types. diff --git a/docs/INSTALL.md b/docs/INSTALL.md index cc8de83..b98e962 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -285,7 +285,7 @@ bash runtime/build.sh # Compile + disassemble a small C demo bash scripts/cDemo.sh -# Run the full smoke test suite (~150 checks, takes ~3 min) +# Run the full smoke test suite (~175 checks, takes a few min) bash scripts/smokeTest.sh ``` diff --git a/docs/USAGE.md b/docs/USAGE.md index 4290cde..c3ad08c 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -353,8 +353,9 @@ See [STATUS.md](../STATUS.md) for the full feature matrix. ## Linking — full reference `link816` produces a flat binary suitable for direct execution (loaded -into a fixed address) or, with `--omf`, an OMF binary that the GS/OS -Loader can load and relocate. +into a fixed address). To get an OMF binary that the GS/OS Loader can +load and relocate, pass `link816`'s output through the separate `omfEmit` +tool (see "Multi-segment OMF (for GS/OS Loader)" below). ### Raw binary (fixed-address load) @@ -1161,10 +1162,14 @@ is a follow-up under the `cxxsmoke` GNO MAME harness. trigger FP-relative addressing. Most programs fit under that limit. See the `frame-rel` discussion in [LLVM_65816_DESIGN.md](../LLVM_65816_DESIGN.md). -- **Three Lua functions** (`luaV_execute`, `symbexec`, `auxsort`) hit - the greedy register allocator's complexity budget. Workaround: - compile those TUs with `-mllvm -regalloc=basic`. Documented in - [`tests/lua/README.md`](../tests/lua/README.md). +- **Register-heavy functions** (e.g. Lua's `luaV_execute`, `symbexec`, + `auxsort`) can exceed what the `greedy` allocator places with the + 65816's single accumulator and abort with `ran out of registers`. + This is handled automatically by `scripts/ccRegallocFallback.sh` + (retries the TU with `-mllvm -regalloc=basic`); the project build + scripts route through it. For a direct `clang` invocation, use the + wrapper or add `-mllvm -regalloc=basic`. See the Quick start note in + [`../README.md`](../README.md) and [`tests/lua/README.md`](../tests/lua/README.md). --- diff --git a/runtime/build.sh b/runtime/build.sh index da397d3..1bf240a 100755 --- a/runtime/build.sh +++ b/runtime/build.sh @@ -31,7 +31,10 @@ cc() { local o="$OUT/$(basename "${c%.c}").o" local extra=("${@:2}") echo " CC $(basename "$c")" - "$CLANG" -target w65816 -O2 -ffunction-sections \ + # Route through the regalloc fallback so a register-heavy runtime TU + # auto-retries with basic regalloc instead of hard-failing the build. + "$PROJECT_ROOT/scripts/ccRegallocFallback.sh" "$CLANG" \ + -target w65816 -O2 -ffunction-sections \ "${extra[@]}" \ -I"$PROJECT_ROOT/runtime/include" \ -c "$c" -o "$o" diff --git a/scripts/ccRegallocFallback.sh b/scripts/ccRegallocFallback.sh new file mode 100755 index 0000000..539b4b9 --- /dev/null +++ b/scripts/ccRegallocFallback.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# Compile one W65816 translation unit, automatically falling back to the +# `basic` register allocator if the default `greedy` allocator hits the +# single-accumulator deadlock ("ran out of registers during register +# allocation"). +# +# Why this exists: the 65816 has exactly one accumulator (the Acc16 +# register class has a single physreg, A). greedy's region-splitting +# heuristic can manufacture two A-pinned live ranges it cannot then +# unspill, and aborts. This is a genuine hardware constraint, not a bug +# we can cheaply fix in the backend (the deadlocking vregs are greedy's +# own split products, so no pre-RA pass prevents them; verified +# 2026-06-29). `basic` regalloc serializes the two values correctly. +# +# greedy stays the default because its codegen is smaller/faster +# everywhere it succeeds; basic is used ONLY for the rare function greedy +# cannot allocate, and ONLY for that translation unit. This replaces the +# old hand-maintained per-file list (tests/lua/build.sh SLOW_FILES) so a +# newly-added register-heavy TU can never silently break the build. +# +# Usage: ccRegallocFallback.sh +# The args are a normal clang `-c file -o out` compile command. + +set -u + +if [ "$#" -lt 1 ]; then + echo "usage: ccRegallocFallback.sh " >&2 + exit 2 +fi + +clang="$1" +shift + +# First attempt with the default (greedy) allocator. Capture combined +# output so we can inspect it; a single-TU compile is quick, so buffering +# until completion is fine. +out="$("$clang" "$@" 2>&1)" +status=$? + +if [ "$status" -eq 0 ]; then + # Success: pass through any warnings clang produced. + [ -n "$out" ] && printf '%s\n' "$out" >&2 + exit 0 +fi + +# Retry with basic regalloc ONLY for the single-accumulator deadlock. +# Any other compile error is surfaced as-is. +if printf '%s' "$out" | grep -q 'ran out of registers'; then + echo " (regalloc: greedy ran out of registers; retrying with -regalloc=basic)" >&2 + exec "$clang" "$@" -mllvm -regalloc=basic +fi + +printf '%s\n' "$out" >&2 +exit "$status" diff --git a/scripts/smokeTest.sh b/scripts/smokeTest.sh index 11185b5..7327fe2 100755 --- a/scripts/smokeTest.sh +++ b/scripts/smokeTest.sh @@ -5003,26 +5003,32 @@ EOF fi rm -f "$cSjeFile" "$oSjeFile" "$oSjeRt" "$oSjeAbi" "$binSjeFile" - # C++ try/throw/catch via clang's -fsjlj-exceptions: COMPILE + - # LINK only. We don't run the binary in MAME from smoke - # because MAME's apple2gs CPU emulation crashes intermittently - # on our SJLJ-prepared exception code (a MAME bug — same - # binary executes correctly when invoked outside the smoke - # harness's I/O environment). The pure-C SJLJ runtime test - # above already exercises the full _Unwind_SjLj_* + __cxa_* - # surface end-to-end; this check just guards that the C++ - # frontend → backend path produces a linkable binary. - log "check: clang++ -fsjlj-exceptions compiles + links a C++ try/catch program" + # C++ try/throw/catch via clang's -fsjlj-exceptions: full + # frontend → backend → SJLJ runtime → MAME execution. Throws an + # int across a recursive call boundary and checks both that the + # catch fired (caught=1) and that the thrown value propagated + # intact (4*100+7 = 407 = 0x0197). This used to be compile+link + # only: the earlier -O2 call_site-store isel mis-route and the + # "intermittent MAME crash" are both resolved, and the binary now + # runs deterministically (verified 20/20 across -O0 and -O2). + log "check: MAME runs clang++ -fsjlj-exceptions C++ throw/catch across a call" cppExcFile="$(mktemp --suffix=.cpp)" oCppExcFile="$(mktemp --suffix=.o)" oExcRt="$(mktemp --suffix=.o)" oExcAbi="$(mktemp --suffix=.o)" binCppExcFile="$(mktemp --suffix=.bin)" cat > "$cppExcFile" <<'EOF' +__attribute__((noinline)) static int deep(int n) { + if (n > 3) throw (n * 100 + 7); + return deep(n + 1); +} int main(void) { - int ok = 0; - try { throw 42; } catch (int e) { if (e == 42) ok = 1; } - *(volatile unsigned short *)0x025000 = (unsigned short)ok; + int caught = 0; + int val = 0; + try { val = deep(0); } + catch (int e) { caught = 1; val = e; } + *(volatile unsigned short *)0x025000 = (unsigned short)caught; + *(volatile unsigned short *)0x025002 = (unsigned short)val; while (1) {} } EOF @@ -5040,6 +5046,10 @@ EOF "$oExcAbi" "$oExcRt" "$oCppExcFile" >/dev/null 2>&1; then die "clang++ -fsjlj-exceptions: C++ try/catch failed to link" fi + if ! bash "$PROJECT_ROOT/scripts/runInMame.sh" "$binCppExcFile" --check \ + 0x025000=0001 0x025002=0197 >/dev/null 2>&1; then + die "MAME: clang++ C++ throw/catch across a call failed (caught!=1 or value!=0x0197)" + fi rm -f "$cppExcFile" "$oCppExcFile" "$oExcRt" "$oExcAbi" "$binCppExcFile" # Real-world: hex dumper using memory-backed file I/O. Reads diff --git a/tests/coremark/build.sh b/tests/coremark/build.sh index 910f782..1407933 100755 --- a/tests/coremark/build.sh +++ b/tests/coremark/build.sh @@ -14,6 +14,9 @@ PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" CM_SRC="$SCRIPT_DIR/coremark-src" OUT="$SCRIPT_DIR/build" CLANG="$PROJECT_ROOT/tools/llvm-mos-build/bin/clang" +# Auto-fall-back to basic regalloc on the single-accumulator deadlock +# instead of hard-failing the build -- see scripts/ccRegallocFallback.sh. +CC="$PROJECT_ROOT/scripts/ccRegallocFallback.sh" ITERATIONS="${ITERATIONS:-1}" @@ -36,11 +39,11 @@ CFLAGS=(--target=w65816 -O2 -ffunction-sections CORE_FILES="core_list_join core_main core_matrix core_state core_util" for f in $CORE_FILES; do echo " CC $f.c" - "$CLANG" "${CFLAGS[@]}" -c "$CM_SRC/$f.c" -o "$OUT/$f.o" + "$CC" "$CLANG" "${CFLAGS[@]}" -c "$CM_SRC/$f.c" -o "$OUT/$f.o" done echo " CC core_portme.c" -"$CLANG" "${CFLAGS[@]}" -c "$SCRIPT_DIR/core_portme.c" -o "$OUT/core_portme.o" +"$CC" "$CLANG" "${CFLAGS[@]}" -c "$SCRIPT_DIR/core_portme.c" -o "$OUT/core_portme.o" echo "" echo "CoreMark built: $(ls "$OUT"/*.o | wc -l) objects, $(du -sh "$OUT" | cut -f1) total" diff --git a/tests/lua/README.md b/tests/lua/README.md index 84693c7..0f2b4b6 100644 --- a/tests/lua/README.md +++ b/tests/lua/README.md @@ -22,17 +22,19 @@ bash tests/lua/runLuaTest.sh # build + link the test wrapper ## Caveats -Three functions (`symbexec` in ldebug.c, `auxsort` in ltablib.c, -`luaV_execute` in lvm.c) hit our greedy register allocator's complexity -budget and fail with an `IndexedMap` assertion. Workaround: compile -these specific TUs with `-mllvm -regalloc=basic` (the simpler-but- -slower allocator). Object size with basic-regalloc + -O2 is ~3.5× -smaller than -O0 (lvm.o: 220KB → 63KB), so this is the preferred -fallback over `-O0`. +A few functions (`symbexec` in ldebug.c, `auxsort` in ltablib.c, +`luaV_execute` in lvm.c, plus `lstrlib.c` under `--layer2`) exceed what +LLVM's `greedy` allocator can place with the 65816's single accumulator +and abort with `ran out of registers during register allocation`. This +is a hardware constraint, not a miscompile. `build.sh` handles it +automatically: it compiles through `scripts/ccRegallocFallback.sh`, which +retries an affected TU with `-mllvm -regalloc=basic` (the simpler, +always-terminating allocator) only when greedy fails. No per-file list +is maintained; a newly register-heavy TU is handled transparently. -These three functions are the largest in Lua and exhibit unusual -register pressure (luaV_execute is a 30+ case VM dispatch). Greedy -regalloc improvements may eventually handle them. +These functions are the largest in Lua and exhibit unusual register +pressure (luaV_execute is a 30+ case VM dispatch). See the regalloc +discussion in [`../../LLVM_65816_DESIGN.md`](../../LLVM_65816_DESIGN.md). ## Runtime execution status @@ -48,7 +50,8 @@ plain shell. ## Files - `lua-5.1.5/` — vendored Lua 5.1.5 source tree (from www.lua.org) -- `build.sh` — compile script (handles per-file regalloc tuning) +- `build.sh` — compile script (auto-falls-back to basic regalloc via + `scripts/ccRegallocFallback.sh` when greedy can't place a TU) - `luaStubs.c` — `strcoll`, `strxfrm`, `freopen` stubs for missing libc - `luaTest.c` — minimal Lua API test (sentinel `0xC0DE` if pass) - `runLuaTest.sh` — full build + run wrapper diff --git a/tests/lua/build.sh b/tests/lua/build.sh index 531c6ce..b1c42ef 100755 --- a/tests/lua/build.sh +++ b/tests/lua/build.sh @@ -1,7 +1,11 @@ #!/usr/bin/env bash -# Build Lua 5.1.5 for the W65816 target. Most files compile at -O2; -# ldebug.c (symbexec) and ltablib.c (auxsort) hit a register-allocation -# limit at -O1+ and need -O0. Outputs object files under build/. +# Build Lua 5.1.5 for the W65816 target. All files compile at -O2. A +# few functions (symbexec in ldebug.c, auxsort in ltablib.c, luaV_execute +# in lvm.c, and luaO_str2d-class code in lstrlib.c under --layer2) exceed +# what greedy regalloc can do with the 65816's single accumulator and +# fall back to basic regalloc automatically -- see +# scripts/ccRegallocFallback.sh. No hand-maintained per-file list. +# Outputs object files under build/. set -eu @@ -23,15 +27,7 @@ done CFLAGS=(-target w65816 -O2 -ffunction-sections -DLUA_USE_C89 -I "$PROJECT_ROOT/runtime/include" -I "$LUA_SRC" "${LAYER2[@]}") -# These functions exceed greedy regalloc's complexity budget — fall back -# to basic regalloc which is simpler but always terminates. Code is -# slower but stays at -O2 so the resulting object is ~3.5× smaller -# than -O0 (lvm.o: 220KB → 63KB). -SLOW_FILES="ldebug ltablib lvm" -if [ ${#LAYER2[@]} -gt 0 ]; then - SLOW_FILES="$SLOW_FILES lstrlib" -fi -SLOW_FLAGS="-mllvm -regalloc=basic" +CC="$PROJECT_ROOT/scripts/ccRegallocFallback.sh" # Core Lua: VM, parser, GC, string/table libs. Skip: liolib (no FILE* # backing in mfs-only mode), loslib (no time), loadlib (no dlsym), @@ -43,19 +39,12 @@ CORE="lapi lcode ldebug ldo ldump lfunc lgc llex lmem lobject lopcodes for f in $CORE; do o="$OUT/${f}.o" - cflags=("${CFLAGS[@]}") - if echo " $SLOW_FILES " | grep -q " $f "; then - # shellcheck disable=SC2206 - cflags=("${cflags[@]}" $SLOW_FLAGS) - echo " CC $f.c (basic regalloc)" - else - echo " CC $f.c" - fi - "$CLANG" "${cflags[@]}" -c "$LUA_SRC/${f}.c" -o "$o" + echo " CC $f.c" + "$CC" "$CLANG" "${CFLAGS[@]}" -c "$LUA_SRC/${f}.c" -o "$o" done # Stubs for libc functions Lua needs that our libc doesn't provide. -"$CLANG" "${CFLAGS[@]}" -c "$SCRIPT_DIR/luaStubs.c" -o "$OUT/luaStubs.o" +"$CC" "$CLANG" "${CFLAGS[@]}" -c "$SCRIPT_DIR/luaStubs.c" -o "$OUT/luaStubs.o" echo " CC luaStubs.c" echo ""