AI_Slop/65816-llvm-mos
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
65816-llvm-mos/docs/INSTALL.md
Scott Duensing 2bec85ffa5 Install cleaned up.
2026-05-26 19:00:55 -05:00

18 KiB
Raw Blame History

Installing llvm816

This document covers everything you need to get from a fresh Ubuntu / Debian install to a working W65816 Clang compiler + Apple IIgs MAME emulator + matching runtime libraries. The entire toolchain installs locally under your repo checkout — nothing goes into /usr/local, /opt, or your home directory beyond a few standard apt packages.

If you've never built LLVM or used a cross-compiler before, follow this document top to bottom. If you're comfortable, the One-command install section gets you running in 5-10 minutes.

What you'll have when it's done

After install, the llvm816/ directory tree contains everything:

Component Disk usage (approx) Purpose
tools/llvm-mos/ 5.0 GB LLVM source tree (clone of llvm-mos). Our backend source is symlinked into here at build time.
tools/llvm-mos-build/ 1.4 GB Compiled clang/llc/llvm-mc binaries. This is where you actually run the compiler from.
tools/llvm-mos-sdk/ 400 MB Prebuilt llvm-mos SDK (the original 6502 distribution). Mostly unused by us; kept as a reference baseline.
tools/calypsi/ 580 MB Commercial Calypsi 5.16 65816 C compiler — installed for output-quality comparisons in compare/.
tools/orca-c/ 10 MB Apple's official ORCA/C compiler source — header reference for the IIgs Toolbox bindings.
tools/gsos/ 13 MB Apple GS/OS 6.0.2 / 6.0.4 disk images for booting under MAME.
tools/mame/roms/ 1.5 MB Apple IIgs ROM 01 + ROM 03 (downloaded from archive.org).
Total ~7-8 GB

Plus a few system-wide apt packages (cmake, ninja, MAME, ...). These are listed up front in System requirements so you can audit them before installing.

The compiler binary itself is at tools/llvm-mos-build/bin/clang — you'll see this path referenced everywhere.

System requirements

  • OS: Ubuntu 22.04 LTS or 24.04 LTS (other Debian-based distros work if you can find equivalents for the apt packages). Pure Arch / Fedora installs need package-name translation but the project itself is distro-neutral.
  • CPU: any 64-bit x86 or ARM Linux machine. We're cross-compiling, so the host CPU only matters for build speed.
  • Disk: ~20 GB free during install (~12 GB peak for LLVM's cmake intermediates, ~7 GB resident after the install + delete the intermediates). If you skip Calypsi (--skip-calypsi), knock 580 MB off the resident size.
  • RAM: 16 GB recommended (LLVM's link step is the memory-heavy one). 8 GB works but the linker may swap, doubling build time.
  • Time: 30-60 minutes end-to-end (LLVM is the long pole). After the first build, incremental edits to the W65816 backend rebuild in ~30 seconds.
  • Network: the install pulls ~500 MB of binaries from GitHub, archive.org, and the Calypsi releases page. No proxy support baked in — set http_proxy / https_proxy if you need one.

apt packages installed

scripts/installDeps.sh runs sudo apt-get install for these packages:

Package Why it's needed
build-essential gcc, make, libc-dev — needed to build LLVM and our linker
cmake, ninja-build LLVM's build system
clang, lld Bootstrap a host clang (faster LLVM build than gcc)
python3, python3-pip Build-time scripting + LLVM's lit test runner
git Cloning the llvm-mos source tree
zlib1g-dev, libedit-dev, libxml2-dev, libncurses-dev LLVM link-time deps
zstd, xz-utils, unzip, tar Unpacking downloaded archives
lua5.4, liblua5.4-dev MAME's autoboot scripting (used by the smoke harness)
curl, ca-certificates Downloading installer payloads
mame, mame-tools Apple IIgs emulator

All packages are installed with --no-install-recommends, so the total apt footprint is bounded. If you want to inspect or audit before running, see scripts/installDeps.sh for the exact list.

Requires sudo: the apt install step needs root. setup.sh prompts via sudo once and then continues without root for everything else.

One-command install

# Replace <REPO-URL> with the actual git URL for this repository.
git clone <REPO-URL> llvm816
cd llvm816
./setup.sh

That's it. setup.sh runs five stages in order:

Stage Script What it does Time
1/5 installDeps.sh sudo apt-get install the packages listed above. ~1 min
2/5 installLlvmMos.sh Clone llvm-mos source (5 GB), download prebuilt llvm-mos SDK (400 MB), apply our W65816 backend (symlinks + patches), build clang/llc/llvm-mc with W65816 target enabled, build link816 + omfEmit, and build the runtime libraries (libc.o, crt0.o, libgcc.o, soft-float, etc.). After this stage you have a working W65816 toolchain end-to-end. ~30-60 min (first time; LLVM build is the long pole)
3/5 installMame.sh Install MAME via apt, download apple2gs.zip (ROM 03) and apple2gsr1.zip (ROM 01) into tools/mame/roms/. ~30 s
4/5 installCalypsi.sh Download Calypsi 5.16 .deb, extract its payload into tools/calypsi/ (no system-wide install). ~30 s
5/5 installOrcaC.sh Shallow clone of byteworksinc's ORCA/C repo into tools/orca-c/ for toolbox header reference. ~15 s

After each stage, the script prints === N/5 stage-name === so you can follow progress. At the end it runs verify.sh which sanity- checks every tool was installed AND end-to-end compiles a tiny C program to confirm clang actually produces W65816 machine code.

A successful install ends with:

[llvm816] all checks passed
[llvm816] setup complete

If verify.sh reports failures, the most common cause is that the LLVM build didn't include the W65816 target. Re-run scripts/applyBackend.sh followed by ninja -C tools/llvm-mos-build clang llc llvm-mc llvm-objdump.

setup.sh flags

Flag Effect
--skip-deps Don't run apt (use if you've already installed the system packages).
--skip-llvm Skip the LLVM clone + build + runtime. Useful for iterating on other parts.
--skip-mame Skip MAME + ROM download.
--skip-calypsi Skip Calypsi (saves 580 MB if you don't need the comparison benchmarks).
--skip-orca Skip ORCA/C (saves ~10 MB; only needed if you regenerate iigs/toolbox.h).
--skip-verify Don't run the post-install verification check.
--verify-only Just run the verification check, don't install anything.
--build-llvm Deprecated alias — the LLVM build is now always part of stage 2/5 (without it we wouldn't have a usable W65816 compiler).

What gets installed where

After a complete setup.sh run, your llvm816/ directory looks like this:

llvm816/                            ← your repo checkout
├── setup.sh                        ← the installer you just ran
├── README.md
├── docs/                           ← documentation (you're reading INSTALL.md)
│   ├── INSTALL.md
│   ├── USAGE.md
│   └── multiSegmentPlan.md
├── src/                            ← OUR backend source (W65816 target)
│   ├── llvm/lib/Target/W65816/     ← ~41 files; symlinked into tools/llvm-mos
│   ├── clang/                      ← clang frontend hooks
│   └── link816/                    ← linker source
├── patches/                        ← upstream-llvm-mos patches
├── runtime/                        ← C standard library + crt0
│   ├── include/                    ← <stdio.h>, <stdlib.h>, <iigs/toolbox.h> ...
│   ├── src/                        ← .c and .s sources
│   └── *.o                         ← built object files (after runtime/build.sh)
├── scripts/                        ← install + run + bench scripts
├── benchmarks/                     ← cycle-count benchmarks
├── compare/                        ← side-by-side ours-vs-Calypsi assembly
├── demos/                          ← example IIgs programs (helloBeep, reversi, ...)
├── tests/                          ← larger compile-only tests (e.g. tests/lua/)
└── tools/                          ← everything installed by setup.sh
    ├── llvm-mos/                       (5.0 GB) — LLVM source tree clone
    │   └── llvm/lib/Target/W65816/     ← symlinks point back to ../../src/llvm/lib/Target/W65816/
    ├── llvm-mos-build/                 (1.4 GB) — cmake build directory
    │   └── bin/
    │       ├── clang  →  clang-23      ← THE COMPILER ⭐
    │       ├── clang++                 ← C++ driver
    │       ├── clang-23                ← actual binary
    │       ├── llc                     ← standalone codegen (.ll → .s)
    │       ├── llvm-mc                 ← standalone assembler
    │       ├── llvm-objdump            ← disassembler
    │       └── ...                     (FileCheck, llvm-readobj, opt, etc.)
    ├── llvm-mos-sdk/                   (400 MB) — prebuilt llvm-mos SDK
    ├── link816                         (~120 KB) — OUR LINKER ⭐
    ├── omfEmit                         (~70 KB) — OMF v2.1 emitter for GS/OS Loader
    ├── cadius/                         — Apple ProDOS / GS/OS disk image tool
    ├── calypsi/                        (580 MB) — Calypsi 5.16 reference compiler
    ├── orca-c/                         (10 MB) — ORCA/C compiler (header reference)
    ├── gsos/                           (13 MB) — GS/OS 6.0.2 / 6.0.4 disk images
    ├── mame/
    │   └── roms/                       (1.5 MB) — apple2gs ROM 01 + ROM 03
    └── venv/                           — Python venv (used by genToolbox.py)

The starred items (clang and link816) are the two binaries you interact with daily. See USAGE.md for how to use them.

Why so much LLVM?

tools/llvm-mos/ is a full LLVM source tree (about 5 GB). We need it because our W65816 backend is part of LLVM — we build it as a target inside LLVM's normal codegen pipeline. Our backend lives in src/llvm/lib/Target/W65816/; scripts/applyBackend.sh symlinks it into the LLVM source tree under tools/llvm-mos/llvm/lib/Target/W65816/ so cmake picks it up.

The size is unavoidable if you want to rebuild the backend. If you don't plan to modify it, you could rm -rf tools/llvm-mos after a successful build — but you'd have to re-clone to fix bugs.

What's outside tools/?

Only the apt packages from System requirements. apt-get remove mame mame-tools cleans those up if you want to fully uninstall.

Importantly, the installer does NOT touch:

  • /usr/local/ (no make install)
  • /opt/ (no FHS-style component install)
  • ~/.mame/ (we use -rompath to point at tools/mame/roms/ instead)
  • ~/.cache/ (downloads go to the repo-local .cache/)

To uninstall completely:

rm -rf llvm816/
sudo apt-get remove mame mame-tools build-essential cmake ninja-build \
    clang lld lua5.4 liblua5.4-dev    # if you want apt deps gone too

Step-by-step (if setup.sh fails)

You can run each install script in isolation if a stage breaks. They're idempotent — running them twice is a no-op (or a "fetch updates" for the LLVM clone).

bash scripts/installDeps.sh           # apt packages
bash scripts/installLlvmMos.sh        # llvm-mos clone + prebuilt SDK
bash scripts/installLlvmMos.sh --build  # also build clang/llc (slow)
bash scripts/installMame.sh           # MAME + apple2gs ROMs
bash scripts/installCalypsi.sh        # reference compiler (optional)
bash scripts/installOrcaC.sh          # reference compiler (optional)
bash scripts/verify.sh                # sanity-check everything

If you only want to build C programs (no benchmarks, no comparisons), installCalypsi.sh and installOrcaC.sh are skippable.

Building W65816 clang from source

setup.sh always builds clang from source — that's the only way to get a clang that actually targets W65816 (the prebuilt llvm-mos SDK in tools/llvm-mos-sdk/ only knows about the 6502 MOS target). The initial build takes 30-60 minutes depending on core count; after that incremental rebuilds are ~30 seconds:

ninja -C tools/llvm-mos-build llc clang

scripts/applyBackend.sh re-runs the symlink-into-LLVM step if you've added new files under src/llvm/lib/Target/W65816/.

Verifying the install

setup.sh automatically runs scripts/verify.sh at the end — it walks every installed tool and checks each runs. If anything fails it shows which step failed (e.g. [FAIL] llvm-mos source tree means the clone didn't land where expected).

To re-verify later:

./setup.sh --verify-only

For a real end-to-end test (compiles and runs a tiny C program through the entire pipeline):

# Build the runtime libraries (libc, libgcc, etc.) — ~30 s
bash runtime/build.sh

# Compile + disassemble a small C demo
bash scripts/cDemo.sh

# Run the full smoke test suite (~150 checks, takes ~3 min)
bash scripts/smokeTest.sh

A successful smoke test ends with:

[llvm816] all smoke checks passed

If smoke passes, your install is good.

Updating

cd llvm816
git pull                            # update our backend source
bash scripts/applyBackend.sh        # re-symlink into tools/llvm-mos
ninja -C tools/llvm-mos-build llc clang
bash runtime/build.sh

If you want a fully clean rebuild (e.g., to chase a "stale .o" bug):

rm -rf tools/llvm-mos-build
./setup.sh --skip-deps --skip-mame --skip-calypsi --skip-orca

Uninstalling

The toolchain is fully contained under llvm816/:

rm -rf llvm816/
sudo apt-get remove mame mame-tools     # if you want MAME gone too
# (also remove build-essential, cmake, etc., if they're not used elsewhere)

Nothing remains outside the repo.

Troubleshooting

cmake: command not found / ninja: command not found

The apt packages aren't installed yet:

bash scripts/installDeps.sh

clang: error: unable to find target 'w65816'

You're invoking a clang that doesn't include our backend. Likely causes:

  1. You're using the system clang (/usr/bin/clang). The system clang doesn't know W65816. Use the full path:

    ./tools/llvm-mos-build/bin/clang --target=w65816 ...

  2. The build didn't complete. Check:

    ls -la tools/llvm-mos-build/bin/clang   # should be a symlink to clang-23
./tools/llvm-mos-build/bin/clang --print-targets | grep -i 65816

    If the binary's missing or no W65816 target appears, rebuild:

    bash scripts/applyBackend.sh
ninja -C tools/llvm-mos-build llc clang

  3. You're using the prebuilt SDK (tools/llvm-mos-sdk/bin/...). That's the original llvm-mos SDK; our W65816 target lives only in our source build at tools/llvm-mos-build/bin/.

linkage error: missing __umulhisi3 / missing __mulsi3 / similar

You forgot to link runtime/libgcc.o. See USAGE.md § Linking. Typical link line:

./tools/link816 -o myprog.bin --text-base 0x1000 \
    runtime/crt0.o runtime/libc.o runtime/libgcc.o myprog.o

ROMs not found at runtime

The apple2gs ROM download from archive.org occasionally fails. Re-run the installer (it's idempotent and skips already-present ROMs):

bash scripts/installMame.sh
ls tools/mame/roms/   # should contain apple2gs.zip and apple2gsr1.zip

If you invoke mame directly without using our runInMame.sh wrapper, pass -rompath tools/mame/roms so it finds them.

MAME pops up a window I don't want

The supplied scripts/runInMame.sh runs headless (-video none plus SDL_VIDEODRIVER=dummy). If you're calling mame yourself, add those flags.

git: refusing to fetch into ... for llvm-mos

installLlvmMos.sh refuses to refresh the LLVM clone if it has local modifications or is on a non-main branch — because our backend symlinks are stitched into the clone and a destructive refresh would stomp them.

If you really want to refresh from upstream:

bash scripts/updateLlvmMos.sh

That script handles the symlinks safely.

Disk fills up during the LLVM build

A full LLVM build needs ~12 GB of temporary build artifacts (cmake's intermediate .o files, .a archives, etc.) on top of the 5 GB source tree. Free ~15 GB before running setup.sh.

Once the build completes, the intermediate artifacts under tools/llvm-mos-build/CMakeFiles/ can be deleted — the binaries under tools/llvm-mos-build/bin/ are self-contained:

rm -rf tools/llvm-mos-build/CMakeFiles tools/llvm-mos-build/lib

But this disables incremental rebuilds. Re-running setup.sh recreates everything.

Calypsi install fails / I don't want it

Calypsi is only used by the compare/ benchmarks for output-quality comparison. Everything else works without it. Skip it:

./setup.sh --skip-calypsi

MAME version mismatch warnings

We're tested against MAME 0.240 ROMs running on whatever MAME apt ships with (typically 0.260+). Cross-version ROMs usually work but you may see "BAD CHECKSUM" warnings. These are warnings, not errors — boot proceeds normally.

If you want a clean match, the ROM 03 set from MAME 0.240 is apple2gs.zip mirrored on archive.org; the installer pulls that exact version.

Smoke test fails on a single check

If scripts/smokeTest.sh fails one check but otherwise looks fine, it might be MAME timing. Try:

MAME_CHECK_FRAME=600 MAME_SECS=12 bash scripts/smokeTest.sh

Some demos (especially the larger toolbox ones) need more wall-clock time to fully draw. Persistent failures are real bugs — file an issue with the failing check's output.

Where to go next