AI_Slop/65816-llvm-mos
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

More codegen improvements.

Browse source
This commit is contained in:
Scott Duensing 2026-10-06 17:01:51 -05:00
parent d4b2f4b535
commit 02ebba390a
85 changed files with 10957 additions and 1202 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

193
STATUS.md
View file

@ -32,7 +32,19 @@ which runs correctly under MAME (apple2gs).
- Loops with goto / break / continue, nested loops, state machines.
- `<stdarg.h>` varargs with int / long / unsigned long long mixed args.
- Heap: `malloc` / `free` (libc.c first-fit allocator) — linked-list
reverse with `cons` works; free-list coalesce verified.
reverse with `cons` works; free-list coalesce verified. Under GS/OS
and GNO the heap is a locked Memory Manager handle (`__heapInitMM`);
bare metal uses link816's window above BSS, stopped at the $C000 I/O
page in banks $00/$01/$E0/$E1. Running the heap out returns NULL
(smoke: bare-metal exhaustion, GS/OS 26 KB BSS + heap + NewHandle).
- `memcpy` / `memmove` / `memset` (libgcc.s): MVN / MVP block moves at
7 cycles a byte (the old C byte loops were ~70 / ~100 / ~55), a 16-bit
`[dp],y` loop below 34 bytes, runs split at every 64 KB bank boundary
of source and destination, a per-call stub on the stack (interrupt- and
thread-safe, any code bank), DBR and D preserved. memset is a 16-bit
store loop (5.9 cycles a byte). Smoke: byte-exact model over every
length class / bank crossing / overlap direction, DBR+D probe, and
injected mid-MVN interrupts whose handler moves memory.
- Strings: hand-rolled `strlen`, `strcmp`, `strcpy`, `strchr`, atoi/itoa
roundtrip.
- Soft-float (single): all four ops + comparisons, MAME-verified.
@ -75,7 +87,16 @@ which runs correctly under MAME (apple2gs).
cosh/tanh. Accuracy is in the ~1e-6 range — good enough for
typical numeric work, far short of glibc-quality. These are
slow (each call is dozens to hundreds of soft-double libcalls)
— pre-compute or cache when possible.
— pre-compute or cache when possible. Exact on the bits (C99
Annex F special values throughout): fmod and remainder (long
division, any quotient), modf / modff, frexp / ldexp (subnormals,
rounded subnormal results, no 16-bit exponent wrap), fpclassify /
isnormal in the argument's own width (subnormals included). hypot
scales instead of overflowing; atan2 has every signed-zero /
infinity case. The soft-float itself flushes subnormal operands
and results to zero, so subnormals survive only these bit-level
functions. Smoke: tests/fpNan/genFpTables.py tables (special,
Annex F, edge) compared bit for bit on MAME.
- `setjmp` / `longjmp` from libgcc.s.
- Static constructors via crt0's init_array walk.
- `<stdio.h>` file I/O with two backends:
@ -85,20 +106,22 @@ which runs correctly under MAME (apple2gs).
- **GS/OS** — `fopen` falls through to `gsosOpen` for any path not
in the mfs table. Routes through the GS/OS class-1 dispatcher
via wrappers in `runtime/src/iigsGsos.s` (Open/Read/Write/Close/
SetMark/GetMark/SetEOF/GetEOF). The full stdio surface
Flush/SetMark/GetMark/SetEOF/GetEOF). stdio keeps no buffer of
its own; `fflush` (and `fflush(NULL)`) issues GS/OS Flush, so
written data and the file's EOF reach the volume without fclose
(smoke: write, fflush, stop MAME, read the file off the image;
without fflush the file is empty). The full stdio surface
(`fread/fwrite/fseek/ftell/fclose/fgetc/fputc/fputs/fgets/ungetc/
feof/ferror/clearerr/rewind/fprintf/vfprintf`) dispatches on
backend. link816 honors weak symbols so programs that don't use
the GS/OS backend don't have to link `iigsGsos.o`.
- **Validation status:** code path compiles, links, and runs under
`runViaFinder.sh --data` injection. `fopen` + `gsosOpen` hangs
when invoked under real GS/OS 6.0.2 (JSL $E100A8 doesn't return);
root cause not yet diagnosed. Stub-dispatcher GS/OS smoke (the
existing one) validates the wrapper contract independently. An
XFAIL'd end-to-end smoke is in `scripts/smokeTest.sh` gated
behind `GSOS_FILE_SMOKE=1` for use after the dispatcher path is
fixed. `runViaFinder.sh --data /PATH=local_file` is the
automated-injection mechanism for runtime-test data files.
- **Validation status:** end to end under real GS/OS 6.0.2 via
`runViaFinder.sh`: fopen / fread of an injected file (smoke "GS/OS
fopen/fread", green since 2026-06-02 -- the old hang was crt0Gsos
re-zeroing BSS over GS/OS state), fwrite + fflush onto the disk
image (smoke "GS/OS fflush"), and a 26 KB-BSS program's file round
trip. `runViaFinder.sh --data /PATH=local_file` injects data files;
`--save-data <file>` hands back the data disk after the run.
- stdin/stdout/stderr route through `putchar` as before.
- `<wchar.h>`: wcslen / wcscmp / wcsncmp / wcscpy / wcsncpy /
wcscat / wcschr / wcsrchr; mbtowc / wctomb / mbstowcs /
@ -455,6 +478,152 @@ Work is now optimization-focused; the toolchain is feature-complete
for the common-case C / minimal-C++ workload. Priority is speed
(cycle counts), not size.
**Recently landed (2026-10-05, speed pass 2):**
CoreMark 21800 -> 20583 ticks (-5.6%, CRCs match); Space Taxi's synced
attract segment 699 -> 751 presents (+7.4%, stRenderFrame -21% of its
cycles); UBER 51/51 hashes, input and sample-pixel rows +21..27%,
sprite draw/save +5..7%, no row slower (two readings 0.3-0.4% lower are
window quantization: traced, their per-iteration cycles went down);
corpus .text 765695 -> 741321 bytes (-3.2%). Each item below has a
W65816_DISABLE_* switch.
- **Global arrays are `lda G+K,x` / `sta G+K,x`** (DBR-relative abs,X;
i32 halves off one X) when the access is based on a sized global:
the index is lowered to 16 bits, K stays in the operand only for an
index known non-negative (GLOBAL_INDEX; global-index-abs.ll;
globalArrFill/Sum 6308/6479 -> 2387 cycles).
- **Multiplies**: a product known to fit 16 bits is a 16-bit multiply
(`sim->spr[idx]`'s u8 x 6-byte record no longer calls __umulhisi3),
and an i32 times a two-signed-digit constant (6, 10, 12, 24, ...) is
shifts and an add (MUL_SHIFT_ADD); NarrowI32Mul does the same for
products it proves narrow across blocks (NARROW_PRODUCT; CoreMark's
bitextract -> __mulhi3, -0.34% cycles).
- **Light calls**: libgcc.s helpers and memcpy/memmove/memset are
JSLlight (clobber A, X, Y, $F0 only), so values stay in IMG0..7
across them; loop homes take loops that call them, and run in
functions with more than 16 hot slots too (LIGHT_CALLS,
LOOP_HOMES_HOT).
- **isLegalAddressingMode**: no negative or scaled offsets, so LSR
stops building `adc #0xff..` pointer math (ADDR_MODES).
- **i8 constants** load as a 16-bit `lda #imm` (I8_IMM16).
- **IMG8..15 homes for far pointers across calls** (StackRelToImg,
after the loop homes): the entry swaps each slot with a free $C0..$CE
word, every return swaps back, FarPtrStageReuse addresses the
pointer in place. Taken only where it pays on every path (cheapest
path, loops once) or twice over by block frequency and a quarter on
every path (CALLEE_SAVED_HOMES).
- **Miscompiles fixed**: a spill's STAfi kept the opcode's implicit
def of A, so MachineCopyPropagation deleted a live `$a = COPY` before
it (spill-stafi-no-a-def.mir); a longjmp skips the returns that give
callers their IMG8..15 words back -- a returns-twice call now
clobbers IMG8..15, so the setjmp function keeps all eight for its
caller (setjmp-img-clobber.ll, tests/regress/setjmpImgSave.c in smoke).
- Rejected: coalescing heuristics (+5.9K..+16.8K instructions, CoreMark
+2.5..3.2%), region split off for single-register classes (+293
instructions), SimplifyCFG sink-common off (a new greedy fallback),
a wider addressing mode (sprite restore -1.1%), IMG8..15 homes
unconditionally (12 UBER rows down, audio tick -18%) or by expected
saving alone (spriteMasks, hookPerTickHigh, computeCollisions
+11..102%), per-loop swaps (+3.2K instructions, +1 present).
Left: Y-indexed versioned pointer loops; short branches where BRL
pairs sit today (~0.3% of Space Taxi, ~0.5% of CoreMark; link816
would have to relax them); CoreMark spends ~14% in multiply helpers.
**Recently landed (2026-10-04, speed pass):**
CoreMark 29450 -> 21800 ticks (-26% time, CRCs match); Space Taxi's
synced attract segment 575 -> 697 presents (+21%); every timed JoeyLib
UBER op at or above the old compiler (51/51 hashes, sprite draws +19..25%,
input +7..14%); corpus .text 935424 -> 764879 bytes (-18%).
- **Far pointer offsets and indexes ride in Y**: an i32 load/store's
second half peels its constant offset into `ldy #` like the first
(constants already lowered to REG_SEQUENCE included); a zero-extended
16-bit index whose known bits keep it in the bank goes into Y for
`[dp],y` instead of a 32-bit add (LD/ST/STB_PTR_IDX).
- **Frame slots ordered by access density** (orderFrameObjects, loop
depth weighted): hot slots sit near SP, so frames past 200 bytes
reach them `,s` instead of through $F6 (big-frame probe -3.2%).
- **StackSlotCleanup**: a slot that only forwards an IMG value is read
from the IMG (LDAimg); stores to slots nothing reads are erased and
the slots leave the frame (the AGI interpreter's vmStepCore frame
940 -> 216 bytes, agiVm .text -51%).
- **Spills and reloads**: a virtual Wide32 spills/reloads both halves
by sub-register (it moved the low half only); reloads keep A when it
is read after them (aLiveFrom, which follows successors and reads a
return's uses); the far ($F6) IMG reload stored its value; the wrap
walk counts non-A STAfi as N/Z corrupters.
- **Miscompile fixed: StackRelToImg's `inc dp` fold** took a tied op
after the store (`and #0x1ff` reading the incremented A) for a write
of A, so `head = (head + 1) & 511` masked the byte just copied instead:
JoeyLib's IIgs serial read never caught up with the ring's tail
(verify-iigs-serial-loss hung). Latent; this pass's new code shape
exposed it. Phase 11's STZ peephole had the same scan order
(dp-inc-fold-tied-use.ll).
- **W65816SplitLoopExits** (IR): a critical loop exit into PHIs gets its
own block unless the value leaving is the one the loop carries
(core_list_find -27%).
- **SinkLoopInc** sees through a short single-predecessor chain;
**NarrowI32Mul** sends sext16 x sext16 to the new `__mulhisi3`.
- **libgcc multiplies and 16-bit divides**: 8x8 quarter squares (table
indexed by Y and X), eight shift-right steps when one operand is below
256, sixteen otherwise (a 16-bit multiplier was ~730 cycles, now ~255);
one 16-bit divide core keeps the remainder in A and starts a dividend
below 256 eight steps in. `tests/regress/genMulDiv.py` (smoke) checks
every helper across operand size classes.
- **Stack releases by cost** (W65816Helpers::emitStackRelease, a cycle
worth two bytes): call cleanups and epilogues pick N/2 PLY,
TSC/CLC/ADC/TCS (A free: void calls, void returns) or TAY ... TYA; the
prologue drops TAY/TYA when no argument comes in A. It pulled up to
seven PLYs (35 cycles) after a call.
- **W65816ReturnPopMerge** (pre-emit): the releases on a path into a
return (last call's cleanup, epilogue, often behind a BRA to a shared
return block) become one, with a return of its own (jlDrawPixel's call
path 28 -> 9 cycles of release).
- **W65816CommuteReload** (pre-emit, last): `sta s ; lda x ; ora s`
(ORA/AND/EOR/ADC) is `sta s ; ora x` -- every far pointer null test.
- Rejected: Sched::Source (greedy allocation fails in 7 corpus TUs,
+982 instructions where it succeeds). Left: DP homes for loops that
call (Space Taxi's render loops call pasteCell per cell; `,s`
accesses are 24% of its cycles), Y-indexed versioned pointer loops.
**Recently landed (2026-10-04):**
- **Float/double compares against constants are integer tests** --
`combineFPSetCCConst` (W65816ISelLowering) rewrites what InstCombine
makes of sign-bit and magnitude tests (`x ==/!= 0.0`, `x </> 0.0`,
`fabs(x) < c`, `isnan`, `x == x`) into exact integer compares,
NaN / signed-zero / subnormal correct; each used to be a soft-float
libcall (float `== 0` 1347 -> 716 cycles, double `fabs(x) < 2.5`
1833 -> 835). i64 SETCC/BR_CC/SELECT_CC lower as one four-word
CMP/SBC chain (the old lowering got 63 i64 predicates wrong).
- **W65816SinkLoopInc** (pre-RA): a loop's pointer/counter step moves
below the last use of the old value, so the pointer is bumped in place
and the back-edge copy goes (sumPtr -31%, copy16 -20.5%, strcpy -20%).
Loop-local far-pointer DP homes now also cover loops whose exits are
shared or carry live A / flags (copy-then-call loops -26..-34%).
- **SpillToX bridges merged spill slots** dead after the reload
(two-loop probe -14.8%); X live-out and a PHP inside the window are
now checked.
- **Address of a weak function** takes the program bank ($BE) when the
function is linked and null when not (it was bank 0, so a call
through it crashed under GS/OS).
- **Four older miscompiles fixed**: StackRelToImg promoted incoming
stack arguments of small PHA-frame functions (libc `ungetc`, so scanf
pushback), and its `lda n,s ; pha ; lda n,s` elision deleted callee-
save restores (wmemcpy, rename, fminf, 11 Lua functions returned with
$C0 clobbered); StackSlotCleanup's wrap walk took byte stores as flag
definers (5th miss of that class); LowerWide32's never-reached
LDAptr32S rewrite was removed.
- Runtime: exact `math.c` edge cases, MVN `memcpy`/`memmove`/`memset`,
`fflush` -> GS/OS Flush, bare-metal heap stops at the I/O page (see
"What works"). CoreMark +2.3% over 2026-09-29, .text -1.0% across
the corpus. Left: Y-indexed versioned pointer loops, DP homes for
non-innermost loops and $F6 frames (the largest remaining staging
pools: snprintf, Space Taxi's render and sim), homes across calls
(measured break-even).
**Recently landed (2026-07-03):**
- **StackSlotCleanup flag-wrap correctness fix** — a conditional branch

BIN
demos/rsrcProbe.apl
View file

Binary file not shown.

4
docs/USAGE.md
View file

@ -246,7 +246,7 @@ Each argument:
| `--text-base 0x1000` | Where the code goes in memory. `0x1000` is conventional (first 4 KB of bank 0 is reserved for stack + zero page). |
| `runtime/crt0.o` | **Must come first.** The C runtime startup — sets up the stack, calls `main`, halts cleanly on return. |
| `runtime/libc.o` | Core C library (`printf`, `malloc`, `strlen`, etc.). |
| `runtime/libgcc.o` | Compiler-provided helpers for things the 65816 can't do natively (16×16 multiply, 32-bit divide, etc.). Required for almost every program. |
| `runtime/libgcc.o` | Compiler-provided helpers for things the 65816 can't do natively (16×16 multiply, 32-bit divide, etc.), plus `memcpy` / `memmove` / `memset` (MVN / MVP block moves; the compiler emits calls to them for struct copies and clears). Required for almost every program. |
| `hello.o` | Your code. |
`link816` will print something like:
@ -388,7 +388,7 @@ the rest.
| `runtime/crt0.o` | **Always.** C runtime startup. |
| `runtime/crt0Gsos.o` | Instead of `crt0.o` for programs launched by the GS/OS Loader. |
| `runtime/libc.o` | `printf`, `malloc`, `strlen`, the usual. Almost always. |
| `runtime/libgcc.o` | Compiler helpers — multiply, divide, shift. Almost always. |
| `runtime/libgcc.o` | Compiler helpers — multiply, divide, shift — and `memcpy` / `memmove` / `memset`. Almost always. |
| `runtime/snprintf.o` | If you use `sprintf` / `snprintf` / `vsnprintf`. |
| `runtime/sscanf.o` | If you use `sscanf` / `vsscanf` / `fscanf`. |
| `runtime/softDouble.o` | If you use `double`-precision arithmetic anywhere. |

10
runtime/include/c++/cmath
View file

@ -10,7 +10,7 @@
// inherited as-is from <math.h>. `std::HUGE_VAL_v` is provided as a
// constexpr alias since macros can't live inside namespaces.
//
// Per the C++ standard, isnan/isinf/isfinite/signbit/fpclassify must be
// Per the C++ standard, isnan/isinf/isfinite/signbit/fpclassify/isnormal must be
// functions when <cmath> is in scope (not C-style type-generic macros).
// We #undef the <math.h> macros and re-declare them as inline functions
// in namespace std and at global scope.
@ -32,6 +32,8 @@
#undef islessequal
#undef islessgreater
#undef isunordered
#undef fpclassify
#undef isnormal
inline bool isnan(double x) { return ::__isnan_d(x) != 0; }
inline bool isinf(double x) { return ::__isinf_d(x) != 0; }
@ -43,6 +45,11 @@ inline bool isless(double x, double y) { return __builtin_isless(x, y); }
inline bool islessequal(double x, double y) { return __builtin_islessequal(x, y); }
inline bool islessgreater(double x, double y) { return __builtin_islessgreater(x, y); }
inline bool isunordered(double x, double y) { return __builtin_isunordered(x, y); }
// fpclassify(double) is <math.h>'s extern "C" function; a float keeps its
// own width (see <math.h>).
inline int fpclassify(float x) { return ::__fpclassify_f(x); }
inline bool isnormal(float x) { return ::__fpclassify_f(x) == FP_NORMAL; }
inline bool isnormal(double x) { return ::fpclassify(x) == FP_NORMAL; }
namespace std {
@ -177,6 +184,7 @@ using ::scalblnf;
// ---- Classification (function form) ---------------------------------
using ::fpclassify;
using ::isnormal;
// ---- Comparison (function form per C++ <cmath>) ---------------------
using ::isgreater;

10
runtime/include/math.h
View file

@ -160,12 +160,20 @@ double scalbln (double x, long n);
float scalblnf(float x, long n);
// ---- Classification ------------------------------------------------
// fpclassify classifies in the argument's own width: a float goes to
// __fpclassify_f, so a float subnormal is FP_SUBNORMAL rather than the
// zero the widening to double would make it (the soft-float flushes
// subnormals). The values must match runtime/src/math.c's copies.
#define FP_NAN 0
#define FP_INFINITE 1
#define FP_NORMAL 2
#define FP_SUBNORMAL 3
#define FP_ZERO 4
int fpclassify(double x);
int (fpclassify)(double x);
int __fpclassify_f(float x);
#define fpclassify(x) (sizeof(x) == sizeof(float) ? \
__fpclassify_f((float)(x)) : (fpclassify)((double)(x)))
#define isnormal(x) (fpclassify(x) == FP_NORMAL)
// ---- Common constants -----------------------------------------------
// (Not in C99 strict, but defined by glibc/BSD math.h and widely used.)

6
runtime/src/iigsGsos.s
View file

@ -50,6 +50,7 @@
.globl gsosRead
.globl gsosWrite
.globl gsosClose
.globl gsosFlush
.globl gsosGetEOF
.globl gsosSetEOF
.globl gsosSetMark
@ -128,6 +129,11 @@ gsosWrite:
gsosClose:
gsosDispatch 0x2014, __gsosClosePb
; FlushGS: write the file's buffered blocks and directory entry (EOF)
; to the volume. libc's fflush reaches it through a hook fopen installs.
gsosFlush:
gsosDispatch 0x2015, __gsosFlushPb
gsosGetEOF:
gsosDispatch 0x2019, __gsosGetEofPb

6
runtime/src/iigsGsosStub.s
View file

@ -92,9 +92,9 @@
; lying about success through the universal-success dispatcher.
;
; NOTE: this file does NOT define per-call symbols (gsosOpen, gsosRead,
; gsosWrite, gsosClose, gsosGetEOF, gsosSetEOF, gsosSetMark, gsosGetMark,
; gsosCreate, gsosDestroy, gsosChangePath, gsosGetPrefix, gsosGetFileInfo,
; gsosGetDirEntry). libc.c declares those as
; gsosWrite, gsosClose, gsosFlush, gsosGetEOF, gsosSetEOF, gsosSetMark,
; gsosGetMark, gsosCreate, gsosDestroy, gsosChangePath, gsosGetPrefix,
; gsosGetFileInfo, gsosGetDirEntry). libc.c declares those as
; undefined-weak, so when the stub is linked instead of iigsGsos.o
; they resolve to address 0 -- the libc-side callers all gate through
; __gsosAvailable() and short-circuit before dereferencing them. Any

117
runtime/src/libc.c
View file

@ -1,6 +1,6 @@
// Minimal libc for the W65816 backend. Provides:
// string.h: memcpy, memset, memmove, memcmp, strlen, strcpy, strcmp,
// strncpy, strncmp, strchr, strrchr
// string.h: memcmp, strlen, strcpy, strcmp, strncpy, strncmp, strchr,
// strrchr (memcpy, memmove and memset are in libgcc.s)
// ctype.h: isdigit, isalpha, isalnum, isspace, isupper, islower,
// toupper, tolower, isxdigit, isprint, ispunct
// stdlib.h: abs, labs, atoi
@ -150,6 +150,7 @@ extern u16 gsosOpen (__GsosOpenParm *p) __attribute__((weak, retain, used))
extern u16 gsosRead (__GsosIORecGS *p) __attribute__((weak, retain, used));
extern u16 gsosWrite (__GsosIORecGS *p) __attribute__((weak, retain, used));
extern u16 gsosClose (__GsosRefNumRecGS *p) __attribute__((weak, retain, used));
extern u16 gsosFlush (__GsosRefNumRecGS *p) __attribute__((weak, retain, used));
extern u16 gsosGetEOF (__GsosEOFRecGS *p) __attribute__((weak, retain, used));
extern u16 gsosSetEOF (__GsosSetPosRecGS *p) __attribute__((weak, retain, used));
extern u16 gsosSetMark(__GsosSetPosRecGS *p) __attribute__((weak, retain, used));
@ -169,7 +170,7 @@ extern u16 gsosGetDirEntry(__GsosDirEntryParm *p) __attribute__((weak, retain, u
// source of truth that distinguishes "real GS/OS available" from
// "stub linked" from "no GS/OS surface at all", so newly-added
// wrappers can refuse to silently lie about success.
extern int __gsosIsRealImpl __attribute__((weak, retain, used));
extern int __gsosIsRealImpl __attribute__((weak));
int __gsosAvailable(void) {
if (&__gsosIsRealImpl == (int *)0) {
@ -179,31 +180,11 @@ int __gsosAvailable(void) {
}
// ---- string.h ----
void *memcpy(void *dst, const void *src, size_t n) {
char *d = (char *)dst;
const char *s = (const char *)src;
while (n--) *d++ = *s++;
return dst;
}
void *memmove(void *dst, const void *src, size_t n) {
char *d = (char *)dst;
const char *s = (const char *)src;
if (d < s) {
while (n--) *d++ = *s++;
} else {
d += n; s += n;
while (n--) *--d = *--s;
}
return dst;
}
void *memset(void *dst, int c, size_t n) {
char *d = (char *)dst;
while (n--) *d++ = (char)c;
return dst;
}
//
// memcpy, memmove and memset are MVN / MVP block moves in libgcc.s.
void *memcpy(void *dst, const void *src, size_t n);
void *memmove(void *dst, const void *src, size_t n);
void *memset(void *dst, int c, size_t n);
int memcmp(const void *a, const void *b, size_t n) {
const u8 *p = (const u8 *)a;
@ -519,8 +500,8 @@ char *stpncpy(char *dst, const char *src, size_t n) {
// Linker-supplied weak symbols; fallback to fixed defaults so a static
// link without crt0 still has SOMETHING.
extern char __heap_start[] __attribute__((weak, retain, used));
extern char __heap_end[] __attribute__((weak, retain, used));
extern char __heap_start[] __attribute__((weak));
extern char __heap_end[] __attribute__((weak));
#define HEAP_DEFAULT_START ((char *)0x4000)
#define HEAP_DEFAULT_END ((char *)0xBF00)
@ -554,10 +535,34 @@ static void heapSetWindow(char *start, char *end) {
}
// Banks $00, $01, $E0 and $E1 have the I/O page at $C000-$CFFF. link816's
// heap window runs from the end of BSS to $FFFE of the entry bank, which is
// plain RAM under GS/OS (where __heapInitMM replaces the window with a
// Memory Manager handle anyway), but a bare-metal crt0.s program lives in
// bank 0, where the window spanned the soft switches: running the heap
// out zeroed them and took the machine down instead of returning NULL. A
// window in such a bank stops at the I/O page.
#define IO_PAGE_START 0xC000UL
#define BANK_MASK 0xFF0000UL
#define OFFSET_MASK 0x00FFFFUL
static int bankHasIoPage(unsigned long a) {
unsigned long bank = a & BANK_MASK;
return bank == 0x000000UL || bank == 0x010000UL ||
bank == 0xE00000UL || bank == 0xE10000UL;
}
static void mallocInitOnce(void) {
if (bumpPtr) return;
heapSetWindow(__heap_start ? __heap_start : HEAP_DEFAULT_START,
__heap_end ? __heap_end : HEAP_DEFAULT_END);
char *start = __heap_start ? __heap_start : HEAP_DEFAULT_START;
char *end = __heap_end ? __heap_end : HEAP_DEFAULT_END;
unsigned long s = (unsigned long)(void *)start;
if (bankHasIoPage(s) && (s & OFFSET_MASK) < IO_PAGE_START &&
((unsigned long)(void *)end & OFFSET_MASK) > IO_PAGE_START) {
end = (char *)(void *)((s & BANK_MASK) | IO_PAGE_START);
}
heapSetWindow(start, end);
}
// GS/OS and GNO: back the malloc heap with a locked Memory-Manager handle
@ -1178,7 +1183,7 @@ typedef struct __sFILE {
// Zero-cost when nothing fails; lets an app report WHY a write/read died
// (fwrite itself can only say "short") without a debugger on the target.
// __gsosLastOp values: 1=Open 2=Create 3=Read 4=Write 5=SetEOF 6=SetMark
// 7=Close 8=GetEOF 9=GetMark.
// 7=Close 8=GetEOF 9=GetMark 10=Flush.
u16 __gsosLastErr;
u16 __gsosLastOp;
u16 __gsosLastRefNum;
@ -1318,7 +1323,50 @@ int fputs(const char *s, FILE *stream) {
return -1;
}
int fflush(FILE *stream) { (void)stream; return 0; }
// fflush's way to GS/OS Flush ($2015), installed by fopen's GS/OS path, so
// a program that never opens a GS/OS file does not link the dispatch
// wrappers through fflush (as __autoDeleteFn below does for remove).
// `volatile` for the same reason as __autoDeleteFn.
typedef u16 (*__GsosFlushFn)(__GsosRefNumRecGS *p);
static __GsosFlushFn volatile __gsosFlushFn = (__GsosFlushFn)0;
// stdio keeps no buffer of its own -- fputc / fwrite hand every byte to
// GS/OS at once -- but the FST keeps written blocks and the file's EOF in
// memory until Flush or Close, so a crash or a power-off loses whatever
// was written since the open. fflush issues Flush for a GS/OS stream;
// other streams have nothing to push. Returns 0, or EOF (-1) with the
// stream's error flag set and __gsosLastOp = 10 when GS/OS fails.
static int fflushOne(FILE *stream) {
if (stream->kind != FILE_KIND_GSOS || !__gsosFlushFn) {
return 0;
}
__GsosRefNumRecGS r = { 1, stream->refNum };
u16 rc = __gsosFlushFn(&r);
if (rc != 0) {
stream->err = 1;
__gsosLatch(10, rc, stream->refNum, 0, 0);
return -1;
}
return 0;
}
// fflush(NULL) flushes every open stream.
int fflush(FILE *stream) {
if (stream) {
return fflushOne(stream);
}
int rc = 0;
// Kept a loop: unrolled it is 21 calls for a rarely used form.
#pragma clang loop unroll(disable)
for (int i = 3; i < MFS_MAX_FILES; i++) {
if (fflushOne(&__mfs[i]) != 0) {
rc = -1;
}
}
return rc;
}
// Indirect hook for tmpfile-auto-delete on fclose. When `tmpfile()`
// is called it installs `remove` here; programs that never call
@ -1627,6 +1675,7 @@ FILE *fopen(const char *path, const char *mode) {
f->unget = -1;
f->path = path;
f->refNum = op.refNum;
__gsosFlushFn = gsosFlush;
if (truncate) {
// "w" / "w+" — truncate to zero length.
__GsosSetPosRecGS e = { 3, op.refNum, 0, 0 };

2
runtime/src/libcGno.c
View file

@ -46,6 +46,7 @@ typedef struct { uint16_t pCount; void *oldPathname; void *newPathname; } GnoCha
#define GSOS_READ 0x2012
#define GSOS_WRITE 0x2013
#define GSOS_CLOSE 0x2014
#define GSOS_FLUSH 0x2015
#define GSOS_SETMARK 0x2016
#define GSOS_GETMARK 0x2017
#define GSOS_SETEOF 0x2018
@ -75,6 +76,7 @@ KEEP uint16_t gsosOpen(GnoOpenParm *p) { return __gnoGsosCall(p, GSOS_OPEN);
KEEP uint16_t gsosRead(GnoIORec *p) { return __gnoGsosCall(p, GSOS_READ); }
KEEP uint16_t gsosWrite(GnoIORec *p) { return __gnoGsosCall(p, GSOS_WRITE); }
KEEP uint16_t gsosClose(GnoRefNumRec *p) { return __gnoGsosCall(p, GSOS_CLOSE); }
KEEP uint16_t gsosFlush(GnoRefNumRec *p) { return __gnoGsosCall(p, GSOS_FLUSH); }
KEEP uint16_t gsosGetEOF(GnoEOFRec *p) { return __gnoGsosCall(p, GSOS_GETEOF); }
KEEP uint16_t gsosSetEOF(GnoEOFRec *p) { return __gnoGsosCall(p, GSOS_SETEOF); }
KEEP uint16_t gsosSetMark(GnoMarkRec *p) { return __gnoGsosCall(p, GSOS_SETMARK); }

1863
runtime/src/libgcc.s
View file

File diff suppressed because it is too large Load diff

551
runtime/src/math.c
View file

@ -49,6 +49,28 @@ static float fFromBits(uint32_t b) {
#define D_SIGN 0x8000000000000000ULL
#define D_INF 0x7FF0000000000000ULL
#define D_ONE 0x3FF0000000000000ULL
#define D_MANT 0x000FFFFFFFFFFFFFULL // the stored mantissa bits
#define D_LEAD 0x0010000000000000ULL // the implicit leading 1
#define D_HUGE2 0x7FE0000000000000ULL // 2^1023: doubling it overflows
#define D_TINY2 0x0020000000000000ULL // 2^-1021: halving it loses a bit
#define D_BIAS 1023
#define D_EMAX 1023 // the largest finite exponent
#define D_EMIN (-1022) // the smallest normal exponent
#define D_MBITS 52 // stored mantissa bits
#define F_SIGN 0x80000000UL
#define F_MANT 0x007FFFFFUL
#define F_EXP 0xFFu // a float's biased exponent field
#define F_BIAS 127
#define F_MBITS 23
// fpclassify's answers. MUST stay in lock-step with math.h, which this
// file cannot include (the smoke test builds it without the runtime's
// include path, where <math.h> would be the host's).
#define FP_NAN 0
#define FP_INFINITE 1
#define FP_NORMAL 2
#define FP_SUBNORMAL 3
#define FP_ZERO 4
// Used before their definitions (this file does not include math.h).
@ -101,6 +123,56 @@ static int isZeroBits(uint64_t b) {
}
// The mantissa of the finite, nonzero magnitude a with its leading 1 at
// bit 52, and its unbiased exponent: a subnormal is shifted up until it
// has one (the soft-float flushes subnormals, so this works on the bits).
static uint64_t dUnpack(uint64_t a, int *eOut) {
int e = (int)(a >> D_MBITS);
uint64_t m = a & D_MANT;
if (e == 0) {
e = 1;
while ((m & D_LEAD) == 0) {
m <<= 1;
e--;
}
} else {
m |= D_LEAD;
}
*eOut = e - D_BIAS;
return m;
}
// sign | m * 2^(e - 52), m's leading 1 at bit 52. Past the largest
// exponent it is an infinity; below the smallest normal one it is rounded
// to nearest, ties to even, into a subnormal or zero (a carry into bit 52
// lands on the smallest normal, as it should). e is a long so that huge
// ldexp scales cannot wrap a 16-bit int.
static double dBuild(uint64_t sign, uint64_t m, long e) {
unsigned shift;
uint64_t q;
uint64_t rest;
uint64_t half;
if (e > D_EMAX) {
return dFromBits(sign | D_INF);
}
if (e >= D_EMIN) {
return dFromBits(sign | ((uint64_t)(e + D_BIAS) << D_MBITS) | (m & D_MANT));
}
if (e < D_EMIN - D_MBITS - 1) {
return dFromBits(sign); // under half the smallest subnormal
}
shift = (unsigned)(D_EMIN - e);
q = m >> shift;
rest = m & (((uint64_t)1 << shift) - 1);
half = (uint64_t)1 << (shift - 1);
if (rest > half || (rest == half && (q & 1))) {
q++;
}
return dFromBits(sign | q);
}
// sqrt and log of an infinity or a NaN: +inf maps to itself, a NaN to
// itself quieted (x + x does both), -inf is a domain error like any other
// negative argument. Without this the series below turned +inf into a
@ -182,10 +254,9 @@ NO_SHRINK(ceil) float ceilf(float x) {
}
// fmod(x, y) = x - trunc(x / y) * y, which takes the sign of x. The
// special cases follow C99 Annex F: a NaN operand, an infinite x or a zero
// y give NaN; an infinite y or a zero x give x back. They are screened on
// the bits, which also replaced a `y == 0.0` libcall compare.
// fmod's special cases follow C99 Annex F: a NaN operand, an infinite x or
// a zero y give NaN; an infinite y gives x back. They are screened on the
// bits, which also replaced a `y == 0.0` libcall compare.
static COLD double fmodSpecial(double x, double y) {
uint64_t xb = dToBits(x);
uint64_t yb = dToBits(y);
@ -195,31 +266,59 @@ static COLD double fmodSpecial(double x, double y) {
if (isNonFinite(xb) || isZeroBits(yb)) {
return __builtin_nan(""); // domain error
}
return x; // infinite y, or a zero x
return x; // infinite y
}
// fmod(x, y) is always exactly representable, so it is computed exactly:
// long division of the mantissas, one quotient bit a step, keeping only
// the remainder (as musl and fdlibm do). It takes x's sign, a zero one
// included. x - trunc(x / y) * y, used before, was off by whole multiples
// of y once x / y rounded (fmod(1e20, 0.1), fmod(-1e300, 7)). The loop
// runs once per bit of exponent difference, so fmod(DBL_MAX, 1e-300) is
// slow but exact.
double fmod(double x, double y) {
uint64_t xb = dToBits(x);
uint64_t yb = dToBits(y);
if (isNonFinite(xb) || isNonFinite(yb) || isZeroBits(yb) || isZeroBits(xb)) {
uint64_t sign;
uint64_t xa;
uint64_t ya;
uint64_t mx;
uint64_t my;
int ex;
int ey;
if (isNonFinite(xb) || isNonFinite(yb) || isZeroBits(yb)) {
return fmodSpecial(x, y);
}
double q = x / y;
// trunc: floor for positives, ceil for negatives. The sign test reads
// the bits, which is cheaper than a `<` libcall compare.
double t;
if (dToBits(q) >> 63) {
t = -floor(-q);
} else {
t = floor(q);
sign = xb & D_SIGN;
xa = xb & ~D_SIGN;
ya = yb & ~D_SIGN;
if (xa <= ya) {
// |x| < |y| is its own remainder (a zero x too); |x| == |y| leaves 0.
return (xa == ya) ? dFromBits(sign) : x;
}
double r = x - t * y;
// An exact multiple leaves x + -x, which is +0; the result takes x's sign.
if (isZeroBits(dToBits(r))) {
return (xb >> 63) ? -0.0 : 0.0;
mx = dUnpack(xa, &ex);
my = dUnpack(ya, &ey);
for (; ex > ey; ex--) {
if (mx >= my) {
mx -= my;
if (mx == 0) {
return dFromBits(sign);
}
return r;
}
mx <<= 1;
}
if (mx >= my) {
mx -= my;
if (mx == 0) {
return dFromBits(sign);
}
}
while ((mx & D_LEAD) == 0) {
mx <<= 1;
ex--;
}
return dBuild(sign, mx, ex);
}
@ -399,6 +498,8 @@ float powf(float x, float y) {
#define M_PI 3.14159265358979323846
#define M_2PI 6.28318530717958647692
#define M_HALF_PI 1.57079632679489661923
#define M_QUART_PI 0.78539816339744830962
#define M_3QUAR_PI 2.35619449019234492885
#define M_LN2 0.69314718055994530942
@ -523,6 +624,18 @@ float expf(float x) {
}
// 1/(2n+1) for the log and atan series: a table of the correctly rounded
// constants (the run-time division gave the same values at ~60000 cycles
// each).
#define LOG_TERMS 8
#define ATAN_TERMS 18 // ~1e-15 at tan(pi/8)
static const double kInvOdd[ATAN_TERMS + 1] = {
1.0, 1.0 / 3.0, 1.0 / 5.0, 1.0 / 7.0, 1.0 / 9.0, 1.0 / 11.0, 1.0 / 13.0,
1.0 / 15.0, 1.0 / 17.0, 1.0 / 19.0, 1.0 / 21.0, 1.0 / 23.0, 1.0 / 25.0,
1.0 / 27.0, 1.0 / 29.0, 1.0 / 31.0, 1.0 / 33.0, 1.0 / 35.0, 1.0 / 37.0
};
// log: x = 2^k * m where m in [1, 2). log(x) = k*ln2 + log(m).
// log(m) for m in [1,2) computed via the substitution u = (m-1)/(m+1)
// giving log(m) = 2 * (u + u^3/3 + u^5/5 + ...) — converges fast for
@ -546,17 +659,11 @@ double log(double x) {
b = (b & ~((uint64_t)0x7FF << 52)) | ((uint64_t)1023 << 52);
double m;
__builtin_memcpy(&m, &b, sizeof(m));
// 1/(2n+1) for the series: a table of the correctly rounded constants
// (the run-time division gave the same values at ~60000 cycles each).
static const double kInvOdd[9] = {
1.0, 1.0 / 3.0, 1.0 / 5.0, 1.0 / 7.0, 1.0 / 9.0, 1.0 / 11.0,
1.0 / 13.0, 1.0 / 15.0, 1.0 / 17.0
};
double u = (m - 1.0) / (m + 1.0);
double u2 = u * u;
double term = u;
double s = term;
for (int n = 1; n <= 8; n++) {
for (int n = 1; n <= LOG_TERMS; n++) {
term = term * u2;
s = s + term * kInvOdd[n];
}
@ -569,37 +676,50 @@ float logf(float x) {
}
// atan: range-reduce |x| > 1 via atan(x) = pi/2 - atan(1/x), then
// further reduce via atan(x) = 2*atan(x / (1 + sqrt(1 + x*x))).
// For the reduced argument, Taylor: x - x^3/3 + x^5/5 - ...
// atan: |x| >= 1 goes through atan(x) = pi/2 - atan(1/x), and from
// tan(pi/8) on, atan(t) = pi/4 + atan((t - 1) / (t + 1)), which leaves
// |t| <= tan(pi/8) ~ 0.414 for the series t - t^3/3 + t^5/5 - ..., whose
// terms then shrink 0.17x apiece. Without the pi/4 step (which the old
// comment promised but the code never did) the series ran at |t| up to 1,
// where 25 terms are only ~1% accurate: atan(1) was 0.795, not 0.785.
#define ATAN_SHIFT_TOP 0x3FDAu // top word of 0.40625, just under tan(pi/8)
double atan(double x) {
uint64_t b;
__builtin_memcpy(&b, &x, sizeof(b));
uint64_t b = dToBits(x);
int sign = (int)(b >> 63) & 1;
if (sign) {
b ^= ((uint64_t)1 << 63);
__builtin_memcpy(&x, &b, sizeof(x));
}
int invert = 0;
// Test |x| > 1 via bit pattern: exponent > 1023 means value >= 1.
int e = (int)((b >> 52) & 0x7FF) - 1023;
if (e >= 0) {
int shift = 0;
double x2;
double term;
double s;
if (isNanBits(b)) {
return x + x;
}
if (sign) {
b ^= D_SIGN;
x = dFromBits(b);
}
// |x| >= 1 from the bits: the exponent is 1023 or more.
if (((topWord(b) >> 4) & 0x7FFu) >= (unsigned)D_BIAS) {
x = 1.0 / x;
invert = 1;
}
// Taylor for |x| <= 1: x - x^3/3 + x^5/5 - x^7/7 + x^9/9 - ...
// Need many terms when |x| close to 1; cap at 30 for ~1e-6 accuracy.
double x2 = x * x;
double term = x;
double s = term;
for (int n = 1; n <= 25; n++) {
term = term * x2;
double denom = (double)(2 * n + 1);
if (n & 1) {
s = s - term / denom;
} else {
s = s + term / denom;
if (topWord(dToBits(x)) >= ATAN_SHIFT_TOP) {
x = (x - 1.0) / (x + 1.0);
shift = 1;
}
x2 = x * x;
term = x;
s = term;
for (int n = 1; n <= ATAN_TERMS; n++) {
term = term * x2;
if (n & 1) {
s = s - term * kInvOdd[n];
} else {
s = s + term * kInvOdd[n];
}
}
if (shift) {
s = M_QUART_PI + s;
}
if (invert) {
s = M_HALF_PI - s;
@ -616,24 +736,47 @@ float atanf(float x) {
}
double atan2(double y, double x) {
// Quadrant adjustment.
uint64_t xb, yb;
__builtin_memcpy(&xb, &x, sizeof(xb));
__builtin_memcpy(&yb, &y, sizeof(yb));
int xNeg = (int)(xb >> 63) & 1;
int yNeg = (int)(yb >> 63) & 1;
// Strip sign of x for the quotient; restore quadrant later.
uint64_t xa = xb & ~((uint64_t)1 << 63);
if (xa == 0) {
// x == 0 (handles +0 and -0): return ±pi/2
return yNeg ? -M_HALF_PI : M_HALF_PI;
// atan2 with a zero, an infinity or a NaN, per C99 F.9.1.4: a NaN operand
// gives NaN; otherwise the result takes y's sign and its magnitude is
// y = inf: pi/4 (x = +inf), 3pi/4 (x = -inf), pi/2 (x finite);
// x = inf: 0 (x = +inf) or pi (x = -inf) for a finite y;
// y = 0: 0 for x > 0 or x = +0, pi for x < 0 or x = -0;
// x = 0: pi/2 for a finite, nonzero y.
// Every x = +-0 case used to give +-pi/2 (atan2(0, 0) too), and the
// infinite pairs went through inf / inf to NaN.
static COLD double atan2Special(double y, double x, uint64_t yb, uint64_t xb) {
uint64_t ya = yb & ~D_SIGN;
uint64_t xa = xb & ~D_SIGN;
int xNeg = (int)(xb >> 63);
double r;
if (isNanBits(xb) || isNanBits(yb)) {
return x + y;
}
double a = atan(y / x);
if (!xNeg) {
if (ya == D_INF) {
r = (xa == D_INF) ? (xNeg ? M_3QUAR_PI : M_QUART_PI) : M_HALF_PI;
} else if (xa == D_INF || ya == 0) {
r = xNeg ? M_PI : 0.0;
} else {
r = M_HALF_PI;
}
return (yb >> 63) ? -r : r;
}
// atan2: atan(y / x) moved to x's half-plane. A y / x that overflows or
// underflows still lands right (atan(+-inf) is +-pi/2, atan(+-0) +-0).
double atan2(double y, double x) {
uint64_t xb = dToBits(x);
uint64_t yb = dToBits(y);
double a;
if (isNonFinite(xb) || isNonFinite(yb) || isZeroBits(xb) || isZeroBits(yb)) {
return atan2Special(y, x, yb, xb);
}
a = atan(y / x);
if ((xb >> 63) == 0) {
return a;
}
return yNeg ? a - M_PI : a + M_PI;
return (yb >> 63) ? a - M_PI : a + M_PI;
}
@ -822,39 +965,21 @@ float fdimf(float x, float y) { return (float)fdim((double)x, (double)y); }
// ---- FP decomposition ----------------------------------------------
// ldexp(x, n) = x * 2^n. Implemented by adjusting the exponent field
// in place. Handles subnormals and overflow only crudely (overflow →
// infinity; underflow → zero).
// ldexp(x, n) = x * 2^n on the bits: exact unless the result leaves the
// double range, where it is +-inf or a correctly rounded subnormal / zero.
// It used to multiply a zero or subnormal x by 2 n times (ldexp(0, 30000)
// ran 30000 soft multiplies, and the soft-float flushed the subnormal to
// 0 on the first), and its 16-bit exponent sum wrapped: ldexp(1, 32767)
// was 0, not inf.
double ldexp(double x, int n) {
uint64_t b = dToBits(x);
int e = (int)((b >> 52) & 0x7FF);
if (e == 0 || e == 0x7FF) {
// 0 or denorm or inf/nan — return as-is for inf/nan, else
// fall back to multiplication (handles subnormals correctly
// enough for typical use).
if (e == 0x7FF) return x;
// For zero / subnormal, multiply by 2^n via repeated *2.
if (n > 0) {
while (n--) x *= 2.0;
uint64_t m;
int e;
if (isNonFinite(b) || isZeroBits(b)) {
return x;
}
if (n < 0) {
while (n++) x *= 0.5;
return x;
}
return x;
}
int newE = e + n;
if (newE >= 0x7FF) {
// Overflow → ±infinity.
return (b >> 63) ? -1e308 * 1e308 : 1e308 * 1e308;
}
if (newE <= 0) {
// Underflow → ±0 (skip subnormal handling).
return (b >> 63) ? -0.0 : 0.0;
}
b = (b & 0x800FFFFFFFFFFFFFULL) | ((uint64_t)newE << 52);
return dFromBits(b);
m = dUnpack(b & ~D_SIGN, &e);
return dBuild(b & D_SIGN, m, (long)e + n);
}
@ -865,15 +990,19 @@ float ldexpf(float x, int n) {
// frexp(x, *e): split x into normalized fraction in [0.5, 1) (or 0)
// and integer exponent. x = frac * 2^e.
// A subnormal x is normalized first (it used to come back unchanged with
// *eOut = 0); zeros, infinities and NaNs come back as they are, *eOut 0.
double frexp(double x, int *eOut) {
uint64_t b = dToBits(x);
int e = (int)((b >> 52) & 0x7FF);
if (e == 0) { *eOut = 0; return x; } // zero or subnormal
if (e == 0x7FF) { *eOut = 0; return x; } // inf or nan
*eOut = e - 1022; // bias adjustment for [0.5,1)
// Force exponent to 1022 (so result is in [0.5, 1)).
b = (b & 0x800FFFFFFFFFFFFFULL) | ((uint64_t)1022 << 52);
return dFromBits(b);
uint64_t m;
int e;
if (isNonFinite(b) || isZeroBits(b)) {
*eOut = 0;
return x;
}
m = dUnpack(b & ~D_SIGN, &e);
*eOut = e + 1;
return dFromBits((b & D_SIGN) | ((uint64_t)(D_BIAS - 1) << D_MBITS) | (m & D_MANT));
}
@ -884,19 +1013,54 @@ float frexpf(float x, int *eOut) {
// modf(x, *iptr): split into integer and fractional parts, both with
// the same sign as x.
// the same sign as x, on the bits. x - trunc(x) made modf(+-inf) a NaN
// (inf - inf) and gave a +0 fraction for -2.0 and -0.0. C99 F.9.3.12:
// modf(+-inf) is +-0 with *iptr = +-inf; a NaN is both parts.
double modf(double x, double *iptr) {
double ip = trunc(x);
*iptr = ip;
return x - ip;
uint64_t b = dToBits(x);
uint64_t sign = b & D_SIGN;
int e = (int)((topWord(b) >> 4) & 0x7FFu) - D_BIAS;
uint64_t frac;
if (e >= D_MBITS) {
*iptr = x; // integral, infinite or NaN
return isNanBits(b) ? x : dFromBits(sign);
}
if (e < 0) {
*iptr = dFromBits(sign); // |x| < 1, subnormals included
return x;
}
frac = D_MANT >> e; // the bits below the unit
if ((b & frac) == 0) {
*iptr = x;
return dFromBits(sign);
}
*iptr = dFromBits(b & ~frac);
return x - *iptr; // exact
}
// modff on the float's own bits: through double, a subnormal float
// flushed to zero on the widening.
float modff(float x, float *iptr) {
double ipd;
double frac = modf((double)x, &ipd);
*iptr = (float)ipd;
return (float)frac;
uint32_t b = fToBits(x);
uint32_t sign = b & F_SIGN;
int e = (int)((b >> F_MBITS) & F_EXP) - F_BIAS;
uint32_t frac;
if (e >= F_MBITS) {
*iptr = x;
return ((b & ~F_SIGN) > ((uint32_t)F_EXP << F_MBITS)) ? x : fFromBits(sign);
}
if (e < 0) {
*iptr = fFromBits(sign);
return x;
}
frac = F_MANT >> e;
if ((b & frac) == 0) {
*iptr = x;
return fFromBits(sign);
}
*iptr = fFromBits(b & ~frac);
return x - *iptr;
}
@ -948,22 +1112,51 @@ float expm1f(float x) { return (float)expm1((double)x); }
// ---- hypot ---------------------------------------------------------
//
// hypot(x, y) = sqrt(x*x + y*y). This implementation does NOT scale
// to avoid overflow — for |x|, |y| < ~1e150 the naive form is fine,
// past that you'd want the standard scale-by-max trick.
// hypot(x, y) = sqrt(x*x + y*y), with both operands first scaled (on the
// bits, exactly) by the power of two that brings the larger into [1, 2),
// so neither square can overflow or underflow: hypot(1e200, 1e200) was
// +inf and hypot(1e-200, 1e-200) was 0. The result is scaled back with
// dBuild, which rounds into the subnormals. A smaller operand more than
// HYPOT_SPAN binary orders below the larger cannot change the rounded
// result. C99 F.9.4.3: an infinite operand gives +inf even when the
// other is a NaN.
#define HYPOT_SPAN 60
double hypot(double x, double y) {
uint64_t xb = dToBits(x);
uint64_t yb = dToBits(y);
// C99 F.9.4.3: an infinite operand gives +inf even when the other is a
// NaN; otherwise a NaN operand gives NaN (through the arithmetic).
if ((xb & ~D_SIGN) == D_INF || (yb & ~D_SIGN) == D_INF) {
uint64_t xa = dToBits(x) & ~D_SIGN;
uint64_t ya = dToBits(y) & ~D_SIGN;
uint64_t t;
uint64_t mx;
uint64_t my;
uint64_t mr;
int ex;
int ey;
int er;
double sx;
double sy;
if (xa == D_INF || ya == D_INF) {
return __builtin_inf();
}
double xx = x * x;
double yy = y * y;
double s = xx + yy;
return sqrt(s);
if (xa > D_INF || ya > D_INF) {
return x + y; // a NaN, quieted
}
if (xa < ya) {
t = xa;
xa = ya;
ya = t;
}
if (ya == 0) {
return dFromBits(xa);
}
mx = dUnpack(xa, &ex);
my = dUnpack(ya, &ey);
if (ex - ey > HYPOT_SPAN) {
return dFromBits(xa);
}
sx = dFromBits(D_ONE | (mx & D_MANT));
sy = dFromBits(((uint64_t)(D_BIAS + ey - ex) << D_MBITS) | (my & D_MANT));
mr = dUnpack(dToBits(sqrt(sx * sx + sy * sy)), &er);
return dBuild(0, mr, (long)er + ex);
}
@ -1012,22 +1205,50 @@ float cbrtf(float x) {
// have. Callers who depend on extra precision must use Kahan summation
// or similar.
// From 2^28 up, x * x + 1 (or - 1) is x * x and log(x + sqrt(x * x)) is
// log(2x) = log(x) + ln 2; computing it that way also keeps x * x from
// overflowing (asinh(1e200) and acosh(1e200) were +inf).
#define HYPER_BIG 0x41B0000000000000ULL
// asinh is odd, so it runs on |x| and takes x's sign: log(x + sqrt(x * x
// + 1)) for a negative x cancels to log(0) = -inf (asinh(-1e10) was -inf,
// asinh(-inf) NaN, asinh(-0) +0).
double asinh(double x) {
// asinh(x) = log(x + sqrt(x*x + 1)) — stable for all finite x.
return log(x + sqrt(x * x + 1.0));
uint64_t b = dToBits(x);
uint64_t a = b & ~D_SIGN;
double ax = dFromBits(a);
double r;
if (isZeroBits(b)) {
return x;
}
if (isNonFinite(b)) {
return x + x;
}
if (a >= HYPER_BIG) {
r = log(ax) + M_LN2;
} else {
r = log(ax + sqrt(ax * ax + 1.0));
}
return (b >> 63) ? -r : r;
}
// acosh(x) = log(x + sqrt(x*x - 1)) for x >= 1, NaN below 1 (and for a
// NaN: the big-x branch takes it, as its bits compare above HYPER_BIG).
double acosh(double x) {
// acosh(x) = log(x + sqrt(x*x - 1)) for x >= 1. Returns NaN for x < 1.
if (x < 1.0) return __builtin_nanf("");
if (dToBits(x) >= HYPER_BIG) {
return log(x) + M_LN2;
}
return log(x + sqrt(x * x - 1.0));
}
double atanh(double x) {
// atanh(x) = 0.5 * log((1+x)/(1-x)) for |x| < 1. Returns +/-inf
// at x = +/-1 and NaN outside [-1, 1].
// at x = +/-1 and NaN outside [-1, 1]; atanh(+-0) is +-0 (the formula
// gives +0 for both).
if (isZeroBits(dToBits(x))) return x;
if (x >= 1.0) return (x == 1.0) ? __builtin_inff() : __builtin_nan("");
if (x <= -1.0) return (x == -1.0) ? -__builtin_inff() : __builtin_nan("");
return 0.5 * log((1.0 + x) / (1.0 - x));
@ -1059,18 +1280,48 @@ double nan(const char *tagp) {
float nanf(const char *tagp) { (void)tagp; return __builtin_nanf(""); }
// IEEE 754 remainder: x - n*y where n = x/y rounded to nearest, ties to
// even. Exact, like fmod, which it starts from (fdlibm's method): r =
// fmod(x, 2|y|) is x - n*y for the even n just below, then one or two
// subtractions of |y| (exact: r is within a factor of two of |y|) land on
// the nearest, ties going to the even n. A |y| so small that |y| / 2 is
// subnormal is scaled up first (ldexp is exact), because the soft-float
// flushes subnormal operands to zero. x - rint(x / y) * y, used before,
// rounded x / y (remainder(1e20, 0.1) was off by units of 0.1) and gave
// remainder(-4, 2) = +0, not -0: a zero result takes x's sign. C99
// F.9.7.2: a NaN operand, an infinite x or a zero y give NaN;
// remainder(x, +-inf) is x for a finite x.
#define REM_SCALE 64
double remainder(double x, double y) {
// IEEE 754 remainder: x - n*y where n = round-to-nearest-even(x/y),
// which is rint (round() rounded ties away: remainder(5, 2) was -1).
// remainder(x, +-inf) is x for a finite x (C99 F.9.7.2); x / y would
// make it 0 * inf = NaN.
if (y == 0.0) return __builtin_nanf("");
uint64_t xb = dToBits(x);
uint64_t yb = dToBits(y);
if ((yb & ~D_SIGN) == D_INF && !isNonFinite(dToBits(x))) {
return x;
uint64_t sign = xb & D_SIGN;
uint64_t ya = yb & ~D_SIGN;
int scaled = ya < D_TINY2;
double ay = dFromBits(ya);
double r;
double half;
if (isNonFinite(xb) || isNonFinite(yb) || isZeroBits(yb)) {
return fmodSpecial(x, y);
}
double quotient = rint(x / y);
return x - quotient * y;
r = (ya < D_HUGE2) ? fmod(x, ldexp(ay, 1)) : x;
r = fabs(r);
if (scaled) {
r = ldexp(r, REM_SCALE);
ay = ldexp(ay, REM_SCALE);
}
half = ay * 0.5;
if (r > half) {
r = r - ay;
if (r >= half) {
r = r - ay;
}
}
if (scaled) {
r = ldexp(r, -REM_SCALE);
}
return dFromBits(dToBits(r) ^ sign);
}
@ -1138,11 +1389,31 @@ double scalbln(double x, long n) { return ldexp(x, (int)n); }
float scalblnf(float x, long n) { return ldexpf(x, (int)n); }
// fpclassify on the bits, in the argument's own width (math.h's macro
// sends a float to __fpclassify_f). Subnormals are FP_SUBNORMAL: a double
// one used to be FP_NORMAL (nothing told them apart), and a float one
// FP_ZERO, because the widening to double flushed it to zero first.
int fpclassify(double x) {
if (__isnan_d(x)) return 0; // FP___builtin_nanf("")
if (__isinf_d(x)) return 1; // FP_INFINITE
if (x == 0.0) return 4; // FP_ZERO
return 2; // FP_NORMAL
// FP_SUBNORMAL (= 3) not distinguished from normal in this minimal
// implementation — subnormals are valid but classified as normal.
uint64_t b = dToBits(x);
unsigned e = (topWord(b) >> 4) & 0x7FFu;
if (e == 0x7FFu) {
return (b & D_MANT) ? FP_NAN : FP_INFINITE;
}
if (e != 0u) {
return FP_NORMAL;
}
return isZeroBits(b) ? FP_ZERO : FP_SUBNORMAL;
}
int __fpclassify_f(float x) {
uint32_t b = fToBits(x);
uint32_t e = (b >> F_MBITS) & F_EXP;
if (e == F_EXP) {
return (b & F_MANT) ? FP_NAN : FP_INFINITE;
}
if (e != 0u) {
return FP_NORMAL;
}
return (b & ~F_SIGN) ? FP_SUBNORMAL : FP_ZERO;
}

11
scripts/runInMame.sh
View file

@ -18,6 +18,14 @@
# 2+ for safe data writes (bank 0 zero page is scribbled by IIgs ROM
# during execution).
#
# Optional environment:
# MAME_CHECK_FRAME frame of the checks (default 300; the load is at 30).
# MAME_SECS emulated seconds before MAME quits (default 6).
# MAME_LUA_EXTRA a Lua file appended to the autoboot script, for a
# check that needs its own hooks (write taps, injected
# interrupts). It runs once at startup, like the rest;
# its MAME-* prints reach the output.
#
# Exit 0 if all reads match, 1 otherwise.
set -euo pipefail
@ -118,6 +126,9 @@ $LUA_CHECKS
end
end)
EOF
if [ -n "${MAME_LUA_EXTRA:-}" ]; then
cat "$MAME_LUA_EXTRA" >> "$LUA_PATH"
fi
# MAME and MAME_ROMPATH let a client project (joeylib) supply its own
# patched MAME and ROM set instead of the ones installMame.sh sets up.

22
scripts/runViaFinder.sh
View file

@ -4,6 +4,7 @@
# specific frames to verify the program executed.
#
# Usage: runViaFinder.sh <omf-file> [--data /VOL/PATH/NAME=local_file]...
# [--save-data <file>]
# --check <addr>=<value>...
# The OMF file is injected as /DATA/HELLO on a separate 800K data
# disk; Lua drives Finder to open the Data volume and launch HELLO.
@ -19,6 +20,10 @@
# readable file via gsosOpen. Pass multiple `--data` options to inject
# more than one file.
#
# --save-data copies the data disk to <file> once MAME has stopped (MAME
# writes the floppy image back when it exits), so a caller can read what
# the program left on it -- e.g. with `cadius EXTRACTFILE`.
#
# Memory checks happen at frame 5400 (~90s emulated, well after the
# launch path completes) and exit 0 / 1 depending on whether each
# requested address holds the requested value.
@ -34,14 +39,20 @@ OMF="$1"
shift
[ -f "$OMF" ] || { echo "missing: $OMF" >&2; exit 2; }
# Collect optional --data injections before --check.
# Collect optional --data injections and --save-data before --check.
USAGE="usage: $0 <omf> [--data /VOL/PATH/NAME=path]... [--save-data <file>] --check <addr>=<val>..."
DATA_INJECTS=()
while [ $# -gt 0 ] && [ "$1" = "--data" ]; do
[ $# -ge 2 ] || { echo "usage: $0 <omf> [--data /VOL/PATH/NAME=path]... --check <addr>=<val>..." >&2; exit 2; }
SAVE_DATA=""
while [ $# -gt 0 ] && { [ "$1" = "--data" ] || [ "$1" = "--save-data" ]; }; do
[ $# -ge 2 ] || { echo "$USAGE" >&2; exit 2; }
if [ "$1" = "--data" ]; then
DATA_INJECTS+=("$2")
else
SAVE_DATA="$2"
fi
shift 2
done
[ "${1:-}" = "--check" ] || { echo "usage: $0 <omf> [--data /VOL/PATH/NAME=path]... --check <addr>=<val>..." >&2; exit 2; }
[ "${1:-}" = "--check" ] || { echo "$USAGE" >&2; exit 2; }
shift
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@ -162,6 +173,9 @@ OUT=$(SDL_VIDEODRIVER=dummy SDL_AUDIODRIVER=dummy timeout 150 mame apple2gs -rom
-video none -nothrottle -sound none \
-seconds_to_run 130 -flop3 "$WORK/disk.po" -flop4 "$WORK/data.po" \
-autoboot_script "$WORK/finder.lua" </dev/null 2>&1)
if [ -n "$SAVE_DATA" ]; then
cp "$WORK/data.po" "$SAVE_DATA"
fi
# Verify each expected value.
fail=0

1086
scripts/smokeTest.sh
View file

File diff suppressed because it is too large Load diff

7
src/link816/link816.cpp
View file

@ -1767,8 +1767,11 @@ struct Linker {
// __heap_start / __heap_end: everything from bss_end to the top
// of the entry segment's BANK.
//
// There is no IO window and no language card to dodge here. Those
// exist only in banks $00/$01/$E0/$E1; GS/OS places bank-aligned
// Under GS/OS there is no IO window and no language card to dodge
// here. (A bare-metal link -- crt0.s, loaded at $00:1000 -- DOES put
// this window in bank 0, across the $C000-$CFFF I/O page; libc's
// heap init stops such a window at $C000 in banks $00/$01/$E0/$E1.)
// Those exist only in banks $00/$01/$E0/$E1; GS/OS places bank-aligned
// OMF segments in free banks (measured under MAME: joeylib's
// examples run in $08-$0B, and bank $00 shows up only in ROM and
// GS/OS glue), and crt0Gsos points DBR at the entry segment's

4
src/llvm/lib/Target/W65816/CMakeLists.txt
View file

@ -36,10 +36,14 @@ add_llvm_target(W65816CodeGen
W65816SjLjFinalize.cpp
W65816LowerWide32.cpp
W65816I32IncFold.cpp
W65816ReturnPopMerge.cpp
W65816CommuteReload.cpp
W65816ImgCalleeSave.cpp
W65816NarrowI32Mul.cpp
W65816CmpPrepare.cpp
W65816SplitLoopExits.cpp
W65816CmpSink.cpp
W65816SinkLoopInc.cpp
W65816PromoteFiToImg.cpp
W65816StackRelToImg.cpp
W65816StackSlotMerge.cpp

29
src/llvm/lib/Target/W65816/W65816.h
View file

@ -102,7 +102,7 @@ FunctionPass *createW65816WidenAcc16();
// Targets fast-regalloc's habit of spilling A unnecessarily; the
// 3x speedup is the biggest single per-iteration win we can get
// without switching to a smarter allocator. See W65816SpillToX.cpp.
FunctionPass *createW65816SpillToX();
FunctionPass *createW65816SpillToX(bool SharedSlots = false);
// Pre-emit peephole: rewrite `LDY #neg ; (LDA|STA) (sr,S),Y` to
// pre-add the offset to the pointer with Y=0. The 65816 spec for
@ -131,6 +131,11 @@ FunctionPass *createW65816NarrowI32Mul();
// one at a time. See W65816CmpPrepare.cpp.
FunctionPass *createW65816CmpPrepare();
// IR pass before ISel: split each critical edge out of a loop into a block
// with PHIs, so PHI elimination's copies run once per exit instead of on
// every trip. See W65816SplitLoopExits.cpp.
FunctionPass *createW65816SplitLoopExits();
// Pre-RA pass, inserted right after the register coalescer: moves the
// instruction that sets a conditional branch's flags down past the back-
// edge COPYs PHI elimination left between them, so the copies (and the
@ -176,11 +181,14 @@ FunctionPass *createW65816StackRelToImg();
// vregs. Without this, greedy/basic regalloc can't fit the pair-
// pressure of i64-via-2-i32-via-Wide32 traffic in i64-heavy
// functions (RegAllocBase crashes during allocatePhysRegs). After
// this pass, only i16 vregs reach regalloc, and the pair structure
// lives only in the LDAptr32S / STAptr32S / STBptr32S pseudos which
// take 2 i16 ptr operands directly.
// this pass, only i16 vregs reach regalloc.
FunctionPass *createW65816LowerWide32();
// Pre-RA, SSA: move a loop-carried increment below the last use of the
// value it increments so the coalescer can merge the two (no back-edge
// copy of a bumped pointer). See W65816SinkLoopInc.cpp.
FunctionPass *createW65816SinkLoopInc();
// Pre-emit peephole: detect the post-PEI 6-instruction `i32 += 1`
// pattern (LDA-ADCi16imm-STA-LDA-ADCEi16imm-STA on consecutive i16
// stack-rel halves) and rewrite to LDA-INA-STA + INC_HI_IF_CARRY.
@ -188,6 +196,17 @@ FunctionPass *createW65816LowerWide32();
// See W65816I32IncFold.cpp.
FunctionPass *createW65816I32IncFold();
// Pre-emit: where a path to a return is only stack releases (the last
// call's argument cleanup, then the epilogue, often behind a branch to a
// shared return block), give the block one release for the total and a
// return of its own. See W65816ReturnPopMerge.cpp.
FunctionPass *createW65816ReturnPopMerge();
// Pre-emit, last: `sta slot ; lda x ; op slot` with a commutative op
// (ORA / AND / EOR / ADC) becomes `sta slot ; op x`. See
// W65816CommuteReload.cpp.
FunctionPass *createW65816CommuteReload();
// Post-RA, pre-PEI pass that emits prologue save + epilogue restore for
// IMG8..IMG15 if the function uses them. Makes IMG8..IMG15 behave as
// callee-saved at the asm level without going through LLVM's CSR
@ -228,7 +247,9 @@ void initializeW65816LowerWide32Pass(PassRegistry &);
void initializeW65816ImgCalleeSavePass(PassRegistry &);
void initializeW65816NarrowI32MulPass(PassRegistry &);
void initializeW65816CmpPreparePass(PassRegistry &);
void initializeW65816SplitLoopExitsPass(PassRegistry &);
void initializeW65816CmpSinkPass(PassRegistry &);
void initializeW65816SinkLoopIncPass(PassRegistry &);
void initializeW65816PromoteFiToImgPass(PassRegistry &);
void initializeW65816StackSlotMergePass(PassRegistry &);
void initializeW65816StackRelToImgPass(PassRegistry &);

52
src/llvm/lib/Target/W65816/W65816AsmPrinter.cpp
View file

@ -532,17 +532,23 @@ void W65816AsmPrinter::emitInstruction(const MachineInstr *MI) {
// no help; INTERSEG is for cross-segment refs. Reading PBR at
// runtime via $BE is the only working mechanism on real hardware.
//
// EXCEPTION — external-weak symbols: a `&weakfn` whose definition may
// EXCEPTION -- external-weak symbols: a `&weakfn` whose definition may
// be absent at link time must be representable as NULL. If we emit
// `lda $BE` for its bank, the runtime PBR (non-zero when GS/OS loads
// the segment in a non-zero bank) makes the pointer always non-null,
// so `if (weakfn) weakfn()` is always taken and jumps to the
// unresolved (=0) target — a crash. For an external-weak global the
// bank must be a literal 0 instead, so an undefined-weak ref is a
// genuine null and a strong link-time def still works (its low16 is
// non-zero so the null test passes and the direct call uses the
// symbol's own reloc, not this bank byte). Defined/normal globals
// and compiler external symbols (libgcc helpers) keep `lda $BE`.
// unresolved (=0) target -- a crash. A literal 0 bank fixed that but
// broke the other case: when a definition IS linked, `p = weakfn;
// p()` called bank 0 (the runtime's `&gsosFlush` hook crashed under
// GS/OS). So the bank depends on the symbol's low 16 bits, which the
// linker resolves to 0 for an undefined weak:
// lda #weakfn ; beq L ; lda $BE ; L:
// giving 0 (a null pointer) or the program bank. A defined weak at
// offset 0 of its bank would read as null; nothing is placed there
// (the entry segment starts with crt0, and link816 sends other
// segments' function pointers through entry-segment trampolines).
// Defined/normal globals and compiler external symbols (libgcc
// helpers) keep the plain `lda $BE`.
//
// CODE symbols in a multi-segment OMF app are NOT an exception, even
// though every overflow segment lives in its own bank: link816
@ -554,10 +560,16 @@ void W65816AsmPrinter::emitInstruction(const MachineInstr *MI) {
// on 2026-09-22 and broke every such pointer (bank of the callee +
// offset of the trampoline). Do not resurrect it.
const MachineOperand &Sym = MI->getOperand(1);
bool WeakUndef = Sym.isGlobal() &&
Sym.getGlobal()->hasExternalWeakLinkage();
if (WeakUndef) {
emitOpImm(W65816::LDA_Imm16, 0);
if (Sym.isGlobal() && Sym.getGlobal()->hasExternalWeakLinkage()) {
MCSymbol *NullSym = OutContext.createTempSymbol();
EmitToStreamer(*OutStreamer,
MCInstBuilder(W65816::LDA_Imm16)
.addOperand(lowerOperand(Sym, MCInstLowering)));
EmitToStreamer(*OutStreamer,
MCInstBuilder(W65816::BEQ).addExpr(
MCSymbolRefExpr::create(NullSym, OutContext)));
emitOpImm(W65816::LDA_DP, kRuntimePbrStashDP);
OutStreamer->emitLabel(NullSym);
} else {
emitOpImm(W65816::LDA_DP, kRuntimePbrStashDP);
}
@ -648,8 +660,11 @@ void W65816AsmPrinter::emitInstruction(const MachineInstr *MI) {
// STA runs in our LDAi8imm-set M=8 mode. Saves 4B / 6cyc per
// hit. We mark the next-SEP-to-skip via a per-AsmPrinter flag
// so the SEP visit drops it.
emitSepM();
emitOpImm(W65816::LDA_Imm8, MI->getOperand(1).getImm() & 0xFF);
//
// Otherwise the byte is a 16-bit `lda #$00nn` with no mode switch at
// all: an i8 value's high byte is don't-care, and nothing reads the
// flags of a constant load (LDAi8imm models no Defs=[P]). 3 bytes /
// 3 cycles against SEP / LDA / REP's 6 / 8.
bool SkipRep = false;
// Walk past mode-neutral MIs (X-flag-only ops, branches, transfers
// that don't touch A) to find the next SEP/REP — same idea as the
@ -701,6 +716,13 @@ void W65816AsmPrinter::emitInstruction(const MachineInstr *MI) {
SkipRep = true;
SkipNextSta8Wrap = true;
}
static const bool Wide16Imm = !getenv("W65816_DISABLE_I8_IMM16");
if (!SkipRep && Wide16Imm) {
emitOpImm(W65816::LDA_Imm16, MI->getOperand(1).getImm() & 0xFF);
return;
}
emitSepM();
emitOpImm(W65816::LDA_Imm8, MI->getOperand(1).getImm() & 0xFF);
if (!SkipRep) {
emitRepM();
}
@ -1042,7 +1064,9 @@ void W65816AsmPrinter::emitInstruction(const MachineInstr *MI) {
return;
}
case W65816::JSLpseudo:
case W65816::JSLpseudo32: {
case W65816::JSLpseudo32:
case W65816::JSLlight:
case W65816::JSLlight32: {
EmitToStreamer(*OutStreamer,
MCInstBuilder(W65816::JSL_Long)
.addOperand(lowerOperand(MI->getOperand(0), MCInstLowering)));

167
src/llvm/lib/Target/W65816/W65816CommuteReload.cpp Normal file
View file

@ -0,0 +1,167 @@
//===-- W65816CommuteReload.cpp - Fold a reload into a commutative op ----===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===---------------------------------------------------------------------===//
//
// Pre-emit peephole, last: A is stored, something else is loaded, and the
// stored word is the other operand of a commutative op:
//
// sta 3,s sta 3,s
// lda 0xd0 => ora 0xd0
// ora 3,s
//
// After the store A equals the slot, so the op can take the loaded operand
// instead and the load goes (4 or 5 cycles, 2 or 3 bytes). The op reads
// memory at the same point the load did, so even an operand that overlaps
// the slot reads the same bytes, and N / Z come out the same. ORA, AND,
// EOR and ADC (a CLC may sit between the load and an ADC; LDA leaves C
// alone either way). The two allocator-assigned operands of a far
// pointer's null test (`stx dp ; sta d,s ; lda dp ; ora d,s`) are the
// common case. The slot is a stack or direct-page word only, never an
// absolute address that could be an I/O register reading back something
// else.
//
// Kill switch: W65816_DISABLE_COMMUTE_RELOAD=1.
//
//===---------------------------------------------------------------------===//
#include "W65816.h"
#include "W65816InstrInfo.h"
#include "W65816Subtarget.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
using namespace llvm;
#define DEBUG_TYPE "w65816-commute-reload"
namespace {
class W65816CommuteReload : public MachineFunctionPass {
public:
static char ID;
W65816CommuteReload() : MachineFunctionPass(ID) {}
bool runOnMachineFunction(MachineFunction &MF) override;
StringRef getPassName() const override {
return "W65816 fold a reload into a commutative op";
}
};
char W65816CommuteReload::ID = 0;
// One addressing mode of LDA and the same mode of ORA / AND / EOR / ADC.
struct ModeRow {
unsigned Lda;
unsigned Ora;
unsigned And;
unsigned Eor;
unsigned Adc;
};
const ModeRow Modes[] = {
{W65816::LDA_Abs, W65816::ORA_Abs, W65816::AND_Abs, W65816::EOR_Abs,
W65816::ADC_Abs},
{W65816::LDA_AbsX, W65816::ORA_AbsX, W65816::AND_AbsX, W65816::EOR_AbsX,
W65816::ADC_AbsX},
{W65816::LDA_AbsY, W65816::ORA_AbsY, W65816::AND_AbsY, W65816::EOR_AbsY,
W65816::ADC_AbsY},
{W65816::LDA_DP, W65816::ORA_DP, W65816::AND_DP, W65816::EOR_DP,
W65816::ADC_DP},
{W65816::LDA_DPInd, W65816::ORA_DPInd, W65816::AND_DPInd,
W65816::EOR_DPInd, W65816::ADC_DPInd},
{W65816::LDA_DPIndLong, W65816::ORA_DPIndLong, W65816::AND_DPIndLong,
W65816::EOR_DPIndLong, W65816::ADC_DPIndLong},
{W65816::LDA_DPIndLongY, W65816::ORA_DPIndLongY, W65816::AND_DPIndLongY,
W65816::EOR_DPIndLongY, W65816::ADC_DPIndLongY},
{W65816::LDA_DPIndX, W65816::ORA_DPIndX, W65816::AND_DPIndX,
W65816::EOR_DPIndX, W65816::ADC_DPIndX},
{W65816::LDA_DPIndY, W65816::ORA_DPIndY, W65816::AND_DPIndY,
W65816::EOR_DPIndY, W65816::ADC_DPIndY},
{W65816::LDA_DPX, W65816::ORA_DPX, W65816::AND_DPX, W65816::EOR_DPX,
W65816::ADC_DPX},
{W65816::LDA_Imm16, W65816::ORA_Imm16, W65816::AND_Imm16,
W65816::EOR_Imm16, W65816::ADC_Imm16},
{W65816::LDA_Imm8, W65816::ORA_Imm8, W65816::AND_Imm8, W65816::EOR_Imm8,
W65816::ADC_Imm8},
{W65816::LDA_Long, W65816::ORA_Long, W65816::AND_Long, W65816::EOR_Long,
W65816::ADC_Long},
{W65816::LDA_LongX, W65816::ORA_LongX, W65816::AND_LongX,
W65816::EOR_LongX, W65816::ADC_LongX},
{W65816::LDA_StackRel, W65816::ORA_StackRel, W65816::AND_StackRel,
W65816::EOR_StackRel, W65816::ADC_StackRel},
{W65816::LDA_StackRelIndY, W65816::ORA_StackRelIndY,
W65816::AND_StackRelIndY, W65816::EOR_StackRelIndY,
W65816::ADC_StackRelIndY},
};
const ModeRow *rowOfLoad(unsigned Opc) {
for (const ModeRow &R : Modes)
if (R.Lda == Opc)
return &R;
return nullptr;
}
// The op of Row with the kind of Opc (an ORA / AND / EOR / ADC in Ref's
// mode), or 0.
unsigned sameKind(unsigned Opc, const ModeRow &Ref, const ModeRow &Row) {
if (Opc == Ref.Ora)
return Row.Ora;
if (Opc == Ref.And)
return Row.And;
if (Opc == Ref.Eor)
return Row.Eor;
if (Opc == Ref.Adc)
return Row.Adc;
return 0;
}
} // namespace
bool W65816CommuteReload::runOnMachineFunction(MachineFunction &MF) {
if (getenv("W65816_DISABLE_COMMUTE_RELOAD"))
return false;
const TargetInstrInfo *TII = MF.getSubtarget().getInstrInfo();
const ModeRow *DpRow = rowOfLoad(W65816::LDA_DP);
const ModeRow *SrRow = rowOfLoad(W65816::LDA_StackRel);
bool Changed = false;
for (MachineBasicBlock &MBB : MF) {
for (auto It = MBB.begin(); It != MBB.end(); ++It) {
unsigned StOpc = It->getOpcode();
if ((StOpc != W65816::STA_DP && StOpc != W65816::STA_StackRel) ||
!It->getOperand(0).isImm())
continue;
const ModeRow *SlotRow = StOpc == W65816::STA_DP ? DpRow : SrRow;
auto Ld = std::next(It);
if (Ld == MBB.end())
continue;
const ModeRow *LdRow = rowOfLoad(Ld->getOpcode());
if (!LdRow)
continue;
auto Op = std::next(Ld);
if (Op != MBB.end() && Op->getOpcode() == W65816::CLC)
++Op;
if (Op == MBB.end() || Op->getNumOperands() == 0 ||
!Op->getOperand(0).isImm() ||
Op->getOperand(0).getImm() != It->getOperand(0).getImm())
continue;
unsigned NewOpc = sameKind(Op->getOpcode(), *SlotRow, *LdRow);
if (!NewOpc)
continue;
MachineInstrBuilder New =
BuildMI(MBB, Op, Op->getDebugLoc(), TII->get(NewOpc));
for (const MachineOperand &MO : Ld->explicit_operands())
New.add(MO);
Op->eraseFromParent();
Ld->eraseFromParent();
Changed = true;
}
}
return Changed;
}
FunctionPass *llvm::createW65816CommuteReload() {
return new W65816CommuteReload();
}

168
src/llvm/lib/Target/W65816/W65816FrameLowering.cpp
View file

@ -16,9 +16,13 @@
#include "W65816InstrInfo.h"
#include "W65816MachineFunctionInfo.h"
#include "W65816Subtarget.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/CodeGen/MachineDominators.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineFunction.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/CodeGen/MachineLoopInfo.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/IR/Function.h"
#include "llvm/Support/CommandLine.h"
@ -101,6 +105,82 @@ bool W65816FrameLowering::hasReservedCallFrame(const MachineFunction &MF) const
return false;
}
// Lay out a large frame with its most-used objects nearest S. A slot more
// than 0xFF bytes above S is out of `off,s` reach and every access to it
// goes through the $F6 frame pointer (`sty $fa ; ldy #off ; lda [$f6],y ;
// ldy $fa`, 18 cycles against 5). PEI places the objects in this list
// from the top of the frame down, so the last ones end up nearest S; the
// default order (creation order) puts the IR's allocas -- arrays among
// them -- below nothing but the spill slots, and which scalars landed out
// of reach was an accident of numbering (Lua's luaV_execute did 400+ far
// accesses in its dispatch loop). Sort ascending by access density:
// uses weighted by loop depth, divided by size, so big arrays (reached
// through a pointer built once) go to the top and hot scalars to the
// bottom. Small frames keep the default order: everything is in reach.
void W65816FrameLowering::orderFrameObjects(
const MachineFunction &MF, SmallVectorImpl<int> &ObjectsToAllocate) const {
const MachineFrameInfo &MFI = MF.getFrameInfo();
uint64_t Total = 0;
for (int FI : ObjectsToAllocate)
Total += MFI.getObjectSize(FI);
if (ObjectsToAllocate.size() < 2 || Total <= 0xC0 ||
getenv("W65816_DISABLE_FRAME_ORDER"))
return;
MachineFunction &MutMF = const_cast<MachineFunction &>(MF);
MachineDominatorTree MDT(MutMF);
MachineLoopInfo MLI(MDT);
// Only the accesses that address the object `off,s` (or `[$f6],y`)
// count: an ADDframe builds its address with `adc #off` from anywhere.
auto isSlotAccess = [](unsigned Opc) {
switch (Opc) {
case W65816::LDAfi: case W65816::STAfi: case W65816::STA8fi:
case W65816::ADCfi: case W65816::ADCEfi: case W65816::SBCfi:
case W65816::SBCEfi: case W65816::ANDfi: case W65816::ORAfi:
case W65816::EORfi: case W65816::CMPfi: case W65816::LDAfi_indY:
case W65816::STAfi_indY: case W65816::STA8fi_indY:
return true;
default:
return false;
}
};
DenseMap<int, uint64_t> Uses;
// Objects some access starts past the end of: that is the PHA-bracket
// idiom (W65816ImgCalleeSave's `pha ; lda dp ; sta slot+2,s ; pla`),
// whose +2 is right only `off,s`, never through $F6, which a PHA does
// not move. They stay nearest S, where creation order (they are made
// last) always put them.
DenseSet<int> PinnedNear;
for (const MachineBasicBlock &MBB : MF) {
// 8x per loop level, capped so a deep nest cannot overflow the sum.
uint64_t W = uint64_t(1) << (3 * std::min(MLI.getLoopDepth(&MBB), 6u));
for (const MachineInstr &MI : MBB) {
if (!isSlotAccess(MI.getOpcode()))
continue;
for (unsigned I = 0, E = MI.getNumOperands(); I != E; ++I) {
const MachineOperand &MO = MI.getOperand(I);
if (!MO.isFI() || MO.getIndex() < 0)
continue;
Uses[MO.getIndex()] += W;
if (I + 1 < E && MI.getOperand(I + 1).isImm() &&
(MI.getOperand(I + 1).getImm() < 0 ||
MI.getOperand(I + 1).getImm() >=
(int64_t)MFI.getObjectSize(MO.getIndex())))
PinnedNear.insert(MO.getIndex());
}
}
}
llvm::stable_sort(ObjectsToAllocate, [&](int A, int B) {
bool PA = PinnedNear.count(A);
bool PB = PinnedNear.count(B);
if (PA != PB)
return PB;
uint64_t SA = std::max<uint64_t>(MFI.getObjectSize(A), 1);
uint64_t SB = std::max<uint64_t>(MFI.getObjectSize(B), 1);
// Uses[A] / SA < Uses[B] / SB, without the division.
return Uses.lookup(A) * SB < Uses.lookup(B) * SA;
});
}
void W65816FrameLowering::determineCalleeSaves(MachineFunction &MF,
BitVector &SavedRegs,
RegScavenger *RS) const {
@ -200,19 +280,24 @@ void W65816FrameLowering::emitPrologue(MachineFunction &MF,
}
if (StackSize > 0) {
// Cycle math: each PHA is 4 cyc; the TSC-sequence (TAY+TSC+SEC+
// SBC+TCS+TYA) is 13 cyc total. N PHAs win on cycles when 4*N <= 13,
// i.e. up to 3 PHAs (6-byte frame). At N=4 (8 bytes): 16 cyc PHAs vs
// 13 cyc TSC-seq → TSC wins. Threshold at 6 bytes for speed.
// (Bytes: N PHAs cost N bytes; TSC-seq costs 8 bytes. We're
// optimizing for speed per the project directive.)
if (StackSize <= 6 && (StackSize % 2) == 0) {
// Cycle math: each PHA is 4 cyc and 1 byte; TSC+SEC+SBC+TCS is 9 cyc
// and 6 bytes, 13 and 8 with the TAY/TYA that keeps an argument in A.
// Under the shared cost model (W65816Helpers::cycleByteCost) PHAs win
// up to 2 of them (4-byte frame) when A is free and up to 3 (6 bytes)
// when it carries an argument in.
bool KeepA = MBB.isLiveIn(W65816::A);
uint64_t Pushes = StackSize / 2;
uint64_t PushCost = W65816Helpers::cycleByteCost(4 * Pushes, Pushes);
uint64_t SubCost = KeepA ? W65816Helpers::cycleByteCost(13, 8)
: W65816Helpers::cycleByteCost(9, 6);
if ((StackSize % 2) == 0 && PushCost <= SubCost) {
// Small frame: N/2 PHAs. Preserves A.
for (uint64_t i = 0; i < StackSize / 2; ++i)
for (uint64_t i = 0; i < Pushes; ++i)
BuildMI(MBB, MBBI, DL, TII.get(W65816::PHA));
} else {
// Larger frame: TAY/TSC/.../TYA bracket. Preserves A via Y as a
// temp.
// Larger frame: TSC/.../TCS, bracketed by TAY/TYA (Y as a temp)
// when A carries an argument in.
if (KeepA)
BuildMI(MBB, MBBI, DL, TII.get(W65816::TAY));
BuildMI(MBB, MBBI, DL, TII.get(W65816::TSC));
BuildMI(MBB, MBBI, DL, TII.get(W65816::SEC));
@ -243,6 +328,7 @@ void W65816FrameLowering::emitPrologue(MachineFunction &MF,
// probe writes before invoking it.
BuildMI(MBB, MBBI, DL, TII.get(W65816::STZ_DP)).addImm(0xF8);
}
if (KeepA)
BuildMI(MBB, MBBI, DL, TII.get(W65816::TYA));
}
}
@ -311,11 +397,8 @@ void W65816FrameLowering::emitEpilogue(MachineFunction &MF,
bool YLive = false;
bool XLive = false;
if (MBBI != MBB.end() && MBBI->isReturn()) {
for (const MachineOperand &MO : MBBI->operands()) {
if (!MO.isReg() || !MO.isImplicit() || !MO.isUse()) continue;
if (MO.getReg() == W65816::Y) YLive = true;
else if (MO.getReg() == W65816::X) XLive = true;
}
YLive = MBBI->readsRegister(W65816::Y, STI.getRegisterInfo());
XLive = MBBI->readsRegister(W65816::X, STI.getRegisterInfo());
}
// VLA cleanup: restore entry SP from DP $F4 (saved in prologue).
@ -376,38 +459,16 @@ void W65816FrameLowering::emitEpilogue(MachineFunction &MF,
// Prologue is always 16-bit M now (see emitPrologue). No 8-bit
// epilogue branch needed.
// 16-bit M epilogue. Mirror the prologue: A holds the return value
// at this point and MUST be preserved. Small frames release via
// N/2 PLY (pop into Y, discard); larger frames use
// TAY/TSC/CLC/ADC #N/TCS/TYA.
// Mirror the prologue threshold (see comment there).
if (StackSize <= 6 && (StackSize % 2) == 0 && !YLive) {
// PLY clobbers Y, which is fine when Y isn't a return reg.
for (uint64_t i = 0; i < StackSize / 2; ++i)
BuildMI(MBB, MBBI, DL, TII.get(W65816::PLY));
return;
}
if (YLive) {
// Y is a return register (i64 / double). Save A via DP $E0
// instead of TAY so Y survives. 4 cyc slower than TAY/TYA but
// correct. X is allowed to be live too — none of these touch X.
BuildMI(MBB, MBBI, DL, TII.get(W65816::STA_DP)).addImm(0xE0);
BuildMI(MBB, MBBI, DL, TII.get(W65816::TSC));
BuildMI(MBB, MBBI, DL, TII.get(W65816::CLC));
BuildMI(MBB, MBBI, DL, TII.get(W65816::ADC_Imm16))
.addImm(StackSize);
BuildMI(MBB, MBBI, DL, TII.get(W65816::TCS));
BuildMI(MBB, MBBI, DL, TII.get(W65816::LDA_DP)).addImm(0xE0);
(void)XLive;
return;
}
BuildMI(MBB, MBBI, DL, TII.get(W65816::TAY));
BuildMI(MBB, MBBI, DL, TII.get(W65816::TSC));
BuildMI(MBB, MBBI, DL, TII.get(W65816::CLC));
BuildMI(MBB, MBBI, DL, TII.get(W65816::ADC_Imm16))
.addImm(StackSize);
BuildMI(MBB, MBBI, DL, TII.get(W65816::TCS));
BuildMI(MBB, MBBI, DL, TII.get(W65816::TYA));
// 16-bit M epilogue. The return value in A (and X / Y for wider
// ones) must survive; a void function's A is free, and then a bare
// TSC/CLC/ADC/TCS (9 cyc) beats both N/2 PLY (5 cyc each) past two of
// them and the TAY/TYA-bracketed form (13 cyc), which in turn beats
// PLYs past three. An i64 / double return keeps Y too, so A goes
// through DP $E0 there.
bool KeepA = MBBI == MBB.end() || !MBBI->isReturn() ||
MBBI->readsRegister(W65816::A, STI.getRegisterInfo());
W65816Helpers::emitStackRelease(MBB, MBBI, DL, TII, StackSize, KeepA,
XLive, YLive);
}
MachineBasicBlock::iterator W65816FrameLowering::eliminateCallFramePseudoInstr(
@ -520,9 +581,16 @@ MachineBasicBlock::iterator W65816FrameLowering::eliminateCallFramePseudoInstr(
reestablishFP();
BuildMI(MBB, I, DL, TII.get(W65816::LDA_DP)).addImm(0xE0);
(void)XLive;
} else if (N > 0 && N <= 14 && (N % 2) == 0 && !usesDpFP) {
for (int i = 0; i < N / 2; ++i)
BuildMI(MBB, I, DL, TII.get(W65816::PLY));
} else if (!usesDpFP) {
// Y is free (no i64 return). When the callee's A result is not
// read (a void call, or one whose value is unused), A is free too
// and TSC/CLC/ADC/TCS releases any size in 9 cycles; otherwise
// N/2 PLY wins up to three of them over TAY/.../TYA. X can carry
// an i32 result's high half, so it is kept.
W65816Helpers::emitStackRelease(
MBB, I, DL, TII, N,
W65816Helpers::aLiveFrom(MBB, std::next(I)),
/*KeepX=*/true, /*KeepY=*/false);
} else {
BuildMI(MBB, I, DL, TII.get(W65816::TAY));
BuildMI(MBB, I, DL, TII.get(W65816::TSC));

3
src/llvm/lib/Target/W65816/W65816FrameLowering.h
View file

@ -35,6 +35,9 @@ public:
RegScavenger *RS) const override;
bool hasReservedCallFrame(const MachineFunction &MF) const override;
void orderFrameObjects(const MachineFunction &MF,
SmallVectorImpl<int> &ObjectsToAllocate) const override;
};
} // namespace llvm

1178
src/llvm/lib/Target/W65816/W65816ISelLowering.cpp
View file

File diff suppressed because it is too large Load diff

14
src/llvm/lib/Target/W65816/W65816ISelLowering.h
View file

@ -56,6 +56,16 @@ public:
SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
void AdjustInstrPostInstrSelection(MachineInstr &MI,
SDNode *Node) const override;
// SELECT_CC is Custom for i64 so that the type legalizer hands its i64
// compare operands to LowerSELECT_CC. A SELECT_CC with an i64 result
// asks here first; leaving Results empty keeps the default expansion,
// which splits it into i32 selects that come back through LowerOperation.
void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue> &Results,
SelectionDAG &DAG) const override {}
// Lock i16 shift amounts to i16 (not i32) even when i32 is a legal
// type. Without this, the DAG combiner promotes i16 shift amounts
// to i32 once i32 is registered as legal, leaving (sra i16, i32:K)
@ -177,6 +187,10 @@ public:
// a libcall is ~12 instructions, while `(mul x, 3)` -> `(add x, (shl
// x, 1))` is 5. i32 stays libcall — the per-half shift+add+chain
// expansion comes out larger than the __mulsi3 call.
bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM,
Type *Ty, unsigned AS,
Instruction *I = nullptr) const override;
bool decomposeMulByConstant(LLVMContext &Context, EVT VT,
SDValue C) const override {
return VT == MVT::i16;

388
src/llvm/lib/Target/W65816/W65816InstrInfo.cpp
View file

@ -14,6 +14,7 @@
#include "W65816InstrInfo.h"
#include "W65816.h"
#include "W65816Subtarget.h"
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/Support/ErrorHandling.h"
@ -36,6 +37,12 @@ void W65816InstrInfo::anchor() {}
// REP coalescer and the AsmPrinter's LDAi8imm mode-tracking behave exactly
// as they did for the old three-instruction form.
bool W65816InstrInfo::expandPostRAPseudo(MachineInstr &MI) const {
if (MI.getOpcode() == W65816::LDAimg) {
BuildMI(*MI.getParent(), MI, MI.getDebugLoc(), get(W65816::LDA_DP))
.addImm(W65816Helpers::imgDPAddr(MI.getOperand(1).getReg()));
MI.eraseFromParent();
return true;
}
unsigned Access;
switch (MI.getOpcode()) {
case W65816::STB_DPIndLongY:
@ -151,6 +158,195 @@ bool hasTiedAcc16Src(const MachineInstr &MI) {
return false;
}
// One block of aLiveFrom: 1 when A is read from From on, 0 when it is
// overwritten first, -1 when the scan reaches the end of the block.
// Conservative -- anything not known to leave A alone or to write all of
// it without reading it counts as a read, as does a SEP / REP (an 8-bit
// load writes half of A). A non-A LDAfi / STAfi further on counts as
// leaving A alone: it saves A itself when A is read after it, and when A
// is not, A is dead here too.
static int scanALive(const MachineBasicBlock &MBB,
MachineBasicBlock::const_iterator From) {
const TargetRegisterInfo *TRI =
MBB.getParent()->getSubtarget().getRegisterInfo();
for (auto It = From; It != MBB.end(); ++It) {
const MachineInstr &N = *It;
if (N.isDebugInstr())
continue;
switch (N.getOpcode()) {
case W65816::LDAfi:
if (N.getOperand(0).getReg() == W65816::A)
return 0;
continue;
case W65816::STAfi:
case W65816::STA8fi:
if (N.getOperand(0).getReg() == W65816::A)
return 1;
continue;
// Write all of A (in 16-bit M) and read nothing of it.
case W65816::LDA_StackRel: case W65816::LDA_DP: case W65816::LDA_Abs:
case W65816::LDA_Long: case W65816::LDA_Imm16: case W65816::LDAi16imm:
case W65816::LDAi16imm_bank: case W65816::LDAabs:
case W65816::LDA_DPIndLongY: case W65816::LDAimg:
case W65816::TXA: case W65816::TYA: case W65816::TSC: case W65816::TDC:
case W65816::PLA:
return 0;
// Leave A alone.
case W65816::LDX_Imm16: case W65816::LDX_DP: case W65816::LDX_Abs:
case W65816::LDY_Imm16: case W65816::LDY_DP: case W65816::LDY_Abs:
case W65816::LDXi16imm:
case W65816::STX_DP: case W65816::STX_Abs:
case W65816::STY_DP: case W65816::STY_Abs:
case W65816::STZ_DP: case W65816::STZ_Abs:
case W65816::INX: case W65816::DEX: case W65816::INY: case W65816::DEY:
case W65816::PHX: case W65816::PHY: case W65816::PLX: case W65816::PLY:
case W65816::CLC: case W65816::SEC: case W65816::PHP: case W65816::PLP:
continue;
default:
break;
}
if (N.readsRegister(W65816::A, TRI))
return 1;
// A call that clobbers A (by def or register mask) ends its life; one
// that preserves it, like a direct branch, leaves A alone.
if (N.isCall()) {
if (N.modifiesRegister(W65816::A, TRI))
return 0;
continue;
}
if (N.isReturn())
return 0;
if (N.isBranch() && !N.isIndirectBranch())
continue;
if (N.isCopy()) {
Register Dst = N.getOperand(0).getReg();
Register Src = N.getOperand(1).getReg();
if (Src == W65816::A)
return 1;
if (Dst == W65816::A)
return 0;
// IMG <-> IMG / X / Y copies leave A alone (IMG to IMG goes
// through A inside PHA / PLA); X <-> Y goes through A.
if ((Dst == W65816::X || Dst == W65816::Y) &&
(Src == W65816::X || Src == W65816::Y))
return 1;
continue;
}
return 1;
}
return -1;
}
bool aLiveFrom(const MachineBasicBlock &MBB,
MachineBasicBlock::const_iterator From) {
int R = scanALive(MBB, From);
if (R >= 0)
return R == 1;
// Off the end: live when any successor reads A before writing it. A
// block re-entered from the top was already scanned from there, so a
// loop back adds nothing; past the budget, assume a read.
SmallVector<const MachineBasicBlock *, 8> Work;
SmallPtrSet<const MachineBasicBlock *, 8> Seen;
const MachineBasicBlock *B = &MBB;
unsigned Budget = 8;
for (;;) {
if (B->succ_empty())
return true;
for (const MachineBasicBlock *S : B->successors())
if (Seen.insert(S).second)
Work.push_back(S);
do {
if (Work.empty())
return false;
if (Budget-- == 0)
return true;
B = Work.pop_back_val();
R = scanALive(*B, B->begin());
if (R == 1)
return true;
} while (R == 0);
}
}
namespace {
// The ways to release stack, in the order a tie in cost prefers them.
enum ReleaseForm { RelNone, RelPly, RelPlx, RelAdc, RelAdcY, RelAdcX, RelAdcDp };
} // namespace
// Cycle and byte counts in 16-bit M and X: PLY / PLX 5 and 1 for two
// bytes; TSC / CLC / ADC # / TCS 9 and 6; TAY / TYA or TAX / TXA around
// that 13 and 8; STA $E0 / LDA $E0 around it 17 and 10.
static ReleaseForm pickRelease(int64_t Bytes, bool KeepA, bool KeepX,
bool KeepY, unsigned &Cost) {
Cost = 0;
if (Bytes <= 0)
return RelNone;
ReleaseForm Best = RelAdcDp;
Cost = cycleByteCost(17, 10);
auto Try = [&](ReleaseForm F, unsigned Cycles, unsigned Size) {
unsigned C = cycleByteCost(Cycles, Size);
if (C < Cost) {
Cost = C;
Best = F;
}
};
unsigned Pulls = Bytes / 2;
if (Bytes % 2 == 0 && Bytes <= 32) {
if (!KeepY)
Try(RelPly, 5 * Pulls, Pulls);
else if (!KeepX)
Try(RelPlx, 5 * Pulls, Pulls);
}
if (!KeepA)
Try(RelAdc, 9, 6);
else if (!KeepY)
Try(RelAdcY, 13, 8);
else if (!KeepX)
Try(RelAdcX, 13, 8);
return Best;
}
unsigned stackReleaseCost(int64_t Bytes, bool KeepA, bool KeepX, bool KeepY) {
unsigned Cost;
pickRelease(Bytes, KeepA, KeepX, KeepY, Cost);
return Cost;
}
void emitStackRelease(MachineBasicBlock &MBB, MachineBasicBlock::iterator I,
const DebugLoc &DL, const TargetInstrInfo &TII,
int64_t Bytes, bool KeepA, bool KeepX, bool KeepY) {
unsigned Cost;
ReleaseForm F = pickRelease(Bytes, KeepA, KeepX, KeepY, Cost);
if (F == RelNone)
return;
if (F == RelPly || F == RelPlx) {
for (int64_t i = 0; i < Bytes / 2; ++i)
BuildMI(MBB, I, DL, TII.get(F == RelPly ? W65816::PLY : W65816::PLX));
return;
}
if (F == RelAdcY)
BuildMI(MBB, I, DL, TII.get(W65816::TAY));
else if (F == RelAdcX)
BuildMI(MBB, I, DL, TII.get(W65816::TAX));
else if (F == RelAdcDp)
BuildMI(MBB, I, DL, TII.get(W65816::STA_DP)).addImm(0xE0);
BuildMI(MBB, I, DL, TII.get(W65816::TSC));
BuildMI(MBB, I, DL, TII.get(W65816::CLC));
BuildMI(MBB, I, DL, TII.get(W65816::ADC_Imm16)).addImm(Bytes);
BuildMI(MBB, I, DL, TII.get(W65816::TCS));
if (F == RelAdcY)
BuildMI(MBB, I, DL, TII.get(W65816::TYA));
else if (F == RelAdcX)
BuildMI(MBB, I, DL, TII.get(W65816::TXA));
else if (F == RelAdcDp)
BuildMI(MBB, I, DL, TII.get(W65816::LDA_DP)).addImm(0xE0);
}
} // namespace W65816Helpers
} // namespace llvm
@ -349,68 +545,77 @@ void W65816InstrInfo::copyPhysReg(MachineBasicBlock &MBB,
llvm_unreachable("W65816: cross-class copyPhysReg not yet implemented");
}
// A STAfi for a spill. The opcode carries Defs = [A] as an over-
// approximation (see its definition), but STAfi never changes A:
// eliminateFrameIndex moves a non-A source through A inside PHA / PLA
// whenever A is still read. The allocator places spills without checking
// A's interference, so one can land after `$a = COPY $img0` with A live
// into the next block; with the implicit def in place, post-RA
// MachineCopyPropagation took the copy as overwritten and deleted it (a
// loop header then read whatever A last held). A spill names no def of A.
MachineInstrBuilder W65816InstrInfo::spillSTAfi(MachineBasicBlock &MBB,
MachineBasicBlock::iterator MI,
const DebugLoc &DL) const {
MachineInstrBuilder B = BuildMI(MBB, MI, DL, get(W65816::STAfi));
MachineInstr *NewMI = B.getInstr();
for (unsigned I = NewMI->getNumOperands(); I-- > 0;) {
const MachineOperand &MO = NewMI->getOperand(I);
if (MO.isReg() && MO.isImplicit() && MO.isDef() &&
MO.getReg() == W65816::A)
NewMI->removeOperand(I);
}
return B;
}
void W65816InstrInfo::storeRegToStackSlot(
MachineBasicBlock &MBB, MachineBasicBlock::iterator MI, Register SrcReg,
bool isKill, int FrameIdx, const TargetRegisterClass *RC, Register VReg,
MachineInstr::MIFlag Flags) const {
// STAfi gets eliminated by W65816RegisterInfo::eliminateFrameIndex into
// a real STA d,S. Source is implicit A; emit the pseudo with the FI
// and zero offset. When regalloc hands us a spill from X or Y, bridge
// through A (TXA / TYA) — same rationale as loadRegFromStackSlot.
// a real STA d,S, moving a source other than A through A.
DebugLoc DL = MI != MBB.end() ? MI->getDebugLoc() : DebugLoc();
// Wide32 spill: split into 2 i16 stores at offsets 0 and 2 of the
// 4-byte spill slot. Bridge each half through A using copyPhysReg.
// Wide32 spill: two i16 stores, offsets 0 and 2 of the 4-byte slot,
// each STAfi straight from its half (eliminateFrameIndex moves a non-A
// half through A, keeping A when it is still read). The allocator
// spills a VIRTUAL Wide32 register through here too (the vregs
// W65816LowerWide32 cannot split -- the pointer PHIs of a loop nest);
// the old code took that for the unpaired-physreg case and stored the
// low half only, so the reload brought back a pointer with whatever
// bank its register pair last held: CoreMark's core_list_mergesort,
// once its loop exits got blocks of their own, walked off the list.
if (RC == &W65816::Wide32RegClass || RC == &W65816::Acc32RegClass ||
RC == &W65816::AnyWide32RegClass) {
Register Lo, Hi;
switch (SrcReg) {
case W65816::AX32: Lo = W65816::A; Hi = W65816::X; break;
case W65816::IMG01: Lo = W65816::IMG0; Hi = W65816::IMG1; break;
case W65816::IMG23: Lo = W65816::IMG2; Hi = W65816::IMG3; break;
case W65816::IMG45: Lo = W65816::IMG4; Hi = W65816::IMG5; break;
case W65816::IMG67: Lo = W65816::IMG6; Hi = W65816::IMG7; break;
case W65816::IMG89: Lo = W65816::IMG8; Hi = W65816::IMG9; break;
case W65816::IMG1011: Lo = W65816::IMG10; Hi = W65816::IMG11; break;
case W65816::IMG1213: Lo = W65816::IMG12; Hi = W65816::IMG13; break;
case W65816::IMG1415: Lo = W65816::IMG14; Hi = W65816::IMG15; break;
default:
// Regalloc occasionally hands us an UNPAIRED single i16 physreg
// (Acc16 / Img16 / Idx16) for a Wide32-class spill — happens when
// only one sub-reg is live at the spill point and the regalloc
// decides to spill it through the Wide32 path anyway. Treat as
// a single i16 store of the lone half at offset 0; the matching
// reload mirrors this (only the lo half is read back). The hi
// half slot at offset 2 is left unwritten — the reload's hi load
// reads zero-init stack memory which is fine because nothing
// genuinely needed the hi value (otherwise the regalloc would
// have allocated a real pair).
if (SrcReg != W65816::A) {
copyPhysReg(MBB, MI, DL, W65816::A, SrcReg, false);
}
BuildMI(MBB, MI, DL, get(W65816::STAfi))
.addReg(W65816::A).addFrameIndex(FrameIdx).addImm(0);
if (SrcReg.isVirtual()) {
spillSTAfi(MBB, MI, DL)
.addReg(SrcReg, {}, llvm::sub_lo)
.addFrameIndex(FrameIdx).addImm(0);
spillSTAfi(MBB, MI, DL)
.addReg(SrcReg, getKillRegState(isKill), llvm::sub_hi)
.addFrameIndex(FrameIdx).addImm(2);
return;
}
// Bridge lo through A, store at offset 0; bridge hi through A,
// store at offset 2. This is brittle in the face of regalloc
// expectations — Wide32 spills are best avoided by keeping the
// pair in registers if at all possible.
if (Lo != W65816::A) {
copyPhysReg(MBB, MI, DL, W65816::A, Lo, false);
}
BuildMI(MBB, MI, DL, get(W65816::STAfi))
.addReg(W65816::A).addFrameIndex(FrameIdx).addImm(0);
copyPhysReg(MBB, MI, DL, W65816::A, Hi, false);
BuildMI(MBB, MI, DL, get(W65816::STAfi))
.addReg(W65816::A).addFrameIndex(FrameIdx).addImm(2);
Register Lo = getRegisterInfo().getSubReg(SrcReg, llvm::sub_lo);
Register Hi = getRegisterInfo().getSubReg(SrcReg, llvm::sub_hi);
if (!Lo || !Hi) {
// Regalloc occasionally hands us an UNPAIRED single i16 physreg for
// a Wide32-class spill (only one half live at the spill point).
// Store the lone half at offset 0; the reload mirrors this.
spillSTAfi(MBB, MI, DL)
.addReg(SrcReg, getKillRegState(isKill))
.addFrameIndex(FrameIdx).addImm(0);
return;
}
if (SrcReg == W65816::X || SrcReg == W65816::Y) {
unsigned XferOp = (SrcReg == W65816::X) ? W65816::TXA : W65816::TYA;
BuildMI(MBB, MI, DL, get(XferOp));
SrcReg = W65816::A;
spillSTAfi(MBB, MI, DL)
.addReg(Lo, getKillRegState(isKill))
.addFrameIndex(FrameIdx).addImm(0);
spillSTAfi(MBB, MI, DL)
.addReg(Hi, getKillRegState(isKill))
.addFrameIndex(FrameIdx).addImm(2);
return;
}
BuildMI(MBB, MI, DL, get(W65816::STAfi))
// X / Y sources included: eliminateFrameIndex transfers them through A
// (a TXA / TYA here clobbered A behind the allocator's back).
spillSTAfi(MBB, MI, DL)
.addReg(SrcReg, getKillRegState(isKill))
.addFrameIndex(FrameIdx)
.addImm(0);
@ -422,66 +627,44 @@ void W65816InstrInfo::loadRegFromStackSlot(MachineBasicBlock &MBB,
const TargetRegisterClass *RC,
Register VReg, unsigned SubReg,
MachineInstr::MIFlag Flags) const {
// LDAfi only knows how to put the value in A. If regalloc asks for
// a spill into X or Y, we have to bridge through A: LDA d,S then
// TAX / TAY. Without this, the MIR has `$x = LDAfi` but the asm
// printer emits just `LDA d,S` (which writes A, not X) — a silent
// miscompile that surfaced as i64 subtract chains using stale X
// values for the second word (caught by udivmod's `a - q*b` mod
// computation).
// LDAfi loads through A; eliminateFrameIndex adds the TAX / TAY / STA dp
// for an X, Y or IMG destination. (Before that existed, `$x = LDAfi`
// printed as a bare `LDA d,S`: i64 subtract chains used stale X words.)
DebugLoc DL = MI != MBB.end() ? MI->getDebugLoc() : DebugLoc();
// Wide32 reload: 2 i16 loads at offsets 0 and 2 of the 4-byte slot.
// Wide32 reload: two i16 loads, the high half first so an AX32 pair's
// X load (through A) comes before A takes the low half. A virtual
// Wide32 register is reloaded by sub-register (see storeRegToStackSlot);
// a physical half other than A is loaded through A by
// eliminateFrameIndex, which keeps A when it is still read (an `lda d,s
// ; tax` / `lda d,s ; sta dp` here clobbered it behind the allocator's
// back).
if (RC == &W65816::Wide32RegClass || RC == &W65816::Acc32RegClass ||
RC == &W65816::AnyWide32RegClass) {
Register Lo, Hi;
switch (DestReg) {
case W65816::AX32: Lo = W65816::A; Hi = W65816::X; break;
case W65816::IMG01: Lo = W65816::IMG0; Hi = W65816::IMG1; break;
case W65816::IMG23: Lo = W65816::IMG2; Hi = W65816::IMG3; break;
case W65816::IMG45: Lo = W65816::IMG4; Hi = W65816::IMG5; break;
case W65816::IMG67: Lo = W65816::IMG6; Hi = W65816::IMG7; break;
case W65816::IMG89: Lo = W65816::IMG8; Hi = W65816::IMG9; break;
case W65816::IMG1011: Lo = W65816::IMG10; Hi = W65816::IMG11; break;
case W65816::IMG1213: Lo = W65816::IMG12; Hi = W65816::IMG13; break;
case W65816::IMG1415: Lo = W65816::IMG14; Hi = W65816::IMG15; break;
default:
// Mirror of the unpaired-spill case in storeRegToStackSlot:
// regalloc handed us a single physreg for a Wide32 reload.
// Just load the lo half from offset 0 into the dest.
BuildMI(MBB, MI, DL, get(W65816::LDAfi), W65816::A)
.addFrameIndex(FrameIdx).addImm(0);
if (DestReg != W65816::A)
copyPhysReg(MBB, MI, DL, DestReg, W65816::A, false);
return;
}
// Lo half: LDA from offset 0, transfer to Lo if needed.
BuildMI(MBB, MI, DL, get(W65816::LDAfi), W65816::A)
.addFrameIndex(FrameIdx).addImm(0);
if (Lo != W65816::A)
copyPhysReg(MBB, MI, DL, Lo, W65816::A, false);
// Hi half: LDA from offset 2, transfer to Hi.
BuildMI(MBB, MI, DL, get(W65816::LDAfi), W65816::A)
if (DestReg.isVirtual()) {
BuildMI(MBB, MI, DL, get(W65816::LDAfi))
.addReg(DestReg, RegState::Define | RegState::Undef, llvm::sub_hi)
.addFrameIndex(FrameIdx).addImm(2);
if (Hi != W65816::A)
copyPhysReg(MBB, MI, DL, Hi, W65816::A, false);
BuildMI(MBB, MI, DL, get(W65816::LDAfi))
.addReg(DestReg, RegState::Define, llvm::sub_lo)
.addFrameIndex(FrameIdx).addImm(0);
return;
}
if (DestReg == W65816::A) {
Register Lo = getRegisterInfo().getSubReg(DestReg, llvm::sub_lo);
Register Hi = getRegisterInfo().getSubReg(DestReg, llvm::sub_hi);
if (!Lo || !Hi) {
// Mirror of the unpaired-spill case in storeRegToStackSlot.
BuildMI(MBB, MI, DL, get(W65816::LDAfi), DestReg)
.addFrameIndex(FrameIdx)
.addImm(0);
.addFrameIndex(FrameIdx).addImm(0);
return;
}
if (DestReg == W65816::X || DestReg == W65816::Y) {
// Load via A, then transfer. A is implicitly clobbered.
BuildMI(MBB, MI, DL, get(W65816::LDAfi), W65816::A)
.addFrameIndex(FrameIdx)
.addImm(0);
unsigned XferOp = (DestReg == W65816::X) ? W65816::TAX : W65816::TAY;
BuildMI(MBB, MI, DL, get(XferOp));
BuildMI(MBB, MI, DL, get(W65816::LDAfi), Hi)
.addFrameIndex(FrameIdx).addImm(2);
BuildMI(MBB, MI, DL, get(W65816::LDAfi), Lo)
.addFrameIndex(FrameIdx).addImm(0);
return;
}
// Fallback: assume A path (covers Acc16 / Wide16 vregs by class).
// Any destination class: eliminateFrameIndex loads A, X, Y or an IMG
// word.
BuildMI(MBB, MI, DL, get(W65816::LDAfi), DestReg)
.addFrameIndex(FrameIdx)
.addImm(0);
@ -761,7 +944,16 @@ unsigned W65816InstrInfo::getInstSizeInBytes(const MachineInstr &MI) const {
// JSLpseudo: jsl is 4 bytes.
case W65816::JSLpseudo:
case W65816::JSLpseudo32:
case W65816::JSLlight:
case W65816::JSLlight32:
return 4;
// LDAi16imm_bank: `lda $BE` (2), or for an external-weak symbol
// `lda #sym ; beq L ; lda $BE` (3 + 2 + 2).
case W65816::LDAi16imm_bank:
return MI.getOperand(1).isGlobal() &&
MI.getOperand(1).getGlobal()->hasExternalWeakLinkage()
? 7
: 2;
default:
break;
}

28
src/llvm/lib/Target/W65816/W65816InstrInfo.h
View file

@ -14,6 +14,7 @@
#define LLVM_LIB_TARGET_W65816_W65816INSTRINFO_H
#include "W65816RegisterInfo.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/CodeGen/TargetInstrInfo.h"
#define GET_INSTRINFO_HEADER
@ -59,6 +60,29 @@ bool isTiedAcc16Consumer(unsigned Opc);
// per-MI operand check the bridge passes perform on every candidate.
bool hasTiedAcc16Src(const MachineInstr &MI);
// Whether A's value may still be read from From on, before something
// overwrites it. Follows the CFG through a few successor blocks and reads
// a return's implicit uses; anything it does not know counts as a read.
bool aLiveFrom(const MachineBasicBlock &MBB,
MachineBasicBlock::const_iterator From);
// Release Bytes of stack (SP += Bytes) before I in 16-bit M and X, keeping
// A, X and Y as asked, in the cheapest form for the 65816: N/2 PLY or PLX,
// TSC/CLC/ADC/TCS, that bracketed by TAY/TYA or TAX/TXA, or through DP $E0.
// Clobbers the flags.
void emitStackRelease(MachineBasicBlock &MBB, MachineBasicBlock::iterator I,
const DebugLoc &DL, const TargetInstrInfo &TII,
int64_t Bytes, bool KeepA, bool KeepX, bool KeepY);
// The cost model the stack-release choices share: a cycle is worth two
// bytes of code.
inline unsigned cycleByteCost(unsigned Cycles, unsigned Bytes) {
return 2 * Cycles + Bytes;
}
// Cost of what emitStackRelease emits for the same request.
unsigned stackReleaseCost(int64_t Bytes, bool KeepA, bool KeepX, bool KeepY);
} // namespace W65816Helpers
class W65816InstrInfo : public W65816GenInstrInfo {
@ -79,6 +103,10 @@ public:
MachineBasicBlock &MBB, MachineBasicBlock::iterator MI, Register SrcReg,
bool isKill, int FrameIndex, const TargetRegisterClass *RC, Register VReg,
MachineInstr::MIFlag Flags = MachineInstr::NoFlags) const override;
// A spill's STAfi, without the opcode's Defs = [A] (see the .cpp).
MachineInstrBuilder spillSTAfi(MachineBasicBlock &MBB,
MachineBasicBlock::iterator MI,
const DebugLoc &DL) const;
void loadRegFromStackSlot(
MachineBasicBlock &MBB, MachineBasicBlock::iterator MI, Register DestReg,
int FrameIdx, const TargetRegisterClass *RC, Register VReg,

114
src/llvm/lib/Target/W65816/W65816InstrInfo.td
View file

@ -45,6 +45,11 @@ def W65816retglue : SDNode<"W65816ISD::RET_GLUE", SDTNone,
def W65816call : SDNode<"W65816ISD::CALL", SDT_W65816Call,
[SDNPHasChain, SDNPOutGlue, SDNPOptInGlue,
SDNPVariadic]>;
// A call to a runtime helper known to touch only A/X/Y, its arguments,
// the $E0..$F0 scratch and the stack (see JSLlight).
def W65816callLight : SDNode<"W65816ISD::CALL_LIGHT", SDT_W65816Call,
[SDNPHasChain, SDNPOutGlue, SDNPOptInGlue,
SDNPVariadic]>;
def W65816callseq_start :
SDNode<"ISD::CALLSEQ_START", SDT_W65816CallSeqStart,
@ -141,6 +146,22 @@ def W65816stPtrOff : SDNode<"W65816ISD::ST_PTR_OFF", SDT_W65816StPtrOff,
def W65816stbPtrOff : SDNode<"W65816ISD::STB_PTR_OFF", SDT_W65816StPtrOff,
[SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
// `Idx` siblings: the deref of `ptr + zext(idx) + off`, with idx + off in
// Y. LowerLoad / LowerStore form them only when idx + off provably fits
// 16 bits, so Y never wraps where the i32 add would have carried.
def SDT_W65816LdPtrIdx
: SDTypeProfile<1, 3, [SDTCisVT<0, i16>, SDTCisVT<1, i32>,
SDTCisVT<2, i16>, SDTCisVT<3, i16>]>;
def SDT_W65816StPtrIdx
: SDTypeProfile<0, 4, [SDTCisVT<0, i16>, SDTCisVT<1, i32>,
SDTCisVT<2, i16>, SDTCisVT<3, i16>]>;
def W65816ldPtrIdx : SDNode<"W65816ISD::LD_PTR_IDX", SDT_W65816LdPtrIdx,
[SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
def W65816stPtrIdx : SDNode<"W65816ISD::ST_PTR_IDX", SDT_W65816StPtrIdx,
[SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
def W65816stbPtrIdx : SDNode<"W65816ISD::STB_PTR_IDX", SDT_W65816StPtrIdx,
[SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
// Trap: produced by LowerTRAP for ISD::TRAP (__builtin_trap,
// -fsanitize-trap=undefined). Pure side-effect node, takes the
// chain in and threads it out — selected from the (W65816trap)
@ -861,6 +882,17 @@ let mayLoad = 1, hasSideEffects = 0, mayStore = 0,
def LDAfi : W65816Pseudo<(outs Acc16:$dst), (ins memfi:$addr),
"# LDAfi $dst, $addr", []>;
}
// A = an IMG register's DP word, as a real instruction instead of a COPY:
// W65816StackSlotCleanup forwards a temp slot's IMG word to its reload
// with it. The reload's A def is often dead in name only (the STA_DP $E0
// that consumes it names no register), and a COPY whose defs are dead is
// deleted at expansion; the explicit IMG use keeps that register's def
// in place for the post-RA passes. expandPostRAPseudo lowers it to
// LDA_DP.
let mayLoad = 1, hasSideEffects = 1, mayStore = 0 in {
def LDAimg : W65816Pseudo<(outs Acc16:$dst), (ins Img16:$src),
"# LDAimg $dst, $src", []>;
}
// STAfi accepts Wide16 src so greedy can park the value in IMGn instead
// of A. When src is in IMGn (or X/Y after class coalescing), eliminate-
// FrameIndex emits a PHA-bracketed sequence (`pha; lda dp; sta d+2,s; pla`)
@ -872,6 +904,12 @@ def LDAfi : W65816Pseudo<(outs Acc16:$dst), (ins memfi:$addr),
// A's was expected — observed as `dadd(1.5,2.5) → 0x4010_0000_3000_3000`
// under full IMG-clobber.
//
// The over-approximation must stay off the STAfis the register allocator
// itself inserts (storeRegToStackSlot drops it): the spiller places them
// without checking A's interference, so a spill can land after `$a = COPY
// $img0` with A still live into the next block, and MachineCopyPropagation
// then deleted that copy as overwritten.
//
// Note: Defs = [A] triggers a greedy-regalloc assertion failure
// (LiveRangeEdit::eliminateDeadDef on a KILL pseudo with non-dead
// implicit-def $a) on functions with many cross-call Acc16 vregs
@ -1164,7 +1202,7 @@ def STAfi_indY : W65816Pseudo<(outs), (ins Acc16:$src, memfi:$addr),
"# STAfi_indY $src, $addr", []>;
}
// Atomic i8 (byte) variant of STAfi_indY. Emitted by the STBptr32S
// Atomic i8 (byte) variant of STAfi_indY. Emitted by the STBptr32
// inserter for Layer-2 (d,S),Y byte stores; eliminateFrameIndex expands
// it with a SEP #$20 / REP #$20 wrap so only one byte is written. Kept a
// single pseudo through register allocation (same reason as STB_DPIndLongY
@ -1290,6 +1328,23 @@ def STBptr32Off : W65816Pseudo<(outs),
(ins Acc16:$val, AnyWide32:$ptr, i16imm:$off),
"# STBptr32Off $val, $ptr, $off", []>;
}
let usesCustomInserter = 1, hasSideEffects = 1, mayLoad = 1,
Defs = [Y, P] in {
def LDAptr32Idx : W65816Pseudo<(outs Acc16:$dst),
(ins AnyWide32:$ptr, Wide16:$idx, i16imm:$off),
"# LDAptr32Idx $dst, $ptr, $idx, $off", []>;
}
let usesCustomInserter = 1, hasSideEffects = 1, mayStore = 1,
Defs = [Y, P] in {
def STAptr32Idx : W65816Pseudo<(outs),
(ins Acc16:$val, AnyWide32:$ptr, Wide16:$idx,
i16imm:$off),
"# STAptr32Idx $val, $ptr, $idx, $off", []>;
def STBptr32Idx : W65816Pseudo<(outs),
(ins Acc16:$val, AnyWide32:$ptr, Wide16:$idx,
i16imm:$off),
"# STBptr32Idx $val, $ptr, $idx, $off", []>;
}
// Direct ptr32 load/store patterns over generic ISD::LOAD / ISD::STORE
// when the address is an i32 (AnyWide32) reg. These are unreachable
@ -1326,29 +1381,16 @@ def : Pat<(W65816stPtrOff Acc16:$val, AnyWide32:$ptr, (i16 timm:$off)),
(STAptr32Off Acc16:$val, AnyWide32:$ptr, imm:$off)>;
def : Pat<(W65816stbPtrOff Acc16:$val, AnyWide32:$ptr, (i16 timm:$off)),
(STBptr32Off Acc16:$val, AnyWide32:$ptr, imm:$off)>;
// Split-pair variants: same semantics as LDAptr32/STAptr32/STBptr32 but
// the ptr is two separate i16 register operands (lo + hi) instead of
// one Wide32 register pair. Used by the W65816LowerWide32 pre-RA pass
// to relieve register-pair allocation pressure: it walks REG_SEQUENCE
// + LDAptr32 chains, decomposes the Wide32 vregs into pairs of i16
// vregs, and rewrites the LDAptr32-family to take the two halves
// directly.
let usesCustomInserter = 1, hasSideEffects = 1, mayLoad = 1,
Defs = [Y, P] in {
def LDAptr32S : W65816Pseudo<(outs Acc16:$dst),
(ins Wide16:$ptrLo, Wide16:$ptrHi),
"# LDAptr32S $dst, $ptrLo, $ptrHi", []>;
}
let usesCustomInserter = 1, hasSideEffects = 1, mayStore = 1,
Defs = [Y, P] in {
def STAptr32S : W65816Pseudo<(outs),
(ins Acc16:$val, Wide16:$ptrLo, Wide16:$ptrHi),
"# STAptr32S $val, $ptrLo, $ptrHi", []>;
def STBptr32S : W65816Pseudo<(outs),
(ins Acc16:$val, Wide16:$ptrLo, Wide16:$ptrHi),
"# STBptr32S $val, $ptrLo, $ptrHi", []>;
}
// Idx variants: a zero-extended 16-bit index folded into Y with the
// offset (see LowerLoad's peelPtr32Index).
def : Pat<(W65816ldPtrIdx AnyWide32:$ptr, Wide16:$idx, (i16 timm:$off)),
(LDAptr32Idx AnyWide32:$ptr, Wide16:$idx, imm:$off)>;
def : Pat<(W65816stPtrIdx Acc16:$val, AnyWide32:$ptr, Wide16:$idx,
(i16 timm:$off)),
(STAptr32Idx Acc16:$val, AnyWide32:$ptr, Wide16:$idx, imm:$off)>;
def : Pat<(W65816stbPtrIdx Acc16:$val, AnyWide32:$ptr, Wide16:$idx,
(i16 timm:$off)),
(STBptr32Idx Acc16:$val, AnyWide32:$ptr, Wide16:$idx, imm:$off)>;
// i8 load via Acc16 pointer producing a true i8 (Acc8) result. Reuses
// the existing zextloadi8 16-bit-LDA-and-mask path: load 2 bytes, mask
@ -1977,7 +2019,11 @@ def : Pat<(store
// regalloc-side patch (taught to never spill between sub-reg defs of
// the same parent vreg) or a backend-side restructure of i64-arg
// passing to use stack slots directly instead of register pairs.
//
// A call that can return twice (setjmp) gets IMG8..IMG15 as implicit defs
// on top (AdjustInstrPostInstrSelection): see W65816ISelLowering.cpp.
let isCall = 1, hasSideEffects = 0, mayLoad = 0, mayStore = 0,
hasPostISelHook = 1,
Defs = [A, X, Y, DPF0,
IMG0, IMG1, IMG2, IMG3, IMG4, IMG5, IMG6, IMG7] in {
def JSLpseudo : W65816Pseudo<(outs), (ins i16imm:$dst),
@ -1995,3 +2041,23 @@ def : Pat<(W65816call (i16 texternalsym:$dst)), (JSLpseudo texternalsym:$dst)>;
// cRELOC opcode rewrites the offset and bank at load time.
def : Pat<(W65816call (i32 tglobaladdr:$dst)), (JSLpseudo32 tglobaladdr:$dst)>;
def : Pat<(W65816call (i32 texternalsym:$dst)), (JSLpseudo32 texternalsym:$dst)>;
// A call to one of libgcc.s's hand-written helpers (16/32-bit multiply,
// divide and shift, memcpy / memmove / memset): each touches only A, X,
// Y, the stack, DBR (saved and restored) and the $E0..$EF scratch, plus
// $F0 for the signed 32-bit divides -- never IMG0..IMG7. Leaving them
// out of Defs lets a value stay in an IMG register across the call
// instead of being spilled (every multiply in a loop used to force the
// loop's live values onto the stack). W65816ISelLowering::isLightCallee
// holds the list; a helper that starts using an IMG word must leave it.
let isCall = 1, hasSideEffects = 0, mayLoad = 0, mayStore = 0,
Defs = [A, X, Y, DPF0] in {
def JSLlight : W65816Pseudo<(outs), (ins i16imm:$dst),
"# JSLlight $dst", []>;
def JSLlight32 : W65816Pseudo<(outs), (ins i32imm:$dst),
"# JSLlight32 $dst", []>;
}
def : Pat<(W65816callLight (i16 tglobaladdr:$dst)), (JSLlight tglobaladdr:$dst)>;
def : Pat<(W65816callLight (i16 texternalsym:$dst)), (JSLlight texternalsym:$dst)>;
def : Pat<(W65816callLight (i32 tglobaladdr:$dst)), (JSLlight32 tglobaladdr:$dst)>;
def : Pat<(W65816callLight (i32 texternalsym:$dst)), (JSLlight32 texternalsym:$dst)>;

50
src/llvm/lib/Target/W65816/W65816LowerWide32.cpp
View file

@ -11,9 +11,7 @@
// functions (`RegAllocBase` crashes during `allocatePhysRegs`) because
// the i64-via-2-i32-via-Wide32 chain produces too many simultaneously
// live register-pair vregs. After this pass, only i16 vregs remain at
// the regalloc input — Wide32 lives only inside this pass and the new
// LDAptr32S / STAptr32S / STBptr32S pseudos that take 2 i16 ptr operands
// directly.
// the regalloc input.
//
// Walks the MIR and:
// 1. Finds REG_SEQUENCE producing Wide32 / Acc32 / AnyWide32; records
@ -23,9 +21,11 @@
// mapped Wide32 vreg; chains the (lo, hi) mapping forward.
// 3. Rewrites EXTRACT_SUBREG of mapped Wide32 vregs by replacing the
// destination vreg with the appropriate half (sub_lo or sub_hi).
// 4. Rewrites LDAptr32 / STAptr32 / STBptr32 with a mapped Wide32 ptr
// to the corresponding LDAptr32S / STAptr32S / STBptr32S pseudo
// with two separate i16 operands.
//
// The far-pointer pseudos (LDAptr32 / STAptr32 / STBptr32) never reach
// this pass: they have custom inserters, which finalize-isel runs right
// after instruction selection, and those take their Wide32 pointer apart
// with sub-register COPYs (rewritten in step 3's partial-COPY case).
//
// Bail / safety: any Wide32 vreg whose def we can't decompose is left
// in place — regalloc may still struggle but no miscompile.
@ -343,53 +343,19 @@ bool W65816LowerWide32::runOnMachineFunction(MachineFunction &MF) {
auto [Lo, Hi] = wideMap[SrcOp.getReg()];
Register Half = (SubIdx == llvm::sub_lo) ? Lo : Hi;
forwardHalf(MI, DstOp.getReg(), Half);
continue;
}
}
// LDAptr32 / STAptr32 / STBptr32 with a mapped Wide32 ptr:
// rewrite to LDAptr32S / STAptr32S / STBptr32S.
unsigned Opc = MI->getOpcode();
bool isPtrOp = (Opc == W65816::LDAptr32 || Opc == W65816::STAptr32 ||
Opc == W65816::STBptr32);
if (isPtrOp) {
Register Ptr = MI->getOperand(1).getReg();
if (Ptr.isVirtual() && wideMap.count(Ptr)) {
Register Lo = current(wideMap[Ptr].first);
Register Hi = current(wideMap[Ptr].second);
unsigned NewOpc = (Opc == W65816::LDAptr32) ? W65816::LDAptr32S
: (Opc == W65816::STAptr32) ? W65816::STAptr32S
: W65816::STBptr32S;
DebugLoc DL = MI->getDebugLoc();
MachineBasicBlock *ParentMBB = MI->getParent();
if (Opc == W65816::LDAptr32) {
Register Dst = MI->getOperand(0).getReg();
BuildMI(*ParentMBB, MI->getIterator(), DL, TII.get(NewOpc), Dst)
.addReg(Lo)
.addReg(Hi);
} else {
Register Val = MI->getOperand(0).getReg();
BuildMI(*ParentMBB, MI->getIterator(), DL, TII.get(NewOpc))
.addReg(Val)
.addReg(Lo)
.addReg(Hi);
}
useToErase.push_back(MI);
continue;
}
}
}
}
// Erase use-side instructions (EXTRACT_SUBREG, LDAptr32-family) first
// Erase use-side instructions (EXTRACT_SUBREG, partial COPYs) first
// so the Wide32 vreg becomes dead.
//
// DBG_VALUE preservation: MRI.replaceRegWith above already redirected
// any debug operands to the new half-vreg (replaceRegWith walks
// reg_operands which includes debug ops). No further work needed for
// the EXTRACT_SUBREG / partial-COPY paths since the user-visible value
// is preserved in the half vreg. For LDAptr32 rewrites we don't move
// a user-named value so DBG_VALUE attribution is unaffected.
// is preserved in the half vreg.
for (auto *MI : useToErase)
MI->eraseFromParent();

89
src/llvm/lib/Target/W65816/W65816NarrowI32Mul.cpp
View file

@ -9,7 +9,10 @@
// IR-level peephole. Detects `mul i32 X, Y` where both X and Y have
// their top 16 bits provably zero (via LLVM's IR-level computeKnownBits)
// and rewrites to a call to `__umulhisi3` — a 16x16 -> 32 unsigned
// multiply (~30% faster than __mulsi3 for `(u32)i * i` patterns).
// multiply (~30% faster than __mulsi3 for `(u32)i * i` patterns) — or
// where both are sign-extended 16-bit values, to `__mulhisi3`, its
// signed sibling. A product that fits 16 bits itself becomes a 16-bit
// multiply, zero-extended.
//
// Why an IR pass instead of a Custom SDAG lowering: LLVM's IndVarSimplify
// loop pass widens narrow induction variables (e.g. an i16 loop counter
@ -53,7 +56,7 @@ public:
W65816NarrowI32Mul() : FunctionPass(ID) {}
StringRef getPassName() const override {
return "W65816 narrow i32 multiplies to __umulhisi3";
return "W65816 narrow i32 multiplies to __umulhisi3 / __mulhisi3";
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
@ -77,13 +80,13 @@ INITIALIZE_PASS_END(W65816NarrowI32Mul, DEBUG_TYPE,
"W65816 narrow i32 multiplies", false, false)
// Get-or-declare `__umulhisi3(i16, i16) -> i32` in the module.
static FunctionCallee getUmulhisi3(Module &M) {
// Get-or-declare `Name(i16, i16) -> i32` (__umulhisi3 / __mulhisi3).
static FunctionCallee getMul16x16(Module &M, StringRef Name) {
LLVMContext &Ctx = M.getContext();
Type *I16 = Type::getInt16Ty(Ctx);
Type *I32 = Type::getInt32Ty(Ctx);
FunctionType *FT = FunctionType::get(I32, {I16, I16}, false);
return M.getOrInsertFunction("__umulhisi3", FT);
return M.getOrInsertFunction(Name, FT);
}
@ -105,13 +108,62 @@ static bool top16Zero(Value *V, const DataLayout &DL, ScalarEvolution &SE) {
}
// True iff the product of two operands whose high words are zero fits
// 16 bits too: by the product's known bits, or by the operands' largest
// values (known bits and ScalarEvolution's unsigned ranges).
static bool productFitsU16(BinaryOperator *BO, const DataLayout &DL,
ScalarEvolution &SE) {
if (computeKnownBits(BO, DL).countMinLeadingZeros() >= 16) {
return true;
}
auto maxOf = [&](Value *V) {
APInt Max = computeKnownBits(V, DL).getMaxValue();
if (SE.isSCEVable(V->getType())) {
Max = APIntOps::umin(Max, SE.getUnsignedRangeMax(SE.getSCEV(V)));
}
return Max;
};
bool Overflow = false;
APInt Product = maxOf(BO->getOperand(0)).umul_ov(maxOf(BO->getOperand(1)),
Overflow);
return !Overflow && Product.getActiveBits() <= 16;
}
// True iff V is the sign extension of its low 16 bits: by known sign
// bits, or by ScalarEvolution's signed range.
static bool fitsS16(Value *V, const DataLayout &DL, ScalarEvolution &SE) {
if (ComputeNumSignBits(V, DL) >= 17) {
return true;
}
if (!SE.isSCEVable(V->getType())) {
return false;
}
ConstantRange R = SE.getSignedRange(SE.getSCEV(V));
return R.getMinSignedBits() <= 16;
}
bool W65816NarrowI32Mul::runOnFunction(Function &F) {
Module *M = F.getParent();
const DataLayout &DL = M->getDataLayout();
Type *I16 = Type::getInt16Ty(F.getContext());
ScalarEvolution &SE = getAnalysis<ScalarEvolutionWrapperPass>().getSE();
SmallVector<BinaryOperator *, 8> Worklist;
// Each multiply with the helper that computes it: __umulhisi3 when both
// operands' high words are zero, else __mulhisi3 when both are
// sign-extended 16-bit values (`(long)a * b` of two shorts -- CoreMark's
// matrix_mul_matrix, the dot products of fixed-point code), whose
// __mulsi3 call took the 32-step loop for every negative operand.
//
// A product that fits 16 bits itself is a 16-bit multiply, zero-extended:
// shifts and adds for a constant (`(u32)byte * 6`, a 6-byte record's
// offset), else __mulhi3, and a high word the rest of the code knows is
// zero.
static const bool NarrowProducts =
!getenv("W65816_DISABLE_NARROW_PRODUCT");
enum class Kind { Unsigned, Signed, Product16 };
SmallVector<std::pair<BinaryOperator *, Kind>, 8> Worklist;
for (Instruction &I : instructions(F)) {
auto *BO = dyn_cast<BinaryOperator>(&I);
if (!BO || BO->getOpcode() != Instruction::Mul) {
@ -120,13 +172,16 @@ bool W65816NarrowI32Mul::runOnFunction(Function &F) {
if (!BO->getType()->isIntegerTy(32)) {
continue;
}
if (!top16Zero(BO->getOperand(0), DL, SE)) {
if (top16Zero(BO->getOperand(0), DL, SE) &&
top16Zero(BO->getOperand(1), DL, SE)) {
bool Fits = NarrowProducts && productFitsU16(BO, DL, SE);
Worklist.push_back({BO, Fits ? Kind::Product16 : Kind::Unsigned});
continue;
}
if (!top16Zero(BO->getOperand(1), DL, SE)) {
continue;
if (fitsS16(BO->getOperand(0), DL, SE) &&
fitsS16(BO->getOperand(1), DL, SE)) {
Worklist.push_back({BO, Kind::Signed});
}
Worklist.push_back(BO);
}
if (Worklist.empty()) {
@ -151,16 +206,22 @@ bool W65816NarrowI32Mul::runOnFunction(Function &F) {
return B.CreateTrunc(V, I16);
};
FunctionCallee Callee = getUmulhisi3(*M);
FunctionCallee UCallee = getMul16x16(*M, "__umulhisi3");
FunctionCallee SCallee = getMul16x16(*M, "__mulhisi3");
SmallPtrSet<Instruction *, 8> MaybeDeadSet;
for (BinaryOperator *BO : Worklist) {
for (auto [BO, K] : Worklist) {
IRBuilder<> B(BO);
Value *AOp = BO->getOperand(0);
Value *BOp = BO->getOperand(1);
Value *A = narrowOperand(AOp, B);
Value *Bv = narrowOperand(BOp, B);
Value *Call = B.CreateCall(Callee, {A, Bv});
BO->replaceAllUsesWith(Call);
Value *Result;
if (K == Kind::Product16) {
Result = B.CreateZExt(B.CreateNUWMul(A, Bv), BO->getType());
} else {
Result = B.CreateCall(K == Kind::Signed ? SCallee : UCallee, {A, Bv});
}
BO->replaceAllUsesWith(Result);
BO->eraseFromParent();
// If the original operands were zext/sext nodes, they may now be
// dead. Add to a SET — `mul %x, %x` would push the same node

3
src/llvm/lib/Target/W65816/W65816PromoteFiToImg.cpp
View file

@ -261,7 +261,8 @@ bool W65816PromoteFiToImg::runOnMachineFunctionDisabled(MachineFunction &MF) {
else continue;
// libgcc.s multiply/divide/shift helpers — verified to only use
// $E0..$E9 internally, no IMG0..7 touch.
if (Name == "__umulhisi3" || Name == "__mulhi3" ||
if (Name == "__umulhisi3" || Name == "__mulhisi3" ||
Name == "__mulhi3" ||
Name == "__mulsi3" || Name == "__udivhi3" ||
Name == "__umodhi3" || Name == "__divhi3" ||
Name == "__modhi3" || Name == "__udivsi3" ||

60
src/llvm/lib/Target/W65816/W65816RegisterInfo.cpp
View file

@ -49,6 +49,18 @@ static int imgRegToDP(Register R) {
}
}
// Whether A's value may still be read after MI, before something
// overwrites it: whether an LDAfi / STAfi that has to move a non-A
// register's value through A must save A around itself. Spill and reload
// code is inserted where the allocator has a value of its own in A (a
// compare's operand kept across a reloaded pointer's copy), so the old
// fixed `lda d,s ; sta dp` reload clobbered it, and the PHA / PLA every
// non-A spill paid is needed only here.
static bool aLiveAfter(const MachineInstr &MI) {
return W65816Helpers::aLiveFrom(*MI.getParent(),
std::next(MI.getIterator()));
}
// Far FI elim via DP frame-pointer ($F6/$F7). Called when an FI's
// effective offset exceeds 0xFF and the function reserved an FP at
// prologue time (StackSize > 200). Stack is always bank 0, so
@ -69,6 +81,14 @@ static bool expandFarFI(MachineInstr &MI, int FPOff,
switch (Opc) {
case W65816::LDAfi: {
Register Dst = MI.getOperand(0).getReg();
// A non-A destination is reached through A (an IMG one by `sta dp`
// after the load, which this path used to leave out: the reload landed
// in A only). Save A around it when A is read afterwards.
int DstDP = imgRegToDP(Dst);
bool SaveA = Dst != W65816::A && aLiveAfter(MI);
if (SaveA)
BuildMI(MBB, II, DL, TII.get(W65816::PHA))
.addReg(W65816::A, RegState::Implicit);
// Mark Y use as Undef: if Y is dead at this insertion point, the
// value we save is "don't care" — we restore the same garbage byte
// later. Without Undef, the verifier rejects when no def reaches
@ -89,6 +109,11 @@ static bool expandFarFI(MachineInstr &MI, int FPOff,
BuildMI(MBB, II, DL, TII.get(W65816::TAX));
else if (Dst == W65816::Y)
BuildMI(MBB, II, DL, TII.get(W65816::TAY));
else if (DstDP >= 0)
BuildMI(MBB, II, DL, TII.get(W65816::STA_DP)).addImm(DstDP);
if (SaveA)
BuildMI(MBB, II, DL, TII.get(W65816::PLA))
.addReg(W65816::A, RegState::ImplicitDefine);
return true;
}
case W65816::STAfi: {
@ -101,7 +126,7 @@ static bool expandFarFI(MachineInstr &MI, int FPOff,
// Without it here, an IMG-source STAfi in a >0xFF frame silently
// clobbered A and the cleanup then deleted the *following* reload,
// so the next `sta $E0` parked the source's value where the caller
// wanted something else. Live symptom: the LDAptr32S inserter
// wanted something else. Live symptom: the LDAptr32 inserter
// built its far pointer with the pointer's BANK byte in the offset
// half, and `lda [$E0],y` read a stray address -- Space Taxi's
// stSimTick read a nonzero byte for sim->collisionPhase and never
@ -248,6 +273,8 @@ static bool expandFarFI(MachineInstr &MI, int FPOff,
#define GET_REGINFO_TARGET_DESC
#include "W65816GenRegisterInfo.inc"
W65816RegisterInfo::W65816RegisterInfo() : W65816GenRegisterInfo(W65816::PC) {}
const MCPhysReg *
@ -345,9 +372,15 @@ bool W65816RegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II,
}
report_fatal_error("W65816: frame offset out of stack-relative range");
}
// A non-A destination is reached through A: save A around it when A
// is still read afterwards (see aLiveAfter).
bool SaveA = Dst != W65816::A && aLiveAfter(MI);
if (SaveA)
BuildMI(*MI.getParent(), II, MI.getDebugLoc(), TII.get(W65816::PHA))
.addReg(W65816::A, RegState::Implicit);
BuildMI(*MI.getParent(), II, MI.getDebugLoc(),
TII.get(W65816::LDA_StackRel))
.addImm(Offset)
.addImm(SaveA ? Offset + 2 : Offset) // PHA moved S down 2
.addReg(W65816::A, RegState::ImplicitDefine);
if (Dst == W65816::X) {
BuildMI(*MI.getParent(), II, MI.getDebugLoc(), TII.get(W65816::TAX));
@ -387,6 +420,9 @@ bool W65816RegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II,
TII.get(W65816::STA_DP)).addImm(dstDP);
}
}
if (SaveA)
BuildMI(*MI.getParent(), II, MI.getDebugLoc(), TII.get(W65816::PLA))
.addReg(W65816::A, RegState::ImplicitDefine);
MI.eraseFromParent();
return true;
}
@ -446,7 +482,25 @@ bool W65816RegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II,
// 16-bit load of the extra byte is a harmless read (the high byte of an i8 value is ignored), so
// LDAfi is left 16-bit. The load side never corrupts anything.
bool ByteSlot = MFI.getObjectSize(FI) == 1;
if (srcDP >= 0 || Src == W65816::X || Src == W65816::Y) {
if ((srcDP >= 0 || Src == W65816::X || Src == W65816::Y) &&
!aLiveAfter(MI)) {
// A is overwritten before anything reads it: no need to keep it.
if (srcDP >= 0) {
BuildMI(*MI.getParent(), II, MI.getDebugLoc(),
TII.get(W65816::LDA_DP)).addImm(srcDP);
} else {
unsigned XferOp = (Src == W65816::X) ? W65816::TXA : W65816::TYA;
BuildMI(*MI.getParent(), II, MI.getDebugLoc(), TII.get(XferOp));
}
if (ByteSlot)
BuildMI(*MI.getParent(), II, MI.getDebugLoc(), TII.get(W65816::SEP)).addImm(0x20);
BuildMI(*MI.getParent(), II, MI.getDebugLoc(),
TII.get(W65816::STA_StackRel))
.addImm(Offset)
.addReg(W65816::A, RegState::Implicit);
if (ByteSlot)
BuildMI(*MI.getParent(), II, MI.getDebugLoc(), TII.get(W65816::REP)).addImm(0x20);
} else if (srcDP >= 0 || Src == W65816::X || Src == W65816::Y) {
// STAfi with non-A source: must clobber A to land the value in
// A and then `sta d,s`. PHA-bracket so A's incoming value is
// preserved across the spill — without this, a regalloc-emitted

325
src/llvm/lib/Target/W65816/W65816ReturnPopMerge.cpp Normal file
View file

@ -0,0 +1,325 @@
//===-- W65816ReturnPopMerge.cpp - Merge stack releases into returns -----===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===---------------------------------------------------------------------===//
//
// Pre-emit: a path that ends in a return releases its stack in pieces.
// The last call's argument cleanup (N/2 PLY, or TSC/CLC/ADC/TCS) runs,
// then the epilogue's own frame release, often in a shared return block
// that branch folding tail-merged the two into, behind a BRA:
//
// jsl f jsl f
// ply tsc
// ply => clc
// bra .Lret adc #10
// .Lret: tcs
// ply rtl
// ply
// ply
// rtl
//
// Here 25 cycles of pulls and 3 of branch become 9. For every block whose
// path to a return is nothing but stack releases (in the block's tail,
// then through blocks made only of releases and reached by fall-through
// or an unconditional branch), add the bytes up and, when one release
// sized for the total is cheaper, give the block its own copy of it and
// of the return. The release keeps what the return reads (A, X, Y by its
// implicit uses) and picks its form by W65816Helpers::emitStackRelease;
// flags are dead at a return. Return blocks left without predecessors
// are deleted.
//
// Kill switch: W65816_DISABLE_RETURN_POP_MERGE=1.
//
//===---------------------------------------------------------------------===//
#include "W65816.h"
#include "W65816InstrInfo.h"
#include "W65816Subtarget.h"
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
using namespace llvm;
#define DEBUG_TYPE "w65816-return-pop-merge"
namespace {
class W65816ReturnPopMerge : public MachineFunctionPass {
public:
static char ID;
W65816ReturnPopMerge() : MachineFunctionPass(ID) {}
bool runOnMachineFunction(MachineFunction &MF) override;
StringRef getPassName() const override {
return "W65816 merge stack releases into returns";
}
};
char W65816ReturnPopMerge::ID = 0;
// What the return reads, which a merged release must keep.
struct KeepSet {
bool A = true;
bool X = true;
bool Y = true;
};
// A run of stack releases ending at a block position.
struct ReleaseRun {
MachineBasicBlock::iterator Begin;
int64_t Bytes = 0;
unsigned Cost = 0;
};
bool isImm(const MachineInstr &MI, unsigned Opc, int64_t Imm) {
return MI.getOpcode() == Opc && MI.getOperand(0).isImm() &&
MI.getOperand(0).getImm() == Imm;
}
// The TSC / CLC / ADC #k / TCS core ending at TCS (It), with its first
// instruction in Begin and k in Bytes.
bool matchAdcCore(MachineBasicBlock &MBB, MachineBasicBlock::iterator It,
MachineBasicBlock::iterator &Begin, int64_t &Bytes) {
if (It->getOpcode() != W65816::TCS || It == MBB.begin())
return false;
MachineBasicBlock::iterator Adc = std::prev(It);
if (Adc->getOpcode() != W65816::ADC_Imm16 || !Adc->getOperand(0).isImm() ||
Adc == MBB.begin())
return false;
MachineBasicBlock::iterator Clc = std::prev(Adc);
if (Clc->getOpcode() != W65816::CLC || Clc == MBB.begin())
return false;
MachineBasicBlock::iterator Tsc = std::prev(Clc);
if (Tsc->getOpcode() != W65816::TSC)
return false;
Begin = Tsc;
Bytes = Adc->getOperand(0).getImm();
// A release, not an allocation written as a wrapping add.
return Bytes > 0 && Bytes < 0x8000;
}
// Walk back from End over the stack releases emitStackRelease and the
// call cleanup produce: PLY / PLX (Y / X not kept), TSC/CLC/ADC/TCS,
// and that bracketed by TAY/TYA, TAX/TXA or STA $E0 / LDA $E0, costed
// by W65816Helpers::cycleByteCost as emitStackRelease's choices are.
ReleaseRun scanRunBack(MachineBasicBlock &MBB, MachineBasicBlock::iterator End,
const KeepSet &Keep) {
ReleaseRun Run;
Run.Begin = End;
while (Run.Begin != MBB.begin()) {
MachineBasicBlock::iterator It = std::prev(Run.Begin);
unsigned Opc = It->getOpcode();
if ((Opc == W65816::PLY && !Keep.Y) || (Opc == W65816::PLX && !Keep.X)) {
Run.Begin = It;
Run.Bytes += 2;
Run.Cost += W65816Helpers::cycleByteCost(5, 1);
continue;
}
MachineBasicBlock::iterator Begin;
int64_t Bytes;
if (matchAdcCore(MBB, It, Begin, Bytes)) {
Run.Begin = Begin;
Run.Bytes += Bytes;
Run.Cost += W65816Helpers::cycleByteCost(9, 6);
continue;
}
// The bracket's two instructions: 4 cycles and 2 bytes for the
// transfers, 8 and 4 for the direct-page store and load.
unsigned Open;
unsigned Cycles;
unsigned Size;
if (Opc == W65816::TYA) {
Open = W65816::TAY;
Cycles = 9 + 4;
Size = 6 + 2;
} else if (Opc == W65816::TXA) {
Open = W65816::TAX;
Cycles = 9 + 4;
Size = 6 + 2;
} else if (isImm(*It, W65816::LDA_DP, 0xE0)) {
Open = W65816::STA_DP;
Cycles = 9 + 8;
Size = 6 + 4;
} else {
break;
}
if (It == MBB.begin() || !matchAdcCore(MBB, std::prev(It), Begin, Bytes) ||
Begin == MBB.begin())
break;
MachineBasicBlock::iterator O = std::prev(Begin);
if (O->getOpcode() != Open ||
(Open == W65816::STA_DP && !isImm(*O, W65816::STA_DP, 0xE0)))
break;
Run.Begin = O;
Run.Bytes += Bytes;
Run.Cost += W65816Helpers::cycleByteCost(Cycles, Size);
}
return Run;
}
// Whether M is 16-bit at the start of a run: the nearest SEP / REP above
// it in the block touching M is not a SEP (blocks start in 16-bit M).
bool sixteenBitMAt(MachineBasicBlock &MBB, MachineBasicBlock::iterator It) {
while (It != MBB.begin()) {
--It;
unsigned Opc = It->getOpcode();
if ((Opc == W65816::SEP || Opc == W65816::REP) &&
It->getOperand(0).isImm() && (It->getOperand(0).getImm() & 0x20))
return Opc == W65816::REP;
}
return true;
}
// Cost of reaching the next block: the unconditional branch's, or none
// for a fall-through.
unsigned branchCost(const MachineInstr *Br) {
if (!Br)
return 0;
switch (Br->getOpcode()) {
case W65816::BRL:
return W65816Helpers::cycleByteCost(4, 3);
case W65816::JMP_Abs:
return W65816Helpers::cycleByteCost(3, 3);
default:
return W65816Helpers::cycleByteCost(3, 2);
}
}
// The block's lone successor and the unconditional branch (or nullptr
// for a fall-through) that reaches it, when that is all its terminators
// do. Null when the block returns, branches conditionally or has other
// successors.
MachineBasicBlock *loneSuccessor(MachineBasicBlock &MBB, MachineInstr *&Br) {
Br = nullptr;
if (MBB.succ_size() != 1)
return nullptr;
MachineBasicBlock *S = *MBB.succ_begin();
MachineBasicBlock::iterator T = MBB.getFirstTerminator();
if (T == MBB.end())
return MBB.isLayoutSuccessor(S) ? S : nullptr;
if (std::next(T) != MBB.end())
return nullptr;
unsigned Opc = T->getOpcode();
if (Opc != W65816::BRA && Opc != W65816::BRL && Opc != W65816::JMP_Abs)
return nullptr;
if (!T->getOperand(0).isMBB() || T->getOperand(0).getMBB() != S)
return nullptr;
Br = &*T;
return S;
}
KeepSet keepFor(const MachineInstr &Ret) {
const TargetRegisterInfo *TRI =
Ret.getMF()->getSubtarget().getRegisterInfo();
KeepSet Keep;
Keep.A = Ret.readsRegister(W65816::A, TRI);
Keep.X = Ret.readsRegister(W65816::X, TRI);
Keep.Y = Ret.readsRegister(W65816::Y, TRI);
return Keep;
}
MachineInstr *returnOf(MachineBasicBlock &MBB) {
MachineBasicBlock::iterator T = MBB.getFirstTerminator();
if (T == MBB.end() || std::next(T) != MBB.end() ||
T->getOpcode() != W65816::RTL)
return nullptr;
return &*T;
}
} // namespace
bool W65816ReturnPopMerge::runOnMachineFunction(MachineFunction &MF) {
if (getenv("W65816_DISABLE_RETURN_POP_MERGE"))
return false;
const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
bool Changed = false;
SmallVector<MachineBasicBlock *, 8> Orphans;
for (MachineBasicBlock &MBB : MF) {
// The release-only chain from MBB to a return, if any.
MachineInstr *Ret = returnOf(MBB);
SmallVector<MachineBasicBlock *, 4> Chain;
SmallVector<MachineInstr *, 4> Branches;
if (!Ret) {
MachineBasicBlock *B = &MBB;
while (!Ret && Chain.size() < 4) {
MachineInstr *Br;
MachineBasicBlock *S = loneSuccessor(*B, Br);
if (!S || S == &MBB || S->isEHPad() || S->hasAddressTaken())
break;
Chain.push_back(S);
Branches.push_back(Br);
Ret = returnOf(*S);
B = S;
}
if (!Ret)
continue;
}
KeepSet Keep = keepFor(*Ret);
int64_t Bytes = 0;
unsigned OldCost = 0;
bool Pure = true;
for (unsigned I = 0; I < Chain.size(); ++I) {
MachineBasicBlock *S = Chain[I];
ReleaseRun Run = scanRunBack(*S, S->getFirstTerminator(), Keep);
if (Run.Begin != S->begin()) {
Pure = false;
break;
}
Bytes += Run.Bytes;
OldCost += Run.Cost + branchCost(Branches[I]);
}
if (!Pure)
continue;
MachineBasicBlock::iterator End = MBB.getFirstTerminator();
ReleaseRun Own = scanRunBack(MBB, End, Keep);
if (!sixteenBitMAt(MBB, Own.Begin))
continue;
Bytes += Own.Bytes;
OldCost += Own.Cost;
// A block of its own pays one byte for its copy of the return.
unsigned NewCost = W65816Helpers::stackReleaseCost(Bytes, Keep.A, Keep.X,
Keep.Y) +
(Chain.empty() ? 0 : 1);
if (NewCost >= OldCost)
continue;
// MBB's tail from the run on goes (its branch, or its own return),
// replaced by the one release and a copy of the return.
DebugLoc DL = Ret->getDebugLoc();
MachineInstr *NewRet = MF.CloneMachineInstr(Ret);
MBB.erase(Own.Begin, MBB.end());
W65816Helpers::emitStackRelease(MBB, MBB.end(), DL, TII, Bytes, Keep.A,
Keep.X, Keep.Y);
MBB.insert(MBB.end(), NewRet);
if (!Chain.empty()) {
MBB.removeSuccessor(Chain.front());
Orphans.push_back(Chain.front());
}
Changed = true;
}
// Delete the chain blocks nothing reaches any more.
SmallPtrSet<MachineBasicBlock *, 8> Erased;
while (!Orphans.empty()) {
MachineBasicBlock *B = Orphans.pop_back_val();
if (Erased.count(B) || !B->pred_empty() || B == &MF.front() ||
B->hasAddressTaken() || B->isEHPad())
continue;
Erased.insert(B);
SmallVector<MachineBasicBlock *, 2> Succs(B->successors());
for (MachineBasicBlock *S : Succs) {
B->removeSuccessor(S);
Orphans.push_back(S);
}
B->eraseFromParent();
}
return Changed;
}
FunctionPass *llvm::createW65816ReturnPopMerge() {
return new W65816ReturnPopMerge();
}

274
src/llvm/lib/Target/W65816/W65816SinkLoopInc.cpp Normal file
View file

@ -0,0 +1,274 @@
//===-- W65816SinkLoopInc.cpp - Sink loop-carried increments -------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// Pre-RA, SSA: move a loop-carried increment below the last use of the
// value it increments, within its block.
//
// `while (p != end) s += *p++;` reaches register allocation as
//
// %p = PHI %p0, %bb.pre, %next, %bb.loop
// %next = ADCi16imm %p, 1 ; low half (the scheduler puts
// %nh = ADCEi16imm %ph, 0 ; high half the step first)
// ... load through %p:%ph ...
// ... compare %next:%nh with end ...
//
// %p is still live when %next is defined, so the coalescer cannot give the
// two one home: PHI elimination's back-edge copy (`%p = COPY %next`, four
// instructions for a far pointer) stays in every trip, and the step cannot
// become an in-place `inc dp ; bne ; inc dp+2` (W65816I32IncFold). Moved
// below the last use of %p, the step starts where %p ends.
//
// Conditions: the step is ADCi16imm / SBCi16imm by a constant, or INA /
// DEA, whose input is a PHI that takes the step's result back from this
// block or one it runs straight into; an ADCE / SBCE right behind it (the high half, reading the carry)
// moves with it and must have the same shape. Nothing between the step
// and its new place may read the step's result. The step lands after the
// old value's last use -- counting a far-pointer access that goes through
// a stack temporary or $E0 / $E2 staged from it -- and after any physical
// A / X / Y live range open there, and only where the flags are dead (the
// steps set them).
//
// Also an i16 counter's `i++` whose old value is read after it (an index,
// a call argument) stops overlapping its successor: Lua's luaV_execute
// allocates with greedy again (it needed the basic-allocator fallback).
//
//===----------------------------------------------------------------------===//
#include "W65816.h"
#include "W65816InstrInfo.h"
#include "W65816Subtarget.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/Support/Debug.h"
using namespace llvm;
#define DEBUG_TYPE "w65816-sink-loop-inc"
namespace {
class W65816SinkLoopInc : public MachineFunctionPass {
public:
static char ID;
W65816SinkLoopInc() : MachineFunctionPass(ID) {}
StringRef getPassName() const override {
return "W65816 sink loop-carried increments";
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.setPreservesCFG();
MachineFunctionPass::getAnalysisUsage(AU);
}
bool runOnMachineFunction(MachineFunction &MF) override;
};
} // namespace
char W65816SinkLoopInc::ID = 0;
INITIALIZE_PASS(W65816SinkLoopInc, DEBUG_TYPE,
"W65816 sink loop-carried increments", false, false)
FunctionPass *llvm::createW65816SinkLoopInc() {
return new W65816SinkLoopInc();
}
// The step `Def = OP Cur, imm` (OP one of Opcs): Cur, or an invalid
// register when MI is not one or Cur is not a PHI taking Def back from MI's
// block.
static Register stepInput(const MachineInstr &MI,
std::initializer_list<unsigned> Opcs,
const MachineRegisterInfo &MRI) {
bool IncDec = MI.getOpcode() == W65816::INA_PSEUDO ||
MI.getOpcode() == W65816::DEA_PSEUDO;
if (!llvm::is_contained(Opcs, MI.getOpcode()) || MI.getNumOperands() < 2 ||
!MI.getOperand(0).isReg() || !MI.getOperand(1).isReg() ||
(!IncDec && (MI.getNumOperands() < 3 || !MI.getOperand(2).isImm())))
return Register();
Register Def = MI.getOperand(0).getReg();
Register Cur = MI.getOperand(1).getReg();
if (!Def.isVirtual() || !Cur.isVirtual())
return Register();
const MachineInstr *Phi = MRI.getVRegDef(Cur);
if (!Phi || !Phi->isPHI())
return Register();
// The back edge may leave from a block MI's block runs straight into:
// a latch whose test was split in two (`while (*p && state != k)`,
// CoreMark's core_state_transition) ends MI's block on the first test
// and the loop on the second.
auto reachedFromMI = [&](const MachineBasicBlock *B) {
for (unsigned Hops = 0; B && Hops < 3; ++Hops) {
if (B == MI.getParent())
return true;
B = B->pred_size() == 1 ? *B->pred_begin() : nullptr;
}
return false;
};
for (unsigned I = 1; I + 1 < Phi->getNumOperands(); I += 2)
if (Phi->getOperand(I).getReg() == Def &&
reachedFromMI(Phi->getOperand(I + 1).getMBB()))
return Cur;
return Register();
}
static bool readsP(const MachineInstr &MI) {
return MI.readsRegister(W65816::P, /*TRI=*/nullptr);
}
static bool definesP(const MachineInstr &MI) {
return MI.definesRegister(W65816::P, /*TRI=*/nullptr);
}
bool W65816SinkLoopInc::runOnMachineFunction(MachineFunction &MF) {
if (skipFunction(MF.getFunction()))
return false;
MachineRegisterInfo &MRI = MF.getRegInfo();
if (!MRI.isSSA())
return false;
bool Changed = false;
for (MachineBasicBlock &MBB : MF) {
SmallVector<MachineInstr *, 4> Steps;
for (MachineInstr &MI : MBB)
if (stepInput(MI,
{W65816::ADCi16imm, W65816::SBCi16imm, W65816::INA_PSEUDO,
W65816::DEA_PSEUDO},
MRI))
Steps.push_back(&MI);
for (MachineInstr *Step : Steps) {
// The group: the step, and the high half reading its carry.
SmallVector<MachineInstr *, 2> Group{Step};
SmallVector<Register, 2> Olds{Step->getOperand(1).getReg()};
SmallVector<Register, 2> News{Step->getOperand(0).getReg()};
auto Next = std::next(Step->getIterator());
while (Next != MBB.end() && Next->isDebugInstr())
++Next;
if (Next != MBB.end() && readsP(*Next)) {
Register Hi =
stepInput(*Next, {W65816::ADCEi16imm, W65816::SBCEi16imm}, MRI);
if (!Hi || !Next->registerDefIsDead(W65816::P, nullptr))
continue;
Group.push_back(&*Next);
Olds.push_back(Hi);
News.push_back(Next->getOperand(0).getReg());
} else if (!Step->registerDefIsDead(W65816::P, nullptr)) {
// Someone might read the step's own flags.
bool Read = false;
for (auto It = std::next(Step->getIterator()); It != MBB.end(); ++It) {
if (readsP(*It)) {
Read = true;
break;
}
if (definesP(*It))
break;
}
if (Read)
continue;
}
// The last use of an old value, before any use of a new one. A far
// pointer access stores the pointer's halves to stack temporaries
// (STAfi), stages them into $E0 / $E2 from there (`$a = LDAfi ;
// STA_DP $E0`) and goes through [$E0],Y, so a read of a slot an old
// value was stored to, and an access through a direct-page word it
// was staged to, are uses of it too.
auto uses = [&](const MachineInstr &MI, ArrayRef<Register> Regs) {
for (Register R : Regs)
if (MI.readsVirtualRegister(R))
return true;
return false;
};
const TargetInstrInfo *TII = MF.getSubtarget().getInstrInfo();
SmallVector<int, 4> OldSlots;
SmallVector<int64_t, 4> OldDP;
bool AFromOld = false;
auto readsOldSlot = [&](const MachineInstr &MI) {
for (const MachineOperand &MO : MI.operands())
if (MO.isFI() && llvm::is_contained(OldSlots, MO.getIndex()))
return true;
return false;
};
auto throughOldDP = [&](const MachineInstr &MI) {
return TII->getName(MI.getOpcode()).contains("_DPInd") &&
MI.getOperand(0).isImm() &&
llvm::is_contained(OldDP, MI.getOperand(0).getImm());
};
MachineInstr *LastOld = nullptr;
for (auto It = std::next(Group.back()->getIterator());
It != MBB.end() && !It->isTerminator(); ++It) {
if (uses(*It, News))
break;
if (It->getOpcode() == W65816::STAfi && It->getOperand(1).isFI() &&
uses(*It, Olds)) {
OldSlots.push_back(It->getOperand(1).getIndex());
LastOld = &*It;
continue;
}
if (It->getOpcode() == W65816::STA_DP && AFromOld &&
It->getOperand(0).isImm()) {
OldDP.push_back(It->getOperand(0).getImm());
LastOld = &*It;
continue;
}
bool Old = uses(*It, Olds) || readsOldSlot(*It) || throughOldDP(*It);
if (It->definesRegister(W65816::A, nullptr))
AFromOld = Old && It->getOpcode() == W65816::LDAfi;
if (Old)
LastOld = &*It;
}
if (!LastOld)
continue;
// Land where A, X and Y are not live as physical registers: the
// staging moves the pointer through A (`$a = LDAfi ; STA_DP $E0,
// implicit $a`), and a step needing A in between cannot be
// allocated. Advance past each such physical live range, never
// past a use of a new value or into the terminators.
const TargetRegisterInfo *TRI = MF.getSubtarget().getRegisterInfo();
auto physReader = [&](MachineBasicBlock::iterator At) -> MachineInstr * {
for (MCRegister R : {W65816::A, W65816::X, W65816::Y})
for (auto It = At; It != MBB.end(); ++It) {
if (It->readsRegister(R, TRI))
return &*It;
if (It->definesRegister(R, TRI))
break;
}
return nullptr;
};
auto IP = std::next(LastOld->getIterator());
bool Blocked = false;
while (MachineInstr *Reader = physReader(IP)) {
for (auto It = IP; !Blocked && It != std::next(Reader->getIterator());
++It)
Blocked = It->isTerminator() || uses(*It, News);
if (Blocked)
break;
IP = std::next(Reader->getIterator());
}
if (Blocked)
continue;
// The flags must be dead where the group lands (INA / DEA model no
// P def but set N / Z all the same). Flags that may flow out of the
// block count as live.
bool PLive = true;
for (auto It = IP; It != MBB.end(); ++It) {
if (readsP(*It))
break;
if (definesP(*It)) {
PLive = false;
break;
}
}
if (PLive)
continue;
for (MachineInstr *MI : Group)
MBB.splice(IP, &MBB, MI->getIterator());
for (Register R : Olds)
MRI.clearKillFlags(R);
LLVM_DEBUG(dbgs() << "sank step to after " << *LastOld);
Changed = true;
}
}
return Changed;
}

112
src/llvm/lib/Target/W65816/W65816SpillToX.cpp
View file

@ -40,6 +40,7 @@
#include "W65816.h"
#include "W65816InstrInfo.h"
#include "W65816Subtarget.h"
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallSet.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineFunction.h"
@ -47,6 +48,7 @@
#include "llvm/CodeGen/MachineInstr.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/CodeGen/TargetRegisterInfo.h"
#include "llvm/Support/CommandLine.h"
using namespace llvm;
@ -57,7 +59,8 @@ namespace {
class W65816SpillToX : public MachineFunctionPass {
public:
static char ID;
W65816SpillToX() : MachineFunctionPass(ID) {}
explicit W65816SpillToX(bool SharedSlots = false)
: MachineFunctionPass(ID), SharedSlots(SharedSlots) {}
StringRef getPassName() const override {
return "W65816 spill-to-X peephole";
}
@ -68,17 +71,31 @@ public:
}
bool runOnMachineFunction(MachineFunction &MF) override;
private:
// Bridge spill slots that other values share too (see the cross-block
// use check). Only the run after W65816StackSlotCleanup does: that pass
// deletes many of the same spill/reload pairs outright, slot and all,
// which beats a bridge, and a bridge made first blocks it (JoeyLib's
// jlpGenericTileCopyMasked kept 36 frame slots instead of 26).
bool SharedSlots;
};
} // namespace
// Run as the post-cleanup invocation (for tests driving the pass alone).
static cl::opt<bool> SpillToXSharedSlots(
"w65816-spill-to-x-shared-slots", cl::Hidden, cl::init(false),
cl::desc("W65816 spill-to-X: also bridge spill slots shared by "
"stack-slot coloring"));
char W65816SpillToX::ID = 0;
INITIALIZE_PASS(W65816SpillToX, DEBUG_TYPE, "W65816 spill-to-X peephole",
false, false)
FunctionPass *llvm::createW65816SpillToX() {
return new W65816SpillToX();
FunctionPass *llvm::createW65816SpillToX(bool SharedSlots) {
return new W65816SpillToX(SharedSlots);
}
// Classifies how an MI interacts with X.
@ -157,6 +174,43 @@ static bool referencesSlot(const MachineInstr &MI, int FI) {
return false;
}
// True when the value spill slot FI holds just after From may still be
// read: some path from there reaches a reference to FI before a full
// overwrite (`STAfi $a, FI, 0`, a 16-bit store over the whole 2-byte
// slot). Every other reference counts as a read.
static bool slotLiveAfter(MachineInstr &From, int FI) {
// -1 for a read, 1 for a full overwrite, 0 for neither.
auto effect = [FI](const MachineInstr &MI) {
if (!referencesSlot(MI, FI))
return 0;
return matchSTAfi(MI) == FI ? 1 : -1;
};
MachineBasicBlock *MBB = From.getParent();
for (auto I = std::next(From.getIterator()); I != MBB->end(); ++I) {
if (int E = effect(*I))
return E < 0;
}
SmallVector<MachineBasicBlock *, 8> Work(MBB->succ_begin(),
MBB->succ_end());
SmallPtrSet<MachineBasicBlock *, 16> Seen;
while (!Work.empty()) {
MachineBasicBlock *B = Work.pop_back_val();
if (!Seen.insert(B).second)
continue;
int E = 0;
for (MachineInstr &MI : *B) {
E = effect(MI);
if (E)
break;
}
if (E < 0)
return true;
if (E == 0)
Work.append(B->succ_begin(), B->succ_end());
}
return false;
}
bool W65816SpillToX::runOnMachineFunction(MachineFunction &MF) {
const W65816Subtarget &STI = MF.getSubtarget<W65816Subtarget>();
const W65816InstrInfo *TII = STI.getInstrInfo();
@ -197,6 +251,10 @@ bool W65816SpillToX::runOnMachineFunction(MachineFunction &MF) {
// (which would invalidate the saved value).
if (touchesX(MI2, TRI)) { xTouched = true; break; }
if (referencesSlot(MI2, FI)) break;
// Unlike the STA, the TAX sets N/Z. A PHP in the window (the
// flag-save wraps exist by the second run) would save the TAX's
// flags instead of the test it brackets.
if (MI2.getOpcode() == W65816::PHP) break;
gapEmpty = false;
}
@ -212,13 +270,21 @@ bool W65816SpillToX::runOnMachineFunction(MachineFunction &MF) {
// a libcall later in the block expects $x intact. Scan from just
// past LDA to end-of-block; if any instr uses X, bail.
bool xUsedAfter = false;
bool xRedefAfter = false;
for (auto Scan = std::next(MachineBasicBlock::iterator(LdaMI));
Scan != MBB.end(); ++Scan) {
const MachineInstr &MI3 = *Scan;
if (MI3.isDebugInstr()) continue;
XEffect eff = xEffect(MI3, TRI);
if (eff & XReads) { xUsedAfter = true; break; }
if (eff & XDefs) break; // X redefined; no longer live
if (eff & XDefs) { xRedefAfter = true; break; }
}
// X still live at the end of the block: the TAX would clobber a
// value a successor reads.
if (!xUsedAfter && !xRedefAfter) {
for (MachineBasicBlock *Succ : MBB.successors()) {
if (Succ->isLiveIn(W65816::X)) { xUsedAfter = true; break; }
}
}
// Also bail if X is live-in to MBB and nothing has defined X
// between MBB start and STA — the live-in value is needed past
@ -235,29 +301,41 @@ bool W65816SpillToX::runOnMachineFunction(MachineFunction &MF) {
}
if (xUsedAfter) continue;
// Cross-block use check: if the slot is referenced anywhere
// OUTSIDE the [STA, LDA] window (including other blocks), the
// STA we'd erase is feeding those other reads — eliding it
// would silently corrupt them. Caught by sumTable() returning
// a stale phi value because the loop's STA-to-merge-slot was
// eliminated; the merge block's LDA then read the bb.0-init 0
// instead of the loop's accumulated sum.
// Cross-block use check: the STA we'd erase must feed no read but
// this LDA, or eliding it would silently corrupt the others.
// Caught by sumTable() returning a stale phi value because the
// loop's STA-to-merge-slot was eliminated; the merge block's LDA
// then read the bb.0-init 0 instead of the loop's accumulated sum.
//
// Strictly: no reference to the slot outside the window. In the
// shared-slot run, a spill slot may also have references in OTHER
// blocks when it is dead after the LDA: stack-slot coloring gives
// one slot to values with disjoint lives (two loops' counters, say),
// and those references never see this STA's value. A reference in
// the same block still bails: there the slot takes part in a pattern
// the later passes rewrite in place once StackRelToImg has moved it
// to a DP word -- an `i32 += 1` (I32IncFold's INC_HI_IF_CARRY), a
// pointer bump, a 32-bit shift's `lsr dp ; ror dp` -- and a bridge
// breaks it (popcount went from 1159 to 2285 cycles a call). Other
// slots never qualify: one whose address is taken can be read
// through a pointer, which no operand scan sees, and a setjmp's
// second return is no CFG edge.
bool Shared = (SharedSlots || SpillToXSharedSlots) &&
MF.getFrameInfo().isSpillSlotObjectIndex(FI) &&
!MF.exposesReturnsTwice();
bool externalUse = false;
for (auto &OtherMBB : MF) {
for (auto &OtherMI : OtherMBB) {
if (&OtherMI == StaMI || &OtherMI == LdaMI) continue;
// Walk inside-window range and skip those refs.
if (&OtherMBB == &MBB) {
// We already verified the gap doesn't reference FI; only
// STA/LDA themselves are allowed users in this block.
}
if (referencesSlot(OtherMI, FI)) {
if (referencesSlot(OtherMI, FI) && (!Shared || &OtherMBB == &MBB)) {
externalUse = true;
break;
}
}
if (externalUse) break;
}
if (!externalUse && Shared)
externalUse = slotLiveAfter(*LdaMI, FI);
if (externalUse) continue;
// Replace STAfi with TAX, LDAfi with TXA.

131
src/llvm/lib/Target/W65816/W65816SplitLoopExits.cpp Normal file
View file

@ -0,0 +1,131 @@
//===-- W65816SplitLoopExits.cpp - Split critical loop-exit edges --------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===---------------------------------------------------------------------===//
//
// IR-level, the last pass before ISel: give each critical edge out of a
// loop into a block with PHIs a block of its own.
//
// PHI elimination puts a PHI's copy at the end of the predecessor the
// value comes from. When that predecessor is a loop block that also
// branches back into the loop, the copy runs on every trip, not just the
// one that leaves: a search loop's `return p` copies p (four instructions
// for a far pointer) and its `return NULL` a zero into the return value's
// home each time round (CoreMark's core_list_find spent a quarter of its
// trip on them). LLVM splits such edges in PHI elimination itself, but
// only through MachineBasicBlock::SplitCriticalEdge, which needs
// analyzeBranch to understand the predecessor's conditional branch, and
// W65816InstrInfo keeps conditional branches opaque on purpose (see
// analyzeBranch). Split here instead: the copy moves into the new block,
// which runs once per exit and ends in a branch to the old target.
// CodeGenPrepare, which would merge the new block back, has already run.
//
//===---------------------------------------------------------------------===//
#include "W65816.h"
#include "llvm/Analysis/CFG.h"
#include "llvm/Analysis/LoopInfo.h"
#include "llvm/IR/Dominators.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/Instructions.h"
#include "llvm/InitializePasses.h"
#include "llvm/Pass.h"
#include "llvm/Transforms/Utils/BasicBlockUtils.h"
using namespace llvm;
#define DEBUG_TYPE "w65816-split-loop-exits"
namespace {
class W65816SplitLoopExits : public FunctionPass {
public:
static char ID;
W65816SplitLoopExits() : FunctionPass(ID) {}
StringRef getPassName() const override {
return "W65816 split critical loop-exit edges";
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<LoopInfoWrapperPass>();
AU.addRequired<DominatorTreeWrapperPass>();
}
bool runOnFunction(Function &F) override;
};
} // namespace
char W65816SplitLoopExits::ID = 0;
INITIALIZE_PASS_BEGIN(W65816SplitLoopExits, DEBUG_TYPE,
"W65816 split critical loop-exit edges", false, false)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_END(W65816SplitLoopExits, DEBUG_TYPE,
"W65816 split critical loop-exit edges", false, false)
// Whether V is what a header PHI of L takes over a back edge: the loop
// keeps it in that PHI's register already, so a copy of it on the way out
// coalesces into that register wherever PHI elimination puts it, and the
// split only moves the allocator's choices around (sumOfSquares' running
// total gained a store a trip from it).
static bool isCarriedValue(Value *V, Loop *L) {
for (PHINode &P : L->getHeader()->phis())
for (unsigned K = 0, E = P.getNumIncomingValues(); K != E; ++K)
if (P.getIncomingValue(K) == V && L->contains(P.getIncomingBlock(K)))
return true;
return false;
}
bool W65816SplitLoopExits::runOnFunction(Function &F) {
if (skipFunction(F) || getenv("W65816_DISABLE_SPLIT_LOOP_EXITS"))
return false;
LoopInfo &LI = getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
DominatorTree &DT = getAnalysis<DominatorTreeWrapperPass>().getDomTree();
// Collect first: splitting adds blocks and edges.
SmallVector<std::pair<Instruction *, unsigned>, 8> Edges;
for (BasicBlock &BB : F) {
Loop *L = LI.getLoopFor(&BB);
Instruction *TI = BB.getTerminator();
if (!L || !TI || TI->getNumSuccessors() < 2 ||
!(isa<CondBrInst>(TI) || isa<SwitchInst>(TI)))
continue;
for (unsigned I = 0, E = TI->getNumSuccessors(); I != E; ++I) {
BasicBlock *Succ = TI->getSuccessor(I);
if (L->contains(Succ) || !isa<PHINode>(Succ->begin()) ||
Succ->getSinglePredecessor() || Succ->isEHPad())
continue;
// Worth it only for a copy that would really run each trip: of a
// constant, or of a value the loop does not carry.
bool Worth = false;
for (PHINode &P : Succ->phis())
if (!isCarriedValue(P.getIncomingValueForBlock(&BB), L))
Worth = true;
if (Worth)
Edges.push_back({TI, I});
}
}
bool Changed = false;
CriticalEdgeSplittingOptions Opts(&DT, &LI);
Opts.setMergeIdenticalEdges();
for (auto [TI, I] : Edges)
if (isCriticalEdge(TI, I) && SplitCriticalEdge(TI, I, Opts))
Changed = true;
return Changed;
}
FunctionPass *llvm::createW65816SplitLoopExits() {
return new W65816SplitLoopExits();
}

614
src/llvm/lib/Target/W65816/W65816StackRelToImg.cpp
View file

@ -48,11 +48,13 @@
#include "W65816Subtarget.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/PostOrderIterator.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineFunction.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/CodeGen/MachineBlockFrequencyInfo.h"
#include "llvm/CodeGen/MachineLoopInfo.h"
#include "llvm/InitializePasses.h"
#include "llvm/IR/DebugInfoMetadata.h"
@ -79,6 +81,7 @@ public:
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<MachineLoopInfoWrapperPass>();
AU.addRequired<MachineBlockFrequencyInfoWrapperPass>();
AU.setPreservesCFG();
MachineFunctionPass::getAnalysisUsage(AU);
}
@ -94,6 +97,7 @@ char W65816StackRelToImg::ID = 0;
INITIALIZE_PASS_BEGIN(W65816StackRelToImg, DEBUG_TYPE,
"W65816 stack-rel to IMG", false, false)
INITIALIZE_PASS_DEPENDENCY(MachineLoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(MachineBlockFrequencyInfoWrapperPass)
INITIALIZE_PASS_END(W65816StackRelToImg, DEBUG_TYPE,
"W65816 stack-rel to IMG", false, false)
@ -135,7 +139,8 @@ static bool isImgSafeCall(const MachineInstr &MI) {
if (MO.isGlobal()) Name = MO.getGlobal()->getName();
else if (MO.isSymbol()) Name = MO.getSymbolName();
else continue;
if (Name == "__umulhisi3" || Name == "__mulhi3" ||
if (Name == "__umulhisi3" || Name == "__mulhisi3" ||
Name == "__mulhi3" ||
Name == "__mulsi3" || Name == "__udivhi3" ||
Name == "__umodhi3" || Name == "__divhi3" ||
Name == "__modhi3" || Name == "__udivsi3" ||
@ -569,15 +574,22 @@ static bool elidePlaPhaPair(MachineFunction &MF) {
// Always-on: drop a redundant LDA when the prior LDA loaded the same
// source and the only intervening instruction was PHA (which reads A
// but doesn't modify it). Emerges from i64 arg-push sequences:
// word and the only intervening instruction was PHA (which reads A but
// doesn't modify it). A direct-page word keeps its address; a stack word
// is 2 bytes further from SP once the PHA has pushed:
//
// LDA off, s
// PHA
// LDA off, s ; A still has this value — redundant
// PHA
// LDA off+2, s ; the same word: A still has this value -- redundant
//
// Saves 1 instruction (3 bytes / 4 cyc) per match.
// `LDA off, s ; PHA ; LDA off, s` reads two different words. Matching
// equal offsets there deleted the load of ImgCalleeSave's restore, which
// reads its save slot inside a PHA bracket right after the return value
// was loaded from the word above it: libc's wmemcpy / wmemmove / rename,
// fminf and eleven Lua functions handed the caller back its callee-saved $C0
// holding their own return value.
//
// Saves 1 instruction (2 bytes / 5 cyc) per match.
static bool elideRedundantLdaAfterPha(MachineFunction &MF) {
bool Changed = false;
for (MachineBasicBlock &MBB : MF) {
@ -594,7 +606,7 @@ static bool elideRedundantLdaAfterPha(MachineFunction &MF) {
MachineOperand &S1 = It->getOperand(0);
MachineOperand &S2 = I2->getOperand(0);
if (!S1.isImm() || !S2.isImm()) continue;
if (S1.getImm() != S2.getImm()) continue;
if (S2.getImm() != S1.getImm() + (IsLdaSr ? 2 : 0)) continue;
ToErase.push_back(&*I2);
}
for (MachineInstr *MI : ToErase) {
@ -729,14 +741,20 @@ static bool elideDeadStaCarry(MachineFunction &MF) {
// the return to the canonical body SP.
static int spDelta(const MachineInstr &MI) {
switch (MI.getOpcode()) {
// PUSH16X is the PHX a call argument already in X is pushed with: left
// out, every slot access between it and the call was taken for the
// neighbouring slot (luaT_init's promoted pointer words read the
// arguments it was pushing).
case W65816::PHA: case W65816::PHX: case W65816::PHY:
case W65816::PUSH16: case W65816::PEA:
case W65816::PUSH16: case W65816::PUSH16X: case W65816::PEA:
case W65816::PHD:
return -2;
case W65816::PLA: case W65816::PLX: case W65816::PLY:
case W65816::PLD:
return 2;
case W65816::PHP:
case W65816::PHP: case W65816::PHB: case W65816::PHK:
return -1;
case W65816::PLP:
case W65816::PLP: case W65816::PLB:
return 1;
default:
return 0;
@ -786,6 +804,62 @@ static bool entrySpAdj(MachineFunction &MF,
}
// The logical offset (in entrySpAdj's terms) where the caller's stack
// arguments start: 4 bytes above the SP at function entry, past the 3-byte
// return address. Relative to the body SP that is FrameSize + 4, which is
// what entrySpAdj measures from when the prologue allocates with TCS (it
// resets the count). A small frame's prologue allocates with leading PHAs
// instead, and entrySpAdj counts those, so there logical offsets are
// relative to the entry SP and the arguments start at 4. Using
// FrameSize + 4 for both let a PHA-frame function promote its own incoming
// arguments to DP words nothing ever stored: libc's ungetc dereferenced a
// FILE pointer it never loaded. A VLA function's prologue saves SP before
// it allocates; take the lower bound there, which only promotes less.
// True when the function takes the address of a stack slot (TSX, or a
// TSC that is not a frame adjust TSC ; CLC|SEC ; ADC|SBC #n ; TCS): a
// pointer -- or a callee handed one -- may then read or write any slot.
// (ADDframe, the LEA, is already TSC ; CLC ; ADC here, after PEI.)
static bool takesStackAddr(const MachineFunction &MF) {
for (const MachineBasicBlock &MBB : MF) {
for (auto It = MBB.begin(), E = MBB.end(); It != E; ++It) {
if (It->getOpcode() == W65816::TSX)
return true;
if (It->getOpcode() != W65816::TSC)
continue;
auto N = std::next(It);
bool FrameAdjust =
N != E &&
(N->getOpcode() == W65816::CLC || N->getOpcode() == W65816::SEC) &&
++N != E &&
(N->getOpcode() == W65816::ADC_Imm16 ||
N->getOpcode() == W65816::SBC_Imm16) &&
++N != E && N->getOpcode() == W65816::TCS;
if (!FrameAdjust)
return true;
}
}
return false;
}
static int argSlotMinOff(MachineFunction &MF) {
const MachineFrameInfo &MFI = MF.getFrameInfo();
int FrameSize = static_cast<int>(MFI.getStackSize());
if (MFI.hasVarSizedObjects())
return 4;
int Pushed = 0;
for (MachineInstr &MI : MF.front()) {
unsigned Opc = MI.getOpcode();
if (Opc == W65816::REP || Opc == W65816::SEP)
continue;
if (Opc != W65816::PHA || Pushed >= FrameSize)
break;
Pushed += 2;
}
return FrameSize + 4 - Pushed;
}
// After promotion, look for `LDA_DP X ; INA ; STA_DP X`
// (or with DEA) patterns and fold to `INC_DP X` (DEC_DP X). These
// are very common in loop body: `i++` and `counter--` both produce
@ -840,8 +914,10 @@ static bool foldDpIncDec(MachineFunction &MF, const TargetInstrInfo *TII) {
}
}
if (Cur->isCall()) defsA = true; // caller-save
if (defsA) return true;
// A read comes first: a tied op (`$a = ANDi16imm $a, k`)
// both reads and writes A, and it reads the INA's result.
if (usesA) return false;
if (defsA) return true;
++Cur;
}
return false;
@ -1094,9 +1170,69 @@ static bool aDeadAtEntry(MachineBasicBlock &MBB) {
}
// True when MBB may end with the accumulator in 8-bit mode: its last
// SEP / REP of the M bit is a SEP.
static bool mayEndInM8(const MachineBasicBlock &MBB) {
for (auto It = MBB.rbegin(); It != MBB.rend(); ++It) {
unsigned Op = It->getOpcode();
if ((Op == W65816::SEP || Op == W65816::REP) &&
It->getOperand(0).isImm() && (It->getOperand(0).getImm() & 0x20))
return Op == W65816::SEP;
}
return false;
}
// True when logical slot S may be read once E is entered: some path from
// E's start reaches an access to S before a 16-bit store over all of it.
// The caller guarantees no access overlaps S from a neighbouring word and
// that no slot's address is taken; blocks start in 16-bit M, as the loop
// copies assume.
static bool slotReadFrom(MachineBasicBlock *E, int64_t S,
const DenseMap<MachineBasicBlock *, int> &EntrySp) {
SmallVector<MachineBasicBlock *, 8> Work{E};
SmallPtrSet<MachineBasicBlock *, 16> Seen;
while (!Work.empty()) {
MachineBasicBlock *B = Work.pop_back_val();
if (!Seen.insert(B).second)
continue;
auto It = EntrySp.find(B);
if (It == EntrySp.end())
return true;
int Sp = It->second;
bool M16 = true;
bool Killed = false;
for (MachineInstr &MI : *B) {
unsigned Op = MI.getOpcode();
if (Op == W65816::TCS) {
Sp = 0;
continue;
}
if ((Op == W65816::SEP || Op == W65816::REP) &&
MI.getOperand(0).isImm() && (MI.getOperand(0).getImm() & 0x20))
M16 = Op == W65816::REP;
if ((isStackRelOp(Op) || carryDpOpcode(Op) || isStackRelIndYOp(Op)) &&
MI.getNumOperands() >= 1 && MI.getOperand(0).isImm() &&
MI.getOperand(0).getImm() + Sp == S) {
if (Op != W65816::STA_StackRel || !M16)
return true;
Killed = true;
break;
}
Sp += spDelta(MI);
}
if (!Killed)
Work.append(B->succ_begin(), B->succ_end());
}
return false;
}
// Loop-local promotion, for a function the whole-function promotion
// below cannot touch because it makes calls: any call may clobber
// IMG0..7. An innermost loop that makes none can still keep its hottest
// IMG0..7. An innermost loop that makes none -- or only calls to the
// libgcc helpers isImgSafeCall lists, which never touch $D0..$DF (a
// multiply in CoreMark's matrix loops) -- can still keep its hottest
// slots -- far pointers first, into adjacent words -- in IMG words the
// function never uses: copied in at the end of the preheader, and the
// ones the loop writes copied back at the start of each exit. The copies
@ -1107,12 +1243,18 @@ static bool aDeadAtEntry(MachineBasicBlock &MBB) {
// access. (JoeyLib's sfx probe inlines its reference copy loop into
// main, which calls: the whole-function promotion never reached it.)
//
// Conservative throughout: a loop with a preheader and dedicated exits;
// nothing in it that calls, moves SP by more than a push, addresses the
// stack as a pointer, or runs in a frame reached through $F6; a slot no
// neighbour's access overlaps; the copy-in where A is dead and nothing
// later in the preheader writes the slot; exits whose A and flags are dead
// on entry. Anything else leaves the loop alone.
// Conservative throughout: a loop with a preheader; nothing in it that
// calls, moves SP by more than a push, addresses the stack as a pointer,
// or runs in a frame reached through $F6; a slot no neighbour's access
// overlaps; the copy-in where A is dead and nothing later in the preheader
// writes the slot. A written home is copied back at an exit only when the
// slot may be read after it; such a home needs a dedicated exit (no other
// way in), and an exit whose A or flags may be live on entry gets its
// copies inside PHP / PHA ... PLA / PLP. Loops failing those two exit
// tests used to be refused outright: a third of the corpus's in-loop far-
// pointer staging, JoeyLib's SCB upload in jlpPresent and every `while
// (n--)` copy (block placement merges the dedicated exit into the zero-
// trip guard's target) among them. Anything else leaves the loop alone.
static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
const TargetInstrInfo *TII, uint8_t ImgBase) {
// A plain slot pays its copies back at a cycle per access per trip; a
@ -1126,10 +1268,13 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
// Logical slots accessed anywhere (a candidate's neighbours must not
// be: a 16-bit access at S +- 1 overlaps S), and the frame-pointer test.
DenseSet<int64_t> AllSlots;
bool AddrTaken = false;
for (MachineBasicBlock &MBB : MF) {
int Sp = EntrySp.lookup(&MBB);
for (MachineInstr &MI : MBB) {
unsigned Op = MI.getOpcode();
if (Op == W65816::ADDframe)
AddrTaken = true;
if (Op == W65816::TCS) {
Sp = 0;
continue;
@ -1164,10 +1309,7 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
L->getUniqueExitBlocks(Exits);
bool Ok = !Exits.empty();
for (MachineBasicBlock *E : Exits) {
if (!EntrySp.count(E) || !aDeadAtEntry(*E))
Ok = false;
for (MachineBasicBlock *P : E->predecessors())
if (!L->contains(P))
if (!EntrySp.count(E))
Ok = false;
}
// The loop's accesses, by logical slot.
@ -1179,7 +1321,8 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
int Sp = EntrySp.lookup(B);
for (MachineInstr &MI : *B) {
unsigned Op = MI.getOpcode();
if (MI.isCall() || MI.isInlineAsm() || isStackRelIndYOp(Op) ||
if ((MI.isCall() && !isImgSafeCall(MI)) || MI.isInlineAsm() ||
isStackRelIndYOp(Op) ||
Op == W65816::TCS || Op == W65816::TSC || Op == W65816::TSX ||
Op == W65816::TXS || Op == W65816::TCD || Op == W65816::PLD ||
Op == W65816::ADDframe || Op == W65816::ALLOCAfi) {
@ -1285,6 +1428,28 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
Home[S] = static_cast<uint8_t>(ImgBase + 2 * Idx);
Taken.set(Idx);
}
// The homes each exit copies back: written in the loop, and maybe
// read after it.
auto copiedOut = [&](MachineBasicBlock *E, int64_t S) {
return Written.count(S) &&
(AddrTaken || slotReadFrom(E, S, EntrySp));
};
// A home an exit must copy back stays on the stack when other paths
// enter that exit too (the copy would store a stale DP word there).
// Block placement merges LoopSimplify's dedicated exits into the
// zero-trip guard's target, so this is the common `while (n--)` shape;
// its pointers and counter are dead after the loop and keep their homes.
for (MachineBasicBlock *E : Exits) {
if (llvm::all_of(E->predecessors(),
[&](MachineBasicBlock *P) { return L->contains(P); }))
continue;
SmallVector<int64_t, 8> Drop;
for (auto &KV : Home)
if (copiedOut(E, KV.first))
Drop.push_back(KV.first);
for (int64_t S : Drop)
Home.erase(S);
}
// Nothing after the copy-in point may write a home's slot.
int Sp = SpIns;
for (auto It = InsPt; It != Pre->end(); ++It) {
@ -1295,13 +1460,20 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
Sp = It->getOpcode() == W65816::TCS ? 0 : Sp + spDelta(*It);
}
// The copies address the slots from where they sit: the offsets must
// fit stack-relative addressing there.
// fit stack-relative addressing there (3 bytes further inside an
// exit's PHP / PHA bracket), and an exit's copies run in 16-bit M.
for (auto &KV : Home) {
int64_t Off = KV.first - SpIns;
bool Fits = Off >= 0 && Off <= 0xFF;
for (MachineBasicBlock *E : Exits) {
int64_t OffE = KV.first - EntrySp.lookup(E);
if (Written.count(KV.first) && (OffE < 0 || OffE > 0xFF))
if (!copiedOut(E, KV.first))
continue;
int64_t OffE = KV.first - EntrySp.lookup(E) +
(aDeadAtEntry(*E) ? 0 : 3);
if (OffE < 0 || OffE > 0xFF ||
llvm::any_of(E->predecessors(), [](MachineBasicBlock *P) {
return mayEndInM8(*P);
}))
Fits = false;
}
if (!Fits)
@ -1324,11 +1496,21 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
for (MachineInstr *MI : Sites[KV.first])
rewriteToDp(*MI, KV.second, TII);
for (MachineBasicBlock *E : Exits) {
int SpE = EntrySp.lookup(E);
auto At = E->begin();
for (auto &KV : Home) {
if (!Written.count(KV.first))
SmallVector<std::pair<int64_t, uint8_t>, 8> Out;
for (auto &KV : Home)
if (copiedOut(E, KV.first))
Out.push_back({KV.first, KV.second});
if (Out.empty())
continue;
bool Save = !aDeadAtEntry(*E);
int SpE = EntrySp.lookup(E) - (Save ? 3 : 0);
auto At = E->begin();
if (Save) {
BuildMI(*E, At, DL, TII->get(W65816::PHP));
BuildMI(*E, At, DL, TII->get(W65816::PHA))
.addReg(W65816::A, RegState::Implicit | RegState::Undef);
}
for (auto &KV : Out) {
BuildMI(*E, At, DL, TII->get(W65816::LDA_DP))
.addImm(KV.second)
.addReg(W65816::A, RegState::ImplicitDefine);
@ -1336,6 +1518,11 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
.addImm(KV.first - SpE)
.addReg(W65816::A, RegState::Implicit);
}
if (Save) {
BuildMI(*E, At, DL, TII->get(W65816::PLA))
.addReg(W65816::A, RegState::ImplicitDefine);
BuildMI(*E, At, DL, TII->get(W65816::PLP));
}
}
// Debug values in the loop that name a promoted slot would read the
// stale stack copy: drop their location, as the whole-function
@ -1364,6 +1551,314 @@ static bool promoteLoopLocal(MachineFunction &MF, MachineLoopInfo &MLI,
}
// The least a path through the function, entry to a return, saves: each
// block's own saving, loops taken once (back edges ignored).
static double minPathSaving(
MachineFunction &MF,
const std::function<double(const MachineBasicBlock &)> &Save) {
ReversePostOrderTraversal<MachineFunction *> RPOT(&MF);
DenseMap<const MachineBasicBlock *, unsigned> Order;
DenseMap<const MachineBasicBlock *, double> Dist;
unsigned N = 0;
for (MachineBasicBlock *MBB : RPOT)
Order[MBB] = N++;
double Best = -1;
Dist[&MF.front()] = 0;
for (MachineBasicBlock *MBB : RPOT) {
auto DIt = Dist.find(MBB);
if (DIt == Dist.end())
continue;
double D = DIt->second + Save(*MBB);
if (MBB->succ_empty() && (Best < 0 || D < Best))
Best = D;
for (MachineBasicBlock *S : MBB->successors()) {
if (Order.lookup(S) <= Order.lookup(MBB))
continue;
auto It = Dist.find(S);
if (It == Dist.end() || D < It->second)
Dist[S] = D;
}
}
return Best < 0 ? 0 : Best;
}
// Whole-function homes in IMG8..IMG15 for a function that makes calls. A
// far pointer the function keeps in two stack slots is staged into $E0 /
// $E2 before every access (18 cycles); in two adjacent direct-page words
// W65816FarPtrStageReuse addresses it where it lives. IMG0..7 do not
// survive a call, but IMG8..15 do -- every function that writes one gives
// the caller its value back (W65816ImgCalleeSave) -- so a word this
// function never touches can carry a pointer across its calls as long as
// the function keeps the caller's value too. It does so in the pointer's
// own slots: the entry SWAPS each slot with its word (the caller's value
// waits in the slot, nothing else reads it), and every return swaps the
// caller's value back. No frame space, no change to the layout PEI fixed.
//
// The swaps cost every call ~60 cycles a pair on the way in and ~25 on
// the way out, so a pair is taken only where its accesses pay for them on
// every path (the cheapest path through the function, loops once), or
// pay twice over by block frequency and a quarter on every path: a big
// function driving loops of calls through a struct pointer (Space Taxi's
// stRenderFrame, -8% of its cycles), not one that returns early from a
// cache hit (spriteMasks) or switches to one of nine inlined hooks
// (hookPerTickHigh), which both ran slower when the expected saving alone
// decided. Loops without calls take their own IMG0..7 homes first
// (promoteLoopLocal, paid only when the loop runs).
//
// Conservative: every exit an RTL, every access a stack-relative op with
// a direct-page form, no stack address taken, no frame reached through
// $F6, no setjmp, slots no neighbouring access overlaps, M=16 and SP at a
// known depth where the swaps go.
static bool promoteCalleeSavedPairs(MachineFunction &MF,
const MachineBlockFrequencyInfo &MBFI,
const TargetInstrInfo *TII) {
static const bool Disabled = getenv("W65816_DISABLE_CALLEE_SAVED_HOMES");
if (Disabled || MF.exposesReturnsTwice() || takesStackAddr(MF))
return false;
DenseMap<MachineBasicBlock *, int> EntrySp;
if (!entrySpAdj(MF, EntrySp))
return false;
DenseMap<int64_t, SmallVector<MachineInstr *, 8>> Sites;
DenseSet<int64_t> AllSlots, Blocked;
// Estimated cycles a home saves: a cycle per access, 9 for each load
// that stages the pointer into $E0 / $E2 (the stage goes). Expected
// per call (block frequency), and per block for the cheapest path.
DenseMap<int64_t, double> Saving;
DenseMap<std::pair<const MachineBasicBlock *, int64_t>, double> BlockSaving;
for (MachineBasicBlock &MBB : MF) {
auto EIt = EntrySp.find(&MBB);
if (EIt == EntrySp.end())
continue;
int Sp = EIt->second;
double Freq = MBFI.getBlockFreqRelativeToEntryBlock(&MBB);
for (auto It = MBB.begin(); It != MBB.end(); ++It) {
MachineInstr &MI = *It;
unsigned Op = MI.getOpcode();
if (Op == W65816::ALLOCAfi || MI.isInlineAsm())
return false;
bool HasImm = MI.getNumOperands() >= 1 && MI.getOperand(0).isImm();
if (HasImm && TII->getName(Op).contains("_DPInd") &&
MI.getOperand(0).getImm() >= 0xF6 &&
MI.getOperand(0).getImm() <= 0xF8)
return false;
if (Op == W65816::TCS) {
Sp = 0;
continue;
}
if (HasImm && (isStackRelOp(Op) || carryDpOpcode(Op))) {
int64_t Slot = MI.getOperand(0).getImm() + Sp;
Sites[Slot].push_back(&MI);
AllSlots.insert(Slot);
auto Next = std::next(It);
bool Stages = Op == W65816::LDA_StackRel && Next != MBB.end() &&
Next->getOpcode() == W65816::STA_DP &&
Next->getOperand(0).isImm() &&
(Next->getOperand(0).getImm() == 0xE0 ||
Next->getOperand(0).getImm() == 0xE2);
Saving[Slot] += Freq * (Stages ? 9.0 : 1.0);
BlockSaving[{&MBB, Slot}] += Stages ? 9.0 : 1.0;
if (!getDpOpcode(Op) && !carryDpOpcode(Op))
Blocked.insert(Slot);
} else if (HasImm && isStackRelIndYOp(Op)) {
AllSlots.insert(MI.getOperand(0).getImm() + Sp);
Blocked.insert(MI.getOperand(0).getImm() + Sp);
}
Sp += spDelta(MI);
}
}
if (Sites.empty())
return false;
// M=16 at an insertion point: the block's last M switch before it is a
// REP (blocks start in M=16).
auto m16At = [](MachineBasicBlock &MBB, MachineBasicBlock::iterator Pt) {
bool M16 = true;
for (auto It = MBB.begin(); It != Pt; ++It)
if ((It->getOpcode() == W65816::SEP ||
It->getOpcode() == W65816::REP) &&
It->getOperand(0).isImm() && (It->getOperand(0).getImm() & 0x20))
M16 = It->getOpcode() == W65816::REP;
return M16;
};
// SP (logical) just before Pt.
auto spAt = [&](MachineBasicBlock &MBB, MachineBasicBlock::iterator Pt) {
int Sp = EntrySp.lookup(&MBB);
for (auto It = MBB.begin(); It != Pt; ++It)
Sp = It->getOpcode() == W65816::TCS ? 0 : Sp + spDelta(*It);
return Sp;
};
// The swap-in point: after the prologue's TCS, or after the leading REP
// and PHA run of a frame allocated by pushes. No access to a slot may
// come before it.
MachineBasicBlock &Entry = MF.front();
MachineBasicBlock::iterator InPt = Entry.end();
for (auto It = Entry.begin(); It != Entry.end(); ++It)
if (It->getOpcode() == W65816::TCS) {
InPt = std::next(It);
break;
}
if (InPt == Entry.end()) {
InPt = Entry.begin();
while (InPt != Entry.end() &&
(InPt->isDebugInstr() || InPt->getOpcode() == W65816::REP ||
InPt->getOpcode() == W65816::PHA))
++InPt;
}
if (InPt == Entry.end() || !m16At(Entry, InPt))
return false;
int SpIn = spAt(Entry, InPt);
DenseSet<MachineInstr *> BeforeIn;
for (auto It = Entry.begin(); It != InPt; ++It)
BeforeIn.insert(&*It);
// The swap-back points: every exit is an RTL, the swap goes in front of
// the frame release that leads to it.
SmallVector<std::pair<MachineBasicBlock *, MachineBasicBlock::iterator>, 4>
OutPts;
double ExitFreq = 0;
for (MachineBasicBlock &MBB : MF) {
if (!MBB.succ_empty())
continue;
auto Last = MBB.getLastNonDebugInstr();
if (Last == MBB.end() || Last->getOpcode() != W65816::RTL)
return false;
if (!EntrySp.count(&MBB))
continue;
MachineBasicBlock::iterator Pt = Last;
while (Pt != MBB.begin()) {
auto Prev = std::prev(Pt);
unsigned Op = Prev->getOpcode();
if (Prev->isDebugInstr() || Op == W65816::TCS || Op == W65816::TSC ||
Op == W65816::CLC || Op == W65816::ADC_Imm16 ||
Op == W65816::TAY || Op == W65816::TYA || Op == W65816::TAX ||
Op == W65816::TXA || Op == W65816::PLY || Op == W65816::PLX) {
Pt = Prev;
continue;
}
break;
}
if (!m16At(MBB, Pt))
return false;
// A return in the entry block must swap back after it swapped in.
if (&MBB == &Entry && (Pt == InPt || BeforeIn.count(&*Pt)))
return false;
OutPts.push_back({&MBB, Pt});
ExitFreq += MBFI.getBlockFreqRelativeToEntryBlock(&MBB);
}
if (OutPts.empty())
return false;
// Pairs, heaviest first, into adjacent free words.
BitVector Taken = usedImgWords(MF, 0xC0u);
DenseMap<int64_t, uint8_t> Home;
auto usable = [&](int64_t S) {
if (!Sites.count(S) || Blocked.count(S) || Home.count(S) ||
AllSlots.count(S - 1) || AllSlots.count(S + 1))
return false;
for (MachineInstr *MI : Sites[S])
if (BeforeIn.count(MI))
return false;
if (S - SpIn + 4 > 0xFF || S - SpIn + 2 < 1)
return false;
for (auto &OP : OutPts) {
int64_t Off = S - spAt(*OP.first, OP.second) + 2;
if (Off < 1 || Off > 0xFF)
return false;
}
return true;
};
// Swap in ~61 cycles a pair (with the A save), back ~27 a return.
const double PairCost = 61.0 + 27.0 * ExitFreq;
for (const FarPtrPair &FP : collectFarPtrPairs(MF, EntrySp, 0xC0u)) {
if (FP.LoIsDp || FP.HiIsDp || FP.Lo == FP.Hi || !usable(FP.Lo) ||
!usable(FP.Hi))
continue;
double MinSave = minPathSaving(MF, [&](const MachineBasicBlock &B) {
return BlockSaving.lookup({&B, FP.Lo}) + BlockSaving.lookup({&B, FP.Hi});
});
double Expected = Saving[FP.Lo] + Saving[FP.Hi];
if (MinSave < PairCost &&
(Expected < 2.0 * PairCost || MinSave < PairCost / 4))
continue;
for (unsigned Idx = 0; Idx + 1 < 8; ++Idx) {
if (!Taken.test(Idx) && !Taken.test(Idx + 1)) {
Home[FP.Lo] = static_cast<uint8_t>(0xC0u + 2 * Idx);
Home[FP.Hi] = static_cast<uint8_t>(0xC0u + 2 * Idx + 2);
Taken.set(Idx);
Taken.set(Idx + 1);
break;
}
}
}
if (Home.empty())
return false;
// Swap in: A kept on the stack, each word through it.
DebugLoc DL;
BuildMI(Entry, InPt, DL, TII->get(W65816::PHA))
.addReg(W65816::A, RegState::Implicit | RegState::Undef);
for (auto &KV : Home) {
BuildMI(Entry, InPt, DL, TII->get(W65816::LDA_DP))
.addImm(KV.second)
.addReg(W65816::A, RegState::ImplicitDefine);
BuildMI(Entry, InPt, DL, TII->get(W65816::PHA))
.addReg(W65816::A, RegState::Implicit);
BuildMI(Entry, InPt, DL, TII->get(W65816::LDA_StackRel))
.addImm(KV.first - SpIn + 4)
.addReg(W65816::A, RegState::ImplicitDefine);
BuildMI(Entry, InPt, DL, TII->get(W65816::STA_DP))
.addImm(KV.second)
.addReg(W65816::A, RegState::Implicit);
BuildMI(Entry, InPt, DL, TII->get(W65816::PLA))
.addReg(W65816::A, RegState::ImplicitDefine);
BuildMI(Entry, InPt, DL, TII->get(W65816::STA_StackRel))
.addImm(KV.first - SpIn + 2)
.addReg(W65816::A, RegState::Implicit);
}
BuildMI(Entry, InPt, DL, TII->get(W65816::PLA))
.addReg(W65816::A, RegState::ImplicitDefine);
// Swap back at each exit, the return value kept on the stack.
for (auto &OP : OutPts) {
int SpOut = spAt(*OP.first, OP.second);
BuildMI(*OP.first, OP.second, DL, TII->get(W65816::PHA))
.addReg(W65816::A, RegState::Implicit | RegState::Undef);
for (auto &KV : Home) {
BuildMI(*OP.first, OP.second, DL, TII->get(W65816::LDA_StackRel))
.addImm(KV.first - SpOut + 2)
.addReg(W65816::A, RegState::ImplicitDefine);
BuildMI(*OP.first, OP.second, DL, TII->get(W65816::STA_DP))
.addImm(KV.second)
.addReg(W65816::A, RegState::Implicit);
}
BuildMI(*OP.first, OP.second, DL, TII->get(W65816::PLA))
.addReg(W65816::A, RegState::ImplicitDefine);
}
for (auto &KV : Home)
for (MachineInstr *MI : Sites[KV.first])
rewriteToDp(*MI, KV.second, TII);
// Debug values naming a promoted slot would read the swapped-out word.
for (MachineBasicBlock &MBB : MF)
for (MachineInstr &MI : MBB) {
if (!MI.isDebugValue())
continue;
const DIExpression *Expr = MI.getDebugExpression();
int64_t Lead = 0;
SmallVector<uint64_t, 4> Rest;
if (!Expr || !Expr->extractLeadingOffset(Lead, Rest) ||
!Home.count(Lead))
continue;
for (MachineOperand &Op : MI.debug_operands())
if (Op.isReg()) {
Op.setReg(0);
Op.setSubReg(0);
}
}
return true;
}
bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
if (skipFunction(MF.getFunction())) return false;
if (MF.getFunction().hasOptNone()) return false;
@ -1410,15 +1905,19 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
}
MachineLoopInfo &MLI =
getAnalysis<MachineLoopInfoWrapperPass>().getLI();
const MachineBlockFrequencyInfo &MBFI =
getAnalysis<MachineBlockFrequencyInfoWrapperPass>().getMBFI();
if (isSelfCall) {
ChangedEarly |= elideStackRelDeadStore(MF);
ChangedEarly |= elideStoreForwarding(MF);
ChangedEarly |= promoteLoopLocal(MF, MLI, STIp.getInstrInfo(), 0xD0u);
ChangedEarly |= promoteCalleeSavedPairs(MF, MBFI, STIp.getInstrInfo());
return ChangedEarly;
}
if (!isImgSafeCall(MI)) {
ChangedEarly |= elideStoreForwarding(MF);
ChangedEarly |= promoteLoopLocal(MF, MLI, STIp.getInstrInfo(), 0xD0u);
ChangedEarly |= promoteCalleeSavedPairs(MF, MBFI, STIp.getInstrInfo());
return ChangedEarly;
}
}
@ -1535,16 +2034,24 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
if (AccessCount[Off] >= kThreshold) ++HotCount;
}
if (HotCount > kMaxHotSlots) {
// Too many hot slots to pick from across the whole function -- but
// its innermost loops each touch few, and every call in the function
// leaves IMG0..7 alone (checked above), so they can keep theirs in
// DP as a calling function's loops do. CoreMark's matrix kernels
// (a multiply per trip) landed here and kept every pointer and
// counter on the stack.
ChangedEarly |= elideStoreForwarding(MF);
if (!getenv("W65816_DISABLE_LOOP_HOMES_HOT"))
ChangedEarly |= promoteLoopLocal(
MF, getAnalysis<MachineLoopInfoWrapperPass>().getLI(),
STIp.getInstrInfo(), 0xD0u);
return ChangedEarly;
}
DenseMap<int64_t, uint8_t> OffsetToDp; // logical offset -> DP byte
BitVector Taken = UsedDp; // DP words spoken for
// Caller-passed arg slots live ABOVE the return address on the stack;
// post-frame-alloc, they're at offsets >= frame_size + 4. Don't
// promote them — caller wrote to the stack location, not to DP.
int FrameSize = static_cast<int>(MF.getFrameInfo().getStackSize());
int ArgSlotMinOff = FrameSize + 4;
// Caller-passed arg slots live ABOVE the return address on the stack.
// Don't promote them: the caller wrote to the stack location, not to DP.
int ArgSlotMinOff = argSlotMinOff(MF);
// Slots Layer 2 ptr32 derefs read as a POINTER (their `(d,S),Y` uses
// require stack-rel addressing) stay put (see PromoteBlocked).
auto promotable = [&](int64_t Off) {
@ -2059,7 +2566,8 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
// Helpers that take one i16 arg in A and one i16 arg at 4,s,
// returning i32 in A:X. __mulhi3 returns i16 not i32 but reads
// same arg layout.
if (Name == "__umulhisi3" || Name == "__mulhi3")
if (Name == "__umulhisi3" || Name == "__mulhisi3" ||
Name == "__mulhi3")
return true;
return false;
}
@ -2481,7 +2989,8 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
(Ops[1]->getOpcode() == W65816::PHA ||
Ops[1]->getOpcode() == W65816::PUSH16) &&
Ops[2]->getOpcode() == W65816::LDA_StackRel &&
Ops[3]->getOpcode() == W65816::JSLpseudo32 &&
(Ops[3]->getOpcode() == W65816::JSLpseudo32 ||
Ops[3]->getOpcode() == W65816::JSLlight32) &&
Ops[4]->getOpcode() == W65816::PLY &&
Ops[5]->getOpcode() == W65816::RTL) {
// JSL target must be a commutative 16x16-multiply helper.
@ -2491,7 +3000,8 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
if (MO.isGlobal()) Name = MO.getGlobal()->getName();
else if (MO.isSymbol()) Name = MO.getSymbolName();
else continue;
if (Name == "__umulhisi3" || Name == "__mulhi3") {
if (Name == "__umulhisi3" || Name == "__mulhisi3" ||
Name == "__mulhi3") {
isCommutativeMul = true;
break;
}
@ -2543,8 +3053,9 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
}
}
if (Cur->isCall()) defsA = true;
if (defsA) return 1;
// A read first: a tied op both reads and writes A (Phase 6).
if (usesA) return -1;
if (defsA) return 1;
++Cur;
}
return 0;
@ -3353,31 +3864,8 @@ bool W65816StackRelToImg::runOnMachineFunction(MachineFunction &MF) {
// takes a stack address (TSX, or a TSC that is not a frame adjust) may
// read H through a pointer and is left alone.
{
bool TakesStackAddr = false;
for (MachineBasicBlock &MBB : MF) {
for (auto It = MBB.begin(), E = MBB.end(); It != E; ++It) {
if (It->getOpcode() == W65816::TSX) {
TakesStackAddr = true;
break;
}
if (It->getOpcode() != W65816::TSC) continue;
// Frame adjust: TSC ; CLC|SEC ; ADC|SBC #n ; TCS.
auto N = std::next(It);
bool FrameAdjust =
N != E && (N->getOpcode() == W65816::CLC ||
N->getOpcode() == W65816::SEC) &&
++N != E && (N->getOpcode() == W65816::ADC_Imm16 ||
N->getOpcode() == W65816::SBC_Imm16) &&
++N != E && N->getOpcode() == W65816::TCS;
if (!FrameAdjust) {
TakesStackAddr = true;
break;
}
}
if (TakesStackAddr) break;
}
DenseMap<MachineBasicBlock *, int> SweepEntrySp;
if (!TakesStackAddr && entrySpAdj(MF, SweepEntrySp)) {
if (!takesStackAddr(MF) && entrySpAdj(MF, SweepEntrySp)) {
DenseSet<int64_t> IncSlots;
DenseSet<int64_t> ReadSlots;
// INC_HI_IF_CARRY_StackRel and STA_StackRel sites, with their

154
src/llvm/lib/Target/W65816/W65816StackSlotCleanup.cpp
View file

@ -819,25 +819,39 @@ bool W65816StackSlotCleanup::runOnMachineFunction(MachineFunction &MF) {
Opc == W65816::BVC || Opc == W65816::BVS)
Branches.push_back(&MI);
}
// Every store: a store sets no flag, and the byte-store pseudos'
// SEP / REP touch only the M bit. A store missing here was taken for
// the flag-defining test by isFlagDefining's default, and the walk
// stopped at it: `cmp #K ; <SRL8A> ; <STA8long> ; lda #0 ; beq` got
// its wrap around the LDA alone, and the BEQ read the SRL8A's AND
// (llvm816's demos/randProbe.c compared rand() with 0x41C6 so).
auto isFlagPreserving = [VLAFunc](unsigned Opc) {
if (VLAFunc) {
// FI store pseudos are flag-corrupting under VLA expansion.
if (Opc == W65816::STAfi || Opc == W65816::STAfi_indY ||
Opc == W65816::STA8fi)
Opc == W65816::STA8fi || Opc == W65816::STA8fi_indY)
return false;
}
switch (Opc) {
case W65816::STA_StackRel: case W65816::STA_StackRelIndY:
case W65816::STAfi: case W65816::STAfi_indY:
case W65816::STA8fi: case W65816::STA8fi_indY:
case W65816::STA_DP: case W65816::STA_DPX:
case W65816::STA_DPInd: case W65816::STA_DPIndX:
case W65816::STA_DPIndY: case W65816::STA_DPIndLong:
case W65816::STA_DPIndLongY: case W65816::STB_DPIndLongY:
case W65816::STA_Abs: case W65816::STA_AbsX: case W65816::STA_AbsY:
case W65816::STA_Long: case W65816::STA_LongX:
case W65816::STAabs: case W65816::STA8abs: case W65816::STA8absX:
case W65816::STA8long:
case W65816::STX_DP: case W65816::STX_DPY: case W65816::STX_Abs:
case W65816::STY_DP: case W65816::STY_DPX: case W65816::STY_Abs:
case W65816::STZ_DP: case W65816::STZ_DPX: case W65816::STZ_Abs:
case W65816::STZ_AbsX:
return true;
default:
return false;
}
return Opc == W65816::STA_StackRel ||
Opc == W65816::STA_StackRelIndY ||
Opc == W65816::STAfi ||
Opc == W65816::STAfi_indY ||
Opc == W65816::STA8fi ||
Opc == W65816::STA_DP ||
Opc == W65816::STA_Abs ||
Opc == W65816::STA_Long ||
Opc == W65816::STX_DP ||
Opc == W65816::STX_Abs ||
Opc == W65816::STY_DP ||
Opc == W65816::STY_Abs;
};
auto isFlagDefining = [](const MachineInstr &MI) {
// Anything that physically writes A, X, Y, or P updates N/Z (or
@ -852,6 +866,7 @@ bool W65816StackSlotCleanup::runOnMachineFunction(MachineFunction &MF) {
case W65816::TCS: case W65816::TXS:
case W65816::TCD:
case W65816::JSLpseudo: case W65816::JSLpseudo32:
case W65816::JSLlight: case W65816::JSLlight32:
case W65816::JSL_Long:
case W65816::JSR_Abs:
case W65816::JMP_Abs:
@ -922,6 +937,7 @@ bool W65816StackSlotCleanup::runOnMachineFunction(MachineFunction &MF) {
// Pseudo wrappers that lower to LDA #imm.
Opc == W65816::LDAfi ||
Opc == W65816::LDAfi_indY ||
Opc == W65816::LDAimg ||
// Pure-logic NZ-setters: AND / ORA / EOR against an immediate
// or absolute operand. These physically set N/Z but
// deliberately do NOT model Defs=[P] (see W65816InstrInfo.td:
@ -1066,8 +1082,21 @@ bool W65816StackSlotCleanup::runOnMachineFunction(MachineFunction &MF) {
MachineInstr *LastCorrupt = nullptr;
for (auto It = std::prev(Br->getIterator()); ; --It) {
MachineInstr &MI = *It;
// An FI store of anything but A moves the value through A (`lda
// dp` / TXA / TYA, perhaps inside PHA ... PLA): it sets N and Z,
// whatever the other stores do. A spill of an IMG register
// between a compare and its BEQ was taken as flag-preserving.
bool NonAStore =
(MI.getOpcode() == W65816::STAfi ||
MI.getOpcode() == W65816::STA8fi) &&
MI.getOperand(0).isReg() && MI.getOperand(0).getReg() != W65816::A;
if (!MI.isDebugInstr()) {
if (isFlagPreserving(MI.getOpcode())) {
if (NonAStore) {
if (TestsNZ) {
if (!LastCorrupt) LastCorrupt = &MI;
FirstCorrupt = &MI;
}
} else if (isFlagPreserving(MI.getOpcode())) {
// skip
} else if (isCarryCorrupting(MI.getOpcode()) ||
(TestsNZ && isLdaLike(MI.getOpcode()))) {
@ -2197,6 +2226,103 @@ bool W65816StackSlotCleanup::runOnMachineFunction(MachineFunction &MF) {
Changed = true;
}
// Pass 2.4: a temp slot that only carries an IMG word to A.
// STAfi $imgN, %T, 0 ... $a = LDAfi %T, 0
// with nothing else touching T and imgN's DP word unchanged in between
// (no def, no call, no direct-page access to it): the reload becomes
// `lda dp` (a cycle less) and Pass 2.5 drops the store. The far-pointer staging inserters route a pointer word to
// $E0 / $E2 this way; with the word in an IMG register and A live, the
// store alone was `pha ; lda dp ; sta n,s ; pla` (400 such slots in
// Lua's luaV_execute, snprintf's format and CoreMark's list code).
{
const TargetInstrInfo *TII = MF.getSubtarget().getInstrInfo();
MachineFrameInfo &FrameInfo = MF.getFrameInfo();
DenseMap<int, unsigned> SlotRefs;
for (MachineBasicBlock &MBB : MF)
for (MachineInstr &MI : MBB)
if (!MI.isDebugInstr())
for (const MachineOperand &MO : MI.operands())
if (MO.isFI())
++SlotRefs[MO.getIndex()];
for (MachineBasicBlock &MBB : MF) {
for (MachineInstr &St : MBB) {
if (St.getOpcode() != W65816::STAfi || !St.getOperand(0).isReg() ||
!St.getOperand(2).isImm() || St.getOperand(2).getImm() != 0)
continue;
Register Img = St.getOperand(0).getReg();
int Dp = W65816Helpers::imgDPAddr(Img);
int FI = St.getOperand(1).getIndex();
if (Dp < 0 || FrameInfo.isFixedObjectIndex(FI) || SlotRefs[FI] != 2)
continue;
for (auto J = std::next(St.getIterator()); J != MBB.end(); ++J) {
if (J->isDebugInstr())
continue;
if (J->getOpcode() == W65816::LDAfi && J->getOperand(1).isFI() &&
J->getOperand(1).getIndex() == FI &&
J->getOperand(2).isImm() && J->getOperand(2).getImm() == 0 &&
J->getOperand(0).isReg() &&
J->getOperand(0).getReg() == W65816::A) {
// LDAimg, not a COPY: see its definition.
BuildMI(MBB, J, J->getDebugLoc(), TII->get(W65816::LDAimg),
W65816::A)
.addReg(Img);
St.getOperand(0).setIsKill(false);
J->eraseFromParent();
Changed = true;
break;
}
bool Touches = J->modifiesRegister(Img, TRI);
for (const MachineOperand &MO : J->operands())
if (MO.isFI() && MO.getIndex() == FI)
Touches = true;
if (TII->getName(J->getOpcode()).contains("_DP"))
for (const MachineOperand &MO : J->operands())
if (MO.isImm() && (MO.getImm() == Dp || MO.getImm() == Dp + 1 ||
MO.getImm() + 1 == Dp ||
MO.getImm() + 2 == Dp))
Touches = true;
if (Touches)
break;
}
}
}
}
// Pass 2.5: stores to a local stack object that nothing reads. The
// far-pointer deref inserters spill the pointer pair to a fresh object
// to stage it into $E0 / $E2 from there; when a later deref reuses the
// staging already in place (the consecutive-deref CSE), its reloads go
// but the stores stay -- and with the pointer in an IMG word and A
// live, each costs `pha ; lda dp ; sta n,s ; pla` (CoreMark's list
// reversal loop paid two such pairs per node). Any frame-index operand
// on anything but a plain store counts as a read (address taken
// included); fixed objects are the caller's. Pass 3 then drops the
// emptied objects from the frame.
{
MachineFrameInfo &FrameInfo = MF.getFrameInfo();
DenseSet<int> ReadSlots;
SmallVector<MachineInstr *, 16> SlotStores;
for (MachineBasicBlock &MBB : MF)
for (MachineInstr &MI : MBB) {
if (MI.isDebugInstr())
continue;
bool PlainStore = MI.getOpcode() == W65816::STAfi ||
MI.getOpcode() == W65816::STA8fi;
for (const MachineOperand &MO : MI.operands())
if (MO.isFI() && !PlainStore)
ReadSlots.insert(MO.getIndex());
if (PlainStore)
SlotStores.push_back(&MI);
}
for (MachineInstr *MI : SlotStores) {
int FI = MI->getOperand(1).getIndex();
if (FrameInfo.isFixedObjectIndex(FI) || ReadSlots.count(FI))
continue;
MI->eraseFromParent();
Changed = true;
}
}
// Pass 3: zero-size unused local frame objects so the
// PrologueEpilogue pass shrinks the prologue PHAs / TSC reservation.
// Walk the MIR collecting which FIs are still referenced; any

21
src/llvm/lib/Target/W65816/W65816TargetMachine.cpp
View file

@ -61,10 +61,12 @@ LLVMInitializeW65816Target() {
initializeW65816SjLjFinalizePass(PR);
initializeW65816LowerWide32Pass(PR);
initializeW65816CmpSinkPass(PR);
initializeW65816SinkLoopIncPass(PR);
initializeW65816I32IncFoldPass(PR);
initializeW65816ImgCalleeSavePass(PR);
initializeW65816NarrowI32MulPass(PR);
initializeW65816CmpPreparePass(PR);
initializeW65816SplitLoopExitsPass(PR);
initializeW65816PromoteFiToImgPass(PR);
initializeW65816StackSlotMergePass(PR);
initializeW65816StackRelToImgPass(PR);
@ -211,6 +213,9 @@ void W65816PassConfig::addISelPrepare() {
// only when optimizing: -O0 keeps the straight lowering.
if (getOptLevel() != CodeGenOptLevel::None)
addPass(createW65816CmpPrepare());
// Loop-exit edges get their own blocks, for the PHI copies on them.
if (getOptLevel() != CodeGenOptLevel::None)
addPass(createW65816SplitLoopExits());
TargetPassConfig::addISelPrepare();
}
@ -239,6 +244,10 @@ void W65816PassConfig::addPreRegAlloc() {
// pass walks Acc16/Idx16/Img16 vregs; running this first means they
// see the decomposed halves uniformly.
addPass(createW65816LowerWide32());
// Loop-carried increments below the old value's last use, before PHI
// elimination and the coalescer see them (see W65816SinkLoopInc.cpp).
if (getOptLevel() != CodeGenOptLevel::None)
addPass(createW65816SinkLoopInc());
// Keep each compare next to its conditional branch once PHI elimination
// has put the back-edge copies between them (see W65816CmpSink.cpp).
if (getOptLevel() != CodeGenOptLevel::None)
@ -277,7 +286,9 @@ void W65816PassConfig::addPostRegAlloc() {
// SpillToX converts STA/LDA pairs to TAX/TXA bridges; StackSlotCleanup
// then deletes still-adjacent redundant spills. A second SpillToX
// invocation collapses any TAX/TXA pair left adjacent by cleanup
// (e.g. when an inner copy between bridge endpoints went away).
// (e.g. when an inner copy between bridge endpoints went away), and
// bridges the pairs in spill slots that stack-slot coloring shares
// between values, once cleanup has deleted the pairs it can.
// Diagnostic lever: W65816_DISABLE_STACKOPT=1 turns off the stack-slot
// rewriting passes (SpillToX / StackSlotCleanup / StackSlotMerge /
// StackRelToImg) wholesale. Used to bisect wrong-VALUES corruption in
@ -286,7 +297,7 @@ void W65816PassConfig::addPostRegAlloc() {
if (!getenv("W65816_DISABLE_STACKOPT")) {
addPass(createW65816SpillToX());
addPass(createW65816StackSlotCleanup());
addPass(createW65816SpillToX());
addPass(createW65816SpillToX(/*SharedSlots=*/true));
}
// Disable MachineCopyPropagation: it eliminates `COPY $img = $a`
// thinking the IMG dest is dead (no explicit physreg use of $img
@ -340,6 +351,9 @@ void W65816PassConfig::addPreEmitPass() {
// because we deliberately KEEP ADCi16imm 1 so this pass can match
// it; the subsequent SepRepCleanup will see only the residual
// (non-fold-eligible) ADCi16imm cases.
// Merge the stack releases on a path into a return (call cleanup plus
// epilogue) into one, before the branch distances are measured.
addPass(createW65816ReturnPopMerge());
addPass(createW65816I32IncFold());
addPass(createW65816BranchExpand());
addPass(createW65816SepRepCleanup());
@ -360,6 +374,9 @@ void W65816PassConfig::addPreEmitPass() {
// stack slots promoted), so the far-pointer staging into $E0..$E3 can be
// tracked across the whole function. Branch distances only shrink.
addPass(createW65816FarPtrStageReuse());
// A load whose only use is a commutative op with the word A was just
// stored to: fold it into the op. Branch distances only shrink.
addPass(createW65816CommuteReload());
}
MachineFunctionInfo *W65816TargetMachine::createMachineFunctionInfo(

143
src/llvm/test/CodeGen/W65816/callee-saved-homes.ll Normal file
View file

@ -0,0 +1,143 @@
; Pin: a far pointer kept in two stack slots across calls gets a home in
; two IMG8..15 words ($C0..$CE survive calls), so W65816FarPtrStageReuse
; addresses it in place (`lda [0xc2], y`) instead of staging it into
; $E0/$E2 before every access. The function keeps its caller's words:
; the entry SWAPS each slot with its word, every return swaps back.
; Taken only where it pays on every path, or where it pays well on
; average and some on every path: a zero-trip guard in front of the loop
; leaves the pointers on the stack. A function that takes a stack
; address keeps them there too.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_CALLEE_SAVED_HOMES=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
declare void @ext(i16)
declare void @sink(ptr)
; q and r (stack arguments) swap with $C2/$C4 and $C6/$C8 after the
; frame is made, A kept; the loop reads them in place; the return swaps
; the caller's words back before the frame goes.
define void @walk(ptr %p, ptr %q, ptr %r, i16 %n) {
; CHECK-LABEL: walk:
; CHECK: tcs
; CHECK-NEXT: pha
; CHECK-NEXT: lda 0xc2
; CHECK-NEXT: pha
; CHECK-NEXT: lda 0x14, s
; CHECK-NEXT: sta 0xc2
; CHECK-NEXT: pla
; CHECK-NEXT: sta 0x12, s
; CHECK: lda 0xc8
; CHECK-NEXT: pha
; CHECK-NEXT: lda 0x1a, s
; CHECK-NEXT: sta 0xc8
; CHECK-NEXT: pla
; CHECK-NEXT: sta 0x18, s
; CHECK-NEXT: pla
; CHECK: jsl ext
; CHECK-NEXT: ldy #0x2
; CHECK-NEXT: lda [0xc2], y
; CHECK: lda [0xc6], y
; CHECK: lda 0x12, s
; CHECK-NEXT: sta 0xc2
; CHECK: lda 0x18, s
; CHECK-NEXT: sta 0xc8
; CHECK-NEXT: pla
; CHECK-NEXT: tsc
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0xc
; CHECK-NEXT: tcs
; CHECK-NEXT: rtl
; OFF-LABEL: walk:
; OFF-NOT: lda [0xc2], y
; OFF: rtl
entry:
br label %loop
loop:
%i = phi i16 [ 0, %entry ], [ %i1, %loop ]
%a = load i16, ptr %p, align 2
call void @ext(i16 %a)
%qb = getelementptr inbounds i16, ptr %q, i32 1
%b = load i16, ptr %qb, align 2
%b1 = add i16 %b, %i
store i16 %b1, ptr %qb, align 2
%rc = getelementptr inbounds i16, ptr %r, i32 2
%cc = load i16, ptr %rc, align 2
%c1 = xor i16 %cc, %b1
store i16 %c1, ptr %rc, align 2
%pc = getelementptr inbounds i16, ptr %p, i32 2
store i16 %c1, ptr %pc, align 2
%i1 = add i16 %i, 1
%e = icmp eq i16 %i1, %n
br i1 %e, label %done, label %loop
done:
ret void
}
; The same loop behind a zero-trip guard: the call that skips it would
; pay for the swaps and save nothing.
define void @guarded(ptr %p, ptr %q, ptr %r, i16 %n) {
; CHECK-LABEL: guarded:
; CHECK-NOT: lda [0xc2], y
; CHECK: rtl
entry:
%c0 = icmp sgt i16 %n, 0
br i1 %c0, label %loop, label %done
loop:
%i = phi i16 [ 0, %entry ], [ %i1, %loop ]
%a = load i16, ptr %p, align 2
call void @ext(i16 %a)
%qb = getelementptr inbounds i16, ptr %q, i32 1
%b = load i16, ptr %qb, align 2
%b1 = add i16 %b, %i
store i16 %b1, ptr %qb, align 2
%rc = getelementptr inbounds i16, ptr %r, i32 2
%cc = load i16, ptr %rc, align 2
%c1 = xor i16 %cc, %b1
store i16 %c1, ptr %rc, align 2
%pc = getelementptr inbounds i16, ptr %p, i32 2
store i16 %c1, ptr %pc, align 2
%i1 = add i16 %i, 1
%e = icmp eq i16 %i1, %n
br i1 %e, label %done, label %loop
done:
ret void
}
; A stack address handed out: the callee may read or write any slot.
define void @addrTaken(ptr %p, ptr %q, ptr %r, i16 %n) {
; CHECK-LABEL: addrTaken:
; CHECK-NOT: lda [0xc2], y
; CHECK: rtl
entry:
%buf = alloca [4 x i16], align 2
call void @sink(ptr %buf)
br label %loop
loop:
%i = phi i16 [ 0, %entry ], [ %i1, %loop ]
%a = load i16, ptr %p, align 2
call void @ext(i16 %a)
%qb = getelementptr inbounds i16, ptr %q, i32 1
%b = load i16, ptr %qb, align 2
%b1 = add i16 %b, %i
store i16 %b1, ptr %qb, align 2
%rc = getelementptr inbounds i16, ptr %r, i32 2
%cc = load i16, ptr %rc, align 2
%c1 = xor i16 %cc, %b1
store i16 %c1, ptr %rc, align 2
%pc = getelementptr inbounds i16, ptr %p, i32 2
store i16 %c1, ptr %pc, align 2
%i1 = add i16 %i, 1
%e = icmp eq i16 %i1, %n
br i1 %e, label %done, label %loop
done:
ret void
}

17
src/llvm/test/CodeGen/W65816/cmp-sink.ll
View file

@ -7,6 +7,12 @@
; end of a CMP / SBC carry chain is better left next to its first half
; (cmp-prepare.ll's postinc_ult pins that shape).
;
; Since W65816SinkLoopInc the pointer's step lands below the load through
; it, so the pointer is bumped in place and only the sum's back-edge COPY
; is left above each compare. (The loop is what is pinned: since
; W65816SplitLoopExits the entry block's copy of the zero-trip result may
; land after its compare, a wrap paid once per call.)
;
; RUN: llc -mtriple=w65816 -O2 -print-after=w65816-cmp-sink -o /dev/null < %s 2>&1 | FileCheck %s --check-prefix=MIR
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
@ -14,18 +20,17 @@ target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-
define i16 @sum16(ptr %p, ptr %end) {
; MIR-LABEL: Machine code for function sum16
; MIR: [[LO:%[0-9]+]]:acc16 = ADCi16imm [[LO]]:acc16(tied-def 0), 2
; MIR-NEXT: [[HI:%[0-9]+]]:acc16 = ADCEi16imm [[HI]]:acc16(tied-def 0), 0
; MIR: ADCfi
; MIR: COPY
; MIR-NEXT: COPY
; MIR-NEXT: COPY
; MIR-NEXT: CMPfi
; MIR-NEXT: CMPfi [[LO]]:acc16
; MIR-NEXT: BNE
; MIR: COPY
; MIR-NEXT: COPY
; MIR-NEXT: COPY
; MIR-NEXT: CMPfi
; MIR-NEXT: CMPfi [[HI]]:acc16
; MIR-NEXT: BNE
; CHECK-LABEL: sum16:
; CHECK: Loop Header
; CHECK-NOT: php
; CHECK: rtl
entry:

4
src/llvm/test/CodeGen/W65816/cmp-via-x.mir
View file

@ -5,7 +5,8 @@
# and C from X - k as CMP did from A - k. It needs X free from the test to
# the end of the block and not live into a successor.
#
# via_x: X free: TAX / CPX, no wrap.
# via_x: X free: TAX / CPX, no wrap. (bb.1 reads the stored slot so the
# store is not dropped as never read.)
# x_live_out: X live into a branch target: the wrap stays.
# x_in_region: the corrupter reads X (TXA): the wrap stays.
# stack_operand: `cmp d,s` has no CPX form: the wrap stays.
@ -49,6 +50,7 @@ body: |
BNE %bb.1, implicit $p
BRA %bb.2
bb.1:
$a = LDAfi %stack.1, 0
RTL
bb.2:
RTL

35
src/llvm/test/CodeGen/W65816/commute-reload.ll Normal file
View file

@ -0,0 +1,35 @@
; Pin: W65816CommuteReload folds a load into the commutative op that
; follows it when the op's other operand is the word A was just stored to:
; `sta 0x1, s ; lda 0xd0 ; ora 0x1, s` is `sta 0x1, s ; ora 0xd0` (A
; already equals the slot). A far pointer's null test, at the top of
; nearly every JoeyLib entry point, is the common case: 4 cycles and 2
; bytes each.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_COMMUTE_RELOAD=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
declare void @f(ptr)
; CHECK-LABEL: nullTest:
; CHECK: stx 0xd0
; CHECK-NEXT: sta 0x1, s
; CHECK-NEXT: ora 0xd0
; CHECK-NEXT: beq
; OFF-LABEL: nullTest:
; OFF: stx 0xd0
; OFF-NEXT: sta 0x1, s
; OFF-NEXT: lda 0xd0
; OFF-NEXT: ora 0x1, s
; OFF-NEXT: beq
define void @nullTest(ptr %p) {
entry:
%c = icmp eq ptr %p, null
br i1 %c, label %done, label %call
call:
call void @f(ptr %p)
br label %done
done:
ret void
}

59
src/llvm/test/CodeGen/W65816/dp-inc-fold-tied-use.ll Normal file
View file

@ -0,0 +1,59 @@
; Pin: StackRelToImg folds `lda dp ; inc a ; sta dp` into `inc dp` only when
; nothing reads A after the store. Its A-liveness scan checked an
; instruction's def of A before its use, so a tied op that reads and
; writes A (`$a = ANDi16imm $a, 511`) counted as overwriting it, and a ring
; index's `head = (head + 1) & 511` became `inc dp ; and #0x1ff ; sta dp`,
; masking whatever A held (the byte just copied). JoeyLib's IIgs serial
; read returned that garbage as its new head and never caught up with the
; tail: SERLOSS hung in its read loop (verify-iigs-serial-loss, scc). The
; Phase 11 STZ peephole's scan had the same order.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S8"
target triple = "w65816"
@gHead = external global i16, align 2
@gTail = external global i16, align 2
@gRing = external global [512 x i8], align 1
@gRead = external global i16, align 2
; CHECK-LABEL: rd:
; CHECK: inc a
; CHECK-NEXT: and #0x1ff
; CHECK-NEXT: sta 0x[[HEAD:[0-9a-f]+]]
define i16 @rd(ptr %buf, i16 zeroext %max) {
entry:
%0 = load volatile i16, ptr @gHead, align 2
%1 = load volatile i16, ptr @gTail, align 2
%cmp12 = icmp ne i16 %max, 0
%cmp113 = icmp ne i16 %0, %1
%2 = select i1 %cmp12, i1 %cmp113, i1 false
br i1 %2, label %for.body, label %for.end
for.body:
%n.015 = phi i16 [ %inc, %for.body ], [ 0, %entry ]
%head.014 = phi i16 [ %and, %for.body ], [ %0, %entry ]
%idxprom = zext i16 %head.014 to i32
%arrayidx = getelementptr inbounds nuw i8, ptr @gRing, i32 %idxprom
%3 = load i8, ptr %arrayidx, align 1
%idxprom2 = zext i16 %n.015 to i32
%arrayidx3 = getelementptr inbounds nuw i8, ptr %buf, i32 %idxprom2
store i8 %3, ptr %arrayidx3, align 1
%add = add i16 %head.014, 1
%and = and i16 %add, 511
%inc = add nuw i16 %n.015, 1
%cmp = icmp ult i16 %inc, %max
%cmp1 = icmp ne i16 %and, %1
%4 = select i1 %cmp, i1 %cmp1, i1 false
br i1 %4, label %for.body, label %for.end
for.end:
%head.0.lcssa = phi i16 [ %0, %entry ], [ %and, %for.body ]
%n.0.lcssa = phi i16 [ 0, %entry ], [ %inc, %for.body ]
store volatile i16 %head.0.lcssa, ptr @gHead, align 2
%5 = load volatile i16, ptr @gRead, align 2
%add4 = add i16 %5, %n.0.lcssa
store volatile i16 %add4, ptr @gRead, align 2
ret i16 %n.0.lcssa
}

65
src/llvm/test/CodeGen/W65816/far-index-y.ll Normal file
View file

@ -0,0 +1,65 @@
; Pin: a far-pointer access at `p + zext(i) + K` puts i + K in Y when that
; sum provably fits 16 bits (`[dp],Y` adds Y to the whole 24-bit pointer,
; as the i32 add carried into the bank byte). It used to compute p + i
; with an i32 add (`clc ; adc ; sta ; lda ; adc #0 ; sta`) and stage that
; into $E0 / $E2. Space Taxi's glyph lookups (`sim->glyphSlot[ch]`, a u8
; index at offset 0x215c) were the motivating case.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
; u8 index plus a field offset: Y = ch + 0x215c, no i32 add.
define i8 @byteIndexOffset(ptr %sim, i8 zeroext %ch) {
; CHECK-LABEL: byteIndexOffset:
; CHECK: and #0xff
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0x215c
; CHECK-NEXT: tay
; CHECK-NOT: adc #0x0
; CHECK: lda [0x{{[0-9a-f]+}}], y
; CHECK: rtl
%f = getelementptr inbounds i8, ptr %sim, i32 8540
%i = zext i8 %ch to i32
%p = getelementptr inbounds i8, ptr %f, i32 %i
%v = load i8, ptr %p, align 1
ret i8 %v
}
; A full u16 index at offset 0: Y = i.
define void @storeWord16(ptr %p, i16 %i, i16 %v) {
; CHECK-LABEL: storeWord16:
; CHECK-NOT: adc #0x0
; CHECK: tay
; CHECK-NOT: adc #0x0
; CHECK: sta [0x{{[0-9a-f]+}}], y
; CHECK: rtl
%i32 = zext i16 %i to i32
%q = getelementptr inbounds i8, ptr %p, i32 %i32
store i16 %v, ptr %q, align 1
ret void
}
; A u16 index plus a nonzero offset could carry past Y's 16 bits: the
; i32 add stays.
define i8 @word16IndexOffset(ptr %p, i16 %i) {
; CHECK-LABEL: word16IndexOffset:
; CHECK: adc #0x0
; CHECK: rtl
%f = getelementptr inbounds i8, ptr %p, i32 4
%i32 = zext i16 %i to i32
%q = getelementptr inbounds i8, ptr %f, i32 %i32
%v = load i8, ptr %q, align 1
ret i8 %v
}
; A signed index may be negative: no fold.
define i8 @signedIndex(ptr %p, i16 %i) {
; CHECK-LABEL: signedIndex:
; CHECK: adc
; CHECK: rtl
%i32 = sext i16 %i to i32
%q = getelementptr inbounds i8, ptr %p, i32 %i32
%v = load i8, ptr %q, align 1
ret i8 %v
}

154
src/llvm/test/CodeGen/W65816/fp-cmp-const-int.ll Normal file
View file

@ -0,0 +1,154 @@
; Pin: a float / double compare that the operand's bits alone decide is an
; integer compare, not a soft-float libcall (combineFPSetCCConst).
; InstCombine makes `(bits & ~sign) == 0` into `fcmp oeq x, 0.0` and a
; magnitude test against the exponent mask into `fcmp uno x, 0.0` or
; `fcmp olt fabs(x), C`; each used to call __eqsf2 / __unordsf2 / __ltsf2.
;
; x == +-0.0 (bits & 0x7fff..) == 0, the sign word masked
; x == C (C != 0) bits == bits(C)
; x uno / ord C (bits & 0x7fff..) > / <= bits(+inf)
; fabs(x) < C, C >= 0 (bits & 0x7fff..) < bits(C), unsigned
; x > 0.0 bits in [1, bits(+inf)]: (bits - 1) < 0x7f800000
; x ult 0.0 (double) !(x >= 0): two compares (bits <= inf, bits == -0),
; still no libcall
;
; Compares of two variables, and relational compares against a constant
; other than a zero (x < 2.5), stay libcalls.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target triple = "w65816"
define i16 @f_oeq0(float %x) {
; CHECK-LABEL: f_oeq0:
; CHECK-NOT: jsl
; CHECK: and #0x7fff
; CHECK: rtl
%c = fcmp oeq float %x, 0.0
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @f_une_m0(float %x) {
; CHECK-LABEL: f_une_m0:
; CHECK-NOT: jsl
; CHECK: and #0x7fff
; CHECK: bne
; CHECK: rtl
%c = fcmp une float %x, -0.0
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @f_oeq1(float %x) {
; CHECK-LABEL: f_oeq1:
; CHECK-NOT: jsl
; CHECK: eor #0x3f80
; CHECK: rtl
%c = fcmp oeq float %x, 1.0
%r = zext i1 %c to i16
ret i16 %r
}
; isnan: |bits| >= 0x7f800001, one CMP/SBC chain.
define i16 @f_uno(float %x) {
; CHECK-LABEL: f_uno:
; CHECK-NOT: jsl
; CHECK: and #0x7fff
; CHECK: cmp #0x1
; CHECK: sbc #0x7f80
; CHECK-NEXT: lda #0x1
; CHECK-NEXT: bcs
; CHECK: rtl
%c = fcmp uno float %x, 0.0
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @f_fabs_olt(float %x) {
; CHECK-LABEL: f_fabs_olt:
; CHECK-NOT: jsl
; CHECK: and #0x7fff
; CHECK: cmp #0x4020
; CHECK-NEXT: bcc
; CHECK: rtl
%a = call float @llvm.fabs.f32(float %x)
%c = fcmp olt float %a, 2.5
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @f_ogt0(float %x) {
; CHECK-LABEL: f_ogt0:
; CHECK-NOT: jsl
; CHECK: adc #0xffff
; CHECK: adc #0xffff
; CHECK: cmp #0x7f80
; CHECK-NEXT: bcc
; CHECK: rtl
%c = fcmp ogt float %x, 0.0
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @f_olt_var(float %x, float %y) {
; CHECK-LABEL: f_olt_var:
; CHECK: jsl __ltsf2
; CHECK: rtl
%c = fcmp olt float %x, %y
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @f_olt25(float %x) {
; CHECK-LABEL: f_olt25:
; CHECK: jsl __ltsf2
; CHECK: rtl
%c = fcmp olt float %x, 2.5
%r = zext i1 %c to i16
ret i16 %r
}
; The sign word (the 4th, on the stack) is masked; all four words ORed.
define i16 @d_oeq0(double %x) {
; CHECK-LABEL: d_oeq0:
; CHECK-NOT: jsl
; CHECK: lda #0x7fff
; CHECK-NEXT: and 0x{{[0-9a-f]+}}, s
; CHECK: ora 0x{{[0-9a-f]+}}, s
; CHECK: rtl
%c = fcmp oeq double %x, 0.0
%r = zext i1 %c to i16
ret i16 %r
}
; !isnan: |bits| < 0x7ff0000000000001 as one four-word CMP/SBC chain.
define i16 @d_ord(double %x) {
; CHECK-LABEL: d_ord:
; CHECK-NOT: jsl
; CHECK: cmp #0x1
; CHECK-NEXT: lda 0xd0
; CHECK-NEXT: sbc #0x0
; CHECK-NEXT: lda 0x6, s
; CHECK-NEXT: sbc #0x0
; CHECK-NEXT: lda 0x1, s
; CHECK-NEXT: sbc #0x7ff0
; CHECK-NEXT: bcc
; CHECK: rtl
%c = fcmp ord double %x, 0.0
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @d_ult0(double %x) {
; CHECK-LABEL: d_ult0:
; CHECK-NOT: jsl
; CHECK: eor #0x8000
; CHECK: sbc #0x7ff0
; CHECK: rtl
%c = fcmp ult double %x, 0.0
%r = zext i1 %c to i16
ret i16 %r
}
declare float @llvm.fabs.f32(float)

92
src/llvm/test/CodeGen/W65816/frame-order-density.mir Normal file
View file

@ -0,0 +1,92 @@
# Pin: W65816FrameLowering::orderFrameObjects lays a large frame out with
# its most-used objects nearest S. A slot more than 0xFF bytes above S is
# out of `off,s` reach, so every access to it goes through the $F6 frame
# pointer (`sty $fa ; ldy #off ; lda [$f6],y ; ldy $fa`, 18 cycles against
# 5). In creation order the 2-byte scalar below sat on top of the 300-byte
# array it was created before; by access density (direct accesses, 8x per
# loop level, per byte) it now sits at S+1 and the array, reached only
# through the address ADDframe builds, goes on top.
#
# pha_pinned: an object accessed past its end is the PHA-bracket idiom
# (W65816ImgCalleeSave saves IMG8..15 with `pha ; lda dp ; sta slot+2,s ;
# pla`), whose +2 holds only stack-relative: through $F6, which the PHA does
# not move, it addresses the neighbour. Such objects stay nearest S however
# rarely they are used. The first cut sorted runtime snprintf's IMG8 save
# slot out of reach: `format` restored IMG8 from its neighbour and fprintf
# wrote nothing (the smoke test's GS/OS fflush check).
#
# RUN: llc -mtriple=w65816 -run-pass=prologepilog -o - %s | FileCheck %s
# RUN: env W65816_DISABLE_FRAME_ORDER=1 llc -mtriple=w65816 -run-pass=prologepilog -o - %s | FileCheck %s --check-prefix=CREATION
--- |
target triple = "w65816"
define void @hot_near() { ret void }
define void @pha_pinned() { ret void }
...
---
# CHECK-LABEL: name: hot_near
# CHECK-NOT: DPIndLongY 246
# CHECK: STA_StackRel 1
# CHECK-NOT: DPIndLongY 246
# CHECK: LDA_StackRel 1
# CREATION-LABEL: name: hot_near
# CREATION: STA_DPIndLongY 246
name: hot_near
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 300, alignment: 1 }
body: |
bb.0:
$a = ADDframe %stack.1, 0
STAfi $a, %stack.0, 0, implicit-def $a
$a = LDAi16imm 0
$a = LDAfi %stack.0, 0
RTL
...
---
# CHECK-LABEL: name: pha_pinned
# CHECK: PHA
# CHECK-NEXT: LDA_DP 192
# CHECK-NEXT: STA_StackRel 3
# CHECK-NEXT: PLA
# The array's 20 loop accesses outweigh the save slot's one (by density),
# which alone would sort the slot above the array, out of reach.
name: pha_pinned
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 300, alignment: 1 }
body: |
bb.0:
successors: %bb.1
PHA implicit $a
LDA_DP 192, implicit-def $a
STAfi $a, %stack.0, 2, implicit-def $a
PLA implicit-def $a
bb.1:
successors: %bb.1, %bb.2
$a = LDAfi %stack.1, 0
$a = LDAfi %stack.1, 2
$a = LDAfi %stack.1, 4
$a = LDAfi %stack.1, 6
$a = LDAfi %stack.1, 8
$a = LDAfi %stack.1, 10
$a = LDAfi %stack.1, 12
$a = LDAfi %stack.1, 14
$a = LDAfi %stack.1, 16
$a = LDAfi %stack.1, 18
$a = LDAfi %stack.1, 20
$a = LDAfi %stack.1, 22
$a = LDAfi %stack.1, 24
$a = LDAfi %stack.1, 26
$a = LDAfi %stack.1, 28
$a = LDAfi %stack.1, 30
$a = LDAfi %stack.1, 32
$a = LDAfi %stack.1, 34
$a = LDAfi %stack.1, 36
$a = LDAfi %stack.1, 38
CMPi16imm $a, 0, implicit-def $p
BNE %bb.1, implicit $p
bb.2:
RTL
...

130
src/llvm/test/CodeGen/W65816/global-index-abs.ll Normal file
View file

@ -0,0 +1,130 @@
; Pin: an access to a global array at a variable index is `lda G+K,x` /
; `sta G+K,x` (DBR-relative absolute indexed, the bank every global lives
; in), not a far pointer `#G` + `lda $BE` staged into $E0/$E2 for
; `lda [$E0],y`. The access is based on G, so a valid program keeps
; the offset inside G (at most 64K): X is that offset's low word. A
; constant K stays in the operand only when the index is known not
; negative; otherwise it is added into X. A global of unknown size
; keeps the far pointer. Space Taxi's per-cell table lookups
; (kCellRow[cell >> 3], gRender.dirtyColMin[row]) were the motivating
; case.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
%struct.R = type { [25 x i8], [25 x i8], [10 x i16] }
@tab = dso_local global [256 x i16] zeroinitializer, align 2
@bytes = dso_local global [125 x i8] zeroinitializer, align 1
@longs = dso_local global [16 x i32] zeroinitializer, align 2
@rec = dso_local global %struct.R zeroinitializer, align 2
@vbytes = dso_local global [8 x i8] zeroinitializer, align 1
@unsized = external global [0 x i8], align 1
; u16 element, u16 index: X = i << 1.
define i16 @wordIndex(i16 zeroext %i) {
; CHECK-LABEL: wordIndex:
; CHECK: asl a
; CHECK-NEXT: tax
; CHECK-NEXT: lda tab, x
; CHECK-NOT: 0xbe
; CHECK: rtl
%idx = zext i16 %i to i32
%p = getelementptr inbounds [256 x i16], ptr @tab, i32 0, i32 %idx
%v = load i16, ptr %p, align 2
ret i16 %v
}
; A byte reads the word and masks. The index is a right shift of a
; zero-extended value: computed in 16 bits.
define i16 @byteShiftIndex(i16 zeroext %cell) {
; CHECK-LABEL: byteShiftIndex:
; CHECK: lsr a
; CHECK-NEXT: lsr a
; CHECK-NEXT: lsr a
; CHECK-NEXT: tax
; CHECK-NEXT: lda bytes, x
; CHECK-NEXT: and #0xff
; CHECK-NOT: 0xbe
; CHECK: rtl
%c = zext i16 %cell to i32
%s = lshr i32 %c, 3
%p = getelementptr inbounds i8, ptr @bytes, i32 %s
%v = load i8, ptr %p, align 1
%r = zext i8 %v to i16
ret i16 %r
}
; A field offset with a known non-negative index stays in the operand.
define void @fieldByteStore(i8 zeroext %r, i8 %v) {
; CHECK-LABEL: fieldByteStore:
; CHECK: tax
; CHECK: sep #0x20
; CHECK-NEXT: sta rec+25, x
; CHECK-NEXT: rep #0x20
; CHECK-NOT: 0xbe
; CHECK: rtl
%i = zext i8 %r to i32
%f = getelementptr inbounds i8, ptr @rec, i32 25
%p = getelementptr inbounds i8, ptr %f, i32 %i
store i8 %v, ptr %p, align 1
ret void
}
; A signed index may be negative: the field offset moves into X.
define i16 @fieldSignedIndex(i16 %i) {
; CHECK-LABEL: fieldSignedIndex:
; CHECK: asl a
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0x32
; CHECK-NEXT: tax
; CHECK-NEXT: lda rec, x
; CHECK: rtl
%s = sext i16 %i to i32
%p = getelementptr inbounds %struct.R, ptr @rec, i32 0, i32 2, i32 %s
%v = load i16, ptr %p, align 2
ret i16 %v
}
; An i32 element: both halves off the same X.
define i32 @longIndex(i16 zeroext %i) {
; CHECK-LABEL: longIndex:
; CHECK: asl a
; CHECK-NEXT: asl a
; CHECK-NEXT: tax
; CHECK-NEXT: lda longs, x
; CHECK: lda longs+2, x
; CHECK-NOT: 0xbe
; CHECK: rtl
%idx = zext i16 %i to i32
%p = getelementptr inbounds [16 x i32], ptr @longs, i32 0, i32 %idx
%v = load i32, ptr %p, align 2
ret i32 %v
}
; A volatile byte keeps its 8-bit access.
define i8 @volatileByte(i8 zeroext %i) {
; CHECK-LABEL: volatileByte:
; CHECK: tax
; CHECK-NEXT: sep #0x20
; CHECK-NEXT: lda vbytes, x
; CHECK-NEXT: rep #0x20
; CHECK: rtl
%idx = zext i8 %i to i32
%p = getelementptr inbounds [8 x i8], ptr @vbytes, i32 0, i32 %idx
%v = load volatile i8, ptr %p, align 1
ret i8 %v
}
; No size, no bound: the far pointer stays.
define i8 @unsizedByte(i16 zeroext %i) {
; CHECK-LABEL: unsizedByte:
; CHECK: lda 0xbe
; CHECK: lda [0xe0], y
; CHECK: rtl
%idx = zext i16 %i to i32
%p = getelementptr inbounds i8, ptr @unsized, i32 %idx
%v = load i8, ptr %p, align 1
ret i8 %v
}

6
src/llvm/test/CodeGen/W65816/i32-compare-chain.ll
View file

@ -434,11 +434,13 @@ f:
}
; Against 0 and -1, and constants whose low half is 0: no chain at all.
; (W65816CommuteReload folds `lda 0xd0 ; ora 0x1, s` after the store of
; A to 0x1, s into `ora 0xd0`.)
define void @brk_eq0(i32 %a) {
; CHECK-LABEL: brk_eq0:
; CHECK-NOT: eor
; CHECK: lda 0xd0
; CHECK-NEXT: ora 0x1, s
; CHECK: sta 0x1, s
; CHECK-NEXT: ora 0xd0
; CHECK-NEXT: beq
; CHECK: rtl
entry:

82
src/llvm/test/CodeGen/W65816/i32-half-offset-fold.ll Normal file
View file

@ -0,0 +1,82 @@
; Pin: both halves of an i32 load or store through a far pointer address
; the pointer's base with the field offset in Y. The high half of an
; i32 load at `p + K` is a load at `(p + K) + 2`; peeling only the outer
; +2 left it addressed through the computed `p + K` (an i32 add and a
; staging of its result into $E0 / $E2 per access). The i32 store path
; did not peel at all: its high half always took `p + 2` through an i32
; add. A `+ K` whose constant was already split into halves by
; LowerI32Constant now peels too. CoreMark's linked-list code
; (list->info, list->next) went -7% total time with this.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
; The pointer field at offset 4: low half at Y=4, high half at Y=6, no
; carry chain (the `adc #0x0` of an i32 address add).
define ptr @getInfo(ptr %n) {
; CHECK-LABEL: getInfo:
; CHECK-NOT: adc #0x0
; CHECK-DAG: ldy #0x4
; CHECK-DAG: ldy #0x6
; CHECK-NOT: adc #0x0
; CHECK: rtl
%f = getelementptr inbounds i8, ptr %n, i32 4
%v = load ptr, ptr %f, align 2
ret ptr %v
}
; An i32 store at offset 8: Y=8 then Y=10 off the incoming pointer.
define void @setCnt(ptr %n, i32 %v) {
; CHECK-LABEL: setCnt:
; CHECK-NOT: adc #0x0
; CHECK: ldy #0x8
; CHECK-NEXT: sta [0x{{[0-9a-f]+}}], y
; CHECK-NOT: adc #0x0
; CHECK: ldy #0xa
; CHECK-NEXT: sta [0x{{[0-9a-f]+}}], y
; CHECK-NOT: adc #0x0
; CHECK: rtl
%f = getelementptr inbounds i8, ptr %n, i32 8
store i32 %v, ptr %f, align 2
ret void
}
; An i32 store at offset 0: the high half is Y=2 off the same base.
define void @setFirst(ptr %n, i32 %v) {
; CHECK-LABEL: setFirst:
; CHECK-NOT: adc #0x0
; CHECK: ldy #0x2
; CHECK-NOT: adc #0x0
; CHECK: rtl
store i32 %v, ptr %n, align 2
ret void
}
; A read-modify-write of an i32 element (CoreMark's transition_count[i]++):
; four [p],Y accesses at Y=8 / Y=10, the only add the increment's.
define void @bump(ptr %tc) {
; CHECK-LABEL: bump:
; CHECK-NOT: adc #0x2
; CHECK-NOT: adc #0x8
; CHECK-DAG: ldy #0x8
; CHECK-DAG: ldy #0xa
; CHECK-NOT: adc #0x2
; CHECK-NOT: adc #0x8
; CHECK: rtl
%p = getelementptr inbounds i8, ptr %tc, i32 8
%v = load i32, ptr %p, align 2
%w = add i32 %v, 1
store i32 %w, ptr %p, align 2
ret void
}
; Offsets that no longer fit Y fall back to the i32 add.
define void @setFar(ptr %n, i32 %v) {
; CHECK-LABEL: setFar:
; CHECK: adc #0xfffe
; CHECK: rtl
%f = getelementptr inbounds i8, ptr %n, i32 65534
store i32 %v, ptr %f, align 2
ret void
}

125
src/llvm/test/CodeGen/W65816/i64-compare-chain.ll Normal file
View file

@ -0,0 +1,125 @@
; Pin: an i64 compare is one four-word CMP/SBC chain (lowerI32Cmp, reached
; from the type legalizer: SETCC / BR_CC / SELECT_CC are Custom for i64),
; not the generic expansion's per-half compares joined by selects.
;
; lda a.w0 ; cmp b.w0 ; lda a.w1 ; sbc b.w1 ; ... ; sbc b.w3 ; bcc/bcs
;
; Signed predicates bias both top words by 0x8000. Constant low words of 0
; drop out of the chain (x < 0x100000000 compares words 2 and 3 only).
; eq / ne stay the type legalizer's own expansion, which ORs the four XORs
; together and tests once. Its expansion of the ordered predicates was ~50
; instructions per compare, and the baseline compiler answered 63 of the
; i64 smoke check's value-form compares against constants wrongly
; (StackRelToImg promoted an incoming argument word to DP; see
; stackrel-to-img-arg-slot.ll).
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target triple = "w65816"
@sink = global i16 0
; a.w0 in A, a.w1 in X, the rest on the stack.
define void @br_ult(i64 %a, i64 %b) {
; CHECK-LABEL: br_ult:
; CHECK: stx 0xd0
; CHECK-NEXT: sta 0xd2
; CHECK-NEXT: cmp 0x8, s
; CHECK-NEXT: lda 0xd0
; CHECK-NEXT: sbc 0xa, s
; CHECK-NEXT: lda 0x4, s
; CHECK-NEXT: sbc 0xc, s
; CHECK-NEXT: lda 0x6, s
; CHECK-NEXT: sbc 0xe, s
; CHECK-NEXT: bcs
; CHECK: rtl
entry:
%c = icmp ult i64 %a, %b
br i1 %c, label %t, label %f
t:
store volatile i16 1, ptr @sink
ret void
f:
store volatile i16 2, ptr @sink
ret void
}
define i16 @va_slt(i64 %a, i64 %b) {
; CHECK-LABEL: va_slt:
; CHECK: lda #0x8000
; CHECK: eor 0x12, s
; CHECK: eor 0xa, s
; CHECK: cmp 0xc, s
; CHECK-NEXT: lda 0xd0
; CHECK-NEXT: sbc 0xe, s
; CHECK-NEXT: lda 0x8, s
; CHECK-NEXT: sbc 0x10, s
; CHECK-NEXT: lda 0xd4
; CHECK-NEXT: sbc 0xd6
; CHECK-NEXT: bcc
; CHECK: rtl
%c = icmp slt i64 %a, %b
%r = zext i1 %c to i16
ret i16 %r
}
; A select on the compare branches straight on the chain's carry.
define i16 @se_uge(i64 %a, i64 %b) {
; CHECK-LABEL: se_uge:
; CHECK: cmp 0x8, s
; CHECK: sbc 0xe, s
; CHECK-NEXT: bcs
; CHECK: lda #0x7
; CHECK: lda #0x9
%c = icmp uge i64 %a, %b
%r = select i1 %c, i16 9, i16 7
ret i16 %r
}
; x > 0x7ff0000000000000 is x >= 0x7ff0000000000001.
define i16 @va_ugtK(i64 %a) {
; CHECK-LABEL: va_ugtK:
; CHECK: cmp #0x1
; CHECK-NEXT: lda 0xd0
; CHECK-NEXT: sbc #0x0
; CHECK-NEXT: lda 0x4, s
; CHECK-NEXT: sbc #0x0
; CHECK-NEXT: lda 0x6, s
; CHECK-NEXT: sbc #0x7ff0
; CHECK-NEXT: bcs
; CHECK: rtl
%c = icmp ugt i64 %a, 9218868437227405312
%r = zext i1 %c to i16
ret i16 %r
}
; Two zero low words: only words 2 and 3 are compared.
define i16 @va_ultHiK(i64 %a) {
; CHECK-LABEL: va_ultHiK:
; CHECK-NOT: 0xd0
; CHECK: lda 0x4, s
; CHECK-NEXT: cmp #0x1
; CHECK-NEXT: txa
; CHECK-NEXT: sbc #0x0
; CHECK-NEXT: bcc
; CHECK: rtl
%c = icmp ult i64 %a, 4294967296
%r = zext i1 %c to i16
ret i16 %r
}
define i16 @va_eq(i64 %a, i64 %b) {
; CHECK-LABEL: va_eq:
; CHECK: tcs
; CHECK-NOT: sbc
; CHECK-DAG: eor 0x12, s
; CHECK-DAG: eor 0x10, s
; CHECK-DAG: eor 0x14, s
; CHECK-DAG: eor 0x16, s
; CHECK-NOT: sbc
; CHECK: beq
; CHECK: rtl
%c = icmp eq i64 %a, %b
%r = zext i1 %c to i16
ret i16 %r
}

33
src/llvm/test/CodeGen/W65816/i8-imm-wide.ll Normal file
View file

@ -0,0 +1,33 @@
; Pin: an i8 constant loaded into A is a 16-bit `lda #imm` (3 cycles),
; not `sep #0x20 ; lda #imm ; rep #0x20` (9): the byte's high half is
; nobody's business, and LDAi8imm is in the flag-wrap pass's isLdaLike
; list either way. A store's own SEP that follows still takes the 8-bit
; form and drops the REP/SEP pair between them.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_I8_IMM16=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
declare i8 @takeByte(i8)
define i8 @retByte() {
; CHECK-LABEL: retByte:
; CHECK: rep #0x30
; CHECK-NEXT: lda #0x7
; CHECK-NEXT: rtl
; OFF-LABEL: retByte:
; OFF: sep #0x20
; OFF-NEXT: lda #0x7
; OFF-NEXT: rep #0x20
ret i8 7
}
define i8 @passByte() {
; CHECK-LABEL: passByte:
; CHECK: rep #0x30
; CHECK-NEXT: lda #0xfb
; CHECK-NEXT: jsl takeByte
%r = call i8 @takeByte(i8 -5)
ret i8 %r
}

75
src/llvm/test/CodeGen/W65816/lda-after-pha-stack-offset.mir Normal file
View file

@ -0,0 +1,75 @@
# Pin: W65816StackRelToImg's redundant-LDA-after-PHA peephole drops the
# second load only when it reads the word the first one did. After the PHA
# a stack word sits 2 bytes further from SP, so `LDA off, s ; PHA ;
# LDA off+2, s` is redundant and `LDA off, s ; PHA ; LDA off, s` is not:
# it reads the word below. The equal-offset match deleted the load in
# W65816ImgCalleeSave's epilogue restore,
#
# lda 3,s (return value) ; pha ; lda 3,s (saved $C0) ; sta $c0 ; pla
#
# so wmemcpy, wmemmove, rename, fminf and eleven Lua functions returned with
# the caller's callee-saved $C0 replaced by their own return value.
#
# restore_after_return: different words; both loads stay.
# same_word: off, then off+2 after the push; the second goes.
# same_dp_word: a direct-page word does not move; the second goes.
#
# RUN: llc -mtriple=w65816 -run-pass=w65816-stackrel-to-img -o - %s | FileCheck %s
--- |
target triple = "w65816"
define void @restore_after_return() { ret void }
define void @same_word() { ret void }
define void @same_dp_word() { ret void }
...
---
# CHECK-LABEL: name: restore_after_return
# CHECK: LDA_StackRel 3
# CHECK-NEXT: PHA
# CHECK-NEXT: LDA_StackRel 3
# CHECK-NEXT: STA_DP 192
# CHECK-NEXT: PLA
name: restore_after_return
tracksRegLiveness: true
body: |
bb.0:
LDA_StackRel 3, implicit-def $a
PHA
LDA_StackRel 3, implicit-def $a
STA_DP 192, implicit $a
PLA implicit-def $a
RTL implicit $a
...
---
# CHECK-LABEL: name: same_word
# CHECK: LDA_StackRel 3
# CHECK-NEXT: PHA
# CHECK-NEXT: PHA
name: same_word
tracksRegLiveness: true
body: |
bb.0:
LDA_StackRel 3, implicit-def $a
PHA
LDA_StackRel 5, implicit-def $a
PHA
PLA implicit-def $a
PLA implicit-def $a
RTL implicit $a
...
---
# CHECK-LABEL: name: same_dp_word
# CHECK: LDA_DP 208
# CHECK-NEXT: PHA
# CHECK-NEXT: PHA
name: same_dp_word
tracksRegLiveness: true
body: |
bb.0:
LDA_DP 208, implicit-def $a
PHA
LDA_DP 208, implicit-def $a
PHA
PLA implicit-def $a
PLA implicit-def $a
RTL implicit $a
...

43
src/llvm/test/CodeGen/W65816/light-call.ll Normal file
View file

@ -0,0 +1,43 @@
; Pin: a call to one of libgcc.s's hand-written helpers (multiply,
; divide, shift, memcpy / memmove / memset) is a JSLlight, which clobbers
; A, X, Y and $F0 only -- the helpers never touch IMG0..IMG7 ($D0..$DE).
; Values live across it stay in those words: here the pointer's high half
; stays in $D0 instead of moving to callee-saved $C0, which costs a save
; in the prologue and a restore before the return.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_LIGHT_CALLS=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
define i16 @sumProducts(ptr %p, ptr %q, i16 %n) {
; CHECK-LABEL: sumProducts:
; CHECK-NOT: 0xc0
; CHECK: stx 0xd0
; CHECK-NOT: 0xc0
; CHECK: jsl __mulhi3
; CHECK-NOT: 0xc0
; CHECK: rtl
; OFF-LABEL: sumProducts:
; OFF: lda 0xc0
; OFF: stx 0xc0
; OFF: jsl __mulhi3
; OFF: sta 0xc0
; OFF: rtl
entry:
br label %loop
loop:
%i = phi i16 [ 0, %entry ], [ %i1, %loop ]
%s = phi i16 [ 0, %entry ], [ %s1, %loop ]
%pa = getelementptr inbounds i16, ptr %p, i16 %i
%qa = getelementptr inbounds i16, ptr %q, i16 %i
%a = load i16, ptr %pa, align 2
%b = load i16, ptr %qa, align 2
%m = mul i16 %a, %b
%s1 = add i16 %s, %m
%i1 = add i16 %i, 1
%e = icmp eq i16 %i1, %n
br i1 %e, label %done, label %loop
done:
ret i16 %s1
}

221
src/llvm/test/CodeGen/W65816/loop-homes-hot-slots.ll Normal file
View file

@ -0,0 +1,221 @@
; Pin: a function with more hot stack slots than the whole-function IMG
; promotion takes (kMaxHotSlots) still gives its innermost loops their own
; IMG0..7 homes (promoteLoopLocal): the byte-copy loop after twenty live
; accumulators walks both pointers in the direct page (`lda [0xd6], y`,
; `inc 0xd6`) instead of staging them through $E0/$E2 every byte.
; Generated from:
; unsigned mix(const unsigned *v, unsigned char *d,
; const unsigned char *s, unsigned n, unsigned m)
; twenty `aK = v[K]` mixed for n rounds, then `d[k] = s[k]` m times.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_LOOP_HOMES_HOT=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
; CHECK-LABEL: mix:
; CHECK: %for.body80.preheader
; CHECK: sta 0xd{{[0-9a-e]}}
; CHECK: %for.body80
; CHECK-NOT: 0xe0
; CHECK: lda [0xd{{[0-9a-e]}}], y
; CHECK-NOT: 0xe0
; CHECK: sta [0xd{{[0-9a-e]}}], y
; CHECK-NOT: 0xe0
; CHECK: inc 0xd{{[0-9a-e]}}
; CHECK: %for.end86
; OFF-LABEL: mix:
; OFF: %for.body80
; OFF: sta 0xe0
; OFF: lda [0xe0], y
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
define dso_local i16 @mix(ptr noundef readonly captures(none) %v, ptr noundef writeonly captures(none) %d, ptr noundef readonly captures(none) %s, i16 noundef %n, i16 noundef %m) local_unnamed_addr {
entry:
%0 = load i16, ptr %v, align 2
%arrayidx1 = getelementptr inbounds nuw i8, ptr %v, i32 2
%1 = load i16, ptr %arrayidx1, align 2
%arrayidx2 = getelementptr inbounds nuw i8, ptr %v, i32 4
%2 = load i16, ptr %arrayidx2, align 2
%arrayidx3 = getelementptr inbounds nuw i8, ptr %v, i32 6
%3 = load i16, ptr %arrayidx3, align 2
%arrayidx4 = getelementptr inbounds nuw i8, ptr %v, i32 8
%4 = load i16, ptr %arrayidx4, align 2
%arrayidx5 = getelementptr inbounds nuw i8, ptr %v, i32 10
%5 = load i16, ptr %arrayidx5, align 2
%arrayidx6 = getelementptr inbounds nuw i8, ptr %v, i32 12
%6 = load i16, ptr %arrayidx6, align 2
%arrayidx7 = getelementptr inbounds nuw i8, ptr %v, i32 14
%7 = load i16, ptr %arrayidx7, align 2
%arrayidx8 = getelementptr inbounds nuw i8, ptr %v, i32 16
%8 = load i16, ptr %arrayidx8, align 2
%arrayidx9 = getelementptr inbounds nuw i8, ptr %v, i32 18
%9 = load i16, ptr %arrayidx9, align 2
%arrayidx10 = getelementptr inbounds nuw i8, ptr %v, i32 20
%10 = load i16, ptr %arrayidx10, align 2
%arrayidx11 = getelementptr inbounds nuw i8, ptr %v, i32 22
%11 = load i16, ptr %arrayidx11, align 2
%arrayidx12 = getelementptr inbounds nuw i8, ptr %v, i32 24
%12 = load i16, ptr %arrayidx12, align 2
%arrayidx13 = getelementptr inbounds nuw i8, ptr %v, i32 26
%13 = load i16, ptr %arrayidx13, align 2
%arrayidx14 = getelementptr inbounds nuw i8, ptr %v, i32 28
%14 = load i16, ptr %arrayidx14, align 2
%arrayidx15 = getelementptr inbounds nuw i8, ptr %v, i32 30
%15 = load i16, ptr %arrayidx15, align 2
%arrayidx16 = getelementptr inbounds nuw i8, ptr %v, i32 32
%16 = load i16, ptr %arrayidx16, align 2
%arrayidx17 = getelementptr inbounds nuw i8, ptr %v, i32 34
%17 = load i16, ptr %arrayidx17, align 2
%arrayidx18 = getelementptr inbounds nuw i8, ptr %v, i32 36
%18 = load i16, ptr %arrayidx18, align 2
%arrayidx19 = getelementptr inbounds nuw i8, ptr %v, i32 38
%19 = load i16, ptr %arrayidx19, align 2
%cmp189.not = icmp eq i16 %n, 0
br i1 %cmp189.not, label %for.cond78.preheader, label %for.body
for.cond78.preheader: ; preds = %for.body, %entry
%a0.0.lcssa = phi i16 [ %0, %entry ], [ %add20, %for.body ]
%a1.0.lcssa = phi i16 [ %1, %entry ], [ %add23, %for.body ]
%a2.0.lcssa = phi i16 [ %2, %entry ], [ %add26, %for.body ]
%a3.0.lcssa = phi i16 [ %3, %entry ], [ %add29, %for.body ]
%a4.0.lcssa = phi i16 [ %4, %entry ], [ %add32, %for.body ]
%a5.0.lcssa = phi i16 [ %5, %entry ], [ %add35, %for.body ]
%a6.0.lcssa = phi i16 [ %6, %entry ], [ %add38, %for.body ]
%a7.0.lcssa = phi i16 [ %7, %entry ], [ %add41, %for.body ]
%a8.0.lcssa = phi i16 [ %8, %entry ], [ %add44, %for.body ]
%a9.0.lcssa = phi i16 [ %9, %entry ], [ %add47, %for.body ]
%a10.0.lcssa = phi i16 [ %10, %entry ], [ %add50, %for.body ]
%a11.0.lcssa = phi i16 [ %11, %entry ], [ %add53, %for.body ]
%a12.0.lcssa = phi i16 [ %12, %entry ], [ %add56, %for.body ]
%a13.0.lcssa = phi i16 [ %13, %entry ], [ %add59, %for.body ]
%a14.0.lcssa = phi i16 [ %14, %entry ], [ %add62, %for.body ]
%a15.0.lcssa = phi i16 [ %15, %entry ], [ %add65, %for.body ]
%a16.0.lcssa = phi i16 [ %16, %entry ], [ %add68, %for.body ]
%a17.0.lcssa = phi i16 [ %17, %entry ], [ %add71, %for.body ]
%a18.0.lcssa = phi i16 [ %18, %entry ], [ %add74, %for.body ]
%a19.0.lcssa = phi i16 [ %19, %entry ], [ %add77, %for.body ]
%cmp79230.not = icmp eq i16 %m, 0
br i1 %cmp79230.not, label %for.end86, label %for.body80
for.body: ; preds = %entry, %for.body
%k.0210 = phi i16 [ %inc, %for.body ], [ 0, %entry ]
%a19.0209 = phi i16 [ %add77, %for.body ], [ %19, %entry ]
%a18.0208 = phi i16 [ %add74, %for.body ], [ %18, %entry ]
%a17.0207 = phi i16 [ %add71, %for.body ], [ %17, %entry ]
%a16.0206 = phi i16 [ %add68, %for.body ], [ %16, %entry ]
%a15.0205 = phi i16 [ %add65, %for.body ], [ %15, %entry ]
%a14.0204 = phi i16 [ %add62, %for.body ], [ %14, %entry ]
%a13.0203 = phi i16 [ %add59, %for.body ], [ %13, %entry ]
%a12.0202 = phi i16 [ %add56, %for.body ], [ %12, %entry ]
%a11.0201 = phi i16 [ %add53, %for.body ], [ %11, %entry ]
%a10.0200 = phi i16 [ %add50, %for.body ], [ %10, %entry ]
%a9.0199 = phi i16 [ %add47, %for.body ], [ %9, %entry ]
%a8.0198 = phi i16 [ %add44, %for.body ], [ %8, %entry ]
%a7.0197 = phi i16 [ %add41, %for.body ], [ %7, %entry ]
%a6.0196 = phi i16 [ %add38, %for.body ], [ %6, %entry ]
%a5.0195 = phi i16 [ %add35, %for.body ], [ %5, %entry ]
%a4.0194 = phi i16 [ %add32, %for.body ], [ %4, %entry ]
%a3.0193 = phi i16 [ %add29, %for.body ], [ %3, %entry ]
%a2.0192 = phi i16 [ %add26, %for.body ], [ %2, %entry ]
%a1.0191 = phi i16 [ %add23, %for.body ], [ %1, %entry ]
%a0.0190 = phi i16 [ %add20, %for.body ], [ %0, %entry ]
%mul = mul i16 %a0.0190, 3
%add = add i16 %mul, %a1.0191
%add20 = add i16 %add, %k.0210
%mul21 = mul i16 %a1.0191, 3
%add22 = add i16 %mul21, %a2.0192
%add23 = add i16 %add22, %k.0210
%mul24 = mul i16 %a2.0192, 3
%add25 = add i16 %mul24, %a3.0193
%add26 = add i16 %add25, %k.0210
%mul27 = mul i16 %a3.0193, 3
%add28 = add i16 %mul27, %a4.0194
%add29 = add i16 %add28, %k.0210
%mul30 = mul i16 %a4.0194, 3
%add31 = add i16 %mul30, %a5.0195
%add32 = add i16 %add31, %k.0210
%mul33 = mul i16 %a5.0195, 3
%add34 = add i16 %mul33, %a6.0196
%add35 = add i16 %add34, %k.0210
%mul36 = mul i16 %a6.0196, 3
%add37 = add i16 %mul36, %a7.0197
%add38 = add i16 %add37, %k.0210
%mul39 = mul i16 %a7.0197, 3
%add40 = add i16 %mul39, %a8.0198
%add41 = add i16 %add40, %k.0210
%mul42 = mul i16 %a8.0198, 3
%add43 = add i16 %mul42, %a9.0199
%add44 = add i16 %add43, %k.0210
%mul45 = mul i16 %a9.0199, 3
%add46 = add i16 %mul45, %a10.0200
%add47 = add i16 %add46, %k.0210
%mul48 = mul i16 %a10.0200, 3
%add49 = add i16 %mul48, %a11.0201
%add50 = add i16 %add49, %k.0210
%mul51 = mul i16 %a11.0201, 3
%add52 = add i16 %mul51, %a12.0202
%add53 = add i16 %add52, %k.0210
%mul54 = mul i16 %a12.0202, 3
%add55 = add i16 %mul54, %a13.0203
%add56 = add i16 %add55, %k.0210
%mul57 = mul i16 %a13.0203, 3
%add58 = add i16 %mul57, %a14.0204
%add59 = add i16 %add58, %k.0210
%mul60 = mul i16 %a14.0204, 3
%add61 = add i16 %mul60, %a15.0205
%add62 = add i16 %add61, %k.0210
%mul63 = mul i16 %a15.0205, 3
%add64 = add i16 %mul63, %a16.0206
%add65 = add i16 %add64, %k.0210
%mul66 = mul i16 %a16.0206, 3
%add67 = add i16 %mul66, %a17.0207
%add68 = add i16 %add67, %k.0210
%mul69 = mul i16 %a17.0207, 3
%add70 = add i16 %mul69, %a18.0208
%add71 = add i16 %add70, %k.0210
%mul72 = mul i16 %a18.0208, 3
%add73 = add i16 %mul72, %a19.0209
%add74 = add i16 %add73, %k.0210
%mul75 = mul i16 %a19.0209, 3
%add76 = add i16 %k.0210, %mul75
%add77 = add i16 %add76, %add20
%inc = add nuw i16 %k.0210, 1
%exitcond.not = icmp eq i16 %inc, %n
br i1 %exitcond.not, label %for.cond78.preheader, label %for.body
for.body80: ; preds = %for.cond78.preheader, %for.body80
%k.1231 = phi i16 [ %inc85, %for.body80 ], [ 0, %for.cond78.preheader ]
%idxprom = zext i16 %k.1231 to i32
%arrayidx81 = getelementptr inbounds nuw i8, ptr %s, i32 %idxprom
%20 = load i8, ptr %arrayidx81, align 1
%arrayidx83 = getelementptr inbounds nuw i8, ptr %d, i32 %idxprom
store i8 %20, ptr %arrayidx83, align 1
%inc85 = add nuw i16 %k.1231, 1
%exitcond232.not = icmp eq i16 %inc85, %m
br i1 %exitcond232.not, label %for.end86, label %for.body80
for.end86: ; preds = %for.body80, %for.cond78.preheader
%xor = xor i16 %a1.0.lcssa, %a0.0.lcssa
%xor87 = xor i16 %xor, %a2.0.lcssa
%xor88 = xor i16 %xor87, %a3.0.lcssa
%xor89 = xor i16 %xor88, %a4.0.lcssa
%xor90 = xor i16 %xor89, %a5.0.lcssa
%xor91 = xor i16 %xor90, %a6.0.lcssa
%xor92 = xor i16 %xor91, %a7.0.lcssa
%xor93 = xor i16 %xor92, %a8.0.lcssa
%xor94 = xor i16 %xor93, %a9.0.lcssa
%xor95 = xor i16 %xor94, %a10.0.lcssa
%xor96 = xor i16 %xor95, %a11.0.lcssa
%xor97 = xor i16 %xor96, %a12.0.lcssa
%xor98 = xor i16 %xor97, %a13.0.lcssa
%xor99 = xor i16 %xor98, %a14.0.lcssa
%xor100 = xor i16 %xor99, %a15.0.lcssa
%xor101 = xor i16 %xor100, %a16.0.lcssa
%xor102 = xor i16 %xor101, %a17.0.lcssa
%xor103 = xor i16 %xor102, %a18.0.lcssa
%xor104 = xor i16 %xor103, %a19.0.lcssa
ret i16 %xor104
}

12
src/llvm/test/CodeGen/W65816/loop-local-dp.ll
View file

@ -9,9 +9,12 @@
; every byte.
;
; walk: calls before and after the loop: promoted.
; call_in_loop: the loop calls: left alone (still staged through $E0).
; call_in_loop: the loop calls: no loop home (still staged through $E0)
; -- the pointer takes an IMG8..15 home for the whole function instead
; (callee-saved-homes.ll), so the pin needs that switched off.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_CALLEE_SAVED_HOMES=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=NOSWAP
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
@ -52,9 +55,12 @@ exit:
define void @call_in_loop(ptr %p, i16 %n) {
; CHECK-LABEL: call_in_loop:
; CHECK: sta 0xe0
; CHECK: lda [0xe0], y
; CHECK: lda [0xc0], y
; CHECK: jsl use
; NOSWAP-LABEL: call_in_loop:
; NOSWAP: sta 0xe0
; NOSWAP: lda [0xe0], y
; NOSWAP: jsl use
entry:
br label %loop
loop:

69
src/llvm/test/CodeGen/W65816/lsr-nonneg-offsets.ll Normal file
View file

@ -0,0 +1,69 @@
; Pin: loop strength reduction keeps a pointer that walks an array of
; records at the record's start. isLegalAddressingMode used to allow any
; signed 16-bit offset, so LSR ran this loop on a pointer to `lastUse`
; and reached `used`, `ptr` and `color` at -4, -3 and -2: each of those
; loads was an i32 add (`adc #0xfffc ; ... adc #0xffff`) and a staging of
; the result into $E0 / $E2, since Y only adds an UNSIGNED offset. Now
; every field is `ldy #K ; lda [dp],y` off the walking pointer. (Space
; Taxi's sprite-cache scan.)
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; CHECK-LABEL: find:
; CHECK-NOT: adc #0xff{{[0-9a-f]+}}
; CHECK: ldy #0x0
; CHECK-NEXT: lda [0x{{[0-9a-f]+}}], y
; CHECK: ldy #0x1
; CHECK-NEXT: lda [0x{{[0-9a-f]+}}], y
; CHECK: ldy #0x2
; CHECK-NEXT: lda [0x{{[0-9a-f]+}}], y
; CHECK-NOT: adc #0xff{{[0-9a-f]+}}
; CHECK: rtl
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S8"
target triple = "w65816"
define dso_local ptr @find(ptr noundef captures(none) %cache, i8 noundef zeroext %ptr, i8 noundef zeroext %color, i16 noundef zeroext %stamp, i8 noundef zeroext %n) local_unnamed_addr {
entry:
%cmp25.not = icmp eq i8 %n, 0
br i1 %cmp25.not, label %cleanup15, label %for.body
for.body: ; preds = %entry, %for.inc
%k.026 = phi i8 [ %inc, %for.inc ], [ 0, %entry ]
%idxprom = zext i8 %k.026 to i32
%arrayidx = getelementptr inbounds nuw [10 x i8], ptr %cache, i32 %idxprom
%0 = load i8, ptr %arrayidx, align 2
%tobool.not = icmp eq i8 %0, 0
br i1 %tobool.not, label %for.inc, label %land.lhs.true
land.lhs.true: ; preds = %for.body
%ptr4 = getelementptr inbounds nuw i8, ptr %arrayidx, i32 1
%1 = load i8, ptr %ptr4, align 1
%cmp7 = icmp eq i8 %1, %ptr
br i1 %cmp7, label %land.lhs.true9, label %for.inc
land.lhs.true9: ; preds = %land.lhs.true
%color10 = getelementptr inbounds nuw i8, ptr %arrayidx, i32 2
%2 = load i8, ptr %color10, align 2
%cmp13 = icmp eq i8 %2, %color
br i1 %cmp13, label %cleanup, label %for.inc
cleanup: ; preds = %land.lhs.true9
%lastUse = getelementptr inbounds nuw i8, ptr %arrayidx, i32 4
store i16 %stamp, ptr %lastUse, align 2
%sprite = getelementptr inbounds nuw i8, ptr %arrayidx, i32 6
%3 = load ptr, ptr %sprite, align 2
br label %cleanup15
for.inc: ; preds = %land.lhs.true9, %land.lhs.true, %for.body
%inc = add nuw i8 %k.026, 1
%exitcond.not = icmp eq i8 %inc, %n
br i1 %exitcond.not, label %cleanup15, label %for.body
cleanup15: ; preds = %for.inc, %entry, %cleanup
%retval.2 = phi ptr [ %3, %cleanup ], [ null, %entry ], [ null, %for.inc ]
ret ptr %retval.2
}

51
src/llvm/test/CodeGen/W65816/mul-const-shift-add.ll Normal file
View file

@ -0,0 +1,51 @@
; Pin: LowerMUL_I32's strength reductions. A product known to fit 16
; bits -- a u8 index times a 6-byte record's size, the address math of
; `sim->spr[idx]` -- is a 16-bit multiply, here shifts and adds
; ((i << 3) - (i << 1)), not a __umulhisi3 call per access. A 32-bit
; multiply by a constant of two signed digits (2^N +- 2^M: 6, 10, 12, 24,
; ...) is two inlined shifts and an add or subtract instead of __mulsi3;
; a constant of more digits keeps the call.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_MUL_SHIFT_ADD=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
define zeroext i8 @recordField(ptr %s, i8 zeroext %idx) {
; CHECK-LABEL: recordField:
; CHECK-NOT: jsl
; CHECK: and #0xff
; CHECK-NEXT: asl a
; CHECK-NEXT: sta 0x1, s
; CHECK-NEXT: asl a
; CHECK-NEXT: asl a
; CHECK-NEXT: sec
; CHECK-NEXT: sbc 0x1, s
; CHECK: lda [0xe0], y
; CHECK: rtl
; OFF-LABEL: recordField:
; OFF: jsl __umulhisi3
%i = zext i8 %idx to i32
%rec = getelementptr inbounds [6 x i8], ptr %s, i32 %i
%f = getelementptr inbounds i8, ptr %rec, i32 2
%v = load i8, ptr %f, align 1
ret i8 %v
}
define i32 @longTimes6(i32 %x) {
; CHECK-LABEL: longTimes6:
; CHECK-NOT: jsl
; CHECK: sbc
; CHECK: rtl
; OFF-LABEL: longTimes6:
; OFF: jsl __mulsi3
%m = mul i32 %x, 6
ret i32 %m
}
define i32 @longTimes1000(i32 %x) {
; CHECK-LABEL: longTimes1000:
; CHECK: jsl __mulsi3
%m = mul i32 %x, 1000
ret i32 %m
}

50
src/llvm/test/CodeGen/W65816/mul-narrow-product.ll Normal file
View file

@ -0,0 +1,50 @@
; Pin: an i32 multiply whose PRODUCT fits 16 bits is a 16-bit multiply,
; zero-extended (W65816NarrowI32Mul): shifts and adds for a constant, else
; __mulhi3, and a high word of 0 -- not __umulhisi3's 16x16->32 call.
; CoreMark's matrix_mul_matrix_bitextract multiplies a 4-bit field by a
; 7-bit one; `(u32)byte * 6` is a 6-byte record's offset. Operands that
; only fit 16 bits each keep __umulhisi3.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_NARROW_PRODUCT=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=OFF
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
define i32 @byteTimes6(i8 zeroext %x) {
; CHECK-LABEL: byteTimes6:
; CHECK-NOT: jsl
; CHECK: asl a
; CHECK: adc
; CHECK: ldx #0x0
; CHECK-NEXT: ply
; OFF-LABEL: byteTimes6:
; OFF: jsl __umulhisi3
%z = zext i8 %x to i32
%m = mul nuw nsw i32 %z, 6
ret i32 %m
}
define i32 @fieldProduct(i16 %a, i16 %b) {
; CHECK-LABEL: fieldProduct:
; CHECK: and #0xf
; CHECK: jsl __mulhi3
; CHECK-NOT: jsl
; CHECK: ldx #0x0
; OFF-LABEL: fieldProduct:
; OFF: jsl __umulhisi3
%x = and i16 %a, 15
%y = and i16 %b, 127
%xz = zext i16 %x to i32
%yz = zext i16 %y to i32
%m = mul nuw nsw i32 %xz, %yz
ret i32 %m
}
define i32 @wideProduct(i16 %a, i16 %b) {
; CHECK-LABEL: wideProduct:
; CHECK: jsl __umulhisi3
%xz = zext i16 %a to i32
%yz = zext i16 %b to i32
%m = mul nuw i32 %xz, %yz
ret i32 %m
}

47
src/llvm/test/CodeGen/W65816/mul-narrow-signed.ll Normal file
View file

@ -0,0 +1,47 @@
; Pin: W65816NarrowI32Mul sends an i32 multiply of two sign-extended
; 16-bit values to __mulhisi3 (signed 16x16 -> 32), as it sends one of two
; zero-extended values to __umulhisi3. `(long)a * b` of two shorts went
; to __mulsi3, whose old loop took 32 steps for any negative operand
; (CoreMark's matrix_mul_matrix: ~330 cycles a call there, 8% of the run).
; A zero-extended byte fits a signed 16-bit value too; a zero-extended
; 16-bit value may not, so that pair keeps __mulsi3.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
define i32 @ss(i16 %a, i16 %b) {
; CHECK-LABEL: ss:
; CHECK: jsl __mulhisi3
%x = sext i16 %a to i32
%y = sext i16 %b to i32
%m = mul i32 %x, %y
ret i32 %m
}
define i32 @uu(i16 %a, i16 %b) {
; CHECK-LABEL: uu:
; CHECK: jsl __umulhisi3
%x = zext i16 %a to i32
%y = zext i16 %b to i32
%m = mul i32 %x, %y
ret i32 %m
}
define i32 @byteTimesShort(i8 %a, i16 %b) {
; CHECK-LABEL: byteTimesShort:
; CHECK: jsl __mulhisi3
%x = zext i8 %a to i32
%y = sext i16 %b to i32
%m = mul i32 %x, %y
ret i32 %m
}
define i32 @wordTimesShort(i16 %a, i16 %b) {
; CHECK-LABEL: wordTimesShort:
; CHECK: jsl __mulsi3
%x = zext i16 %a to i32
%y = sext i16 %b to i32
%m = mul i32 %x, %y
ret i32 %m
}

40
src/llvm/test/CodeGen/W65816/setjmp-img-clobber.ll Normal file
View file

@ -0,0 +1,40 @@
; Pin: a call that can return twice (setjmp) clobbers IMG8..IMG15
; ($C0..$CE). The longjmp that comes back through it skips the returns
; that give each caller its IMG8..15 words back, so the function calling
; setjmp keeps no value of its own in them across the call and saves and
; restores all eight for its caller (W65816ImgCalleeSave sees the defs).
; tests/regress/setjmpImgSave.c is the run-time check.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
@jb = dso_local global [8 x i8] zeroinitializer, align 1
declare i16 @setjmp(ptr) returns_twice
declare void @work(ptr, ptr)
define i16 @guard(ptr %p, ptr %q) {
; CHECK-LABEL: guard:
; CHECK: lda 0xc0
; CHECK: lda 0xc2
; CHECK: lda 0xc4
; CHECK: lda 0xc6
; CHECK: lda 0xc8
; CHECK: lda 0xca
; CHECK: lda 0xcc
; CHECK: lda 0xce
; CHECK: jsl setjmp
; CHECK: sta 0xce
; CHECK: rtl
entry:
%r = call i16 @setjmp(ptr @jb)
%z = icmp eq i16 %r, 0
br i1 %z, label %body, label %out
body:
call void @work(ptr %p, ptr %q)
br label %out
out:
%v = phi i16 [ 0, %body ], [ 1, %entry ]
ret i16 %v
}

125
src/llvm/test/CodeGen/W65816/sink-loop-inc.ll Normal file
View file

@ -0,0 +1,125 @@
; Pin: W65816SinkLoopInc moves a loop-carried step below the last use of
; the value it steps, so the old and new values do not overlap and the
; coalescer gives them one home: no back-edge copy, and a far pointer's
; `p++` becomes the in-place `inc dp ; bne ; inc dp+2`.
;
; sumPtr (`while (p != end) s += *p++;`): the scheduler put the pointer's
; ADC / ADCE pair above the load through the old pointer. The pair must
; land after the load ([$E0],Y with $E0 / $E2 staged from the old value),
; and the loop then bumps the pointer's DP home in place. It used to step
; into a second pair of words and copy them back every trip (+8
; instructions a trip; MAME: 169079 -> 115088 cycles for 2047 bytes).
;
; counterArg: an i16 `i++` whose old value is still passed after it. The
; INA lands after the old value's last use (the PUSH16 of the argument)
; and the counter's home is incremented in place.
;
; splitLatch: the back edge leaves from a second test block the step's
; block runs straight into (`while (*++p && state != 1)`, CoreMark's
; core_state_transition). The step was skipped because the PHI took it
; from that block; the pointer was then stepped into a second pair of
; words and copied back every trip.
;
; RUN: llc -mtriple=w65816 -O2 -stop-after=w65816-sink-loop-inc < %s | FileCheck %s --check-prefix=MIR
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target triple = "w65816"
@gOut16 = global i16 0
define i16 @sumPtr(ptr %p, ptr %end) {
; MIR-LABEL: name: sumPtr
; MIR: bb.2.while.body:
; MIR: PHI
; MIR-NEXT: [[LO:%[0-9]+]]:acc16 = PHI
; MIR-NEXT: [[HI:%[0-9]+]]:acc16 = PHI
; MIR: LDA_DPIndLongY 224
; MIR: ADCi16imm [[LO]], 1
; MIR-NEXT: ADCEi16imm [[HI]], 0
; CHECK-LABEL: sumPtr:
; CHECK: [[LOOP:.LBB0_[0-9]+]]: ; %while.body{{$}}
; CHECK-NEXT: ; =>This Inner Loop Header
; CHECK-NEXT: lda [0x[[PLO:d[0-9a-f]]]], y
; CHECK: inc 0x[[PLO]]
; CHECK-NEXT: bne
; CHECK-NEXT: inc 0x{{d[0-9a-f]}}
; CHECK-NOT: sta 0x[[PLO]]
; CHECK: bne [[LOOP]]
entry:
%cmp.not4 = icmp eq ptr %p, %end
br i1 %cmp.not4, label %while.end, label %while.body
while.body:
%s.06 = phi i16 [ %add, %while.body ], [ 0, %entry ]
%p.addr.05 = phi ptr [ %incdec.ptr, %while.body ], [ %p, %entry ]
%incdec.ptr = getelementptr inbounds nuw i8, ptr %p.addr.05, i32 1
%0 = load i8, ptr %p.addr.05, align 1
%conv = zext i8 %0 to i16
%add = add i16 %s.06, %conv
%cmp.not = icmp eq ptr %incdec.ptr, %end
br i1 %cmp.not, label %while.end, label %while.body
while.end:
%s.0.lcssa = phi i16 [ 0, %entry ], [ %add, %while.body ]
store volatile i16 %s.0.lcssa, ptr @gOut16, align 2
ret i16 %s.0.lcssa
}
define i16 @counterArg(i16 %n) {
; MIR-LABEL: name: counterArg
; MIR: [[I:%[0-9]+]]:acc16 = PHI
; MIR: $a = COPY [[I]]
; MIR-NEXT: PUSH16
; MIR-NEXT: {{%[0-9]+}}:acc16 = INA_PSEUDO [[I]]
; CHECK-LABEL: counterArg:
; CHECK: lda 0x[[C:d[0-9a-f]]]
; CHECK-NEXT: pha
; CHECK-NEXT: inc a
; CHECK-NEXT: sta 0x[[C]]
; CHECK-NEXT: jsl __mulhi3
entry:
br label %loop
loop:
%i = phi i16 [ 0, %entry ], [ %i.next, %loop ]
%s = phi i16 [ 0, %entry ], [ %s.next, %loop ]
%i.next = add i16 %i, 1
%t = mul i16 %i.next, %i
%s.next = add i16 %s, %t
%c = icmp ult i16 %i.next, %n
br i1 %c, label %loop, label %done
done:
ret i16 %s.next
}
define i16 @splitLatch(ptr %s, i16 %st0) {
; CHECK-LABEL: splitLatch:
; CHECK: Loop Header
; CHECK: lda [0x[[P:d[0-9a-f]]]], y
; CHECK-NEXT: tax
; CHECK-NEXT: inc 0x[[P]]
; CHECK-NOT: sta 0x[[P]]
; CHECK: rtl
entry:
%c0 = load i8, ptr %s, align 1
%z0 = icmp eq i8 %c0, 0
br i1 %z0, label %exit, label %body
body:
%p = phi ptr [ %s, %entry ], [ %pn, %latch2 ]
%st = phi i16 [ %st0, %entry ], [ %stn, %latch2 ]
%stn = xor i16 %st, 3
%pn = getelementptr inbounds i8, ptr %p, i32 1
%c = load i8, ptr %pn, align 1
%nz = icmp ne i8 %c, 0
br i1 %nz, label %latch2, label %exit
latch2:
%go = icmp ne i16 %stn, 1
br i1 %go, label %body, label %exit
exit:
%r = phi i16 [ %st0, %entry ], [ %stn, %body ], [ %stn, %latch2 ]
ret i16 %r
}

135
src/llvm/test/CodeGen/W65816/slot-img-forward.mir Normal file
View file

@ -0,0 +1,135 @@
# Pin: W65816StackSlotCleanup's temp-slot forwarding (Pass 2.4) and its
# never-read store sweep (Pass 2.5).
#
# The far-pointer staging inserters move a pointer word to $E0 / $E2 through
# a fresh stack object: `STAfi src, T ; ... ; LDAfi T ; STA_DP $E2`. With the
# word in an IMG register and A live, the store alone cost
# `pha ; lda dp ; sta n,s ; pla`. The reload now reads the IMG word itself
# (LDAimg, `lda dp` after expansion) and the store, read by nothing any more,
# goes; so do stores to objects nothing ever read (consecutive-deref CSE left
# those behind: CoreMark's list reversal paid two pairs per node).
#
# forward: IMG word unchanged between store and reload: forwarded,
# store dropped.
# img_redefined: the IMG register is written in between: both stay.
# dp_touched: a direct-page access to the IMG word in between: both stay.
# two_reads: the slot is read twice: both stay.
# never_read: a store to an object nothing reads: dropped.
# addr_taken: an ADDframe of the object counts as a read: kept.
#
# RUN: llc -mtriple=w65816 -run-pass=w65816-stack-slot-cleanup -o - %s | FileCheck %s
--- |
target triple = "w65816"
define void @forward() { ret void }
define void @img_redefined() { ret void }
define void @dp_touched() { ret void }
define void @two_reads() { ret void }
define void @never_read() { ret void }
define void @addr_taken() { ret void }
...
---
# CHECK-LABEL: name: forward
# CHECK: $img0 = COPY $x
# CHECK-NOT: STAfi
# CHECK-NOT: LDAfi
# CHECK: $a = LDAimg $img0
# CHECK-NEXT: STA_DP 226
name: forward
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
body: |
bb.0:
$img0 = COPY $x
STAfi $img0, %stack.0, 0, implicit-def dead $a
$a = LDAi16imm 7
STA_DP 224, implicit $a
dead $a = LDAfi %stack.0, 0
STA_DP 226
RTL
...
---
# CHECK-LABEL: name: img_redefined
# CHECK: STAfi $img0, %stack.0, 0
# CHECK: $a = LDAfi %stack.0, 0
name: img_redefined
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
body: |
bb.0:
$img0 = COPY $x
STAfi $img0, %stack.0, 0, implicit-def dead $a
$img0 = COPY $y
$a = LDAfi %stack.0, 0
STA_DP 226, implicit $a
RTL
...
---
# CHECK-LABEL: name: dp_touched
# CHECK: STAfi $img1, %stack.0, 0
# CHECK: $a = LDAfi %stack.0, 0
name: dp_touched
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
body: |
bb.0:
$img1 = COPY $x
STAfi $img1, %stack.0, 0, implicit-def dead $a
INC_DP 210
$a = LDAfi %stack.0, 0
STA_DP 226, implicit $a
RTL
...
---
# CHECK-LABEL: name: two_reads
# CHECK: STAfi $img0, %stack.0, 0
# CHECK: $a = LDAfi %stack.0, 0
# CHECK: $a = LDAfi %stack.0, 0
name: two_reads
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
body: |
bb.0:
$img0 = COPY $x
STAfi $img0, %stack.0, 0, implicit-def dead $a
$a = LDAfi %stack.0, 0
STA_DP 224, implicit $a
$a = LDAi16imm 1
STA_DP 228, implicit $a
$a = LDAfi %stack.0, 0
STA_DP 226, implicit $a
RTL
...
---
# CHECK-LABEL: name: never_read
# CHECK: $a = LDAi16imm 5
# CHECK-NOT: STAfi
# CHECK: RTL
name: never_read
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
body: |
bb.0:
$a = LDAi16imm 5
STAfi $a, %stack.0, 0, implicit-def $a
RTL
...
---
# CHECK-LABEL: name: addr_taken
# CHECK: STAfi $a, %stack.0, 0
# CHECK: $a = ADDframe %stack.0, 0
name: addr_taken
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
body: |
bb.0:
$a = LDAi16imm 5
STAfi $a, %stack.0, 0, implicit-def $a
$a = ADDframe %stack.0, 0
RTL
...

87
src/llvm/test/CodeGen/W65816/spill-a-live.mir Normal file
View file

@ -0,0 +1,87 @@
# Pin: eliminateFrameIndex moves a non-A STAfi (a spill of an IMG word, X
# or Y) through A and keeps A around it -- PHA ... PLA, the store 2 bytes
# further up -- only when A's value is still read afterwards (aLiveAfter).
# Every such spill used to pay the PHA / PLA; the reloads, the other half
# of the change, kept none and clobbered A where the allocator had a value
# of its own in it (wide32-spill-subreg.ll pins that through CoreMark's
# core_list_mergesort: a non-A LDAfi is not legal MIR input here).
#
# spill_a_dead: A rewritten first: `lda dp ; sta d,s`, no bracket.
# spill_a_live: A read after: bracketed, offset + 2.
# nona_store_wrap: the store sets N and Z through A, so between a compare
# and its BNE it is wrapped in PHP / PLP (the wrap pass
# took every store for flag-preserving).
#
# RUN: llc -mtriple=w65816 -run-pass=prologepilog -o - %s | FileCheck %s
# RUN: llc -mtriple=w65816 -run-pass=w65816-stack-slot-cleanup -o - %s | FileCheck %s --check-prefix=WRAP
--- |
target triple = "w65816"
define void @spill_a_dead() { ret void }
define void @spill_a_live() { ret void }
define void @nona_store_wrap() { ret void }
...
---
# CHECK-LABEL: name: spill_a_dead
# CHECK: TCS
# CHECK-NOT: PHA
# CHECK: LDA_DP 210
# CHECK-NEXT: STA_StackRel 9
# CHECK-NEXT: $a = LDAi16imm 0
name: spill_a_dead
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 8, alignment: 1 }
body: |
bb.0:
STAfi $img1, %stack.0, 0, implicit-def $a
$a = LDAi16imm 0
STA_DP 224, implicit $a
RTL
...
---
# CHECK-LABEL: name: spill_a_live
# CHECK: PHA
# CHECK-NEXT: LDA_DP 210
# CHECK-NEXT: STA_StackRel 11
# CHECK-NEXT: PLA
name: spill_a_live
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 8, alignment: 1 }
body: |
bb.0:
$a = LDAi16imm 5
STAfi $img1, %stack.0, 0, implicit-def $a
STA_DP 224, implicit $a
RTL
...
---
# WRAP-LABEL: name: nona_store_wrap
# WRAP: CMPi16imm killed $a, 3, implicit-def $p
# WRAP-NEXT: PHP
# WRAP-NEXT: STAfi $img1, %stack.1, 0
# WRAP-NEXT: PLP
# WRAP-NEXT: BNE %bb.1, implicit $p
name: nona_store_wrap
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 2, alignment: 1 }
body: |
bb.0:
successors: %bb.1, %bb.2
$a = LDAfi %stack.0, 0
CMPi16imm killed $a, 3, implicit-def $p
STAfi $img1, %stack.1, 0, implicit-def $a
BNE %bb.1, implicit $p
BRA %bb.2
bb.1:
liveins: $x
$a = LDAfi %stack.1, 0
RTL
bb.2:
liveins: $x
RTL
...

74
src/llvm/test/CodeGen/W65816/spill-stafi-no-a-def.mir Normal file
View file

@ -0,0 +1,74 @@
# Pin: a spill STAfi the register allocator inserts names no def of A.
# STAfi carries Defs = [A] as an over-approximation (it never changes A:
# eliminateFrameIndex moves an IMG / X / Y source through A inside PHA /
# PLA when A is still read), but the spiller places spills without
# checking A's interference. Here ten values outlive a call that keeps
# only IMG8..15, so two go to the stack through $img0 while the incoming
# A is still live (it is copied again right after the spill). With the
# implicit def on the spill, A looked clobbered in the middle of its own
# live range, and post-RA MachineCopyPropagation deleted an earlier
# `$a = COPY` it took as overwritten -- snprintf's format() lost its
# format pointer's bank and stopped at the first conversion.
#
# RUN: llc -mtriple=w65816 -run-pass=greedy,virtregrewriter -o - %s | FileCheck %s
# CHECK-LABEL: name: f
# CHECK: $img0 = COPY $a
# CHECK-NEXT: STAfi killed $img0, %stack.{{[0-9]+}}, 0{{$}}
# CHECK-NEXT: $img0 = COPY killed $a
# CHECK-NEXT: STAfi killed $img0, %stack.{{[0-9]+}}, 0{{$}}
# CHECK: $img0 = LDAfi %stack.{{[0-9]+}}, 0
--- |
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S8"
target triple = "w65816"
declare void @ext()
define i16 @f(i16 %a) { ret i16 %a }
...
---
name: f
tracksRegLiveness: true
registers:
- { id: 0, class: acc16 }
- { id: 1, class: acc16 }
- { id: 10, class: img16 }
- { id: 11, class: img16 }
- { id: 12, class: img16 }
- { id: 13, class: img16 }
- { id: 14, class: img16 }
- { id: 15, class: img16 }
- { id: 16, class: img16 }
- { id: 17, class: img16 }
- { id: 18, class: img16 }
- { id: 19, class: img16 }
stack:
- { id: 0, size: 2, alignment: 2 }
body: |
bb.0:
liveins: $a
%0:acc16 = COPY $a
%10:img16 = COPY %0
%11:img16 = COPY %0
%12:img16 = COPY %0
%13:img16 = COPY %0
%14:img16 = COPY %0
%15:img16 = COPY %0
%16:img16 = COPY %0
%17:img16 = COPY %0
%18:img16 = COPY %0
%19:img16 = COPY %0
ADJCALLSTACKDOWN 0, 0, implicit-def dead $sp, implicit $sp
JSLpseudo32 @ext, implicit-def dead $a, implicit-def dead $x, implicit-def dead $y, implicit-def dead $dpf0, implicit-def dead $img0, implicit-def dead $img1, implicit-def dead $img2, implicit-def dead $img3, implicit-def dead $img4, implicit-def dead $img5, implicit-def dead $img6, implicit-def dead $img7
ADJCALLSTACKUP 0, 0, implicit-def dead $sp, implicit $sp
STAfi %10, %stack.0, 0, implicit-def dead $a
STAfi %11, %stack.0, 0, implicit-def dead $a
STAfi %12, %stack.0, 0, implicit-def dead $a
STAfi %13, %stack.0, 0, implicit-def dead $a
STAfi %14, %stack.0, 0, implicit-def dead $a
STAfi %15, %stack.0, 0, implicit-def dead $a
STAfi %16, %stack.0, 0, implicit-def dead $a
STAfi %17, %stack.0, 0, implicit-def dead $a
STAfi %18, %stack.0, 0, implicit-def dead $a
STAfi %19, %stack.0, 0, implicit-def dead $a
$a = COPY %0
RTL implicit $a
...

199
src/llvm/test/CodeGen/W65816/spill-to-x-merged-slot.mir Normal file
View file

@ -0,0 +1,199 @@
# Pin: W65816SpillToX bridges a spill through X (TAX ... TXA) when stack-
# slot coloring has given the slot to other values too, as long as the
# stored value is dead after the reload.
#
# merged_dead: `for (..) gOut = gV; for (..) gOut = gW;`. Coloring shares
# slot 1 between loop 1's counter and gW's copy, and slot 0
# between n and loop 2's counter. The pass used to bail on
# any reference outside the STA..LDA window, so both counters
# went through memory (sta/lda, 8 cycles a trip, after
# StackRelToImg) instead of tax/txa (4). Both are dead after
# their reloads: the next reference on every path is a store.
# merged_live: the reload is followed by another read of the slot in a
# successor, so the store must stay.
# x_live_out: X, set in the block, is live into the successor; the TAX
# would clobber it (only reads within the block were checked).
# php_window: a PHP between the store and the reload would save the TAX's
# N/Z (the store sets none) in place of the test it brackets.
#
# Shared slots are bridged only by the run after StackSlotCleanup (which
# deletes many such pairs outright, a better result than a bridge); the
# option makes a lone -run-pass behave like that run. The first run keeps
# merged_dead's spills (FIRST).
#
# RUN: llc -mtriple=w65816 -run-pass=w65816-spill-to-x -w65816-spill-to-x-shared-slots -o - %s | FileCheck %s
# RUN: llc -mtriple=w65816 -run-pass=w65816-spill-to-x -o - %s | FileCheck %s --check-prefix=FIRST
--- |
target triple = "w65816"
@gV = global i16 0
@gW = global i16 0
@gOut = global i16 0
define void @merged_dead() { ret void }
define void @merged_live() { ret void }
define void @x_live_out() { ret void }
define void @php_window() { ret void }
...
---
# FIRST-LABEL: name: merged_dead
# FIRST: bb.3:
# FIRST-NOT: TAX
# FIRST: bb.4:
# FIRST-NOT: TAX
# FIRST: RTL
# CHECK-LABEL: name: merged_dead
# CHECK: bb.3:
# CHECK: TAX
# CHECK-NEXT: $a = LDAfi %stack.2, 0
# CHECK-NEXT: STAabs killed $a, @gOut
# CHECK-NEXT: TXA
# CHECK: bb.4:
# CHECK: TAX
# CHECK-NEXT: $a = LDAfi %stack.1, 0
# CHECK-NEXT: STAabs killed $a, @gOut
# CHECK-NEXT: TXA
name: merged_dead
tracksRegLiveness: true
liveins:
- { reg: '$a' }
stack:
- { id: 0, type: spill-slot, size: 2, alignment: 2 }
- { id: 1, type: spill-slot, size: 2, alignment: 2 }
- { id: 2, type: spill-slot, size: 2, alignment: 2 }
body: |
bb.0:
successors: %bb.1, %bb.5
liveins: $a
STAfi $a, %stack.0, 0, implicit-def $a
$a = LDAfi %stack.0, 0
$a = EORi16imm killed $a, -32768
CMPi16imm killed $a, -32767, implicit-def $p
dead $a = LDAfi %stack.0, 0
BCC %bb.5, implicit $p
BRA %bb.1
bb.1:
successors: %bb.3
$a = LDAabs @gV
STAfi killed $a, %stack.2, 0, implicit-def $a
$a = LDAfi %stack.0, 0
BRA %bb.3
bb.2:
successors: %bb.4
$a = LDAabs @gW
STAfi killed $a, %stack.1, 0, implicit-def $a
$a = LDAfi %stack.0, 0
BRA %bb.4
bb.3:
successors: %bb.2, %bb.3
liveins: $a
STAfi killed $a, %stack.1, 0, implicit-def $a
$a = LDAfi %stack.2, 0
STAabs killed $a, @gOut
$a = LDAfi %stack.1, 0
$a = DEA_PSEUDO killed $a
CMPi16imm $a, 0, implicit-def $p
BEQ %bb.2, implicit $p
BRA %bb.3
bb.4:
successors: %bb.5, %bb.4
liveins: $a
STAfi killed $a, %stack.0, 0, implicit-def $a
$a = LDAfi %stack.1, 0
STAabs killed $a, @gOut
$a = LDAfi %stack.0, 0
$a = DEA_PSEUDO killed $a
CMPi16imm $a, 0, implicit-def $p
BNE %bb.4, implicit $p
BRA %bb.5
bb.5:
RTL
...
---
# CHECK-LABEL: name: merged_live
# CHECK: bb.0:
# CHECK-NOT: TAX
# CHECK: STAfi $a, %stack.0, 0
# CHECK: $a = LDAfi %stack.0, 0
# CHECK: bb.1:
name: merged_live
tracksRegLiveness: true
liveins:
- { reg: '$a' }
stack:
- { id: 0, type: spill-slot, size: 2, alignment: 2 }
body: |
bb.0:
successors: %bb.1
liveins: $a
STAfi $a, %stack.0, 0, implicit-def $a
$a = LDAabs @gV
STAabs killed $a, @gOut
$a = LDAfi %stack.0, 0
STAabs killed $a, @gW
BRA %bb.1
bb.1:
$a = LDAfi %stack.0, 0
STAabs killed $a, @gOut
RTL
...
---
# CHECK-LABEL: name: x_live_out
# CHECK: bb.0:
# CHECK-NOT: TAX
# CHECK: STAfi $a, %stack.0, 0
# CHECK: bb.1:
name: x_live_out
tracksRegLiveness: true
liveins:
- { reg: '$a' }
stack:
- { id: 0, type: spill-slot, size: 2, alignment: 2 }
body: |
bb.0:
successors: %bb.1
liveins: $a
$x = LDXi16imm 7
STAfi $a, %stack.0, 0, implicit-def $a
$a = LDAabs @gV
STAabs killed $a, @gOut
$a = LDAfi %stack.0, 0
BRA %bb.1
bb.1:
liveins: $a, $x
STAabs killed $a, @gW
TXA implicit-def $a, implicit $x
STAabs killed $a, @gOut
RTL
...
---
# CHECK-LABEL: name: php_window
# CHECK: bb.0:
# CHECK-NOT: TAX
# CHECK: STAfi $a, %stack.0, 0
# CHECK: RTL
name: php_window
tracksRegLiveness: true
liveins:
- { reg: '$a' }
stack:
- { id: 0, type: spill-slot, size: 2, alignment: 2 }
body: |
bb.0:
liveins: $a
STAfi $a, %stack.0, 0, implicit-def $a
$a = LDAabs @gV
CMPi16imm killed $a, 5, implicit-def $p
PHP
$a = LDAabs @gW
STAabs killed $a, @gOut
PLP
$a = LDAfi %stack.0, 0
STAabs killed $a, @gW
RTL
...

83
src/llvm/test/CodeGen/W65816/split-loop-exits.ll Normal file
View file

@ -0,0 +1,83 @@
; Pin: W65816SplitLoopExits gives a critical edge out of a loop into a
; block with PHIs a block of its own, so PHI elimination's copies run once
; per exit instead of on every trip. LLVM does that split in PHI
; elimination only through MachineBasicBlock::SplitCriticalEdge, which
; needs analyzeBranch to understand the conditional branch -- this target
; keeps conditional branches opaque. A search loop's two exits (found:
; return p; end of list: return NULL) copied p into the return value's
; words and zero into them on every trip (CoreMark's core_list_find:
; -27% with the copies gone).
;
; RUN: llc -mtriple=w65816 -O2 -print-after=w65816-split-loop-exits -o /dev/null < %s 2>&1 | FileCheck %s --check-prefix=IR
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
%struct.node = type { ptr, ptr }
; IR-LABEL: define {{.*}}@find(
; IR: br i1 %cmp.not, label %land.rhs.while.end_crit_edge, label %while.body
; IR: land.rhs.while.end_crit_edge:
; IR-NEXT: br label %while.end
; IR: br i1 %tobool.not, label %while.body.while.end_crit_edge, label %land.rhs
; IR: while.body.while.end_crit_edge:
; IR-NEXT: br label %while.end
; IR: %r = phi ptr [ null, %entry ], [ %list.addr.05, %land.rhs.while.end_crit_edge ], [ null, %while.body.while.end_crit_edge ]
; The loop keeps neither the return value's copy of p nor its zero; the
; copies sit in the exit blocks.
; CHECK-LABEL: find:
; CHECK: Loop Header
; CHECK-NOT: lda #0x0
; CHECK: bne .LBB0_[[LOOP:[0-9]+]]
; CHECK-NEXT: ; %bb.{{[0-9]+}}: ; %while.body.while.end_crit_edge
; CHECK-NEXT: lda #0x0
define ptr @find(ptr %list, i16 %want) {
entry:
%tobool.not4 = icmp eq ptr %list, null
br i1 %tobool.not4, label %while.end, label %land.rhs
land.rhs:
%list.addr.05 = phi ptr [ %next, %while.body ], [ %list, %entry ]
%info = getelementptr inbounds nuw i8, ptr %list.addr.05, i32 4
%0 = load ptr, ptr %info, align 2
%idx = getelementptr inbounds nuw i8, ptr %0, i32 2
%1 = load i16, ptr %idx, align 2
%cmp.not = icmp eq i16 %1, %want
br i1 %cmp.not, label %while.end, label %while.body
while.body:
%next = load ptr, ptr %list.addr.05, align 2
%tobool.not = icmp eq ptr %next, null
br i1 %tobool.not, label %while.end, label %land.rhs
while.end:
%r = phi ptr [ null, %entry ], [ null, %while.body ], [ %list.addr.05, %land.rhs ]
ret ptr %r
}
; A running total leaves the loop as the value the loop carries: that
; copy coalesces into the loop's own register wherever it sits, so the
; edge stays as it is (splitting it cost sumOfSquares a store a trip).
; IR-LABEL: define {{.*}}@total(
; IR-NOT: crit_edge
; IR: ret i32
define i32 @total(i16 %n) {
entry:
%z = icmp eq i16 %n, 0
br i1 %z, label %done, label %loop
loop:
%i = phi i16 [ 1, %entry ], [ %i1, %loop ]
%t = phi i32 [ 0, %entry ], [ %t1, %loop ]
%w = zext i16 %i to i32
%sq = mul nuw i32 %w, %w
%t1 = add i32 %t, %sq
%i1 = add i16 %i, 1
%c = icmp ugt i16 %i1, %n
br i1 %c, label %done, label %loop
done:
%r = phi i32 [ 0, %entry ], [ %t1, %loop ]
ret i32 %r
}

4
src/llvm/test/CodeGen/W65816/srl8a-flag-corrupt-wrap.mir
View file

@ -23,6 +23,9 @@
# The wrapped reload keeps its offset: eliminateFrameIndex adds the byte
# PHP pushed (php-wrap-frame-index.mir).
#
# srl8a reads %stack.2 after the branch so its store survives the pass's
# never-read slot sweep.
#
# The branch targets take X live in, so the wrap cannot be replaced by the
# TAX ... CPX #k form that StackSlotCleanup uses when X is free (pinned in
# cmp-via-x.mir); these cases keep testing the PHP / PLP wrap itself.
@ -61,6 +64,7 @@ body: |
BRA %bb.2
bb.1:
liveins: $x
$a = LDAfi %stack.2, 0
RTL
bb.2:
liveins: $x

132
src/llvm/test/CodeGen/W65816/stack-release.ll Normal file
View file

@ -0,0 +1,132 @@
; Pin: stack releases (a call's argument cleanup, the epilogue's frame
; release) and the prologue's frame allocation take the cheapest form for
; what must survive them, by W65816Helpers::cycleByteCost (a cycle is worth
; two bytes): N/2 PLY (5 cycles each), TSC/CLC/ADC/TCS (9) when A is free,
; that inside TAY/TYA (13) when A holds a value. The old fixed thresholds
; pulled up to seven PLYs after a call (35 cycles) and kept A through
; TAY/TYA in every epilogue, void ones included.
;
; W65816ReturnPopMerge folds the releases on a path into a return into one:
; `tailish` pays 9 cycles where it paid 18 (and, before the cost model, 15
; plus 15 in PLYs).
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
; RUN: env W65816_DISABLE_RETURN_POP_MERGE=1 llc -mtriple=w65816 -O2 < %s | FileCheck %s --check-prefix=NOMERGE
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S16"
declare void @take3(i16, i16, i16, i16)
declare i16 @get3(i16, i16, i16, i16)
declare i16 @get4(i16, i16, i16, i16, i16)
declare void @use(ptr)
@sink = global i16 0
; A void call: A is dead after it, so three words go in one TSC/CLC/ADC/TCS.
; CHECK-LABEL: voidCall:
; CHECK: jsl take3
; CHECK-NEXT: tsc
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0x6
; CHECK-NEXT: tcs
; CHECK-NEXT: lda #0x1
define void @voidCall(i16 %a, i16 %b) {
entry:
call void @take3(i16 %a, i16 %b, i16 %a, i16 %b)
store volatile i16 1, ptr @sink
ret void
}
; The call's value is used: three words are three PLYs (15 cycles, 3 bytes
; against 13 and 8) ...
; CHECK-LABEL: valCall3:
; CHECK: jsl get3
; CHECK-NEXT: ply
; CHECK-NEXT: ply
; CHECK-NEXT: ply
; CHECK-NEXT: sta sink
define i16 @valCall3(i16 %a, i16 %b) {
entry:
%r = call i16 @get3(i16 %a, i16 %b, i16 %a, i16 %b)
store volatile i16 %r, ptr @sink
ret i16 %r
}
; ... four go through TAY ... TYA.
; CHECK-LABEL: valCall4:
; CHECK: jsl get4
; CHECK-NEXT: tay
; CHECK-NEXT: tsc
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0x8
; CHECK-NEXT: tcs
; CHECK-NEXT: tya
define i16 @valCall4(i16 %a, i16 %b) {
entry:
%r = call i16 @get4(i16 %a, i16 %b, i16 %a, i16 %b, i16 %a)
store volatile i16 %r, ptr @sink
ret i16 %r
}
; A void function's epilogue keeps nothing in A; its argument does come in
; in A, so the prologue keeps it.
; CHECK-LABEL: voidFrame:
; CHECK: tay
; CHECK-NEXT: tsc
; CHECK-NEXT: sec
; CHECK-NEXT: sbc #0xa
; CHECK-NEXT: tcs
; CHECK-NEXT: tya
; CHECK: sta sink
; CHECK-NEXT: tsc
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0xa
; CHECK-NEXT: tcs
; CHECK-NEXT: rtl
define void @voidFrame(i16 %a) {
entry:
%buf = alloca [8 x i8], align 1
call void @use(ptr %buf)
store volatile i16 %a, ptr @sink
ret void
}
; Call, then return: one release for the arguments and the frame.
; CHECK-LABEL: tailish:
; CHECK: jsl take3
; CHECK-NEXT: tsc
; CHECK-NEXT: clc
; CHECK-NEXT: adc #0x12
; CHECK-NEXT: tcs
; CHECK-NEXT: rtl
; NOMERGE-LABEL: tailish:
; NOMERGE: jsl take3
; NOMERGE-NEXT: tsc
; NOMERGE-NEXT: clc
; NOMERGE-NEXT: adc #0x6
; NOMERGE-NEXT: tcs
; NOMERGE-NEXT: tsc
; NOMERGE-NEXT: clc
; NOMERGE-NEXT: adc #0xc
; NOMERGE-NEXT: tcs
; NOMERGE-NEXT: rtl
define void @tailish(i16 %a) {
entry:
%buf = alloca [6 x i8], align 1
store volatile i16 %a, ptr %buf
call void @take3(i16 %a, i16 %a, i16 %a, i16 %a)
ret void
}
; No argument in A: no TAY / TYA in the prologue either.
; CHECK-LABEL: noArgFrame:
; CHECK: rep #0x30
; CHECK-NEXT: tsc
; CHECK-NEXT: sec
; CHECK-NEXT: sbc #0x8
; CHECK-NEXT: tcs
define void @noArgFrame() {
entry:
%buf = alloca [8 x i8], align 1
call void @use(ptr %buf)
ret void
}

52
src/llvm/test/CodeGen/W65816/stackrel-to-img-arg-slot.ll Normal file
View file

@ -0,0 +1,52 @@
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
;
; W65816StackRelToImg must not promote the caller's stack arguments to DP
; words. It keys slots by "logical" offsets that count the PHAs of a
; small-frame prologue, so there the arguments start at logical offset 4,
; not at FrameSize + 4 as with a TCS prologue. Measuring from the body SP
; in both cases let this function (libc's ungetc: a 4-byte PHA frame, the
; FILE pointer read four times) move its pointer argument to $D0/$D2 as a
; far-pointer home that nothing ever stored, then write through it.
;
; The pointer must be read from its stack slots (0x8,s / 0xa,s here) every
; time, and no DP word may stand in for it.
target triple = "w65816"
define i16 @ungetcLike(i16 %c, ptr %stream) {
; CHECK-LABEL: ungetcLike:
; CHECK: pha
; CHECK-NEXT: pha
; CHECK-NOT: tcs
; CHECK: lda 0x8, s
; CHECK-NEXT: ora 0xa, s
; CHECK: lda 0x8, s
; CHECK-NEXT: sta 0xe0
; CHECK-NEXT: lda 0xa, s
; CHECK-NEXT: sta 0xe2
; CHECK: lda [0xe0], y
; CHECK-NOT: [0xd{{[0-9a-f]}}], y
; CHECK: rtl
entry:
%tobool = icmp eq ptr %stream, null
%cmp = icmp slt i16 %c, 0
%or.cond = or i1 %cmp, %tobool
br i1 %or.cond, label %return, label %if.end
if.end:
%unget = getelementptr inbounds nuw i8, ptr %stream, i32 22
%0 = load i16, ptr %unget, align 2
%cmp1 = icmp sgt i16 %0, -1
br i1 %cmp1, label %return, label %if.end3
if.end3:
%and = and i16 %c, 255
store i16 %and, ptr %unget, align 2
%eof = getelementptr inbounds nuw i8, ptr %stream, i32 2
store i8 0, ptr %eof, align 2
br label %return
return:
%retval.0 = phi i16 [ %and, %if.end3 ], [ -1, %entry ], [ -1, %if.end ]
ret i16 %retval.0
}

15
src/llvm/test/CodeGen/W65816/stafi-far-nona-pha-bracket.mir
View file

@ -7,7 +7,7 @@
# invalidating its "A still holds slot N" fact, on the strength of that bracket.
# The far path used to emit a bare `LDA dp`, so an IMG-source STAfi in a >0xFF
# frame silently clobbered A and the cleanup then deleted the FOLLOWING reload.
# Live symptom: the LDAptr32S inserter built its far pointer with the pointer's
# Live symptom: the LDAptr32 inserter built its far pointer with the pointer's
# BANK byte in the offset half, so `lda [$E0],y` read a stray address -- JoeyLib's
# Space Taxi read a nonzero byte for a zero struct field and never ran its
# physics.
@ -17,7 +17,14 @@
# far_acc: offset > 0xFF, A source -> NO bracket (A is the source; the
# +7 cycles would be pure cost).
#
# RUN: llc -mtriple=w65816 -run-pass=prologepilog -o - %s | FileCheck %s
# The returns read A (a value-returning function), so A is live across the
# store; the bracket is left out where nothing reads A after it.
#
# The frame layout is pinned with W65816_DISABLE_FRAME_ORDER: the far cases
# need their 2-byte slot above the big array, where creation order puts it
# and the access-density order would not.
#
# RUN: env W65816_DISABLE_FRAME_ORDER=1 llc -mtriple=w65816 -run-pass=prologepilog -o - %s | FileCheck %s
--- |
target triple = "w65816"
define void @far_img() { ret void }
@ -41,7 +48,7 @@ stack:
body: |
bb.0:
STAfi $img8, %stack.0, 0, implicit-def $a
RTL
RTL implicit $a
...
---
# CHECK-LABEL: name: near_img
@ -57,7 +64,7 @@ stack:
body: |
bb.0:
STAfi $img8, %stack.0, 0, implicit-def $a
RTL
RTL implicit $a
...
---
# A is already the source, so the far store needs no bracket at all.

119
src/llvm/test/CodeGen/W65816/store-flag-preserving-wrap.mir Normal file
View file

@ -0,0 +1,119 @@
# Pin: every store counts as flag-PRESERVING in the PHP/PLP wrap
# (W65816StackSlotCleanup Pass -2.5): the backward walk from a branch steps
# over it to the real test, and wraps every corrupter in between.
#
# A store sets no flag (the byte-store pseudos' SEP / REP touch only the M
# bit), but only the 16-bit STA / STX / STY forms were listed. Any other
# store fell to isFlagDefining's default and was taken as THE test, so the
# walk stopped at it and left the corrupters before it unwrapped:
#
# cmp #$41C6 ; xba ; and #$ff ; sep ; sta $71 ; rep ; php ; lda #0 ; plp ; beq
#
# -- llvm816's demos/randProbe.c: `(r1 == 0x41C6u) ? 0 : 0x99` next to a
# byte store of r1 >> 8. The BEQ read the AND's Z, not the compare's.
#
# byte_store_long: the randProbe shape (SRL8A, STA8long, LDAi8imm).
# byte_store_ptr: a far-pointer byte store (STB_DPIndLongY) between a
# reload and the branch.
# stz: an STZ_Abs between the corrupter and the branch (it stays
# outside the wrap, which ends after the last corrupter).
#
# RUN: llc -mtriple=w65816 -run-pass=w65816-stack-slot-cleanup -o - %s | FileCheck %s
--- |
target triple = "w65816"
@g = global i16 0
define void @byte_store_long() { ret void }
define void @byte_store_ptr() { ret void }
define void @stz() { ret void }
...
---
# CHECK-LABEL: name: byte_store_long
# CHECK: CMPi16imm $a, 16838, implicit-def $p
# CHECK-NEXT: PHP
# CHECK-NEXT: $a = SRL8A killed $a
# CHECK-NEXT: STA8long killed $a, 113
# CHECK-NEXT: $a = LDAi8imm 0
# CHECK-NEXT: PLP
# CHECK-NEXT: BEQ %bb.2, implicit $p
name: byte_store_long
tracksRegLiveness: false
body: |
bb.0:
successors: %bb.1, %bb.2
liveins: $a
STA8long $a, 112
CMPi16imm $a, 16838, implicit-def $p
$a = SRL8A killed $a
STA8long killed $a, 113
$a = LDAi8imm 0
BEQ %bb.2, implicit $p
BRA %bb.1
bb.1:
liveins: $x
RTL
bb.2:
liveins: $x
RTL
...
---
# CHECK-LABEL: name: byte_store_ptr
# CHECK: CMPi16imm killed $a, 7, implicit-def $p
# CHECK-NEXT: PHP
# CHECK-NEXT: $a = LDAfi %stack.1, 0
# CHECK-NEXT: LDY_Imm16 2
# CHECK-NEXT: STB_DPIndLongY 224
# CHECK-NEXT: $a = LDAi16imm 1
# CHECK-NEXT: PLP
# CHECK-NEXT: BNE %bb.1, implicit $p
name: byte_store_ptr
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 2, alignment: 1 }
body: |
bb.0:
successors: %bb.1, %bb.2
$a = LDAfi %stack.0, 0
CMPi16imm killed $a, 7, implicit-def $p
$a = LDAfi %stack.1, 0
LDY_Imm16 2, implicit-def $y
STB_DPIndLongY 224, implicit-def $p, implicit $a, implicit $y
$a = LDAi16imm 1
BNE %bb.1, implicit $p
BRA %bb.2
bb.1:
liveins: $x
RTL
bb.2:
liveins: $x
RTL
...
---
# CHECK-LABEL: name: stz
# CHECK: CMPi16imm killed $a, 9, implicit-def $p
# CHECK-NEXT: PHP
# CHECK-NEXT: $a = LDAfi %stack.1, 0
# CHECK-NEXT: PLP
# CHECK-NEXT: STZ_Abs @g
# CHECK-NEXT: BEQ %bb.1, implicit $p
name: stz
tracksRegLiveness: false
stack:
- { id: 0, size: 2, alignment: 1 }
- { id: 1, size: 2, alignment: 1 }
body: |
bb.0:
successors: %bb.1, %bb.2
$a = LDAfi %stack.0, 0
CMPi16imm killed $a, 9, implicit-def $p
$a = LDAfi %stack.1, 0
STZ_Abs @g
BEQ %bb.1, implicit $p
BRA %bb.2
bb.1:
liveins: $x
RTL
bb.2:
liveins: $x
RTL
...

65
src/llvm/test/CodeGen/W65816/weak-fn-address-bank.ll Normal file
View file

@ -0,0 +1,65 @@
; Pin: the bank half of `&sym` for an external-weak symbol is the program
; bank when the symbol is linked and 0 when it is not, decided at run time
; from the low half the linker resolved (0 for an undefined weak):
;
; lda #weakFn ; beq L ; lda $BE ; L:
;
; It used to be a literal 0, which made an undefined weak a null pointer
; but sent every call through a stored pointer to a DEFINED weak function
; to bank 0: the runtime's `&gsosFlush` hook crashed under GS/OS, where the
; program does not run in bank 0. Other symbols -- a weak definition, a
; strong declaration, a local function -- read the program bank from $BE.
;
; RUN: llc -mtriple=w65816 -O2 < %s | FileCheck %s
target triple = "w65816"
@gWeak = global ptr null
@gWeakDef = global ptr null
@gStrong = global ptr null
@gStatic = global ptr null
declare extern_weak void @weakFn()
declare void @strongFn()
define weak void @weakDef() {
ret void
}
define internal void @staticFn() {
ret void
}
define void @take() {
; CHECK-LABEL: take:
; CHECK: lda #weakFn
; CHECK-NEXT: sta gWeak{{$}}
; CHECK: lda #weakFn
; CHECK-NEXT: beq [[NULL:.Ltmp[0-9]+]]
; CHECK-NEXT: lda 0xbe
; CHECK-NEXT: [[NULL]]:
; CHECK-NEXT: sta gWeak+2
; CHECK: rtl
store ptr @weakFn, ptr @gWeak
store ptr @weakDef, ptr @gWeakDef
store ptr @strongFn, ptr @gStrong
store ptr @staticFn, ptr @gStatic
ret void
}
; Each of the other three gets its bank from $BE alone.
define void @takeOthers() {
; CHECK-LABEL: takeOthers:
; CHECK-NOT: beq
; CHECK-NOT: lda #0x0
; CHECK-DAG: sta gWeakDef+2
; CHECK-DAG: sta gStrong+2
; CHECK-DAG: sta gStatic+2
; CHECK-NOT: beq
; CHECK-NOT: lda #0x0
; CHECK: rtl
store ptr @weakDef, ptr @gWeakDef
store ptr @strongFn, ptr @gStrong
store ptr @staticFn, ptr @gStatic
ret void
}

156
src/llvm/test/CodeGen/W65816/wide32-spill-subreg.ll Normal file
View file

@ -0,0 +1,156 @@
; Pin: a VIRTUAL Wide32 register (one W65816LowerWide32 could not split:
; the pointer PHIs of a loop nest) spills and reloads both halves, by
; sub-register. load/storeRegToStackSlot took a virtual register for the
; unpaired-physreg case and moved the low half only (`STAfi $a` /
; `%n:wide32 = COPY $a`), so a reloaded pointer kept whatever bank its
; register pair last held, and the reload went through A behind the
; allocator's back. CoreMark's core_list_mergesort (this function) walked
; off its list once its loop exits got blocks of their own; with the old
; code that only happened to work.
;
; The reload of a pointer into an IMG pair goes through A, which it keeps
; (PHA ... PLA) where A may still be read after it: spill-a-live.mir pins
; that.
;
; RUN: llc -mtriple=w65816 -O2 -stop-after=greedy -o - < %s | FileCheck %s
; CHECK-LABEL: name: core_list_mergesort
; CHECK-NOT: wide32 = COPY $a
; CHECK: STAfi %{{[0-9]+}}.sub_lo, %stack.[[S:[0-9]+]], 0
; CHECK-NEXT: STAfi %{{[0-9]+}}.sub_hi, %stack.[[S]], 2
; CHECK-NOT: wide32 = COPY $a
; CHECK: undef %[[R:[0-9]+]].sub_hi:wide32 = LDAfi %stack.{{[0-9]+}}, 2
; CHECK-NEXT: %[[R]].sub_lo:wide32 = LDAfi %stack.{{[0-9]+}}, 0
; CHECK-NOT: wide32 = COPY $a
target datalayout = "e-m:e-p:32:16-i16:16-i32:16-i64:16-f32:16-f64:16-a:8-n8:16-S8"
target triple = "w65816"
%struct.list_data_s = type { i16, i16 }
define dso_local ptr @core_list_mergesort(ptr noundef %list, ptr noundef readonly captures(none) %cmp, ptr noundef %res) local_unnamed_addr {
entry:
br label %while.cond
while.cond: ; preds = %while.end37, %entry
%list.addr.0 = phi ptr [ %list, %entry ], [ %list.addr.2.ph, %while.end37 ]
%insize.0 = phi i32 [ 1, %entry ], [ %mul, %while.end37 ]
call void @llvm.assume(i1 true) [ "nonnull"(ptr %list.addr.0) ]
%cmp374 = icmp sgt i32 %insize.0, 0
br label %while.body2
while.cond1.loopexit: ; preds = %lor.rhs
%tobool.not = icmp eq ptr %q.2, null
br i1 %tobool.not, label %while.end37, label %while.body2
while.body2: ; preds = %while.cond, %while.cond1.loopexit
%nmerges.085 = phi i32 [ 0, %while.cond ], [ %inc, %while.cond1.loopexit ]
%tail.084 = phi ptr [ null, %while.cond ], [ %tail.1, %while.cond1.loopexit ]
%p.083 = phi ptr [ %list.addr.0, %while.cond ], [ %q.2, %while.cond1.loopexit ]
%list.addr.182 = phi ptr [ null, %while.cond ], [ %list.addr.2.ph, %while.cond1.loopexit ]
%inc = add nuw nsw i32 %nmerges.085, 1
br i1 %cmp374, label %for.body, label %while.cond7.preheader
for.cond: ; preds = %for.body
%cmp3 = icmp slt i32 %inc6, %insize.0
br i1 %cmp3, label %for.body, label %while.cond7.preheader
for.body: ; preds = %while.body2, %for.cond
%i.077 = phi i32 [ %inc6, %for.cond ], [ 0, %while.body2 ]
%q.075 = phi ptr [ %0, %for.cond ], [ %p.083, %while.body2 ]
%inc6 = add nuw nsw i32 %i.077, 1
%0 = load ptr, ptr %q.075, align 2
%tobool5.not = icmp eq ptr %0, null
br i1 %tobool5.not, label %while.cond7.preheader, label %for.cond
while.cond7.preheader: ; preds = %for.cond, %for.body, %while.body2
%q.2.ph = phi ptr [ %p.083, %while.body2 ], [ %0, %for.cond ], [ null, %for.body ]
%psize.2.ph = phi i32 [ 0, %while.body2 ], [ %inc6, %for.body ], [ %inc6, %for.cond ]
br label %while.cond7.outer
while.cond7.outer: ; preds = %if.end31, %while.cond7.preheader
%list.addr.2.ph = phi ptr [ %list.addr.182, %while.cond7.preheader ], [ %e.0, %if.end31 ]
%p.1.ph = phi ptr [ %p.083, %while.cond7.preheader ], [ %p.2, %if.end31 ]
%q.2.ph88 = phi ptr [ %q.2.ph, %while.cond7.preheader ], [ %q.3, %if.end31 ]
%tail.1.ph = phi ptr [ %tail.084, %while.cond7.preheader ], [ %e.0, %if.end31 ]
%psize.2.ph89 = phi i32 [ %psize.2.ph, %while.cond7.preheader ], [ %psize.3, %if.end31 ]
%qsize.0.ph = phi i32 [ %insize.0, %while.cond7.preheader ], [ %qsize.1, %if.end31 ]
br label %while.cond7
while.cond7: ; preds = %while.cond7.outer, %if.then33
%p.1 = phi ptr [ %p.2, %if.then33 ], [ %p.1.ph, %while.cond7.outer ]
%q.2 = phi ptr [ %q.3, %if.then33 ], [ %q.2.ph88, %while.cond7.outer ]
%tail.1 = phi ptr [ %e.0, %if.then33 ], [ %tail.1.ph, %while.cond7.outer ]
%psize.2 = phi i32 [ %psize.3, %if.then33 ], [ %psize.2.ph89, %while.cond7.outer ]
%qsize.0 = phi i32 [ %qsize.1, %if.then33 ], [ %qsize.0.ph, %while.cond7.outer ]
%cmp8 = icmp sgt i32 %psize.2, 0
br i1 %cmp8, label %if.else, label %lor.rhs
lor.rhs: ; preds = %while.cond7
%cmp9 = icmp sgt i32 %qsize.0, 0
%tobool10 = icmp ne ptr %q.2, null
%1 = select i1 %cmp9, i1 %tobool10, i1 false
br i1 %1, label %while.body11, label %while.cond1.loopexit
while.body11: ; preds = %lor.rhs
%cmp12 = icmp eq i32 %psize.2, 0
br i1 %cmp12, label %if.then13, label %if.else
if.then13: ; preds = %while.body11
%2 = load ptr, ptr %q.2, align 2
%dec = add nsw i32 %qsize.0, -1
br label %if.end31
if.else: ; preds = %while.cond7, %while.body11
%cmp15 = icmp ne i32 %qsize.0, 0
%tobool16 = icmp ne ptr %q.2, null
%or.cond = select i1 %cmp15, i1 %tobool16, i1 false
br i1 %or.cond, label %if.else20, label %if.then17
if.then17: ; preds = %if.else
%3 = load ptr, ptr %p.1, align 2
%dec19 = add nsw i32 %psize.2, -1
br label %if.end31
if.else20: ; preds = %if.else
%info = getelementptr inbounds nuw i8, ptr %p.1, i32 4
%4 = load ptr, ptr %info, align 2
%info21 = getelementptr inbounds nuw i8, ptr %q.2, i32 4
%5 = load ptr, ptr %info21, align 2
%call = tail call i32 %cmp(ptr noundef %4, ptr noundef %5, ptr noundef %res)
%cmp22 = icmp slt i32 %call, 1
br i1 %cmp22, label %if.then23, label %if.else26
if.then23: ; preds = %if.else20
%6 = load ptr, ptr %p.1, align 2
%dec25 = add nsw i32 %psize.2, -1
br label %if.end31
if.else26: ; preds = %if.else20
%7 = load ptr, ptr %q.2, align 2
%dec28 = add nsw i32 %qsize.0, -1
br label %if.end31
if.end31: ; preds = %if.then17, %if.else26, %if.then23, %if.then13
%p.2 = phi ptr [ %p.1, %if.then13 ], [ %6, %if.then23 ], [ %p.1, %if.else26 ], [ %3, %if.then17 ]
%q.3 = phi ptr [ %2, %if.then13 ], [ %q.2, %if.then23 ], [ %7, %if.else26 ], [ %q.2, %if.then17 ]
%e.0 = phi ptr [ %q.2, %if.then13 ], [ %p.1, %if.then23 ], [ %q.2, %if.else26 ], [ %p.1, %if.then17 ]
%psize.3 = phi i32 [ 0, %if.then13 ], [ %dec25, %if.then23 ], [ %psize.2, %if.else26 ], [ %dec19, %if.then17 ]
%qsize.1 = phi i32 [ %dec, %if.then13 ], [ %qsize.0, %if.then23 ], [ %dec28, %if.else26 ], [ %qsize.0, %if.then17 ]
%tobool32.not = icmp eq ptr %tail.1, null
br i1 %tobool32.not, label %while.cond7.outer, label %if.then33
if.then33: ; preds = %if.end31
store ptr %e.0, ptr %tail.1, align 2
br label %while.cond7
while.end37: ; preds = %while.cond1.loopexit
store ptr null, ptr %tail.1, align 2
%cmp39 = icmp eq i32 %nmerges.085, 0
%mul = shl nsw i32 %insize.0, 1
br i1 %cmp39, label %if.then40, label %while.cond
if.then40: ; preds = %while.end37
ret ptr %list.addr.2.ph
}

4
tests/coremark/coreMark.manifest.json
View file

@ -7,7 +7,7 @@
"num": 1,
"name": "SEG1",
"base": "0x001000",
"size": 44791,
"size": 43655,
"bss_start": "0x00d800",
"bss_size": 2034,
"image": "coreMark.bin",
@ -17,7 +17,7 @@
"num": 2,
"name": "SEG2",
"base": "0x040000",
"size": 10368,
"size": 4571,
"bss_start": "0x000000",
"bss_size": 0,
"image": "coreMark.seg2.bin",

217
tests/fpNan/genFpCmpConst.py Normal file
View file

@ -0,0 +1,217 @@
#!/usr/bin/env python3
# genFpCmpConst.py - emits the C source of the smoke check "float/double
# compares against constants", with kExp (the expected answers) computed
# here from Python's IEEE 754 compares.
#
# python3 tests/fpNan/genFpCmpConst.py f # float
# python3 tests/fpNan/genFpCmpConst.py d 1 2 # double, first half
# python3 tests/fpNan/genFpCmpConst.py d 2 2 # double, second half
#
# PART / NPARTS split the tests so each binary stays well below the IIgs
# I/O window at $C000 (the whole double table is ~54 KB of code).
#
# W65816's combineFPSetCCConst turns a compare the operand's bit pattern
# alone decides (x against a zero, an infinity or any non-NaN constant for
# == / !=, fabs(x) against a constant >= 0, x against itself, the ordered /
# unordered tests) into integer compares instead of a soft-float libcall.
# This check runs every C comparison operator, its negation (LLVM's
# unordered predicates), isunordered and islessgreater and their negations
# against a set of constants, on x and on fabs(x), plus the classification
# builtins, each as a branch and as a 0/1 value, on 34 values: signed
# zeros, subnormals, normals and their neighbours one ulp away, the largest
# finite values, infinities and quiet, signaling and all-ones NaNs of both
# signs. Compares the combine leaves alone run the libcall, so the table
# also pins the helpers' NaN contract.
#
# Mailbox: 0x025000 = mismatches, 0x025002 = first failing case
# (function index * NV + value index), 0x025004 = 0xC0DE when done.
import math
import struct
import sys
KIND = sys.argv[1] if len(sys.argv) > 1 else "f"
PART = int(sys.argv[2]) if len(sys.argv) > 2 else 1
NPARTS = int(sys.argv[3]) if len(sys.argv) > 3 else 1
if KIND not in ("f", "d") or not 1 <= PART <= NPARTS:
sys.exit("usage: genFpCmpConst.py f|d [PART NPARTS]")
if KIND == "f":
T = "float"
BITS_T = "u32"
SUF = "UL"
FABS = "__builtin_fabsf"
VALS = [
0x00000000, 0x80000000, 0x00000001, 0x80000001, 0x007FFFFF,
0x807FFFFF, 0x00800000, 0x80800000, 0x3F7FFFFF, 0x3F800000,
0x3F800001, 0xBF7FFFFF, 0xBF800000, 0xBF800001, 0x401FFFFF,
0x40200000, 0x40200001, 0xC01FFFFF, 0xC0200000, 0xC0200001,
0x7F7FFFFF, 0xFF7FFFFF, 0x7F800000, 0xFF800000, 0x7FC00000,
0xFFC00000, 0x7F800001, 0xFF800001, 0x7FFFFFFF, 0xFFFFFFFF,
0x7F7FFFFE, 0x00000002, 0x7F000000, 0xFF000000,
]
def decode(b):
return struct.unpack("<f", struct.pack("<I", b))[0]
# C literal (exact hex float) and its value.
def lit(x):
if math.isinf(x):
return "(-__builtin_inff())" if x < 0 else "__builtin_inff()"
return x.hex() + "f"
CONSTS = [0.0, -0.0, 1.0, -1.0, 2.5, -2.5, math.inf, -math.inf,
decode(0x00000001), decode(0x7F7FFFFF)]
ABS_CONSTS = [0.0, -0.0, 1.0, 2.5, decode(0x00000001),
decode(0x7F7FFFFF), math.inf, -1.0]
else:
T = "double"
BITS_T = "u64"
SUF = "ULL"
FABS = "__builtin_fabs"
VALS = [
0x0000000000000000, 0x8000000000000000, 0x0000000000000001,
0x8000000000000001, 0x000FFFFFFFFFFFFF, 0x800FFFFFFFFFFFFF,
0x0010000000000000, 0x8010000000000000, 0x3FEFFFFFFFFFFFFF,
0x3FF0000000000000, 0x3FF0000000000001, 0xBFEFFFFFFFFFFFFF,
0xBFF0000000000000, 0xBFF0000000000001, 0x4003FFFFFFFFFFFF,
0x4004000000000000, 0x4004000000000001, 0xC003FFFFFFFFFFFF,
0xC004000000000000, 0xC004000000000001, 0x7FEFFFFFFFFFFFFF,
0xFFEFFFFFFFFFFFFF, 0x7FF0000000000000, 0xFFF0000000000000,
0x7FF8000000000000, 0xFFF8000000000000, 0x7FF0000000000001,
0xFFF0000000000001, 0x7FFFFFFFFFFFFFFF, 0xFFFFFFFFFFFFFFFF,
0x7FEFFFFFFFFFFFFE, 0x0000000000000002, 0x0000000100000000,
0x8000000100000000,
]
def decode(b):
return struct.unpack("<d", struct.pack("<Q", b))[0]
def lit(x):
if math.isinf(x):
return "(-__builtin_inf())" if x < 0 else "__builtin_inf()"
return x.hex()
CONSTS = [0.0, -0.0, 1.0, -1.0, 2.5, -2.5, math.inf, -math.inf,
decode(0x0000000000000001), decode(0x7FEFFFFFFFFFFFFF)]
ABS_CONSTS = [0.0, -0.0, 1.0, 2.5, decode(0x0000000000000001),
decode(0x7FEFFFFFFFFFFFFF), math.inf, -1.0]
def isun(a, b):
return math.isnan(a) or math.isnan(b)
# (C expression with A and B for the operands, Python predicate).
PREDS = [
("A < B", lambda a, b: a < b),
("A <= B", lambda a, b: a <= b),
("A > B", lambda a, b: a > b),
("A >= B", lambda a, b: a >= b),
("A == B", lambda a, b: a == b),
("A != B", lambda a, b: a != b),
("!(A < B)", lambda a, b: not a < b),
("!(A <= B)", lambda a, b: not a <= b),
("!(A > B)", lambda a, b: not a > b),
("!(A >= B)", lambda a, b: not a >= b),
("__builtin_isunordered(A, B)", isun),
("!__builtin_isunordered(A, B)", lambda a, b: not isun(a, b)),
("__builtin_islessgreater(A, B)", lambda a, b: a < b or a > b),
("!__builtin_islessgreater(A, B)", lambda a, b: not (a < b or a > b)),
]
# Every test: (C expression of a, Python predicate of a).
tests = []
for c in CONSTS:
for expr, fn in PREDS:
tests.append((expr.replace("A", "a").replace("B", lit(c)),
(lambda fn, c: lambda a: fn(a, c))(fn, c)))
for c in ABS_CONSTS:
for expr, fn in PREDS:
tests.append((expr.replace("A", FABS + "(a)").replace("B", lit(c)),
(lambda fn, c: lambda a: fn(abs(a), c))(fn, c)))
for expr, fn in PREDS:
tests.append((expr.replace("A", "a").replace("B", "a"),
(lambda fn: lambda a: fn(a, a))(fn)))
tests += [
("__builtin_isnan(a)", lambda a: math.isnan(a)),
("!__builtin_isnan(a)", lambda a: not math.isnan(a)),
("__builtin_isinf(a)", lambda a: math.isinf(a)),
("__builtin_isfinite(a)", lambda a: math.isfinite(a)),
]
per = (len(tests) + NPARTS - 1) // NPARTS
tests = tests[(PART - 1) * per:PART * per]
nv = len(VALS)
nf = 2 * len(tests)
bits = []
for expr, fn in tests:
for form in range(2):
for v in VALS:
bits.append(1 if fn(decode(v)) else 0)
exp = bytearray((len(bits) + 7) // 8)
for i, b in enumerate(bits):
if b:
exp[i >> 3] |= 1 << (i & 7)
out = []
w = out.append
w("// Float / double compares against constants, generated by")
w("// tests/fpNan/genFpCmpConst.py %s %d %d; see there." % (KIND, PART, NPARTS))
w("typedef unsigned char u8;")
w("typedef unsigned short u16;")
w("typedef unsigned long u32;")
w("typedef unsigned long long u64;")
w('#define BAR() __asm__ volatile("" ::: "memory")')
w("#define NOI __attribute__((noinline)) static")
w("#define NV %d" % nv)
w("#define NF %d" % nf)
w("typedef %s T;" % T)
w("static const %s kBits[NV] = {" % BITS_T)
for i in range(0, nv, 4):
w(" " + " ".join("0x%X%s," % (v, SUF) for v in VALS[i:i + 4]))
w("};")
w("static const u8 kExp[%d] = {" % len(exp))
for i in range(0, len(exp), 16):
w(" " + " ".join("0x%02X," % b for b in exp[i:i + 16]))
w("};")
for k, (expr, fn) in enumerate(tests):
w("NOI u16 b%d(T a) { if (%s) { BAR(); return 1u; } return 0u; }"
% (k, expr))
w("NOI u16 v%d(T a) { return (u16)(%s); }" % (k, expr))
w("typedef u16 (*Fn)(T);")
w("static Fn const kFn[NF] = {")
for k in range(len(tests)):
w(" b%d, v%d," % (k, k))
w("};")
w("static u16 gBad;")
w("static u16 gFirst = 0xFFFFu;")
w("static T val(u16 i) {")
w(" union { %s u; T x; } c;" % BITS_T)
w(" c.u = kBits[i];")
w(" return c.x;")
w("}")
w("int main(void) {")
w(" u16 f;")
w(" u16 i;")
w(" u16 bit = 0u;")
w(" for (f = 0u; f < NF; f++) {")
w(" for (i = 0u; i < NV; i++) {")
w(" u16 want = (u16)((kExp[bit >> 3] >> (bit & 7u)) & 1u);")
w(" if ((kFn[f](val(i)) != 0u) != want) {")
w(" if (gBad == 0u) {")
w(" gFirst = bit;")
w(" }")
w(" gBad++;")
w(" }")
w(" bit++;")
w(" }")
w(" }")
w(" *(volatile u16 *)0x025000UL = gBad;")
w(" *(volatile u16 *)0x025002UL = gFirst;")
w(" *(volatile u16 *)0x025004UL = 0xC0DEu;")
w(" for (;;) {")
w(" }")
w(" return 0;")
w("}")
print("\n".join(out))

185
tests/fpNan/genFpTables.py
View file

@ -5,6 +5,7 @@
#
# python3 tests/fpNan/genFpTables.py cmp # kFBits / kDBits / kExp
# python3 tests/fpNan/genFpTables.py special # kCases
# python3 tests/fpNan/genFpTables.py edge # kEdge
#
# Comparisons come from Python's own float compares, which are IEEE 754
# (every ordered compare with a NaN is false). Float operands are widened
@ -23,6 +24,14 @@
# pow with a non-integer exponent, sinh, cbrt, ...) are only ~1e-6
# accurate, so their ordinary-value cases carry an `approx` flag: the check
# accepts a result within 1e-6 relative of the host's value.
#
# The edge table is math.c's exact and bit-level functions, checked bit for
# bit against Python's (C99 Annex F) answers: fmod and remainder must be
# exact for any quotient, modf / modff keep the sign of zero and split the
# infinities, atan2 has every signed-zero and infinity case, fpclassify /
# isnormal tell subnormals apart in both widths, hypot scales instead of
# overflowing or underflowing, frexp / ldexp handle subnormals and huge
# exponents, and asinh / acosh / atanh keep signs and do not overflow.
import math
import struct
@ -583,7 +592,177 @@ def annexFCases():
return out
# ---- math.c edge table -------------------------------------------------
EDGE_OPS = ['modfFrac', 'modfInt', 'modffFrac', 'modffInt', 'atan2',
'clsD', 'clsF', 'hypot', 'fmod', 'remainder', 'frexpM', 'frexpE',
'ldexp', 'asinh', 'acosh', 'atanh']
FP_NAN_CLS, FP_INFINITE_CLS, FP_NORMAL_CLS, FP_SUBNORMAL_CLS, FP_ZERO_CLS = 0, 1, 2, 3, 4
D_MIN_SUB = 5e-324
F_MIN_SUB_BITS = 0x00000001
F_MAX_SUB_BITS = 0x007FFFFF
D_MAX_SUB = dval(0x000FFFFFFFFFFFFF)
def clsBits(bits, expBits, mantBits):
e = (bits >> mantBits) & ((1 << expBits) - 1)
m = bits & ((1 << mantBits) - 1)
if e == (1 << expBits) - 1:
c = FP_NAN_CLS if m else FP_INFINITE_CLS
elif e:
c = FP_NORMAL_CLS
else:
c = FP_SUBNORMAL_CLS if m else FP_ZERO_CLS
return c | ((1 if c == FP_NORMAL_CLS else 0) << 4)
def cmodf(x):
if math.isnan(x):
return (NAN, NAN)
return math.modf(x)
def cfmodExact(x, y):
if math.isnan(x) or math.isnan(y) or math.isinf(x) or y == 0:
return NAN
if math.isinf(y):
return x
return math.fmod(x, y)
def catan2(y, x):
return NAN if math.isnan(x) or math.isnan(y) else math.atan2(y, x)
def chypot(x, y):
if math.isinf(x) or math.isinf(y):
return INF
return math.hypot(x, y)
def cldexp(x, n):
try:
return math.ldexp(x, n)
except OverflowError:
return math.copysign(INF, x)
def cacosh(x):
return NAN if math.isnan(x) or x < 1 else math.acosh(x)
def catanh(x):
if math.isnan(x) or abs(x) > 1:
return NAN
if abs(x) == 1:
return math.copysign(INF, x)
return math.atanh(x)
# (op, a, b, approx); a and b are doubles, or floats for the float ops.
EDGE_CASES = [
('modfFrac', -2.0, 0, 0), ('modfInt', -2.0, 0, 0), ('modfFrac', -0.0, 0, 0),
('modfInt', -0.0, 0, 0), ('modfFrac', INF, 0, 0), ('modfInt', INF, 0, 0),
('modfFrac', -INF, 0, 0), ('modfInt', -INF, 0, 0), ('modfFrac', NAN, 0, 0),
('modfInt', NAN, 0, 0), ('modfFrac', -3.75, 0, 0), ('modfInt', -3.75, 0, 0),
('modfFrac', 0.5, 0, 0), ('modfInt', 0.5, 0, 0), ('modfFrac', 1e300, 0, 0),
('modfInt', 4503599627370497.5, 0, 0), ('modfFrac', 4503599627370497.5, 0, 0),
('modfFrac', D_MIN_SUB, 0, 0), ('modfInt', -D_MIN_SUB, 0, 0),
('modffFrac', -2.0, 0, 0), ('modffInt', -2.0, 0, 0), ('modffFrac', -0.0, 0, 0),
('modffFrac', INF, 0, 0), ('modffInt', -INF, 0, 0), ('modffFrac', NAN, 0, 0),
('modffInt', NAN, 0, 0), ('modffFrac', -3.75, 0, 0), ('modffInt', -3.75, 0, 0),
('modffFrac', 8388609.5, 0, 0), ('modffFrac', 1e-45, 0, 0),
('atan2', 0.0, 0.0, 0), ('atan2', 0.0, -0.0, 0), ('atan2', -0.0, 0.0, 0),
('atan2', -0.0, -0.0, 0), ('atan2', 0.0, -1.0, 0), ('atan2', -0.0, -1.0, 0),
('atan2', 0.0, 1.0, 0), ('atan2', -0.0, 1.0, 0), ('atan2', 1.0, 0.0, 0),
('atan2', -1.0, -0.0, 0), ('atan2', 1.0, -INF, 0), ('atan2', -1.0, -INF, 0),
('atan2', 1.0, INF, 0), ('atan2', -1.0, INF, 0), ('atan2', INF, 1.0, 0),
('atan2', -INF, -1.0, 0), ('atan2', INF, -INF, 0), ('atan2', -INF, -INF, 0),
('atan2', INF, INF, 0), ('atan2', -INF, INF, 0), ('atan2', NAN, 1.0, 0),
('atan2', 1.0, NAN, 0), ('atan2', 1.0, 1.0, 1), ('atan2', -1.0, -2.0, 1),
('atan2', 2.0, -1.0, 1), ('atan2', 1e300, 1e-300, 0), ('atan2', 1e-300, -1e300, 0),
('clsD', 1.0, 0, 0), ('clsD', -0.0, 0, 0), ('clsD', D_MIN_SUB, 0, 0),
('clsD', -D_MAX_SUB, 0, 0), ('clsD', 2.2250738585072014e-308, 0, 0),
('clsD', INF, 0, 0), ('clsD', NAN, 0, 0), ('clsD', -1e300, 0, 0),
('clsF', 1.0, 0, 0), ('clsF', -0.0, 0, 0), ('clsF', fval(F_MIN_SUB_BITS), 0, 0),
('clsF', -fval(F_MAX_SUB_BITS), 0, 0), ('clsF', 1.1754943508222875e-38, 0, 0),
('clsF', INF, 0, 0), ('clsF', NAN, 0, 0), ('clsF', -3e38, 0, 0),
('hypot', 1e200, 1e200, 1), ('hypot', 3e200, -4e200, 1), ('hypot', 1e-200, 1e-200, 1),
('hypot', -3e-300, 4e-300, 1), ('hypot', 1e308, 1e308, 0), ('hypot', 1e300, 1.0, 0),
('hypot', D_MIN_SUB, D_MIN_SUB, 0), ('hypot', -0.0, -0.0, 0), ('hypot', -0.0, -5.0, 0),
('hypot', -INF, NAN, 0), ('hypot', NAN, 2.0, 0), ('hypot', 3.0, 4.0, 1),
('fmod', 1e300, 3.0, 0), ('fmod', -1e300, 7.0, 0), ('fmod', 1e20, 0.1, 0),
('fmod', 5.5, -2.0, 0), ('fmod', -5.5, -2.0, 0), ('fmod', -6.0, 3.0, 0),
('fmod', 6.0, -3.0, 0), ('fmod', 1.0, 3.0, 0), ('fmod', 3 * D_MIN_SUB, 2 * D_MIN_SUB, 0),
('fmod', 1.5, D_MIN_SUB, 0), ('fmod', 1.7976931348623157e308, 1e-300, 0),
('fmod', 2.0 ** 60 + 2.0 ** 8, 2.0 ** -40, 0), ('fmod', 7.5, 2.0, 0),
('fmod', -0.0, 3.0, 0), ('fmod', 1.0, -INF, 0), ('fmod', INF, 1.0, 0),
('fmod', NAN, 1.0, 0), ('fmod', 1.0, 0.0, 0),
('remainder', 1e300, 3.0, 0), ('remainder', -1e300, 7.0, 0), ('remainder', 1e20, 0.1, 0),
('remainder', -4.0, 2.0, 0), ('remainder', 4.0, -2.0, 0), ('remainder', 3.0, 2.0, 0),
('remainder', -3.0, 2.0, 0), ('remainder', 5.0, -2.0, 0), ('remainder', 1.5, 1e-300, 0),
('remainder', 3 * D_MIN_SUB, 2 * D_MIN_SUB, 0), ('remainder', 1.7976931348623157e308, -1.7e308, 0),
('remainder', 1.0, INF, 0), ('remainder', INF, 1.0, 0), ('remainder', 1.0, -0.0, 0),
('frexpM', D_MIN_SUB, 0, 0), ('frexpE', D_MIN_SUB, 0, 0), ('frexpM', -D_MAX_SUB, 0, 0),
('frexpE', -D_MAX_SUB, 0, 0), ('frexpM', 1e300, 0, 0), ('frexpE', 1e300, 0, 0),
('frexpM', -0.0, 0, 0), ('frexpE', -0.0, 0, 0), ('frexpM', -INF, 0, 0),
('frexpM', 0.75, 0, 0), ('frexpE', 0.75, 0, 0),
('ldexp', 1.0, 1023, 0), ('ldexp', 1.0, 1024, 0), ('ldexp', -1.0, 32767, 0),
('ldexp', 1.0, -32768, 0), ('ldexp', 0.0, 30000, 0), ('ldexp', -0.0, -30000, 0),
('ldexp', 1.0, -1074, 0), ('ldexp', 1.5, -1074, 0), ('ldexp', 1.0, -1075, 0),
('ldexp', 3.0, -1075, 0), ('ldexp', D_MIN_SUB, 1074, 0), ('ldexp', -D_MAX_SUB, 1, 0),
('ldexp', 0.75, -1022, 0), ('ldexp', INF, -5, 0), ('ldexp', NAN, 5, 0),
('asinh', -0.0, 0, 0), ('asinh', 0.0, 0, 0), ('asinh', -INF, 0, 0), ('asinh', INF, 0, 0),
('asinh', NAN, 0, 0), ('asinh', -1e10, 0, 1), ('asinh', 1e200, 0, 1), ('asinh', -1e300, 0, 1),
('asinh', 0.5, 0, 1), ('asinh', -2.0, 0, 1),
('acosh', 1e200, 0, 1), ('acosh', INF, 0, 0), ('acosh', 1.0, 0, 0), ('acosh', 0.5, 0, 0),
('acosh', -INF, 0, 0), ('acosh', NAN, 0, 0), ('acosh', 2.0, 0, 1),
('atanh', -0.0, 0, 0), ('atanh', 0.0, 0, 0), ('atanh', -0.5, 0, 1), ('atanh', 1.0, 0, 0),
]
EDGE_MODEL = {
'modfFrac': lambda x, y: cmodf(x)[0], 'modfInt': lambda x, y: cmodf(x)[1],
'atan2': catan2, 'hypot': chypot, 'fmod': cfmodExact, 'remainder': cremainder,
'frexpM': lambda x, y: x if not math.isfinite(x) else math.frexp(x)[0],
'ldexp': lambda x, n: x if not math.isfinite(x) else cldexp(x, n),
'asinh': lambda x, y: math.asinh(x), 'acosh': lambda x, y: cacosh(x),
'atanh': lambda x, y: catanh(x),
}
def edgeCase(name, x, y, approx):
op = EDGE_OPS.index(name)
if name in ('modffFrac', 'modffInt'):
xf = fround32(x)
r = cmodf(xf)[0 if name == 'modffFrac' else 1]
return (op, fbits(xf), 0, nanFold(fbits(fround32(r)), True), approx)
if name == 'clsF':
b = fbits(fround32(x)) if not math.isnan(x) else F_DNAN
return (op, b, 0, clsBits(b, 8, 23), approx)
if name == 'clsD':
b = dbits(x) if not math.isnan(x) else D_DNAN
return (op, b, 0, clsBits(b, 11, 52), approx)
if name == 'frexpE':
return (op, dbits(x), 0, math.frexp(x)[1] & 0xFFFF, approx)
if name == 'ldexp':
return (op, dbits(x), y & 0xFFFF, nanFold(dbits(EDGE_MODEL[name](x, y)), False), approx)
want = EDGE_MODEL[name](x, y)
return (op, dbits(x), dbits(y), nanFold(dbits(want), False), approx)
def emitEdge():
table = [edgeCase(*c) for c in EDGE_CASES]
print('#define NE %d' % len(table))
print('static const CaseT kEdge[NE] = {')
for op, a, b, want, approx in table:
print(' { %-14s 0x%016XULL, 0x%016XULL, 0x%016XULL, %d },' %
('E_' + EDGE_OPS[op].upper() + ',', a, b, want, approx))
print('};')
if __name__ == '__main__':
if len(sys.argv) != 2 or sys.argv[1] not in ('cmp', 'special'):
sys.exit('usage: genFpTables.py cmp|special')
emitCmp() if sys.argv[1] == 'cmp' else emitSpecial()
modes = {'cmp': emitCmp, 'special': emitSpecial, 'edge': emitEdge}
if len(sys.argv) != 2 or sys.argv[1] not in modes:
sys.exit('usage: genFpTables.py cmp|special|edge')
modes[sys.argv[1]]()

18
tests/lua/luaTest.manifest.json
View file

@ -7,9 +7,9 @@
"num": 1,
"name": "SEG1",
"base": "0x001000",
"size": 44531,
"size": 45004,
"bss_start": "0x00e300",
"bss_size": 4248,
"bss_size": 4252,
"image": "luaTest.bin",
"entry_offset": "0x0000"
},
@ -17,7 +17,7 @@
"num": 2,
"name": "SEG2",
"base": "0x040000",
"size": 44940,
"size": 44949,
"bss_start": "0x000000",
"bss_size": 0,
"image": "luaTest.seg2.bin",
@ -27,21 +27,11 @@
"num": 3,
"name": "SEG3",
"base": "0x050000",
"size": 41839,
"size": 32420,
"bss_start": "0x000000",
"bss_size": 0,
"image": "luaTest.seg3.bin",
"entry_offset": "0x0000"
},
{
"num": 4,
"name": "SEG4",
"base": "0x060000",
"size": 28814,
"bss_start": "0x000000",
"bss_size": 0,
"image": "luaTest.seg4.bin",
"entry_offset": "0x0000"
}
]
}

122
tests/memBlock/memBlockIrq.c Normal file
View file

@ -0,0 +1,122 @@
// MVN / MVP stay correct when an interrupt lands mid-move and its handler
// moves memory too. memBlockIrq.lua injects a native-mode interrupt (to
// irqEntry, memBlockIrqEntry.s) whenever the CPU is executing an armed
// outer call's block-move stub on the stack; the handler runs irqNested,
// whose memcpy / memmove / memset use other banks and their own stubs, and
// after RTI the outer move resumes. A move whose bank operands lived in
// shared code would resume with the handler's banks.
// Mailbox: 0x025000 = bad outer results, 0x025002 = bad nested results,
// 0x025004 = 0xC0DE done, 0x025006 = 1 when at least MIN_SHOTS interrupts
// landed mid-move and every handler finished, 0x025020 = irqEntry (set
// here), 0x025022 = armed, 0x025024 = interrupts injected (Lua), 0x025026
// = handlers completed, 0x025028 = a handler is running (Lua sets it, the
// handler clears it).
#include "memPattern.h"
#define MB_BAD (*(volatile u16 *)0x025000UL)
#define MB_BAD_IN (*(volatile u16 *)0x025002UL)
#define MB_DONE (*(volatile u16 *)0x025004UL)
#define MB_SHOT_OK (*(volatile u16 *)0x025006UL)
#define MB_IRQ (*(volatile u16 *)0x025020UL)
#define MB_ARM (*(volatile u16 *)0x025022UL)
#define MB_SHOTS (*(volatile u16 *)0x025024UL)
#define MB_HANDLED (*(volatile u16 *)0x025026UL)
#define MB_BUSY (*(volatile u16 *)0x025028UL)
#define MB_DONE_MARK 0xC0DEu
#define ROUNDS 2u
#define MIN_SHOTS 8u
#define OUTER_LEN 20000UL
#define INNER_LEN 300UL
#define SEED_OUT 0x3Cu
#define SEED_IN 0x96u
#define MOVE_SHIFT 0x123UL
#define IN_SET_VAL 0xA7
// Outer: one MVN (bank 6 to 7), several runs (the source across 08/09, the
// destination across 0A/0B), a backward overlapping MVP across 0C/0D.
#define OUT_SRC1 0x061000UL
#define OUT_DST1 0x072000UL
#define OUT_SRC2 (0x090000UL - 11111UL)
#define OUT_DST2 (0x0B0000UL - 7777UL)
#define OUT_MOVE (0x0D0000UL - 15000UL)
// Inner: banks 0E-11, apart from the outer ones.
#define IN_SRC (0x0F0000UL - 500UL)
#define IN_DST 0x0E4000UL
#define IN_MOVE (0x110000UL - 700UL)
#define IN_SET 0x103000UL
static int filledWith(u32 a, u32 n, u8 c);
void irqEntry(void);
void irqNested(void);
static u16 gBadIn;
static int filledWith(u32 a, u32 n, u8 c) {
const u8 *p = (const u8 *)a;
u16 i;
for (i = 0; i < (u16)n; i++) {
if (p[i] != c) {
return 0;
}
}
return 1;
}
// Called from irqEntry, on the interrupted program's stack.
void irqNested(void) {
fill(IN_SRC, INNER_LEN, SEED_IN);
memset((void *)IN_DST, 0, INNER_LEN);
if (memcpy((void *)IN_DST, (const void *)IN_SRC, INNER_LEN) != (void *)IN_DST ||
!holds(IN_DST, IN_SRC, INNER_LEN, SEED_IN)) {
gBadIn++;
}
fill(IN_MOVE, INNER_LEN + MOVE_SHIFT, SEED_IN);
memmove((void *)(IN_MOVE + MOVE_SHIFT), (const void *)IN_MOVE, INNER_LEN);
if (!holds(IN_MOVE + MOVE_SHIFT, IN_MOVE, INNER_LEN, SEED_IN)) {
gBadIn++;
}
memset((void *)IN_SET, IN_SET_VAL, INNER_LEN);
if (!filledWith(IN_SET, INNER_LEN, (u8)IN_SET_VAL)) {
gBadIn++;
}
MB_BAD_IN = gBadIn;
MB_HANDLED = (u16)(MB_HANDLED + 1u);
MB_BUSY = 0u;
}
int main(void) {
u16 bad = 0u;
u16 r;
MB_IRQ = (u16)(u32)irqEntry;
fill(OUT_SRC1, OUTER_LEN, SEED_OUT);
fill(OUT_SRC2, OUTER_LEN, SEED_OUT);
for (r = 0; r < ROUNDS; r++) {
memset((void *)OUT_DST1, 0, OUTER_LEN);
memset((void *)OUT_DST2, 0, OUTER_LEN);
fill(OUT_MOVE, OUTER_LEN + MOVE_SHIFT, SEED_OUT);
MB_ARM = 1u;
memcpy((void *)OUT_DST1, (const void *)OUT_SRC1, OUTER_LEN);
memcpy((void *)OUT_DST2, (const void *)OUT_SRC2, OUTER_LEN);
memmove((void *)(OUT_MOVE + MOVE_SHIFT), (const void *)OUT_MOVE, OUTER_LEN);
MB_ARM = 0u;
if (!holds(OUT_DST1, OUT_SRC1, OUTER_LEN, SEED_OUT)) {
bad++;
}
if (!holds(OUT_DST2, OUT_SRC2, OUTER_LEN, SEED_OUT)) {
bad++;
}
if (!holds(OUT_MOVE + MOVE_SHIFT, OUT_MOVE, OUTER_LEN, SEED_OUT)) {
bad++;
}
MB_BAD = bad;
}
MB_SHOT_OK = (u16)(MB_SHOTS >= MIN_SHOTS && MB_HANDLED == MB_SHOTS);
MB_DONE = MB_DONE_MARK;
for (;;) {
}
return 0;
}

31
tests/memBlock/memBlockIrq.lua Normal file
View file

@ -0,0 +1,31 @@
-- Inject a native-mode interrupt into memBlockIrq while an armed outer
-- move is executing its block-move stub on the stack (PB 0, PC below the
-- program at $1000), never into a handler and at most every third frame,
-- so the outer moves make progress between handlers.
local cpu = manager.machine.devices[":maincpu"]
local mem = cpu.spaces["program"]
local irqFrame = 0
emu.register_frame_done(function()
irqFrame = irqFrame + 1
if irqFrame % 3 ~= 0 then
return
end
if mem:read_u16(0x025022) ~= 1 or mem:read_u16(0x025028) ~= 0 then
return
end
local pc = cpu.state["PC"].value
if cpu.state["PB"].value ~= 0 or pc < 0x0100 or pc >= 0x1000 then
return
end
local s = cpu.state["S"].value
local p = cpu.state["P"].value
mem:write_u8(s, cpu.state["PB"].value)
mem:write_u8(s - 1, (pc >> 8) & 0xFF)
mem:write_u8(s - 2, pc & 0xFF)
mem:write_u8(s - 3, p)
cpu.state["S"].value = s - 4
cpu.state["P"].value = (p | 0x04) & 0xF7
cpu.state["PC"].value = mem:read_u16(0x025020)
mem:write_u16(0x025028, 1)
mem:write_u16(0x025024, mem:read_u16(0x025024) + 1)
end)

28
tests/memBlock/memBlockIrqEntry.s Normal file
View file

@ -0,0 +1,28 @@
; irqEntry -- the interrupt memBlockIrq.lua injects: a native-mode handler
; that saves every register, gives the C code its own direct page (any
; interrupt code that runs C must: C keeps its scratch in the direct
; page) and DBR 0, runs irqNested and returns with RTI.
IRQ_D = 0x0200
.text
.globl irqEntry
irqEntry:
rep #0x30
pha
phx
phy
phb
phd
pea 0
plb
plb
lda #IRQ_D
tcd
jsl irqNested
pld
plb
ply
plx
pla
rti

124
tests/memBlock/memBlockRegs.s Normal file
View file

@ -0,0 +1,124 @@
; memBlockRegs -- memcpy / memmove / memset keep the caller's DBR and D.
;
; unsigned short memBlockRegs(void) calls each routine down each of its
; paths (short loop, one MVN / MVP, several runs; memset short, one pass
; and over 64 KB) with DBR = $05 and D = $0100, so the routines' direct-
; page scratch lands at $01E0, and sets one bit at $02:5008 for each call
; that came back with DBR, D and dst in A:X intact. It returns that word:
; 0x0FFF, all 12 bits, is a pass.
REGS_DBR = 0x05
REGS_D = 0x0100
REGS_OUT = 0x025008
.text
.globl memBlockRegs
; The caller pushed the arguments after dst (nWords words); call fn with
; dst in A:X, drop the arguments and check what came back.
.macro regsCall fn, dstLo, dstHi, nWords, bit
lda #\dstLo
ldx #\dstHi
jsl \fn
tay
tsc
clc
adc #(\nWords * 2)
tcs
cpy #\dstLo
bne .Lskip\bit
cpx #\dstHi
bne .Lskip\bit
tdc
cmp #REGS_D
bne .Lskip\bit
phb
sep #0x20
.a8
pla
cmp #REGS_DBR
rep #0x20
.a16
bne .Lskip\bit
lda #(1 << \bit)
ora REGS_OUT
sta REGS_OUT
.Lskip\bit:
.endm
memBlockRegs:
rep #0x30
phb
phd
lda #0
sta REGS_OUT
pea REGS_DBR * 0x101 ; DBR twice: plb, plb
plb
plb
lda #REGS_D
tcd
; memcpy: short, one MVN, runs (source across 03/04)
pea 0
pea 20
pea 0x0002
pea 0x6200
regsCall memcpy, 0x6100, 0x0002, 4, 0
pea 0
pea 300
pea 0x0002
pea 0x6200
regsCall memcpy, 0x6800, 0x0002, 4, 1
pea 0
pea 600
pea 0x0003
pea 0xff00
regsCall memcpy, 0x7000, 0x0002, 4, 2
; memmove backward: short, one MVP, runs (both across 03/04)
pea 0
pea 20
pea 0x0002
pea 0x6100
regsCall memmove, 0x6105, 0x0002, 4, 3
pea 0
pea 300
pea 0x0002
pea 0x6100
regsCall memmove, 0x6105, 0x0002, 4, 4
pea 0
pea 600
pea 0x0003
pea 0xff00
regsCall memmove, 0xff40, 0x0003, 4, 5
; memmove forward: short, one MVN, runs
pea 0
pea 20
pea 0x0002
pea 0x6105
regsCall memmove, 0x6100, 0x0002, 4, 6
pea 0
pea 300
pea 0x0002
pea 0x6105
regsCall memmove, 0x6100, 0x0002, 4, 7
pea 0
pea 600
pea 0x0003
pea 0xff40
regsCall memmove, 0xff00, 0x0003, 4, 8
; memset: short, one pass, over 64 KB
pea 0
pea 5
pea 0x77
regsCall memset, 0x6100, 0x0002, 3, 9
pea 0
pea 300
pea 0x77
regsCall memset, 0x6100, 0x0002, 3, 10
pea 1
pea 0x10
pea 0x77
regsCall memset, 0x0000, 0x0004, 3, 11
pld
plb
lda REGS_OUT
rtl

196
tests/memBlock/memBlockTest.c Normal file
View file

@ -0,0 +1,196 @@
// memcpy / memmove / memset against a byte-exact model: every length
// class (0, 1, short, both sides of the MVN threshold, long, over 64 KB),
// odd and even source and destination addresses, buffers that straddle a
// bank boundary (source, destination, both), overlapping memmove in both
// directions (short and long, across a bank edge), memset of each class
// and of a value above 255, the returned pointer, and guard bytes on both
// sides; first (memBlockRegs.s) the caller's DBR and D across every path.
// Mailbox: 0x025000 = mismatches, 0x025002 = first failing case (0xFFFF:
// none), 0x025004 = 0xC0DE done, 0x025006 = cases run, 0x025008 = 0x0FFF
// when DBR and D survived all 12 register probes; the first failure's
// dst / src (or c) / n | op << 28 at 0x025010 / 0x025014 / 0x025018.
#include "memPattern.h"
#define MB_BAD (*(volatile u16 *)0x025000UL)
#define MB_FIRST (*(volatile u16 *)0x025002UL)
#define MB_DONE (*(volatile u16 *)0x025004UL)
#define MB_CASES (*(volatile u16 *)0x025006UL)
#define MB_DST (*(volatile u32 *)0x025010UL)
#define MB_SRC (*(volatile u32 *)0x025014UL)
#define MB_N (*(volatile u32 *)0x025018UL)
#define MB_DONE_MARK 0xC0DEu
#define MB_NO_FAIL 0xFFFFu
#define GUARD 8u
#define SEED_SRC 0xC3u
#define SEED_DST 0x5Au
#define SET_BYTE 0xA5
#define SET_BIG 0x3C
#define SET_HIGH 0x100
#define BIG_LEN 70000UL
#define N_LENS 27u
#define N_SHIFTS 10u
#define N_ALIGNS 2u
#define OP_COPY 0x10000000UL
#define OP_MOVE 0x20000000UL
#define OP_SET 0x30000000UL
// Plain buffers in bank 2; bank edges at 03/04, 04/05, 05/06, 06/07 and
// 07/08 for the straddling ones.
#define PLAIN_DST 0x026000UL
#define PLAIN_SRC 0x027000UL
#define PLAIN_MOVE 0x026200UL
#define EDGE_SRC 0x040000UL
#define EDGE_DST 0x050000UL
#define EDGE_MOVE 0x060000UL
#define EDGE_SET 0x070000UL
#define BIG_SRC 0x058000UL
#define BIG_DST 0x074000UL
#define BIG_UP 0x058123UL
#define BIG_DOWN 0x057F00UL
#define BIG_SET 0x09FFFFUL
static void begin(u32 dst, u32 src, u32 n);
static void caseCopy(u32 dst, u32 src, u32 n);
static void caseMove(u32 dst, u32 src, u32 n);
static void caseSet(u32 dst, int c, u32 n);
static void fail(void);
static int guarded(u32 a, u32 n, u8 seed);
static int holdsByte(u32 a, u32 n, u8 c);
u16 memBlockRegs(void);
static const u16 kLens[N_LENS] = {
0, 1, 2, 3, 4, 5, 7, 8, 15, 16, 17, 22, 23, 24, 25, 26, 31, 32, 33, 34,
35, 64, 100, 255, 256, 257, 1000
};
static const short kShifts[N_SHIFTS] = { -100, -7, -3, -2, -1, 1, 2, 3, 7, 100 };
static u16 gBad;
static u16 gCase;
static u32 gDst;
static u32 gSrc;
static u32 gN;
static void begin(u32 dst, u32 src, u32 n) {
gDst = dst;
gSrc = src;
gN = n;
gCase++;
MB_CASES = gCase;
}
static void caseCopy(u32 dst, u32 src, u32 n) {
begin(dst, src, n | OP_COPY);
fill(dst - GUARD, n + 2u * GUARD, SEED_DST);
fill(src, n, SEED_SRC);
if (memcpy((void *)dst, (const void *)src, n) != (void *)dst ||
!holds(dst, src, n, SEED_SRC) || !guarded(dst, n, SEED_DST)) {
fail();
}
}
// One region holds both blocks: after the move [dst, dst + n) holds what
// [src, src + n) held and the rest of the window is untouched.
static void caseMove(u32 dst, u32 src, u32 n) {
u32 lo = (dst < src) ? dst : src;
u32 hi = ((dst < src) ? src : dst) + n;
begin(dst, src, n | OP_MOVE);
fill(lo - GUARD, hi - lo + 2u * GUARD, SEED_SRC);
if (memmove((void *)dst, (const void *)src, n) != (void *)dst ||
!holds(dst, src, n, SEED_SRC) ||
!holds(lo - GUARD, lo - GUARD, dst - (lo - GUARD), SEED_SRC) ||
!holds(dst + n, dst + n, hi + GUARD - (dst + n), SEED_SRC)) {
fail();
}
}
static void caseSet(u32 dst, int c, u32 n) {
begin(dst, (u32)c, n | OP_SET);
fill(dst - GUARD, n + 2u * GUARD, SEED_DST);
if (memset((void *)dst, c, n) != (void *)dst ||
!holdsByte(dst, n, (u8)c) || !guarded(dst, n, SEED_DST)) {
fail();
}
}
static void fail(void) {
if (gBad == 0u) {
MB_FIRST = gCase;
MB_DST = gDst;
MB_SRC = gSrc;
MB_N = gN;
}
gBad++;
MB_BAD = gBad;
}
// The GUARD bytes on each side of [a, a + n) still hold their own pattern.
static int guarded(u32 a, u32 n, u8 seed) {
return holds(a - GUARD, a - GUARD, GUARD, seed) && holds(a + n, a + n, GUARD, seed);
}
static int holdsByte(u32 a, u32 n, u8 c) {
const u8 *p = (const u8 *)a;
while (n != 0u) {
u16 run = (n > 0x8000UL) ? 0x8000u : (u16)n;
u16 i;
for (i = 0; i < run; i++) {
if (p[i] != c) {
return 0;
}
}
p += run;
n -= run;
}
return 1;
}
int main(void) {
u16 li;
u16 d;
u16 s;
u16 k;
// DBR and D survive every path (memBlockRegs.s writes 0x025008 itself).
(void)memBlockRegs();
for (li = 0; li < N_LENS; li++) {
u32 n = kLens[li];
u32 half = n / 2u;
for (d = 0; d < N_ALIGNS; d++) {
for (s = 0; s < N_ALIGNS; s++) {
// Plain, source across 03/04, destination across 04/05,
// both across (at different points of the run).
caseCopy(PLAIN_DST + d, PLAIN_SRC + s, n);
caseCopy(PLAIN_DST + d, EDGE_SRC - half + s, n);
caseCopy(EDGE_DST - half + d, PLAIN_SRC + s, n);
caseCopy(EDGE_DST - half + d, EDGE_SRC - n / 3u + s, n);
}
}
for (k = 0; k < N_SHIFTS; k++) {
caseMove(PLAIN_MOVE + (long)kShifts[k], PLAIN_MOVE, n);
caseMove(EDGE_MOVE - half + (long)kShifts[k], EDGE_MOVE - half, n);
}
caseSet(PLAIN_DST + (li & 1u), SET_BYTE, n);
caseSet(EDGE_SET - half, SET_HIGH + li, n);
}
// Over 64 KB: several MVN / MVP runs, every bank byte stepping.
caseCopy(BIG_DST, BIG_SRC, BIG_LEN);
caseMove(BIG_UP, BIG_SRC, BIG_LEN);
caseMove(BIG_DOWN, BIG_SRC, BIG_LEN);
caseSet(BIG_SET, SET_BIG, BIG_LEN);
MB_BAD = gBad;
if (gBad == 0u) {
MB_FIRST = MB_NO_FAIL;
}
MB_DONE = MB_DONE_MARK;
for (;;) {
}
return 0;
}

75
tests/memBlock/memPattern.h Normal file
View file

@ -0,0 +1,75 @@
// memPattern.h - the byte pattern the memBlock tests fill and check.
//
// The byte at address a is a's low byte xor a mix of its page and bank, so
// a copy off by one byte, one page or one bank reads wrong. Walked a page
// at a time so the per-byte work stays 16-bit.
#ifndef MEM_PATTERN_H
#define MEM_PATTERN_H
typedef unsigned char u8;
typedef unsigned short u16;
typedef unsigned long u32;
typedef unsigned long size_t;
#define PAGE_BYTES 256u
static void fill(u32 a, u32 n, u8 seed);
static int holds(u32 a, u32 from, u32 n, u8 seed);
void *memcpy(void *dst, const void *src, size_t n);
void *memmove(void *dst, const void *src, size_t n);
void *memset(void *dst, int c, size_t n);
static u8 mixOf(u32 a, u8 seed);
// [a, a + n) = the pattern of its own addresses.
static void fill(u32 a, u32 n, u8 seed) {
u8 *p = (u8 *)a;
while (n != 0u) {
u8 lo = (u8)a;
u8 m = mixOf(a, seed);
u16 run = (u16)(PAGE_BYTES - lo);
u16 i;
if ((u32)run > n) {
run = (u16)n;
}
for (i = 0; i < run; i++) {
p[i] = (u8)(lo ^ m);
lo++;
}
p += run;
a += run;
n -= run;
}
}
// 1 when [a, a + n) holds the pattern of [from, from + n).
static int holds(u32 a, u32 from, u32 n, u8 seed) {
const u8 *p = (const u8 *)a;
while (n != 0u) {
u8 lo = (u8)from;
u8 m = mixOf(from, seed);
u16 run = (u16)(PAGE_BYTES - lo);
u16 i;
if ((u32)run > n) {
run = (u16)n;
}
for (i = 0; i < run; i++) {
if (p[i] != (u8)(lo ^ m)) {
return 0;
}
lo++;
}
p += run;
from += run;
n -= run;
}
return 1;
}
static u8 mixOf(u32 a, u8 seed) {
return (u8)((u8)((u8)(a >> 8) * 7u) ^ (u8)((u8)(a >> 16) * 13u) ^ seed);
}
#endif

148
tests/regress/genI64Cmp.py Normal file
View file

@ -0,0 +1,148 @@
#!/usr/bin/env python3
# Emits the C source of the smoke check "i64 compares", with kExp holding
# the expected results computed here (C semantics).
#
# Every i64 predicate (eq ne ult ule ugt uge slt sle sgt sge) on values
# that sit on each 16-bit word boundary and both sides of the sign bit: as
# a branch, as a 0/1 value and as an i16 select against registers, and as
# a branch and a value against ten constants (some with zero low words,
# which lowerI32Cmp drops from its CMP/SBC chain), then loops whose i64
# counters cross a 32-bit boundary. Pins the four-word CMP/SBC chain
# (SETCC / BR_CC / SELECT_CC on i64 are lowered by lowerI32Cmp).
#
# Mailbox: 0x025000 = mismatches, 0x025002 = first failing case,
# 0x025004 = 0xC0DE when done.
V = [
0x0000000000000000, 0x0000000000000001, 0x000000000000FFFF,
0x0000000000010000, 0x00000000FFFFFFFF, 0x0000000100000000,
0x0000000100000001, 0x0000FFFFFFFFFFFF, 0x0001000000000000,
0x0000800000000000, 0x7FF0000000000000, 0x7FF0000000000001,
0x0001FFFF00000000, 0x7FFFFFFFFFFFFFFF, 0x8000000000000000,
0x8000000000000001, 0xFFFF000000000000, 0xFFFFFFFF00000000,
0xFFFFFFFFFFFF0000, 0xFFFFFFFFFFFFFFFF,
]
K = [
0x0000000000000000, 0x0000000000000001, 0x00000000FFFFFFFF,
0x0000000100000000, 0x7FF0000000000000, 0x7FF0000000000001,
0x7FFFFFFFFFFFFFFF, 0x8000000000000000, 0xFFFF000000000000,
0xFFFFFFFFFFFFFFFF,
]
PREDS = [
("eq", "u64", "=="), ("ne", "u64", "!="), ("ult", "u64", "<"),
("ule", "u64", "<="), ("ugt", "u64", ">"), ("uge", "u64", ">="),
("slt", "s64", "<"), ("sle", "s64", "<="), ("sgt", "s64", ">"),
("sge", "s64", ">="),
]
def s64(x):
return x - (1 << 64) if x & (1 << 63) else x
def ev(t, op, a, b):
if t == "s64":
a, b = s64(a), s64(b)
return {"==": a == b, "!=": a != b, "<": a < b, "<=": a <= b,
">": a > b, ">=": a >= b}[op]
bits = []
for _, t, op in PREDS:
for a in V:
for b in V:
bits.append(ev(t, op, a, b))
for _, t, op in PREDS:
for k in K:
for a in V:
bits.append(ev(t, op, a, k))
exp = bytearray((len(bits) + 7) // 8)
for i, b in enumerate(bits):
if b:
exp[i >> 3] |= 1 << (i & 7)
o = []
w = o.append
w("// i64 compares, generated by tests/regress/genI64Cmp.py; see there.")
w("typedef unsigned char u8;")
w("typedef unsigned short u16;")
w("typedef unsigned long long u64;")
w("typedef long long s64;")
w("#define NV %du" % len(V))
w("#define NK %du" % len(K))
w("#define NP %du" % len(PREDS))
w('#define BAR() __asm__ volatile("" ::: "memory")')
w("#define NOI __attribute__((noinline)) static")
w("static volatile u16 gSink;")
w("static const u64 kV[NV] = {")
for i in range(0, len(V), 3):
w(" " + " ".join("0x%016XULL," % v for v in V[i:i + 3]))
w("};")
w("static const u8 kExp[%d] = {" % len(exp))
for i in range(0, len(exp), 16):
w(" " + " ".join("0x%02X," % b for b in exp[i:i + 16]))
w("};")
for name, t, op in PREDS:
w("NOI u16 br_%s(u64 a, u64 b) { if ((%s)a %s (%s)b) { BAR(); return 1u; } return 0u; }" % (name, t, op, t))
w("NOI u16 va_%s(u64 a, u64 b) { return (u16)((%s)a %s (%s)b); }" % (name, t, op, t))
w("NOI u16 se_%s(u64 a, u64 b) { return (%s)a %s (%s)b ? 0x5A5Au : 0xA5A5u; }" % (name, t, op, t))
for i, k in enumerate(K):
w("NOI u16 bk_%s_%d(u64 a) { if ((%s)a %s (%s)0x%016XULL) { BAR(); return 1u; } return 0u; }" % (name, i, t, op, t, k))
w("NOI u16 vk_%s_%d(u64 a) { return (u16)((%s)a %s (%s)0x%016XULL); }" % (name, i, t, op, t, k))
w("typedef u16 (*Fn2)(u64, u64);")
w("typedef u16 (*Fn1)(u64);")
for p in ("br", "va", "se"):
w("static Fn2 const k%s[NP] = { %s };" % (p.capitalize(), ", ".join("%s_%s" % (p, n) for n, _, _ in PREDS)))
for p in ("bk", "vk"):
w("static Fn1 const k%s[NP * NK] = {" % p.capitalize())
for n, _, _ in PREDS:
w(" " + " ".join("%s_%s_%d," % (p, n, i) for i in range(len(K))))
w("};")
w("NOI u16 loopUlt(u64 s, u64 e) { u16 n = 0u; u64 i; for (i = s; i < e; i++) { gSink = (u16)i; n++; } return n; }")
w("NOI u16 loopSge(s64 s, s64 e) { u16 n = 0u; s64 i; for (i = e; i >= s; i--) { gSink = (u16)i; n++; } return n; }")
w("NOI u16 loopUgtDown(u64 s, u64 e) { u16 n = 0u; u64 i; for (i = e; i > s; i--) { gSink = (u16)i; n++; } return n; }")
w("static u16 gBad;")
w("static u16 gFirst = 0xFFFFu;")
w("static void check(u16 got, u16 want, u16 id) {")
w(" if (got != want) {")
w(" if (gBad == 0u) {")
w(" gFirst = id;")
w(" }")
w(" gBad++;")
w(" }")
w("}")
w("int main(void) {")
w(" u16 p;")
w(" u16 i;")
w(" u16 j;")
w(" u16 bit = 0u;")
w(" for (p = 0u; p < NP; p++) {")
w(" for (i = 0u; i < NV; i++) {")
w(" for (j = 0u; j < NV; j++) {")
w(" u16 want = (u16)((kExp[bit >> 3] >> (bit & 7u)) & 1u);")
w(" check(kBr[p](kV[i], kV[j]), want, bit);")
w(" check(kVa[p](kV[i], kV[j]), want, (u16)(bit | 0x4000u));")
w(" check(kSe[p](kV[i], kV[j]) == 0x5A5Au, want, (u16)(bit | 0x8000u));")
w(" bit++;")
w(" }")
w(" }")
w(" }")
w(" for (p = 0u; p < NP * NK; p++) {")
w(" for (i = 0u; i < NV; i++) {")
w(" u16 want = (u16)((kExp[bit >> 3] >> (bit & 7u)) & 1u);")
w(" check(kBk[p](kV[i]), want, bit);")
w(" check(kVk[p](kV[i]), want, (u16)(bit | 0x4000u));")
w(" bit++;")
w(" }")
w(" }")
w(" check(loopUlt(0x00000000FFFFFFF0ULL, 0x0000000100000010ULL), 32u, 0xC000u);")
w(" check(loopUlt(0x7FFFFFFFFFFFFFF0ULL, 0x8000000000000010ULL), 32u, 0xC001u);")
w(" check(loopSge(-5LL, 5LL), 11u, 0xC002u);")
w(" check(loopUgtDown(0x0000FFFFFFFFFFF8ULL, 0x0001000000000008ULL), 16u, 0xC003u);")
w(" *(volatile u16 *)0x025000UL = gBad;")
w(" *(volatile u16 *)0x025002UL = gFirst;")
w(" *(volatile u16 *)0x025004UL = 0xC0DEu;")
w(" for (;;) {")
w(" }")
w(" return 0;")
w("}")
print("\n".join(o))

240
tests/regress/genMulDiv.py Executable file
View file

@ -0,0 +1,240 @@
#!/usr/bin/env python3
# Emits the C source of the smoke check "mul/div libcalls", with kExp
# holding the expected checksums computed here (C semantics).
#
# Every multiply and divide helper in runtime/src/libgcc.s on N operand
# pairs per operation: an xorshift32 stream (shifts and XORs only, so the
# generator needs no helper it tests), each operand masked to a size class
# (full 32 bits, 16, 9, 8 bits, 4 bits, sign-extended 16 / 8 bits) so every
# fast path and its boundaries are taken, plus the word-boundary values in
# kEdge. Results fold into one checksum per operation.
#
# Mailbox: 0x025000 = bitmask of operations whose checksum differs,
# 0x025002 = 0xC0DE when done.
import sys
N = int(sys.argv[1]) if len(sys.argv) > 1 else 300
EDGE = [0x00000000, 0x00000001, 0x00000002, 0x00000003, 0x0000007F,
0x00000080, 0x000000FF, 0x00000100, 0x00000101, 0x000001FF,
0x00007FFF, 0x00008000, 0x00008001, 0x0000FFFF, 0x00010000,
0x00010001, 0x0001FFFF, 0x00FFFFFF, 0x7FFFFFFF, 0x80000000,
0x80000001, 0xFFFF0000, 0xFFFF7FFF, 0xFFFF8000, 0xFFFFFF80,
0xFFFFFFFE, 0xFFFFFFFF, 0x12345678, 0xFEDCBA98, 0x0000ABCD]
M32 = 0xFFFFFFFF
def xs(x):
x ^= (x << 13) & M32
x ^= x >> 17
x ^= (x << 5) & M32
return x
def cls(v, c):
# Size class c (0..7) of the raw 32-bit value v.
if c == 0:
return v
if c == 1:
return v & 0xFFFF
if c == 2:
return v & 0x1FF
if c == 3:
return v & 0xFF
if c == 4:
return v & 0xF
if c == 5:
return (v & 0xFFFF) | (0xFFFF0000 if v & 0x8000 else 0)
if c == 6:
return (v & 0xFF) | (0xFFFFFF00 if v & 0x80 else 0)
return v & 0x7FFFFFFF
def s32(x):
x &= M32
return x - (1 << 32) if x & 0x80000000 else x
def s16(x):
x &= 0xFFFF
return x - 0x10000 if x & 0x8000 else x
def tdiv(a, b):
q = abs(a) // abs(b)
return -q if (a < 0) != (b < 0) else q
def tmod(a, b):
return a - tdiv(a, b) * b
OPS = ['mulU32', 'mulU16x', 'mulS16x', 'mulU16', 'divU32', 'modU32',
'divS32', 'modS32', 'divU16', 'modU16', 'divS16', 'modS16']
def ev(op, a, b):
if op == 'mulU32':
return (a * b) & M32
if op == 'mulU16x':
return ((a & 0xFFFF) * (b & 0xFFFF)) & M32
if op == 'mulS16x':
return (s16(a) * s16(b)) & M32
if op == 'mulU16':
return (a * b) & 0xFFFF
if op == 'divU32':
return a // b
if op == 'modU32':
return a % b
if op == 'divS32':
return tdiv(s32(a), s32(b)) & M32
if op == 'modS32':
return tmod(s32(a), s32(b)) & M32
if op == 'divU16':
return (a & 0xFFFF) // (b & 0xFFFF)
if op == 'modU16':
return (a & 0xFFFF) % (b & 0xFFFF)
if op == 'divS16':
return tdiv(s16(a), s16(b)) & 0xFFFF
return tmod(s16(a), s16(b)) & 0xFFFF
def fix(op, a, b):
# The divisor must not be zero, and INT_MIN / -1 is undefined.
if op.startswith('div') or op.startswith('mod'):
w16 = op.endswith('16')
if (b & 0xFFFF if w16 else b) == 0:
b |= 1
if op.endswith('S32') and a == 0x80000000 and b == M32:
b = 1
if op.endswith('S16') and (a & 0xFFFF) == 0x8000 and (b & 0xFFFF) == 0xFFFF:
b = 1
return a, b
def rot(h):
return ((h << 5) | (h >> 27)) & M32
exp = []
for op in OPS:
h = 0
x = 0x2545F491
pairs = [(a, b) for a in EDGE for b in EDGE]
for i in range(N):
x = xs(x)
a = cls(x, (x >> 8) & 7)
x = xs(x)
b = cls(x, (x >> 8) & 7)
pairs.append((a, b))
for a, b in pairs:
a, b = fix(op, a, b)
h = rot(h) ^ ev(op, a, b)
exp.append(h)
print('''// Generated by tests/regress/genMulDiv.py: every multiply / divide helper
// on the word-boundary values and an xorshift32 stream of size-classed
// operands; one checksum per operation against kExp (host C semantics).
// Mailbox: 0x025000 = bitmask of mismatching operations, 0x025002 =
// 0xC0DE when done. int is 16 bits and long 32 here.
typedef unsigned short u16;
typedef short s16;
typedef unsigned long u32;
typedef long s32;
#define NOI __attribute__((noinline)) static
#define NOPS %d
#define NRAND %d
#define NEDGE %d
static const u32 kEdge[NEDGE] = { %s };
static const u32 kExp[NOPS] = { %s };
NOI u32 mulU32(u32 a, u32 b) { return a * b; }
NOI u32 mulU16x(u16 a, u16 b) { return (u32)a * (u32)b; }
NOI s32 mulS16x(s16 a, s16 b) { return (s32)a * (s32)b; }
NOI u16 mulU16(u16 a, u16 b) { return (u16)(a * b); }
NOI u32 divU32(u32 a, u32 b) { return a / b; }
NOI u32 modU32(u32 a, u32 b) { return a %% b; }
NOI s32 divS32(s32 a, s32 b) { return a / b; }
NOI s32 modS32(s32 a, s32 b) { return a %% b; }
NOI u16 divU16(u16 a, u16 b) { return a / b; }
NOI u16 modU16(u16 a, u16 b) { return a %% b; }
NOI s16 divS16(s16 a, s16 b) { return a / b; }
NOI s16 modS16(s16 a, s16 b) { return a %% b; }
static u32 xs(u32 x) {
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
return x;
}
static u32 cls(u32 v, u16 c) {
switch (c) {
case 0: return v;
case 1: return v & 0xFFFFUL;
case 2: return v & 0x1FFUL;
case 3: return v & 0xFFUL;
case 4: return v & 0xFUL;
case 5: return (v & 0xFFFFUL) | ((v & 0x8000UL) ? 0xFFFF0000UL : 0UL);
case 6: return (v & 0xFFUL) | ((v & 0x80UL) ? 0xFFFFFF00UL : 0UL);
default: return v & 0x7FFFFFFFUL;
}
}
static u32 ev(u16 op, u32 a, u32 b) {
if (op >= 4) {
u16 w16 = (op >= 8);
if ((w16 ? (b & 0xFFFFUL) : b) == 0UL) {
b |= 1UL;
}
if ((op == 6 || op == 7) && a == 0x80000000UL && b == 0xFFFFFFFFUL) {
b = 1UL;
}
if ((op == 10 || op == 11) && (a & 0xFFFFUL) == 0x8000UL && (b & 0xFFFFUL) == 0xFFFFUL) {
b = 1UL;
}
}
switch (op) {
case 0: return mulU32(a, b);
case 1: return mulU16x((u16)a, (u16)b);
case 2: return (u32)mulS16x((s16)a, (s16)b);
case 3: return mulU16((u16)a, (u16)b);
case 4: return divU32(a, b);
case 5: return modU32(a, b);
case 6: return (u32)divS32((s32)a, (s32)b);
case 7: return (u32)modS32((s32)a, (s32)b);
case 8: return divU16((u16)a, (u16)b);
case 9: return modU16((u16)a, (u16)b);
case 10: return (u16)divS16((s16)a, (s16)b);
default: return (u16)modS16((s16)a, (s16)b);
}
}
int main(void) {
u16 bad = 0;
u16 op;
*(volatile u16 *)0x025002UL = 0u;
for (op = 0; op < NOPS; op++) {
u32 h = 0UL;
u32 x = 0x2545F491UL;
u16 i;
u16 j;
for (i = 0; i < NEDGE; i++) {
for (j = 0; j < NEDGE; j++) {
h = ((h << 5) | (h >> 27)) ^ ev(op, kEdge[i], kEdge[j]);
}
}
for (i = 0; i < NRAND; i++) {
u32 a;
u32 b;
x = xs(x);
a = cls(x, (u16)((x >> 8) & 7UL));
x = xs(x);
b = cls(x, (u16)((x >> 8) & 7UL));
h = ((h << 5) | (h >> 27)) ^ ev(op, a, b);
}
if (h != kExp[op]) {
bad |= (u16)(1u << op);
}
}
*(volatile u16 *)0x025000UL = bad;
*(volatile u16 *)0x025002UL = 0xC0DEu;
for (;;) {
}
return 0;
}''' % (len(OPS), N, len(EDGE), ', '.join('0x%08XUL' % v for v in EDGE),
', '.join('0x%08XUL' % v for v in exp)))

88
tests/regress/setjmpImgSave.c Normal file
View file

@ -0,0 +1,88 @@
// Regression test: a longjmp must not hand a caller back IMG8..IMG15
// ($C0..$CE) words that a function it unwound overwrote.
//
// IMG8..IMG15 survive calls because every function that writes one gives
// its caller the old value back on return (W65816ImgCalleeSave, and the
// entry/return swaps of W65816StackRelToImg's IMG8..15 pointer homes). A
// longjmp skips those returns. walk() keeps q and r in $C2..$C8 and
// longjmps out of ext(); outer() keeps ITS q and r in the same words
// across the call to f0(). Before the fix outer() came back from f0()
// walking walk()'s r (bufF) instead of its own (bufE).
//
// Fix: a call that can return twice (setjmp) clobbers IMG8..IMG15
// (W65816TargetLowering::AdjustInstrPostInstrSelection), so f0() keeps
// nothing in them across setjmp and saves and restores all eight for
// outer().
//
// outer(bufB, bufC, bufE, 2) = 2 + bufC[1] (5) + bufE[2] (9) = 16; the
// broken build gave 9, 5, 8.
#include <setjmp.h>
static jmp_buf gJb;
volatile int gSink;
int bufA[4] = {5, 0, 0, 0};
int bufB[4] = {0, 1, 2, 3};
int bufC[4] = {0, 4, 5, 6};
int bufD[4] = {9, 9, 9, 9};
int bufE[4] = {0, 0, 8, 0};
int bufF[4] = {0, 0, 0, 0};
__attribute__((noinline)) void ext(int v) {
gSink = v;
if (v == 7) {
longjmp(gJb, 1);
}
}
// q and r live across the calls to ext in IMG8..15 homes.
__attribute__((noinline)) void walk(int *p, int *q, int *r, int n) {
int i = 0;
do {
ext(p[0]);
q[1] += i;
r[2] ^= q[1];
p[2] = r[2];
p[0]++;
} while (++i != n);
}
__attribute__((noinline)) int f0(int *p, int *q, int *r) {
if (setjmp(gJb)) {
return 1;
}
walk(p, q, r, 3);
return 0;
}
// Its own q and r in the same IMG8..15 words across the call to f0.
__attribute__((noinline)) int outer(int *p, int *q, int *r, int n) {
int i = 0;
int t = 0;
do {
t += f0(bufA, bufD, bufF);
q[1] += i;
r[2] ^= q[1];
p[3] = r[2];
} while (++i != n);
return t + q[1] + r[2];
}
int main(void) {
volatile int *out = (volatile int *)0x025000;
out[1] = outer(bufB, bufC, bufE, 2); // 0x0010
out[2] = bufC[1]; // 0x0005
out[3] = bufE[2]; // 0x0009
out[0] = 0xC0DE;
while (1) {
}
return 0;
}