Matching Techniques & Blockers

This page documents everything we've learned about producing byte-exact C matches with agbcc (a GCC 2.95 fork targeting ARM7TDMI Thumb). Of 522 functions, 143 are fully matched (27.4%). The remaining 379 are understood (470+ have semantic names and docstrings) but blocked by compiler code generation differences. Additionally, 61 IWRAM globals and 59 ROM data tables have been identified and named.

The actual Nintendo SDK compiler (mid-kid/arm-000512) has been evaluated and produces identical output to agbcc with -fprologue-bugfix, confirming our toolchain is correct. Our -ftst and -fcmp-elim patches go beyond what the base SDK provides.

Current State by Module

system.c7 of 12 matched(ReadKeyInput, FixedMul8, FreeAllDecompBuffers, ...)
math.c8 of 9 matched(FixedDivShift8, FixedReciprocal, FixedMul4, FixedDivUnit, ...)
engine.c5 of 27 matched(OAM clears, video state, fade effect, UpdateWindowCircleEffect, ...)
code_0.c1 of 24 matched(InitOamEntries)
code_1.c8 of 109 matched(TransitionToWorldMap, TransitionSoftReset, ...)
code_3.c8 of 88 matched(IsSelectButtonPressed, GetEntityLookupData, RollRandomLevelVariant, ...)
gfx.c48 of 124 matched(DMA-fill, stream commands, scroll, level defaults, HBlank wait, ...)
m4a.c + m4a_1.c52 of 113 matched(SoundContextRef, SoundInfoInit, VoiceLookupAndApply, ply_*, ...)
syscalls.c5 of 5 matched(all BIOS call wrappers)
util.c1 of 11 matched(EepromTimerCallback; rest blocked by loop patterns)

Solved Techniques

These techniques allow us to control agbcc's code generation to match the original ROM.

1. ASM Barrier for Constant Load Scheduling SOLVED

Problem: agbcc loads literal pool constants after computing ALU operations on arguments. The original code loads constants before the ALU operations (filling pipeline stall slots).

Original (ROM)
ldr r1, =TABLE    ; load first
lsls r0, r0, #2   ; then compute
adds r0, r0, r1
agbcc default
lsls r0, r0, #2   ; compute first
ldr r1, =TABLE    ; then load
adds r0, r0, r1

Solution: Use an inline asm barrier to force the constant into a register at a specific point in the code:

u32 addr = 0x08189B4C;
u32 *table;
asm("" : "=r"(table) : "0"(addr));
// compiler now loads addr via ldr BEFORE any subsequent code

Matched: FinalizeLevelLayerSetup, LoadAndDecompressStream, TransitionToWorldMap, FadeOutController, SetupGfxCallbacks, all m4a stream commands, and ~20 more.

2. Shift-Pair Interleaving SOLVED

Problem: When zero-extending a parameter to u16 (lsls r0, #16; lsrs r0, #13), the original interleaves constant loads between the two shifts. agbcc keeps the shifts together.

Original
lsls r0, r0, #16   ; start truncation
ldr r2, =BASE      ; interleaved load
ldr r1, =TABLE     ; interleaved load
lsrs r0, r0, #13   ; finish truncation
Solution
u32 shifted = idx << 16;
asm("" : "+r"(shifted)); // force materialization
asm("" : "=r"(base) : "0"(a0));
asm("" : "=r"(table) : "0"(a1));
shifted >>= 13; // now after the loads

Matched: GetEntityLookupData, InitSceneState, m4aSongNumStop, m4aMPlayCommand.

3. Integer Arithmetic for adds Operand Order SOLVED

Problem: table + offset compiles to adds rTABLE, rTABLE, rOFF (result in table's register), but the original has adds rOFF, rOFF, rTABLE (result in offset's register).

Solution: Use integer arithmetic instead of pointer arithmetic:

// Instead of: entry = table + offset;
// Write:
offset += (u32)table;
entry = (u8 *)offset;
// Result stays in offset's register

Matched: GetEntityLookupData, InitSceneState, m4aSongNumStop, m4aMPlayCommand.

4. Forced Dead Copy via register asm SOLVED

Problem: GCC 2.95 sometimes generates a copy instruction (adds r1, r0, #0) before modifying a value. Modern optimization would eliminate this dead copy. We need it for matching.

s32 result = (s32)a * (s32)b;
register s32 shifted asm("r1");
shifted = result;                // forces: adds r1, r0, #0
asm("" : "+r"(shifted));         // prevent optimization
if (result < 0) shifted += 0xFF; // modifies r1, checks r0

Matched: FixedMul8 (8.8 fixed-point multiply).

5. Memory-Before-Constant Scheduling SOLVED

Problem: When loading both a memory value and a constant, the compiler may schedule the constant load first. The original loads from memory first.

u32 val = sceneCtrl[0];       // memory load
asm("" : "+r"(val));          // force it to materialize NOW
u32 addr = 0x03005498;
u8 *fadeCounter;
asm("" : "=r"(fadeCounter) : "0"(addr)); // constant load AFTER

Matched: FadeOutController.

6. Return Type for Pop Register Selection SOLVED

Problem: Non-leaf functions end with pop {rN}; bx rN. Most use r0, but some use r1 (to preserve the return value in r0). agbcc always uses r0 for void functions.

Solution: Declare the function and its callee as returning u32. The compiler then preserves r0 for the return value and pops to r1.

u32 MidiProcessEvent(void);     // declare as returning u32
u32 MPlayMain_SetAndProcess(u32 val) {
    // ...
    return MidiProcessEvent();  // compiler uses pop {r1}; bx r1
}

Matched: MPlayMain_SetAndProcess.

7. -fprologue-bugfix Compiler Flag SOLVED

Problem: agbcc generates push {lr}; ... pop {r0}; bx r0 for leaf functions that don't need to save lr. The original uses direct bx lr.

Solution: Add -fprologue-bugfix to CC1FLAGS. This makes the compiler emit bx lr for true leaf functions (no calls, no callee-saved registers needed).

Matched: All leaf functions without callee-saved registers (ReadUnalignedU16, etc.).

8. Register Pinning via asm Calls SOLVED

Problem: agbcc's register allocator chooses different registers than the original for struct member access and multi-argument function calls. This causes widespread mismatches.

Solution: Use asm("") barriers to force the compiler to materialize values in specific registers at specific points. This pins the register allocation without changing the generated instructions:

register u32 val asm("r4") = someExpr;
asm("" : "+r"(val));  // force r4 to hold val here
// subsequent code uses r4 as the original did

Matched: Many m4a functions including SoundInfoInit, ply_voice, ply_bend, ply_bendr, ply_lfos, ply_lfodl, ply_mod, and numerous gfx stream commands.

9. -ftst Compiler Flag SOLVED

Problem: The original ROM uses tst Rn, Rm (test without storing result) for bitflag checks. agbcc generates ands Rd, Rn + cmp Rd, #0 instead.

Solution: A custom -ftst flag was added to the Dream-Atelier agbcc fork. It adds a *andsi3_tst combiner pattern in thumb.md that merges and + cmp #0 into tst. Enabled per-compilation-unit for m4a_1.c.

Matched: VoiceLookupAndApply and other m4a functions that check bit flags.

10. Integer Arithmetic for adds Operand Order SOLVED

Problem: base + offset compiles to adds rOFF, rOFF, rBASE (result in offset's register), but the original has adds rOFF, rBASE, rOFF or vice versa.

// Instead of: entry = (u8 *)(tbl + off);
// Write:
off += (u32)tbl;
entry = (u8 *)off;
// Controls which register gets the adds result

Matched: StreamCmd_SetBGScroll, ProcessHBlankWait.

11. Early Return for Branch Direction SOLVED

Problem: if (x < 0) { cleanup; return 0; } return 1; generates bge .Lstill_waiting (skip cleanup), but the original has blt .Lcleanup (jump to cleanup).

Solution: Use early return 1; before the cleanup code so the "still waiting" path is the fall-through, forcing the compiler to use blt for the forward branch to cleanup.

Matched: ProcessHBlankWait.

12. Variable Reuse for Register Reuse SOLVED

Problem: Sequential constants go to different registers. The original reuses r0 for multiple constants: movs r0, #0; strh; movs r0, #0xE8; ...

u16 val = 0;
buf[4] = val;     // movs r0, #0; strh r0, [r1, #8]
val = 0xE80;
buf[8] = val;     // movs r0, #0xE8; lsls r0, #4; strh r0, [r1, #16]
// Same variable → same register

Matched: InitLevelStateDefaults.

Remaining Blockers

These patterns prevent matching the remaining ~379 functions. Each affects a different subset of the codebase. Solving any one of them would unlock dozens of new matches.

A. tst vs ands for Bitflag Checks ~60 functions

When checking if (flags & MASK), the original generates tst r0, r1 (test without storing result). agbcc generates ands r0, r1 (AND with store). Both set the same flags, but encode as different opcodes.

Original
movs r0, #0x80
tst r0, r1      ; 4208 — test only
beq skip
agbcc generates
movs r1, #0x80
ands r0, r1     ; 4008 — AND + store
cmp r0, #0
beq skip

Affected: SoundContextRef, VoiceLookupAndApply, MPlayContinue, many entity behavior functions, all functions that check bit flags.

Status: Partially solved by -ftst flag (see Technique 9). Remaining issue: original also uses ands; beq (no redundant cmp) which agbcc cannot produce.

B. Leaf Functions with Unnecessary push {rN, lr} ~40 functions

Leaf functions (no bl calls) that use callee-saved registers (r4-r7) have push {r4, lr} in the original. With -fprologue-bugfix, agbcc generates push {r4} (without lr) and pop {r4}; bx lr. The original has push {r4, lr}; ... pop {r4}; pop {r0}; bx r0.

Affected: EnableInterruptsAfterGfxSetup, StreamCmd_SetSoundReverb, all stream command leaf handlers that use r4+, DmaSpriteToObjVram, SetPaletteAnimEntry.

Dilemma: Removing -fprologue-bugfix would fix these but break all the simpler leaf functions (bx lr) that currently match. Per-function compiler flags are not supported by the build system.

C. Non-Word-Aligned Function Starts ~20 functions

Some functions start at 2-byte-aligned addresses (non_word_aligned_thumb_func_start). C functions always start at 4-byte-aligned addresses. Replacing these with C adds 2 bytes of alignment padding, shifting all subsequent code.

Affected: StreamCmd_StopSound, StreamCmd_Nop3, StreamCmd_ClearRenderMode, HBlankScrollUpdate, AcknowledgeInterrupt, FixedMul4, many m4a command handlers (MPlayCmd_SetBendRange, MPlayCmd_SetLFOSpeed, etc.).

Status: Fundamentally impossible from C. These functions can only be kept as inline assembly.

D. Shared Literal Pools ~10 functions

Adjacent functions share literal pool entries. The last 2 bytes of function A's literal pool happen to be the first instruction of function B (or vice versa). Decompiling function A to C generates a full 4-byte pool entry, adding 2 bytes that break function B's ldr displacement.

Original (shared)
FreeGfxBuffer:
  ...
  bx r0
  .short 0x0000      ; padding
  .short 0x34A0      ; low 2 bytes of pool
DeadCode:             ; starts mid-pool!
  lsls r0, r0, #12   ; = 0x0300 = high bytes
C compiled (broken)
FreeGfxBuffer:
  ...
  bx r0
  .short 0x0000      ; padding
  .word 0x030034A0   ; full 4-byte pool
DeadCode:             ; now 2 bytes later!
  lsls r0, r0, #12   ; shifted

Affected: FreeGfxBuffer, LoadBGTileData, StreamCmd_SetBlendMode, StreamCmd_DmaSpriteData, several adjacent stream command pairs.

Status: Can only be solved by decompiling both functions simultaneously (not supported by INCLUDE_ASM macro).

E. Small Constant Pool Loads ~5 functions

Some functions load small constants (like 4) from the literal pool via ldr r0, =4 instead of using movs r0, #4. This happens when the constant participates in a u16 truncation pattern. agbcc always uses movs for constants 0-255, which is 4 bytes smaller (no pool entry needed).

Affected: StopAllMusicPlayers, StopSoundEffects, m4aSoundInit_Impl.

Status: No known way to force a pool load for a small constant from C.

F. Register Allocation for Struct Member Stores ~100+ functions

When a function stores multiple values to struct members, the compiler's register allocator may choose different registers than the original. This affects which register holds the struct pointer, which holds intermediate values, and the order of operations.

The register type var asm("rN") constraint can sometimes fix this, but it often causes cascading changes in other register assignments. The more constrained registers, the more likely other parts of the code break.

Affected: InitLevelStateDefaults, TransitionReturnToWorldMap, TransitionToSceneSelect, TransitionToTitleScreen, most functions with 3+ struct member stores.

Status: Case-by-case manual tuning. No systematic solution.

G. Loop Increment Byte Truncation Placement ~15 functions

When incrementing a u8 loop counter, the original truncates at the increment point (adds r0, r6, #1; lsls r0, #24; lsrs r6, r0, #24). agbcc with register u8 asm("r6") either: (a) increments in-place (adds r6, #1) without truncation, or (b) truncates at the comparison point instead.

Affected: SaveGameRetry, entity spawn loops, any loop with u8 counter.

Status: No known workaround. The compiler's truncation placement is not controllable.

H. Hand-Written Assembly Patterns ~15 functions

Some functions (especially in the m4a sound engine) use hand-written assembly patterns that no C compiler would generate:

Affected: InstrumentLookup, SoundEffectChain, MidiDecodeByte, several m4a command handlers.

Status: These can only remain as inline assembly. They were never C to begin with.

Function Breakdown

Approximate categorization of all 522 functions:

CategoryCountStatus
Matched to C143Done
Blocked by tst vs ands~60Partially solved (-ftst -fcmp-elim)
Blocked by leaf prologue~40Many actually match (see InitOamEntries)
Non-word-aligned~20Impossible from C
Hand-written asm~15Impossible from C
Shared literal pools~10Needs paired decompilation
Register allocation / scheduling~100+Case-by-case with asm barriers
Large/complex functions~134Need more investigation

The Nintendo SDK compiler (mid-kid/arm-000512) produces identical output to agbcc, confirming the remaining mismatches are from source-level differences, not compiler differences. Progress depends on per-function C tuning using the techniques above.