[DRAFT] integrate/faithful-adopt-caner: SPU lifter faithful adoption + LBP SPU/SPURS + RSX textures - #82
Draft
sp00nznet wants to merge 207 commits into
Draft
[DRAFT] integrate/faithful-adopt-caner: SPU lifter faithful adoption + LBP SPU/SPURS + RSX textures#82sp00nznet wants to merge 207 commits into
sp00nznet wants to merge 207 commits into
Conversation
The FP update forms (lfsu/lfdu/stfsu/stfdu) were lifted identically to their non-update siblings: EA computed from rA+disp but never written back to rA. gcc walks float arrays with `lfsu f,4(rN)`, so every later access through rN silently re-read element 0. wave's _XyToPolar loaded y via lfsu then x through the "advanced" base -- the lift computed sqrt(x*x+x*x) instead of sqrt(x*x+y*y), turning the hue-palette disc into a sqrt(2)-wide vertical band (the "square colour wheel"). Same class of bug fixed alongside: indexed FP update forms (lfsux/lfdux/stfsux/stfdux) were unhandled entirely, indexed integer update stores (stbux/sthux/stwux) lacked the writeback, and ldux/stdux were missing from the indexed maps. New mem_fp_update KAT section in torture_mem.c covers value + EA for all of the above (they fail against the old lifter, pass now). The O0 switch-section noise in the torture run is the pre-existing layout-sensitive jump-table lift bug shifting with the new guest code; no update-form instruction occurs outside the new KATs in either variant. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The vblank ticker advanced its beat with GetTickCount64 + 16ms integer steps = exactly 62.5Hz, which leaked into every flip-synced title as 62-63 fps. Accumulate QPC ticks at qpf*1001/60000 instead: long-run rate is the PS3's NTSC 60000/1001 Hz (titlebar now reads 59/60), independent of the host display. The ~1ms high-res drain cadence is unchanged. rsx backend: TEX_SAVE=1 dumps the first few converted ARGB texture uploads as rgb+alpha BMP pairs -- ground truth for "is the guest texture wrong or the sampling wrong" (this is what proved wave's colour wheel was baked wrong by the guest, not mis-sampled: alpha inside-region was a vertical band, not a disc). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cellPngDec.c was written but never exercised -- no prior sample decoded a PNG. DeferredShading (skin.png + font_tex.png) is the first, and it faulted in cellPngDecCreate writing the output handle to host address 0x8EF88: the HLE bridge forwards raw gpr values, so a `CellPngDecMainHandle*` arrives as a big-endian GUEST address, not a host pointer. The whole file assumed host-native little-endian struct access. Three ABI bugs, all against the real SDK <cell/codec/pngdec.h>: - No translation/byte-swap. Every pointer param is now a u32 guest address; struct fields are marshalled with vm_read32/vm_write32/vm_read64 at explicit big-endian offsets rather than overlaying a host struct. - CellPngDecSrc layout was wrong: the repo header modelled fileName as an inline char[1024], but the SDK has it as a char* at +4 -- which mis-placed fileOffset/streamPtr/streamSize by ~1KB. fileName is now read as a guest string; the working copy is host-side only. - cellPngDecDecodeData had the wrong arity: the real SDK takes 5 args with dataCtrlParam (carrying outputBytesPerLine, the destination pitch) BEFORE dataOutInfo. The 4-arg version read dataOutInfo from what was actually the pitch struct. DecodeData now honours the guest pitch and writes decoded pixels row-by-row into the guest output buffer via vm_base. Result: both DeferredShading textures decode + upload correctly (256x256 ARGB, verified in-log). No other port calls cellPngDec, so no regression surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
read_vp_vertex passed the raw vertex-array offset to cellGcmResolveOffset, but the NV4097_SET_VERTEX_DATA_ARRAY_OFFSET register carries the context-DMA location in bit 31: 0 = LOCAL (VRAM), 1 = MAIN (IO-mapped system memory). DeferredShading is the first sample to put its meshes in the main heap, so every attrib offset was 0x80xxxxxx -- resolveOffset computed page 0x80x, missed the IO table, and read garbage VRAM (vertices came back as -7.992 => degenerate => empty G-buffer, nothing rasterized). Strip bit 31 and resolve MAIN via cellGcmResolveLocated; local offsets keep the old path, so no change for wave/gcmcube/cellmark/vkcube (all VRAM verts). DeferredShading geometry now rasterizes (was entirely absent). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The SET_VERTEX_DATA_ARRAY_FORMAT parser dropped bits [31:16] -- the vertex frequency divisor -- and SET_FREQUENCY_DIVIDER_OPERATION (0x1FC0) was unhandled, so instanced geometry was fetched as if every attribute advanced per-vertex. DeferredShading's cube rings confirmed it: attrib0/attrib2 (the 24-vertex cube mesh) have freq=24 with the divider-op MODULO bit set (repeat the mesh per instance), and attrib9 (per-instance transform) has freq=72 in DIVIDE mode (advance once per instance). Reading attrib9 per vertex gave every instance a garbage transform. read_vp_vertex now maps the fetch index through the divisor: freq>1 uses vertex%freq (MODULO) or vertex/freq (DIVIDE) per the per-attribute op bit; freq 0/1 stays per-vertex, so the non-instanced samples (cellmark/wave/ gcmcube, all freq=0 -- verified) are unaffected. Instancing infrastructure only: DeferredShading's cubes are still blocked downstream by garbage G-buffer MVP constants (separate issue). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A title that double-buffers the display (DeferredShading clears both 0x0 and 0x440000 per frame, plus a HUD pass) issues several display-buffer clears per real frame. d3d12_clear treated every one as a frame boundary and presented the accumulated batch, so the 1-2-draw intermediates strobed black between the full ~190-draw frames -- violent black/white flicker. Anchor the boundary to the FLIP: a display clear only presents once a cellGcmSetFlip has landed since the last present, and only when the batch is a substantial fraction of the running-max frame size (so an intermediate that arrives before the full frame's clear keeps accumulating instead of presenting alone). Titles that flip once per frame (cellmark/gcmcube, both verified 14/14) and titles that never call SetFlipCommand (fc stays 0) keep the old clear-only heuristic -- no behavior change for them. Also: VP_NOFREQ instancing kill-switch, FLIP_DBG trace env. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vmrghw/vmrglw and the byte/halfword variants were unhandled -> silent no-ops. DeferredShading's Vectormath::Aos::Matrix4 assembles its projection rows with vmrghw (sgSceneEnvironment::setCameraProjection), so the projection matrix vector kept whatever stale VRAM bytes were in the register -- the guest uploaded 0xC0FFC0FF (a VRAM pointer, = -7.99f) into the transform constants, so every G-buffer mesh transformed offscreen and the instanced cube scene was empty (only the baseplate survived, via the garbage-projection fallback). Merge interleaves the HIGH (elements 0..n/2-1) or LOW half of vA and vB per big-endian element order (byte 0 = element 0, matching vsldoi/ vspltw); temps guard vD aliasing vA/vB (`vmrghw v0,v0,v13`). With it the transform constants come out a correct perspective matrix (0.97, 1.73, -1.02, -2.02, -1.0) and the scene geometry submits (prim=6 strips). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The FP decompiler formatted embedded constants with %g, which prints the bare tokens "nan"/"inf" for non-finite values. HLSL rejects those (X3004 undeclared identifier 'nan'), so any fragment program with a NaN/Inf constant failed to compile and its pass rendered black -- DeferredShading's light passes embed one. Reproduce non-finite lanes bit-exactly via asfloat(0xBITS); finite values keep %g. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Capstone decodes the VA-form vector FMAs as [vD, vA, vC, vB], so ops[2] is the multiplicand (vC) and ops[3] is the addend (vB). The lifter had these swapped, computing vA*vB+vC instead of vA*vC+vB, corrupting every VMX matrix multiply (Vectormath::Aos::Matrix4). DeferredShading's transformHierarchy world*view multiply produced huge 1e37 model-eye matrices as a result. After the fix the transform constants are clean (garbage slots 40+ -> 0), with no regression (gcmcube 10/12, vkcube 11/12, cellmark text intact). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Purpose-built RSX frame capture ("rsxcap"): RSX_CAP=start[:count[:stride]]
snapshots whole frames to <RSX_CAP_DIR>/frame_NN/ -- a text manifest of every
op in submission order (surface targets, viewport, FP/VP, textures, colour
mask, blend) plus a BMP of the backbuffer and every offscreen colour RT used
that frame, captured atomically. Also dumps each geometry draw's VP constant
bank (candidate MVP) for transform debugging. Replaces cross-referencing draw
lists across separate runs, which is unreliable once a title cycles scenes.
current_rt_off gains an RT_OFFDBG surface-routing trace.
cmask: RenderTargetWriteMask used (cmask ? cmask : 0xF), turning a guest's
explicit all-channels-off mask (DeferredShading's depth-only shadow pass) into
write-all, so the depth pass splattered fragment colour onto its target. cmask
is already 0xF by default when unset, so a literal 0 is genuine -- honour it.
Capture confirmed DeferredShading's G-buffer geometry (142 MRT draws to
0xCC0000) rasterizes zero fragments even under FP_FORCE (camera transform is
off-screen) while the light-space shadow pass draws fine; c[0..3] of those
draws is zero with real constants up to slot 467.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Frame capture now dumps the full set of non-zero VP constant slots for the first offscreen geometry draw (not just c[0..3]), which pinpointed the projection block at c[256..259]. TCONST_DBG gains a slot 254..260 filter (TCONST_ALL for the old first-64 behaviour) so the projection upload is visible past the early flood. Together these traced DeferredShading's black screen to the camera projection: c[256].x/c[257].y are uploaded correctly (0.9743/1.7321) for the first ~10 frames then collapse to 0.0667 for both axes in steady state -- a guest-side miscompute in setCameraProjection (values arrive wrong at the TCONST upload, so not a backend bug), squashing all geometry to a sub-pixel centre strip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… repair Three lifter-correctness fixes surfaced by the DeferredShading (first VMX matrix-math title) and LittleBigPlanet (first retail game) bring-ups. 1. VMX float byte-order. ctx->vr holds RAW big-endian guest bytes (lvx is a plain 16-byte memcpy). Value-interpreting ops read (float*)&ctx->vr, which reads BE bytes as host little-endian -> a BE 1.0 (0x3F800000) becomes the denormal 0x0000803F. Added ppu_vldf4/ppu_vstf4/ppu_vldu4/ppu_vstu4 helpers (per-lane bswap) and rewrote every value op: FMA, add/sub/mul, FP compares, recip/rsqrt/floor, int<->float convert, splat-imm, word shifts. Byte/word MOVE ops (vperm, vmrgh*, vsldoi, vspltw, vsel, bitwise) are endian-correct on the raw bytes and stay untouched. 2. FMA operand order (vmaddfp/vnmsubfp). The lifter uses ppu_disasm.py, whose VA-form printer emits ENCODING-field order "vD, vA, vB, vC" -- ops[2] is the addend (vB), ops[3] the multiplicand (vC). The prior fix assumed capstone's ISA order (vD,vA,vC,vB) and swapped them, so the Vectormath dot-product square `vmaddfp vD,vA,v0zero,vA` (= vA*vA+0) degenerated into a copy (vA*0+vA) -> len^2 collapsed -> rsqrt -> NaN view matrix -> black screen. Byte-proven: 0x1e1a8 = 0x11AC4B2E enc(vD=13,vA=12,vB=9,vC=12). Corrected to vb=ops[2], vc=ops[3] (d = vA*vC + vB). 3. Fall-through gap repair. IDA exports sometimes truncate a function's end mid-body on a non-terminator (LBP malloc wrapper 0x5E7808 exported 8 bytes long, real body 48). The fragment falls through, and with no function at the end address the emit fallback trampolined to the next listed function -- silently skipping the tail (the `bl mspace_malloc`), so malloc returned NULL -> operator new threw bad_alloc -> terminate -> abort at boot. New pass: for any bound whose last insn is not b/ba/blr/bctr/rfi and whose end has no function, synthesize a tail fn [end, next_start), to fixpoint. Systemic fix for the fragmented/r1-drift class (obviates cellmark's hand-patches). Verified: DeferredShading renders its 3D scene; LBP boots through system init to the render window; cellmark clean re-lift runs + draws text with no hand-patches; gcmcube/wave/vkcube use no VMX float value ops (no-op for them). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…le strips Driven by DeferredShading's G-buffer deferred renderer (screenshots showed a flat-lavender normals view, blown-out light passes, and missing tentacle meshes vs the Sony reference). - MRT C/D binding. SET_SURFACE_COLOR_TARGET 0x17/0x1F (MRT2/MRT3 = A+B+C[+D]) bound only A+B before, so the G-buffer's normal/position targets were never written -> normals view = the untouched clear colour and lighting read garbage. rt_off2 -> rt_mrt[3] threaded through the draw record, current_rt_off, the RT warm-up, the PSO (nrt via dr_num_rts), the submit loop (OMSetRenderTargets up to 4), ordered clears, the sampled-RT transition guard, and rsxcap. - Guest blend factors/equation. The PSO hardcoded straight alpha; deferred's light accumulation is additive (ONE,ONE), so each spot light blew the frame to white. rsx_blend_key() packs the guest SFACTOR/DFACTOR/EQUATION (GL enums) into D3D12 blend enums as a PSO key, with a legacy straight-alpha fallback when factors are never programmed (dbgfont text). - Indexed TRIANGLE_STRIP/FAN (prim 6/7) were skipped -> the spline-tube meshes (octopus tentacles) never drew. Expand CPU-side to a triangle list (upload_strip_vp_indexed; CullMode is NONE so strip winding parity is moot). - RTT_DUMP now skips empty init frames so PNG-decode-heavy titles don't burn the capture budget before the first real draw. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…user paths LittleBigPlanet boot progression. Several HLE functions dereferenced their pointer parameters as host pointers, but the generic adapter passes PPC registers raw so those are GUEST effective addresses -- each was a host access violation the moment the title called it. - cellGameDataCheck: printed dirName (raw EA) and returned ERROR_NOTFOUND for a missing game-data dir; translate before use and return CELL_GAME_RET_NONE (firmware/RPCS3 semantics) so the title creates its data instead of bailing. - cellSysCacheMount: marshalled CellSysCacheParam (cacheId in @+0, path out @+0x20) instead of strncpy through the raw EA. - cellSysutilGetBgmPlaybackStatus, cellDiscGameGetBootDiscInfo: vm_write the out-params. - cellSaveData: translate dirName in AutoSave/AutoLoad (+_2) and add the User* wrappers (userId after version; the 9th arg is on the guest stack). - cellRtc: dual host/guest accessors (guest fields are big-endian) for GetCurrentTick/Clock*/Time_t/SetTick/GetTick, plus the missing cellRtcGetCurrentClock. - cellSysmodule: the module-ID table was scrambled (a stray SYNC2=0x0E shifted everything after FS); rewrite it from the firmware enum, and accept the 0xF0xx extended range (LBP's NP_TROPHY = 0xF035 was rejected every boot). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Diagnostics used to root-cause the LBP boot flow (all env-gated or bounded, no cost when off): - ppu_log_host_chain(): resolves the host call stack to guest function addrs via the lifted dispatch table (function_table). The direct-call model leaves guest LR slots stale, so a host backtrace mapped back to guest addresses is the only reliable caller chain -- this is what pinned the boot state-machine stage that starves (sub_439E50) and the abort caller. - PS3_CALLTRACE=N (ppu_loader): log the first N indirect calls (target + r3/r4 + tid) -- vtable/callback/job-body dispatch visibility. - POLLSITE (sys_timer): dump the guest chain of the 1ms poll site. - sys_timer usleep trace gains lr/cia; sys_semaphore_post gains a [WAKE] trace; sys_lwmutex create/destroy counters (LWM_COUNT). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion) build_spu_workloads.py generalizes the flOw-hardcoded gen_spu_workloads.py into a title-agnostic tool: for a directory of SPU ELFs it lifts each image under a per-image C symbol prefix (so dozens of images -- each defining spu_func_*/spu_recomp_register/a static function table -- link without collisions), rewrites the lifter's depth-sensitive ../../runtime/spu/*.h includes to bare includes (resolved via a build -I path, so the lifted tree is location-independent), and emits one registration C file that registers every image with the runtime workload registry by FNV-1a-64 content fingerprint (byte-identical to runtime/spu/spu_workload.c) under a distinct image id, with an optional startup constructor. Proven on LittleBigPlanet's 23 embedded SPU images: all lift + syntax-check clean and link into the title exe (+5.5MB). cellSpurs AddWorkload/CreateTask/ RunJobChain dispatch these by fingerprint once un-gated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ooks
Two coupled VFS fixes that unblock LittleBigPlanet's boot (it exited(0) at a
bringup stage that polls cellFsStat('/dev_hdd0/game/<title>/USRDIR/') ~15ms and
requires the dir to exist):
- runtime/ppu/ppu_fs.cpp: cellFsMkdir was a no-op stub (returned CELL_OK,
created nothing) -> games that create then poll a directory silently stalled.
Implement it as a recursive host mkdir. Add env-gated PS3_FSLOG tracing to
stat/opendir/mkdir.
- libs/system/cellGame.c: cellGameContentPermit created the game-data dir only
at ./gamedata/dev_hdd0/game/... , but ppu_fs.cpp host_path maps
/dev_hdd0/game/<title>/ to $ppu_vfs_root/game/<title>/ -- a different host
location cellFs never reads, so cellFsStat always returned ENOENT. Also create
the dir at the VFS-mapped location so the two agree.
LBP now boots from the early exit(0) all the way to D3D12 window-open + network
init (148 -> 399 log lines) before a separate cellHttp crash. The 0x95BF60 job
spin was a red herring (the stage gates on the dir, not the job).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cellHttpCreateClient wrote *clientId directly, but the generic HLE adapter passes r3 as a raw guest effective address -> writing through it faulted on a host pointer (host AV at 0x47100FA0 during LBP boot). Translate via vm_base (GUEST_PTR), matching the other libs. LBP now boots past network init into its main engine loading threads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two systemic PPC->C lifter correctness fixes (both benefit every port), plus the matching runtime helpers and env-gated debug tooling. 1. Capped-tail continuation (the LBP "Pool possibly corrupt" abort storm). A gap/mid-entry tail lifted past _MAX_MID_TAIL (0x6000) ends mid-stream on a non-terminator with fallthrough_to = <cut addr>, and no function covers that address. The emit fallback then chained to the NEXT function in ADDRESS order -- for an interior join that is a *backwards* teleport. LBP: sub_32CC18 (0x69BC bytes, absent from the IDA function export) was gap-lifted and capped at 0x332C18; every string-build/reset join teleported back into func_0032F154, re-running the shrink-to-SSO reset WITHOUT its cap>15 free-guard -> HIBYTE(heap ptr)=0 truncated a live 0x47xxxxxx pointer to 0x00xxxxxx -> free(garbage) -> "doesn't belong to pool 32768" x1328 -> all engine threads hang. Fix: _register_continuation() promotes a dangling capped-tail continuation into branch_targets (window-guarded) so a later mid-function pass lifts a real func_<cont>; the emit fallback now chains to next-in-order ONLY when it starts exactly at fallthrough_to, else emits a halt + lift-time WARNING (halting a thread is strictly safer than teleporting it). Verified: LBP pool-corrupt count 1328 -> 0, PT-detector clean. 2. Atomic stwcx./stdcx. store-conditional. The old codegen was a plain conditional write that always succeeded and never validated the reservation value, so lock-free sequences (free-list CAS, refcounts) lost concurrent updates. Now calls ppu_stwcx32/ppu_stdcx64 (runtime), which do an atomic CAS on the raw big-endian guest word. Correct for both compare-swap and read-modify-write. Torture suite (3793 KATs x -O0/-O2) A/B-verified regression-free: the failing set (stmw, u64 var-shift, sw_dense jump tables, setjmp) is byte-identical with and without these changes. Debug tooling in ppu_loader.cpp is all env-gated / error-string-gated and harmless when idle (PT persistent-truncation detector, pool/OOM host-chain trace, value/addr watchpoints). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cellGameContentPermit always returned /dev_hdd0/game/<title>, so a disc title (BootCheck type=1) that permits its boot content got the game-data dir instead of the disc. LBP then hunted for data.farc on /dev_hdd0 (ENOENT), every resource load failed, and the loader stalled forever polling its completion semaphore (sem=7). Model a check session: BootCheck / DataCheck record whether the checked content is disc (CELL_GAME_GAMETYPE_DISC) or game-data; the following ContentPermit reports that content's paths -- /dev_bdvd/PS3_GAME[/USRDIR] for disc, /dev_hdd0/game/<dir>[/USRDIR] otherwise -- and only ensures the game-data dirs exist for the non-disc case. Result: data.farc (180 MB) is found and opened, blurayguids.map resolves, and the loader advances into resource loading. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cellFsLseek used fseek/ftell (long = 32-bit on Windows), truncating offsets in files > 2 GB. Use _fseeki64/_ftelli64 (POSIX fseeko/ftello). Extend the guest-allocator error trace (sys_tty_write path) to also fire on "out of memory on request" so the host->guest call chain is dumped when the game's AllocatorPlatform reports an allocation failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…der-decode Two more PPC->C lifter correctness fixes, both surfaced bringing up LBP's resource loader (each benefits every port). 1. Callee-save memory-snapshot misfire on an address-escaped frame slot. The robust callee-save pass, for a pure mid-function tail-entry, assumes a slot it never itself stores to still holds the original saved value at entry and rewrites `ld rN,off(r1)` -> `rN = _cs_N` (snapshot at entry). But a slot whose ADDRESS was taken (`addi rN,r1,off`) and passed to a call is written by that CALLEE (an out-param) -- invisible to the store scan -- so the `ld` after the call is a LIVE reload, not a restore. Snapshotting it resurrected stale pre-call stack garbage. LBP loc_4A52B0: `addi r5,r1,var_80; bl <fstat>` writes the file size to var_80, then `ld r28,var_80` reloads it; the snapshot gave r28 a stale stack address, so a std::vector grow requested 2x garbage -> "AllocatorPlatform out of memory on request size -805238272" and the loader stalled. Fix: `_CS_ADDR_OF_RE` collects offsets whose address is computed into a register; the memory-snapshot branch skips them (the genuine load stays). 2. Jump-table discovery under-decoded, and the emit dropped out-of-range cases. (a) Count detection took the nearest `cmpwi` to the bctr, which grabbed an unrelated sibling-block compare (`cmpwi r9,0` / a per-case `cmpwi r7,1`) -> count 0/1 -> only the default case decoded. Now it matches compares of the RAW INDEX register (traced back through the index shift) and takes the LARGEST immediate (the bounds check; per-case tests use smaller values). (b) The bctr emit only kept switch arms with `func.start<=case<func.end`, so cases past an IDA-truncated function end were silently dropped -> the runtime bctr hit an unlifted address. Now out-of-range cases are tail-called as their own func_<case> and registered so the mid-function pass lifts them. LBP sub_422A40 (a 0x2A-case "GMTb" resource dispatcher, cases past its truncated end 0x422BEC) decoded 1 case -> `unresolved indirect call 0x422CA0`; now decodes all 42 and dispatches. Torture suite (3793 KATs x -O0/-O2) regression-free: failing set (stmw, u64 var-shift, sw_dense, setjmp) byte-identical to baseline -- notably sw_dense (the jump-table KAT) is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ee as cellFs sys_fs_translate_path mapped /dev_bdvd/X -> <root>/dev_bdvd/X, a directory that doesn't exist, while the cellFs layer (ppu_fs.cpp) strips the mount prefix to <root>/X. A title that opens disc content through the raw sys_fs path then failed even when the file is present -- LBP's Bink videos (gamedata/videos/ localisation_test.bik etc.) went through sys_fs and returned ENOENT. Strip the known mount prefixes (/dev_bdvd, /app_home, /dev_hdd0, /dev_hdd1, /dev_flash, /host_root, /dev_usb*) up front, matching ppu_fs; keep the non-leading app_home spelling + extracted-dump fallback. Video opens now succeed (0 sys_fs ENOENT). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The store-conditional helpers were a bare value-CAS, which is ABA-vulnerable: for a lock-free free-list pop (lwarx head; next=*head; stwcx next,head), a concurrent pop+push that restores head's old value lets a stale stwcx succeed, publishing a dangling next -> heap corruption. Real PPC loses the reservation on ANY store to the granule, so that stale stwcx would fail. Model that: threads register their ppu_context in a reservation set (ppu_resv_register: host thread proc + the main entry thread); a SUCCESSFUL stwcx/stdcx breaks every other thread's reservation on that word by clobbering its reserve_addr to an impossible value (bit 32 set), so its own reserve_addr==ea guard fails and it retries. Recheck+CAS+break are serialized under address-sharded spinlocks (1024 slots by 16-byte block; a single global lock convoyed the LBP loader's workers), and the caller's own reservation is re-validated inside the critical section -- exact PPC semantics for stwcx-vs-stwcx races. PPU_RESV_STORE=1 additionally breaks reservations on plain vm_write32/64 stores to a reserved word (full granule semantics; default off keeps the hot store path to one predictable branch). PPU_RESV_OFF=1 reverts to the bare CAS for A/B diagnosis. RESV_DIAG=1 flags stwcx from unregistered contexts. Fixes LBP's nondeterministic loader heap corruption (OOM aborts + hangs from a corrupted lock-free free list). Also env-gated debug tooling: ALLOCPROBE (insane-size allocation probe with guest-stack + host-chain attribution), LBP_BREADCRUMB (per-tid last indirect-call + count table, dumped from the semaphore wait path so it fires during hangs), ppu_dbg4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sys_lwmutex_lock was a no-op stamp ("boot is single-threaded") -- once a title
spins up worker threads, every lwmutex-protected structure races; LBP's
dlmalloc mspace behind its big-allocator fell apart and reported OOM on tiny
requests with 100+ MB free. Each guest lwmutex now maps to a host
CRITICAL_SECTION (recursive, blocking) keyed by guest address in an
open-addressed table; trylock returns EBUSY on contention; lwcond_wait
releases the paired lwmutex, sleeps briefly and reacquires (poll-style: guests
re-check their predicate loops; signals stay no-ops). Contended locks log
[LWM-BLOCK]/[LWM-GOT] (bounded; uncapped inside a g_nd_inpump probe window,
default 0 -- a title's diagnostic code may pulse it around a suspect wait).
sys_ppu_thread_get_id wrote a fixed id of 1, breaking every
am-I-the-designated-thread check in multithreaded titles -- LBP's job system
routes work by thread identity, so its network-init queues were never
serviced and boot parked forever. Return the caller's real thread id.
hle_ppu_thread_create/exit now store the syscall's return into r3: the
ctx-aware dispatch doesn't propagate handler returns, so r3 kept the incoming
&tid out-pointer and guest wrappers (LBP sub_52613C) read nonzero as "create
failed" and handed tid 0 back to their callers.
sys_net offline model: LBP's net tick (sub_11A864) drains its UDP socket with
non-blocking recvfrom until it returns -1 (EWOULDBLOCK on empty socket).
These NIDs were unresolved, and the unresolved-NID default of r3=0 reads as
"received a 0-byte packet": the drain loop spun forever holding the
net-manager lwmutex, wedging boot at the Network init node. recvfrom/recv/
recvmsg now return -1 with errno EWOULDBLOCK (kept in a guest scratch cell,
since _sys_net_errno_loc returns a pointer the game dereferences); poll and
select report 0 ready. NIDs from PSL1GHT libnet exports.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cellNetCtlGetState claimed IPObtained unconditionally, so titles took their online boot path and waited on PSN jobs that can never complete. Offline (the default) now FAILS the query with NOT_INITIALIZED and reports the link down, matching a console with no connection -- and LBP's working RPCS3 boot, where exactly one failing GetState sends the game down its offline path. Returning OK+Disconnected is not enough: LBP's connect job (sub_F433C) polls GetState until IPObtained and only exits on a negative return. PS3_NET_ONLINE=1 restores the old fake-connected behaviour for online experiments. sceNp.h carried invented 0x8055xxxx error codes; replace them with the official SDK values (0x8002aa01..). Games compare exact codes: LBP special-cases NOT_INITIALIZED (0x8002aa01) from sceNpManagerGetStatus as "NP not up -> treat as offline" and cleanly skips NP init. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…eExec DISC, open PSID) Four more members of the raw-host-write bug class (the guest is big-endian and HLE pointer args are guest effective addresses): - cellUserInfo: CellUserInfoUserStat carried a homeDir[128] member the real SDK struct doesn't have (homeDir comes from cellUserInfoGetHomeDir), making sizeof()=196 while guests reserve ~80 bytes on their STACK: GetStat's memset+strncpy smashed the caller's frame, wiping the PPC CR save slot (nonvolatile cr2-4) -- in LBP this silently derailed the bringup init walker so the Video/RenderTargets/GfxMemPools nodes never ran. Use the real 68-byte layout, and write all u32 out-params byte-swapped (a count of 1 was being read back as 0x01000000). - sceNpTrophyGetRequiredDiskSpace wrote its u64 host-LE: 1 MB read back as 2^44, underflowing LBP's save-data free-space accounting (sub_379EC0) into a GameData ERROR-B boot bail. - cellGameExec: all three entry points dereferenced raw guest EAs (host AV at 0xD000FDF0-class addresses); cellGameGetBootGameInfo also claimed an HDD boot -- report DISC, matching cellGame's disc-content check sessions, which unlocks the disc-boot init sublist. - syscall 872 sys_ss_get_open_psid was unimplemented, leaving the out-param as heap garbage that an LBP boot job then read. Fill zeros (RPCS3's unconfigured console_psid) and return CELL_OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A disc title patched to e.g. v1.30 runs the update's EBOOT and reads its patchN.farc from /dev_hdd0/game/<title>/ while base data stays on /dev_bdvd -- the PS3/RPCS3 layout. PS3_HDD0_ROOT names the host dir /dev_hdd0 maps into (the one containing game/<title>/); both path translators (cellFs host_path and raw sys_fs_translate_path) honor it ahead of the generic mount-prefix strip, so both layers hit the same host tree. Also accept cellFsAllocateFileAreaWithoutZeroFill (LBP's cache warm-up spams it): host filesystems grow files on write, so no preallocation is needed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…orensics - The sys_process_exit back-chain dump now always fires for a NONZERO exit code (LBP's rare loader-thread abort is a race we can't reproduce on demand); clean code=0 exits stay quiet, FLOW_EXITCHAIN still forces all. - The hang watchdog's stack scan trusted VirtualQuery blindly and read through guard pages / a mid-create thread's garbage Rsp; the diagnostic AV was then caught by the crash filter, killing otherwise-healthy runs. Skip the scan unless the region is committed and readable. - Semaphore forensics (all env-gated): SEMTID adds thread ids to [WAIT]/[WAKE] and lifts the print cap; SEMCHAIN dumps guest call chains at the sem=7 poll / sem=3 post; LBP_BREADCRUMB dumps the per-tid indirect-call table from the wait path (fires during hangs); and lbp_hang_census / lbp_unstick_once (POKESEM/POKE7) read LBP's resource-manager queue depths out of guest memory and cross-check each queue's semaphore, to prove or disprove lost-wakeup hypotheses from the watchdog. - sys_ppu_thread_create's log line now includes the new tid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…imports The CRITICAL_SECTION-backed lwmutex (3002f18) enforced Windows CS semantics: only the owning thread may release. Guest code doesn't play by that rule -- LBP's job system passes lwmutex ownership between threads, and threads exit while holding. One cross-thread LeaveCriticalSection silently no-ops, the CS stays owned forever, and the next EnterCriticalSection parks that thread for good. In practice this parked LBP's bringup thread at a random point a few seconds into the Network init node (whichever lock it touched first after the poison), which masqueraded as a network-init hang for a long chain of plausible-but-wrong theories (busy-flag stick, frozen guest clock, starved quiesce timeout, stuck async component -- each eliminated by measurement: the clock is 1:1, the compare is correct, the state machine and present path are healthy right up until they touch the lock). The watchdog's "last HLE call = 0x1573DC3F" (sys_lwmutex_lock) plus every thread sitting in ntdll was the giveaway. Replace the CS with a binary semaphore, which any thread may release: - lock: recursive re-lock via the guest owner/recur stamps (race-free: only the holder ever writes owner=self), else try-wait then block; the [LWM-BLOCK]/[LWM-GOT] contention probes stay. - trylock: 0-timeout wait, EBUSY on contention, recursion honored. - unlock: recur>1 counts down; final release clears the guest fields and posts the semaphore -- from whichever thread calls it. Over-release of a free mutex fails harmlessly at max count 1. - create: force-signals the slot so a recreate at a reused guest address can't inherit a dead holder's locked state. - lwcond_wait: save/clear the guest fields, release, sleep, reacquire, restore (poll-style wait, signalers stay no-ops). Also register the rest of LBP's 13 sys_net imports so none fall to the unresolved-NID default: netInitializeNetworkEx/netBind/netSetSockOpt/ netClose/netFinalizeNetwork/netFreethreadContext return 0, netSocket hands out distinct fds (the default gave every socket fd 0, aliasing them all), netSendTo reports the full length sent (RPCS3-offline oracle: packets vanish into the void), netGetHostByName returns NULL (DNS down). Verified: with this fix LBP's Network init node completes deterministically (2/2 runs) -- the state machine ticks continuously, the game's own 30-second offline quiesce timeout fires right on schedule, and the boot walker advances past Network into a previously-unreached boot phase (DetermineMain -> NGfx -> NHUD -> Input -> LoadingScreen -> GameData) with a live render loop drawing UI quads. The old DebugVariable wild-pointer crash is also gone (same lock corruption, different victim). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LBP spun forever printing its own diagnostic:
ERROR - LIBAUDIO DROPOUT - LIBAUDIO POSITION NO LONGER INCREMENTING!!
3387 of 3792 log lines. Its watchdog (sub_48411C) says exactly what it wants:
lwz r11, 0x21C(r3) ; readIndexAddr, held as a 32-bit pointer
ld r0, 0(r11) ; 64-bit load from it
cmpw cr7, r9, r0 ; 32-bit compare -> LOW word only
beq cr7, ... ; unchanged -> strike; >8 strikes -> dropout
Two bugs, both ours:
1. CellAudioPortConfig was marshalled as though every address field were 64
bits. sys_addr_t is uintptr_t, which on the 32-bit PPU ABI is FOUR bytes,
so writing readIndexAddr as a u64 clobbered status and shifted every field
after it. The game read readIndexAddr as a u32 at +0 and got the high --
zero -- half of that write, then `ld r0, 0(0)`. The VM demand-commits and
returns zeros for a wild pointer rather than faulting, so this read quietly
forever instead of crashing. Correct layout:
readIndexAddr u32@0, status u32@4, nChannel u64@8,
nBlock u64@16, portSize u32@24, portAddr u32@28 -> 32 bytes
2. The counter itself is a 64-bit big-endian value; `ld` + `cmpw` means the
live bits are bytes [4..7]. Publishing it with vm_write32 only ever wrote
bytes [0..3] -- the half the game never looks at. Use vm_write64, and widen
the PortOpen initialiser to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ointer The decode output went straight into `data` and the status into `dataInfo`, both of which arrive as raw guest effective addresses (the generic HLE dispatcher passes GPRs through untranslated). Dereferencing them as host pointers writes decoded pixels to an arbitrary host address -- a crash or silent corruption -- while the guest's real output buffer never receives the image. The input side already translated (`sub->src_data = vm_base + guest_addr`); only the output was left raw. Translate both through vm_base / vm_write32, mirroring cellPngDecDecodeData (the one file in libs/codec that follows the convention correctly). cellJpgDec is imported by LBP (7 NIDs). Verified: no boot regression across 4 runs (104k-119k log lines, 0 crashes). Correct-by-construction against the cellPngDec reference; not positively exercised because LBP does not reach a JPEG decode during the observable boot window. The sibling raw-pointer functions in this file (Create/Open/ReadHeader/ SetParameter) share the pattern and are left for a codec-wide pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cellKbGetInfo/cellKbGetData and cellMouseGetData/cellMouseGetDataList wrote their result structs through the raw argument pointer -- but HLE struct args are guest EAs, so the memset/field stores dereferenced a guest address as host memory. LBP crashed the moment its frontend polled the keyboard (cellKbGetInfo memset at guest-stack EA 0x0FEFF814 -> host AV), which was the very first thing it did after surviving the loading phase. Write through the vm accessors big-endian instead (the cellPad.c idiom, same as the earlier cellMouseGetInfo fix), with the SDK's guest layouts: CellKbInfo status[] is BYTES at +0xC (not u32s), CellKbData is led/mkey/len/keycode[62] at 0/4/8/0xC, CellMouseData is 6 used bytes + 2 pad, CellMouseDataList is u32 count + 8 entries. The NO_DEVICE paths now zero the guest struct instead of memsetting host memory. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…counter
The HLE was a static counter advancing 1 ms PER CALL ("so callers see time
progress") -- a boot-era stopgap that made every guest clock built on it run
at call-rate instead of wall-time. LBP's Bink movie clock (sysGetSystemTime,
liblv2 NID 0x8461E528, resolves here via the computed-NID registration) paced
the whole intro off it: video decoded at <1 fps with ~13 IO submissions per
run, the movie audio preload threshold effectively never filled (the port was
opened and configured but cellAudioPortStart never fired -> silence), and
frontend animations timed off the same clock crawled.
Return real monotonic microseconds (QPC on Windows, CLOCK_MONOTONIC
elsewhere).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sys_spu_segment.src is a u32 effective address at offset +0x10 -- the guest reads read32(seg+0x10) and uses it directly as the DMA source EA. We were writing it as if it were a big-endian u64 (hi=0 at +0x10, lo=addr at +0x14), so the guest read the zero high word: every runtime-imported SPU image DMA'd from address 0. Found via LBP's FMOD SPU mixer, which materializes its DSP-plugin overlays with sys_spu_image_import and then DMAs each into LS from seg[+0x10]; with the address in the wrong word the overlays loaded as zeros, and the mixer branched into empty LS (unresolved indirect branch, loading hang after intro-skip). Writing the address at +0x10 (and keeping +0x14 = addr, harmless for any u64-low reader) makes the overlay bytes land correctly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cellFsSdataOpen (0xB1840B53) was unresolved and silently faked (default
CELL_OK, no handle). LBP stores its SPU job code modules -- the physics/mesh
jobs (VERLET, SKIN, SPRING, SQUISH) -- zlib-compressed inside patch.sdat and
data.sdat (NPD/SDATA containers), opened via this call. Faking it meant the
containers were never read, zero job modules ever reached memory, every SPU
job dispatched into an empty code buffer, and the loader deadlocked waiting on
completions that could never come. Confirmed: the module ila-header signatures
inside these files match the resident modules in an RPCS3 save state exactly.
Implement it for real:
- runtime/ppu/sdata_decrypt.h: self-contained AES-128 (ECB + CBC) and the
SDATA block cipher. Per 0x4000-byte block:
crypt_key = dev_hash ^ SDAT_KEY
key_result = AES_ECB(crypt_key, dev_hash[0:12] || be32(block))
key_final = (flags & ENCRYPTED_KEY 0x08) ? AES_CBC(EDAT_KEY_0, key_result)
: key_result
plaintext = AES_CBC(key_final, iv = npd.digest, ciphertext)
Algorithm and keys ported from RPCS3 Crypto/unedat.cpp; validated by
decrypting LBP's data.sdat to its plaintext FSHb container.
- cellFsSdataOpen: open the file; if it's an already-decrypted container
(hdd0 patch.sdat is plain FSHb) serve it directly; if NPD-encrypted,
decrypt the whole file in memory and serve the plaintext from a tmpfile.
Registered by the literal import NID (the friendly-name hash didn't match).
With this the SPU job modules finally load (they never did before). The loader
now proceeds into real job execution and stalls on a separate, downstream
job-completion issue -- but the code is present now, which it never was.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A disc title patched to e.g. v1.30 runs the update's EBOOT and reads its patchN.farc from /dev_hdd0/game/<title>/ while base data stays on /dev_bdvd -- the PS3/RPCS3 layout. PS3_HDD0_ROOT names the host dir /dev_hdd0 maps into (the one containing game/<title>/); both path translators (cellFs host_path and raw sys_fs_translate_path) honor it ahead of the generic mount-prefix strip, so both layers hit the same host tree. Also accept cellFsAllocateFileAreaWithoutZeroFill (LBP's cache warm-up spams it): host filesystems grow files on write, so no preallocation is needed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…user paths LittleBigPlanet boot progression. Several HLE functions dereferenced their pointer parameters as host pointers, but the generic adapter passes PPC registers raw so those are GUEST effective addresses -- each was a host access violation the moment the title called it. - cellGameDataCheck: printed dirName (raw EA) and returned ERROR_NOTFOUND for a missing game-data dir; translate before use and return CELL_GAME_RET_NONE (firmware/RPCS3 semantics) so the title creates its data instead of bailing. - cellSysCacheMount: marshalled CellSysCacheParam (cacheId in @+0, path out @+0x20) instead of strncpy through the raw EA. - cellSysutilGetBgmPlaybackStatus, cellDiscGameGetBootDiscInfo: vm_write the out-params. - cellSaveData: translate dirName in AutoSave/AutoLoad (+_2) and add the User* wrappers (userId after version; the 9th arg is on the guest stack). - cellRtc: dual host/guest accessors (guest fields are big-endian) for GetCurrentTick/Clock*/Time_t/SetTick/GetTick, plus the missing cellRtcGetCurrentClock. - cellSysmodule: the module-ID table was scrambled (a stray SYNC2=0x0E shifted everything after FS); rewrite it from the firmware enum, and accept the 0xF0xx extended range (LBP's NP_TROPHY = 0xF035 was rejected every boot). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eExec DISC, open PSID) Four more members of the raw-host-write bug class (the guest is big-endian and HLE pointer args are guest effective addresses): - cellUserInfo: CellUserInfoUserStat carried a homeDir[128] member the real SDK struct doesn't have (homeDir comes from cellUserInfoGetHomeDir), making sizeof()=196 while guests reserve ~80 bytes on their STACK: GetStat's memset+strncpy smashed the caller's frame, wiping the PPC CR save slot (nonvolatile cr2-4) -- in LBP this silently derailed the bringup init walker so the Video/RenderTargets/GfxMemPools nodes never ran. Use the real 68-byte layout, and write all u32 out-params byte-swapped (a count of 1 was being read back as 0x01000000). - sceNpTrophyGetRequiredDiskSpace wrote its u64 host-LE: 1 MB read back as 2^44, underflowing LBP's save-data free-space accounting (sub_379EC0) into a GameData ERROR-B boot bail. - cellGameExec: all three entry points dereferenced raw guest EAs (host AV at 0xD000FDF0-class addresses); cellGameGetBootGameInfo also claimed an HDD boot -- report DISC, matching cellGame's disc-content check sessions, which unlocks the disc-boot init sublist. - syscall 872 sys_ss_get_open_psid was unimplemented, leaving the out-param as heap garbage that an LBP boot job then read. Fill zeros (RPCS3's unconfigured console_psid) and return CELL_OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…CELL_OK
gen_hle_nids.py resolves each module name to libs/**/<name>.c. A name that
matched nothing printed "warning: <lib>.c not found" and carried on -- the worst
available outcome, because every NID that module exports then stays
unregistered, and the dispatcher's unresolved path answers CELL_OK with
untouched out-params. The title runs on fabricated success, and the only clue
scrolls past in the build noise.
The trap: these are PRX names, but the files are named after the LIBRARY.
LBP's list carried six that match no file at all --
sys_io, sys_fs, sys_net, sceNp2, cellSysutilAvconfExt, cellDiscGame
-- so every NID behind them has been faked from the start. sys_io is the one
that bit: cellMouseInit, cellMouseGetInfo and cellKbInit are all fully
implemented in libs/input/cellMouse.c and cellKb.c, and none were ever
registered. LBP polls the mouse during boot, got CELL_OK and an untouched
CellMouseInfo every time, and wandered off down a path that never reaches SPU
init -- which is why runs kept "nondeterministically" failing to reach SPURS.
Fail loudly instead, listing each unmatched name with the nearest real file.
Registering the mouse then exposed why the module had never worked: it had never
executed. cellMouseGetInfo took a CellMouseInfo* and memset/assigned through it,
dereferencing the raw guest EA as a host pointer -- an instant AV, and exactly
what happened on the first run after registering it. The local struct also
declares u32 vendor_id/product_id/status where hardware has u16/u16/u8, so even
a translated pointer would lay the fields out wrong. Rewritten against the SDK
layout (cell/mouse/mouse_codes.h, CELL_MAX_MICE=127), writing BE fields through
vm_write*.
cellMouseInit also asserted port 0 was always connected. That reports hardware
that is not there: a title is entitled to open a mouse-driven path and wait on
input that can never arrive. Report what actually exists -- ports connect when
the host injects events, and an unplugged port is a normal thing to see.
Verified: NID table 527 -> 546 handlers, nothing lost (the six dead names
contributed nothing, so dropping them changes no registration). The mouse crash
is gone and boot reaches 8,063 log lines, past the 7,445 it managed while
faking. The remaining five dead names are now loud, for a deliberate decision
rather than a silent fake.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…l NID table gen/ppu_hle_nids.cpp is the NID->handler table; an unregistered import silently fakes CELL_OK with untouched out-params. After the file was lost it was regenerated with only cellFs/cellGcmSys/cellSysmodule (3 modules, 110 handlers), so the game called cellSysutilCheckCallback unregistered, got a faked CELL_OK, and its boot diverged: the loading threads exited early, cellSpursInitialize was never reached, and the loader spun (finish#3). Regenerate with all 88 PRX modules the runtime implements (1032 handlers). cellSysutil / sceNp / cellUserInfo / cellGame / ... now resolve and LBP boots through to the SPURS workload + WWS job-manager path again (27x AddWorkload, wid=12, farc asset loads, no early loading-thread exit). - gen_hle_nids.py: add --all, which discovers every PRX module compiled into ps3recomp_runtime rather than a hand-typed list that can silently drop a whole library. --all reproduces the identical 1032-handler set. - track gen/ppu_hle_nids.cpp. Its siblings gen/jobmods_register.c and gen/spu_workloads.c are already tracked; this one never was, which is why a stray `git clean` deleted only it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two LBP WWS physics jobmods (e0A40, e0A20) embed a zlib-inflate error-string table plus float const tables after their code, inside the executable segment. Those data bytes decode as in-range branches (br/brsl/brhz) whose targets seed spurious "functions", which then disassemble the tail as code -- 97 `.word` (undecodable) each, 194 unsupported total. Reachability from the entry can't bound it (the image entry is an indirect dispatch stub), so the boundary is supplied per image. - spu_lifter.py: --code-end ADDR drops any auto-detected function starting at/after ADDR and caps any function's end there, leaving the tail as data. - lift_jobmods.py: per-module CODE_END (e0A40=0xAE68, e0A20=0xAE48 -- the real code ends at the last unconditional `br`, located by disassembly). Re-lift is 0 unsupported across all 89 modules (was 194). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… now passes
The cri decode task ran ~323 instructions and exited via `stop 0` because its
in-LS context validator (func_00026E80/F18/FC4, the same checks as libsre's PPU
task-add) rejected our reconstructed SpursTasksetContext with 0x80410911.
Diffed our LS 0x2700/0x2FB0 against a WORKING reference captured with caner's
RPCS3 guest-memory-dumper fork (github.com/canersaka/rpcs3-guest-memory-dumper):
YDKJ BLUS30569, SPU0 "CellSpursKernel0" mid-cri-decode, file offset == LS address.
Mismatched fields the validator reads:
- moduleId @0x2840 = "SPURSTASK MODULE" (we never wrote it)
- TI_LS_PATTERN @0x27A0 = 0 (build_context forced all-ones "full coverage";
the validator does andc(r20,r21) on it, and all-ones forced the error branch)
- task descriptor @0x2FB0 word0/word1 = 0/0 (we planted 0xFFFFFFFF/0x400)
- dmaTagId @0x27D0 = 0x1F (we wrote 0)
Applied CRI-SCOPED (image 22 only) in the dispatch after build_context -- putting
them in the shared build_context breaks the audio SPURS task (BGM crash @0x2DF320).
Result: the task no longer hits the validator/error path. It now runs the real
body (0x257xx -> func_00026DE0) for 373 steps and branches to the task-API syscall
trampoline (0xA70) -- i.e. it passed validation and reached the work/yield call.
Next gate: HLE that task-API syscall in the pure interpreter (the lifted path
intercepts 0xA70 via spu_channels.c; the interp does not yet).
Reference dump made possible by caner's dumper fork -- a genuinely good idea.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…erpreter With the RPCS3-dump context fix, the cri task passes validation and reaches its task-API call: func_00026DE0 reads syscallAddr @0x27C4 (0xA70) and branches there. The lifted path intercepts 0xA70 via spu_indirect_branch, but the pure interpreter (image_id<0) executed empty LS at 0xA70 and stopped 0. Intercept pc==0xA70 in spu_interp_run (gated by the syscallAddr sentinel that only spurs_pm_build_context writes), call spu_spurs_taskset_syscall (now non-static), and resume at the task link (r0). Present image_id==22 across the call so the handler returns on EXIT(num=0) instead of longjmp-ing out of a context the interp loop isn't wrapped for. Result: the task runs its body, makes the task-syscall (num=0, resumes at 0x26E18) and continues past it instead of stopping. Still no decode (video_dma=0) -- it now loops post-syscall; next is to find that loop and wire the producer-side stream. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A task stuck in an infinite work/wait loop never halts, so the trace ring (which dumps on halt) never fires. YDKJ_SPU_STEPCAP=N forces a one-shot ring dump after N steps -- used to locate the cri task's post-syscall wait-for-work spin (0x26E18 -> func_00026E80 -> 0x26E18). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…sses naturally The cri create-task pre-check (libsre 0x300158C4) STATs 0x8041090F unless the task event-flag @0x006B4600 has +0xC==0xFF. That magic is stamped by _cellSpursEventFlagInitialize (NID 0x5EF96465) -- but that NID was pinned in YDKJ_FORCE_HLE, and the HLE stub doesn't write +0xC. (It was force-HLE'd because libsre's version once STAT'd and tripped CRI's CellSpursTaskset.cc:423 assertion -- but that predates the taskset/attr fixes.) Confirmed against a working RPCS3 PPU-RAM dump (caner's guest-memory-dumper fork): the cri descriptor @0x006B4600 is fully populated there (+0xC set), so the flag IS libsre-inited before create on real hw. Route ONLY the cri event flag to real libsre (the eventFlag EA arrives in r5 for this internal variant; matched against YDKJ_CRI_EVFLAG, default 0x006B4600). Audio flags stay on the HLE stub -- routing them makes the game wait for an audio SPU task we don't dispatch (timer_usleep hang). Result WITHOUT the CRI_CREATE_OK hack: cri create RETURNS 0, the taskset is populated with the movie ELF, the game keeps rendering (879 flips), and the SPU cri task runs (passes validation, makes its task syscall). The game returns to the q2/q3 decode-completion wait -- next the task needs movie data fed to decode+post. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…S3 dump) Comparing our spinning cri task's LS 0x2700 SpursTasksetContext against the working RPCS3 SPU0 dump: ours had RUNNING(0x00)=0 and PENDING(0x20)=0x80000000, RPCS3 had RUNNING=0x80000000 and PENDING=0. Our CRI_READY only did ready|=pending; complete the kernel's full transition (RUNNING|=pending, PENDING=0) so the state matches. Also: YDKJ_SPU_STEPCAP now dumps the LS 0x2700 bitsets so the task's context can be diffed against the dump without halting. This aligns the bitsets, but the task still busy-spins in func_00026DE0/E80: it re-reads its STATIC LS context (build_context runs once) for a work-ready change that never arrives, because the interpreter runs the task in isolation from the game's ongoing SPU-directed work delivery (the SPURS kernel re-DMA/signal path). That kernel<->task work-delivery connection is the remaining gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Option (c): the cri task's r3 (CellSpursTaskArgument) points to different work structures than the working RPCS3 SPU0 dump. Ours: r3=[00400000 006B4500 006B457C 006B6C00] where arg3@0x006B6C00 references SPURS internals (instance 0x40009F00, taskset 0x40131000). RPCS3: [007A0000 3002EE44 3002ED00 005C3580] where arg3@0x005C3580 is the cri decode buffer-descriptor table (sizes 0x180/0xC0/0x170 + buffer ptrs 005C1xxx/30008A90) and arg0 points to float coefficients. So the game's cri_mpv setup builds the task's work-args differently in our recomp: the task gets pointers to SPURS internals instead of the movie decode buffers, so it never finds real work and busy-spins. The remaining gap is upstream in the game cri initialization that computes these args. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ctness batch (16 commits) # Conflicts: # runtime/syscalls/sys_fs.c
…nsition matching the RPCS3 dump) # Conflicts: # libs/system/cellSysutil.c # runtime/ppu/ppu_fs.cpp
discover_jump_tables chose the table-base register for `lwzx rD, rIdx, rBase` as "first operand whose TOC slot reads back". Both operands are frequently TOC-loaded, so an unrelated `lwz rIdx, d(r2)` earlier in the window reads back a valid-but-wrong address and wins, and the table decodes from the wrong base. The offset-table idiom always ends in `add rD, rOff, rBase` feeding mtctr, so the base is unambiguous there. Prefer the lwzx operand that this add combines with the (sign-extended) loaded offset; fall back to operand order otherwise. Factored the offset-register set out as _extended_from() and reused it for both the base disambiguation and the existing is_offset test (identical semantics). flOw's app-loop state machine @0x89300 decoded its table from r9's global 0x10157F30 instead of r11's real base 0x00089304, so the switch fell through to an unresolved bctr and unwound out of the main loop. With the fix flOw lifts 101 jump-table dispatchers / 1005 case targets (was 88 / 914) and drives its app loop instead of exiting after one iteration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PhyreEngine titles (flOw) use the lv2 sys_fs syscalls directly, not the cellFs HLE, so the existing YDKJ_FSDBG in ppu_fs.cpp never fires. Add FLOW_FSDBG to sys_fs_read: fd, path, position, requested/actual bytes, first 4 bytes, LR. In sys_tty_write's FLOW_PSSGTRACE hook, match "Mystery" too (flOw prints "Mystery Numbers: %d, %d" once per app-loop iteration -- its loop heartbeat) and call ppu_guest_callstack(): the guest back-chain reads 0 under the DRAIN/fragment model, but the host-backtrace-to-guest-function mapper is reliable. This is what surfaced the unresolved bctr fixed in the previous commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… of caner's decode + LBP SPU/SPURS + RSX textures # Conflicts: # libs/audio/cellAudio.c # libs/spurs/cellSpurs.c # libs/spurs/spurs_pm.c # libs/system/cellGame.c # libs/system/cellSaveData.c # libs/system/cellSysutil.c # libs/video/rsx_d3d12_backend.c # runtime/ppu/ppu_fs.cpp # runtime/ppu/ppu_hle.cpp # runtime/ppu/ppu_loader.cpp # runtime/ppu/ppu_sysprx.cpp # runtime/spu/spu_channels.c # runtime/spu/spu_context.h # runtime/spu/spu_dma.h # runtime/spu/spu_workload.c # runtime/syscalls/lv2_register.c # runtime/syscalls/sys_event.c # runtime/syscalls/sys_fs.c # runtime/syscalls/sys_ppu_thread.c # tools/gen_hle_nids.py # tools/ppu_lifter.py # tools/spu_disasm.py # tools/spu_lifter.py
…struct sagemono's boot_main.cpp added a sys_ppu_thread.h include (->lv2_syscall_table.h ->ppu_context.h) after the lifted ppu_recomp.h already defined struct ppu_context, causing a redefinition when a title is built with a lifter whose header uses #pragma once (no PPU_CONTEXT_H). Claim the guard right after ppu_recomp.h.
Take sagemono's version of every runtime/spu file and drop the base-only interpreter split (spu_interp.c/.h/_tables.inc, spu_fn_registry.c/.h + selftests) that sagemono consolidated into spu_channels.c. Re-add ppu_guest_callstack (kept ydkj diagnostics reference it) and drop the ydkj ppu_in_guest_callback re-entrancy guard in ppu_gcm_pump (no guest-call-depth counter in sagemono's runtime; the local guard holds). Makes the merged runtime link when a title builds against it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #82 (faithful-adopt-caner) is a from-scratch re-derivation of the SPU decoder/lifter + runtime that sp00nz and Caner Saka originally authored. Their original commit history stands on its own; this credits them as the originators of the underlying design that sagemono's branch faithfully re-adopts, so the attribution is preserved when the re-adoption folds in. Co-Authored-By: sp00nz <sp00nz@gmail.com> Co-Authored-By: Caner Saka <c33.saka@gmail.com> Co-Authored-By: sagemono <sewshee@tuta.io> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Aug 10, 2026
sp00nznet
added a commit
that referenced
this pull request
Aug 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Attribution: this branch is a from-scratch re-derivation of the SPU decoder/lifter + runtime originally authored by @sp00nznet and @canersaka; credit for the underlying design belongs to them (their original commits stand on their own). sagemono authored the re-adoption. See CONTRIBUTORS.md.
DRAFT -- compile-verified, NOT yet re-lift/run verified. Do not merge before the validation below.
Folds
sagemono/spu/faithful-adopt-caner(150 commits) on top of #81 (integrate/fold-2026-07-24). Brings a faithful re-adoption of caner's decode/lift work + LBP SPU/SPURS bring-up + RSX texture support (G8B8/DXT, deswizzle). -- @sagemono, adopting @canersaka's decode work.Why this was hard (and the resolution principle)
The two lineages independently evolved the SAME subsystems (PPU/SPU lifter, FP/VMX handlers, SPU runtime, HLE) with DIFFERENT helper vocabularies -- e.g. lane-safe BE VMX floats via
vrf/vstf(ours) vsppu_vldf4/ppu_vstf4(theirs);_mem_baserA=0 vs rawctx->gpr[base]. Cherry-picking hunks across them would reference helpers defined on only one side. So this is a "which lineage is canonical" decision, not a hunk merge. 72 conflict hunks across 23 files.Canonical = sagemono's faithful-adopt-caner (the deliberate, curated adoption). Resolution:
ppu_lifter.py+ spu_lifter/spu_disasm/gen_hle_nids): took theirs, then RE-APPLIED our jump-table-base fix (theirdiscover_jump_tableshad the identical "first TOC-loaded operand wins" bug that dropped flow's app-loop dispatcher).ydkj_host_btre-added (ppu_fs.cpp calls it),ydkj_release_pending_threadsunioned (sys_event.c calls it),park_on_empty_inmbox,TUNERFIX.sys_ppu_thread.c(ydkj threadgate + sagemono main-thread unique-id fix),lv2_register.c(both back-chain diagnostics),rsx_d3d12_backend.c(SYNTHESIZED: TRIANGLE_STRIP/FAN from theirs AND our VP-microcode guard),ppu_fs.cpp/sys_fs.c(superset of hdd0-overlay + dev_flash + USRDIR shortcut + mount-strip).What IS verified
gitclean.ppu_recomp-independent runtime lib compiles clean (0 errors,ps3recomp_runtime.libproduced) -- so every C resolution is syntactically/type-correct.ast.parse).What is NOT verified (BLOCKERS before merge)
Recommend: re-lift flow -> build+run flow (ocean scene + SPU sim) AND run LBP, THEN mark ready.
🤖 Generated with Claude Code