fix(jaj): cap the PCIe FFC link at Gen2 - #120
Conversation
C7 drives the 16-pin FFC header, whose Raspberry Pi 5 pinout carries one lane over unshielded flat flex. Raspberry Pi rates that connector at 5 GT/s and treats 8 GT/s as opt-in and unguaranteed; nothing in the stock DT pins the rate, so the link trains at 8 GT/s with no margin and runs a continuous correctable-AER storm.
|
Runtime confirmation on the testlaptop9 JAJ fixture. Forced with
The control is what rules out a lucky retrain. Reads are Three things the numbers add to the PR:
Orthogonal, but it settles whether this and ark_jetson_kernel#118 are alternatives: with the link left at 8 GT/s and only the console quieted, One consequence worth naming before this merges: afterwards nothing on a healthy unit will ever show anything about that link's margin. The correctable-AER count in the post-mortem digest on ark_scripts#79 becomes the only sentinel for it. |
Summary
Pins C7 — the controller behind JAJ's 16-pin PCIe FFC header — to 5 GT/s.
Problem
Schematic sheet 14 ("Pi 5 PCIE Connector") wires J7 with a single lane:
PCIE2_CLK_P/N,PCIE2_RX0_P/N,PCIE2_TX0_P/Nand nothing else.PCIE2_RX1/PCIE2_TX1reach the SODIMM but are unrouted. So this is one lane over unshielded flat flex, on a pinout Raspberry Pi rates at 5 GT/s and documents 8 GT/s as opt-in and unguaranteed — over a run far shorter than the ~6" cable on the bench fixture.Nothing pins the rate today.
tegra234.dtsigives C7num-lanes = <8>and the stock p3768 carrier dtsi supplies two P2U phys without overriding it, so the link trains at 8 GT/s. On the fixture that produces a continuous correctable-AER storm — RxErr and BadDLLP at 210-240 events/s with the drive idle, summed over both ends of the link. Counted as one bit error apiece, which is the most generous reading, that is a BER near 3e-8: four to five orders of magnitude above the 1e-12 the Gen3 channel budget assumes. Measuring this off the console understates it by about 7x — the UART tops out at ~28 events/s and drops the rest — so take it fromaer_dev_correctable, which increments whether or not the message reaches the wire. A PCIe link in L0 never goes quiet (logical-idle blocks plus periodic UpdateFC DLLPs), so that rate is the raw channel BER rather than anything about NVMe traffic. Every error is correctable and the link replays, but there is no margin left for temperature, voltage or cable flex.The board contributes two more things at 8 GT/s. D23/D24/D25 put ESD arrays on all three high-speed pairs, which is shunt capacitance and a stub at 4 GHz Nyquist unless they are ultra-low-cap parts. AC coupling is fine and not a suspect: C171/C172 are 0.22 µF on the TX pair at the connector, inside the spec window.
Worth correcting a claim in ark_scripts#79 while this is in view: the x1 width is nominal, not a downgrade. The root port's x8 is the
num-lanesdefault and the endpoint's x4 is the drive's own capability; neither describes what the FFC routes.Solution
max-link-speed = <2>onpcie@141e0000in the JAJ override. Halving the symbol rate drops Nyquist to 2.5 GHz, which buys a large amount of loss and jitter margin and cuts what the ESD capacitance costs — it aligns JAJ with the operating point the connector's originator actually certifies, rather than suppressing the symptom. Costs roughly 400 MB/s of ceiling on that link, ample for the FFC SSD's logging role. C1, C4 and C5 are untouched, so the M.2 Key-M slot keeps Gen3 x4.Confirmed on the testlaptop9 fixture by the equivalent runtime retrain (
setpcion Link Control 2): zero corrected errors over 90 s at 5 GT/s and over 60 s at 2.5 GT/s, against 209-229/s at 8 GT/s, and a control back at 8 GT/s brings them straight back. Numbers in the comment below. One gap the retrain cannot cover: it re-trains a link that already equalized at Gen3, so readcurrent_link_speedand anaer_dev_correctabledelta once on the first image built from this.