[hipSPARSE] Windows gfx1151/gfx110x-all: bsrsv2 pre_checkin unit-check failure (CPU vs GPU .y diff exceeds compare_val) — regression in a608060..880166b

[AmosLewis]

Problem Description

CI regression observed when bumping rocm-libraries in TheRock from a608060 to 880166b. hipsparse test job fails on Windows gfx1151 (release) and Windows gfx110X-all (release).

Failure: pre_checkin/bsrsv2.level2 tests fail unit-check. Multiple bsrsv2 parameter variants (e.g. f64_c, filename "nos7", different fill_mode/solve_policy/dirA) report:

The difference between hCPU[i + j * lda].y and hGPU[i + j * lda].y is 3.3306690738754688e-15, which exceeds compare_val.y
— in rocm-libraries/projects/hipsparse/clients/common/unit.cpp:264.

CI job (full log):
gfx1151: https://github.com/ROCm/TheRock/actions/runs/23139971395/job/67290574098?pr=3985
gfx110x-all: https://github.com/ROCm/TheRock/actions/runs/23139971395/job/67320587002?pr=3985
TheRock bump PR rocm-libraries: ROCm/TheRock#3985
Commit range: rocm-libraries a608060..880166b.

Operating System

Windows

CPU

strixhalo

GPU

gfx1151

ROCm Version

7.13.0

ROCm Component

hipSPARSE

Steps to Reproduce

Run TheRock CI for PR #3985, Windows gfx1151/gfx110x-all release → Test hipsparse. Or build rocm-libraries at 880166b and run hipsparse pre_checkin bsrsv2 tests on Windows gfx1151.

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

Additional Information

...
2026-03-16T20:05:51.2313086Z [  FAILED  ] known_bug/csric02.precond/f64_r_1_row_use_level_0b_t7, where GetParam() = { filename: "shipsec1", function: "csric02", category: "known_bug", M: 1, N: -1, K: -1, nnz: -1, block_dim: 2, row_block_dimA: 2, col_block_dimA: 2, row_block_dimB: 2, col_block_dimB: 2, lda: 1, ldb: 1, ldc: 1, batch_count: -1, index_type_I: "i32", index_type_J: "i32", compute_type: "f64_r", alpha: 1.0, alphai: 0.0, beta: 0.0, betai: 0.0, threshold: 1.0, percentage: 0.0, c: 1.0, s: 1.0, transA: "NT", transB: "NT", baseA: "0b", baseB: "0b", baseC: "0b", baseD: "0b", action: "num", part: "auto", diag_type: "ND", fill_mode: "L", solve_policy: "use_level", dirA: "row", orderA: "col", orderB: "col", orderC: "col", formatA: "csr", formatB: "csr", csr2csc_alg: "default", dense2sparse_alg: "default", sparse2dense_alg: "default", sddmm_alg: "default", spgemm_alg: "default", spmm_alg: "default", spmv_alg: "default", spsm_alg: "default", spsv_alg: "default", csr2csc_alg: 0, dense2sparse_alg: 0, sparse2dense_alg: 0, sddmm_alg: 0, spgemm_alg: 0, spmm_alg: 0, spmv_alg: 0, spsm_alg: 0, spsv_alg: 0, numeric_boost: 0, boosttol: 0.0, boostval: 1.0, boostvali: 0.0, slice_size: 2, ell_width: 0, permute: 0, gtsv_alg: 0, gpsv_alg: 0, unit_check: 1, timing: 0, iters: 10, graph_test: false }
2026-03-16T20:05:51.2316878Z 
2026-03-16T20:05:51.2316884Z 
2026-03-16T20:05:51.2316960Z 88 FAILED TESTS
2026-03-16T20:05:51.6843027Z Traceback (most recent call last):
2026-03-16T20:05:51.6851216Z   File "C:\runner\_work\TheRock\TheRock\build_tools\github_actions\test_executable_scripts\test_hipsparse.py", line 56, in <module>
2026-03-16T20:05:51.6851806Z     subprocess.run(
2026-03-16T20:05:51.6852580Z   File "C:\runner\_work\_tool\Python\3.12.10\x64\Lib\subprocess.py", line 571, in run
2026-03-16T20:05:51.6975187Z     raise CalledProcessError(retcode, process.args,
2026-03-16T20:05:51.6976072Z subprocess.CalledProcessError: Command '['./build/bin/hipsparse-test', '--gtest_filter=--gtest_filter=*quick*:-known_bug*:*spmm*']' returned non-zero exit status 1.
2026-03-16T20:05:51.7106421Z ##[error]Process completed with exit code 1.

[AmosLewis]

Same error in bump pr rocm-systems ROCm/TheRock#3984

[arjun-raj-kuppala]

Update: Spoke to Nico on this issue. He is trying to get a machine to check on this. He will be talking to Stan to get the machine and work on this one.
@ntrost57 Again to re-iterate here, this is blocking our bump PR's, if you any need help on pushing on the priority please let us know.

[ntrost57]

PR #5605 should unblock you for now while we investigate. I moved the failing tests to the known issues category, so you can filter them out with gtest_filter=-known_bug* filter. Let me know if that works for you.