Conversation
…y its own code Profiling QuaTrEx (github.com/quatrex/quatrex) on an Alps GH200 with its real carbon-nanotube GW inputs turned up an awkward result: there is no hot GPU kernel. Over 4 SCBA iterations the GPU is busy 2.4% of the time -- 597 ms of kernels across 221,508 launches, mean kernel 2.4 us -- and doubling the block size pushes utilisation DOWN to 1.5%, because the added work lands on the CPU. Application-owned CUDA is 1.8% of that GPU time, i.e. ~0.05% of wall. The wall time is 78% open-boundary-condition solve, and perf plus nsys python sampling put that squarely on LAPACK's complex non-symmetric eigensolve (zlaqr5/zlahqr/zlarfg, reached because eig_compute_location defaults to "numpy"). Moving it to cuSOLVER buys 9%, so the cost is latency across many small dense eigenproblems, not flops. zgeev and zgemm are the strictly hottest symbols, but they are OpenBLAS/LAPACK and extracting them reproduces a library (and gemm is already in the corpus). The hottest thing that is QuaTrEx's own mathematics is the RGF selected solve: 35% of a CPU-mode iteration and the source of the ~45% ZGEMM. Ported from qttools/greens_function_solver/rgf.py::RGF.selected_solve. It is a block-tridiagonal selected inversion -- forward Schur-complement sweep, then a backward sweep reusing the stored diagonal blocks -- which is why it sits under sparse_linear_algebra despite dense blocks: the block-tridiagonal structure is what makes it O(NB*BS^3) rather than O((NB*BS)^3). Simplifications (all documented in the kernel docstring): obc_blocks folded in by the caller, both triangles written, no Meir-Wingreen current, no energy batching, uniform block size. Upstream supports non-uniform blocks; fixing BS keeps the declared shapes an honest ABI contract. Validated four ways: bit-identical to a frozen transcription of the upstream expressions; X^r equals the dense inverse's diagonal blocks and X^< equals the dense congruence A^-1 S^< A^-H to ~1e-16, both fully independent of the port; the NEGF anti-Hermitian symmetries hold; and BS/NB/NE are exercised as three genuinely independent sizes. The c, cpp and fortran backends all reproduce the numpy reference.
Main renamed the three tracks (hpc -> scientific_computing, foundation -> loop_level_reasoning, ml -> machine_learning). Git saw the kernel as a plain add under hpc/ and main's work as a rename of everything else, so it merged both without conflict and left the kernel stranded on a track that no longer exists. test_tree_structure's test_top_level_is_only_the_three_tracks and test_every_kernel_resolves_under_a_track both fail on the merge commit. Moved the kernel directory and updated taxonomy.track. The manifest carries no relative_path -- it is derived from the tree -- so nothing else needed touching. test_tree_structure, test_yaml_style, test_levels and the six benchmark-local tests pass, as do the c/cpp/fortran e2e backends.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.