sync: rvck #105: Backport subject : 'Svvptc extension to remove preventive sfence.vma'#165
Merged
Conversation
added 3 commits
April 8, 2026 14:57
mainline inclusion from mainline-v6.9-rc1 commit 01261e24cfab69c65043e1e61168348ae23a64c2 category: performance bugzilla: -------------------------------- We used to emit a flush_icache_all() whenever a dirty executable mapping is set in the page table but we can instead call flush_icache_mm() which will only send IPIs to cores that currently run this mm and add a deferred icache flush to the others. The number of calls to sbi_remote_fence_i() (tested without IPI support): With a simple buildroot rootfs: * Before: ~5k * After : 4 (!) Tested on HW, the boot to login is ~4.5% faster. With an ubuntu rootfs: * Before: ~24k * After : ~13k Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20240202124711.256146-1-alexghiti@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn> (cherry picked from commit 7dc32f7)
mainline inclusion from mainline-v6.11 commit 503638e0babf364061bc50fca5103b00a56cc50a category: feature bugzilla: RVCK-Project/rvck#104 ------------------------------- In 6.5, we removed the vmalloc fault path because that can't work (see [1] [2]). Then in order to make sure that new page table entries were seen by the page table walker, we had to preventively emit a sfence.vma on all harts [3] but this solution is very costly since it relies on IPI. And even there, we could end up in a loop of vmalloc faults if a vmalloc allocation is done in the IPI path (for example if it is traced, see [4]), which could result in a kernel stack overflow. Those preventive sfence.vma needed to be emitted because: - if the uarch caches invalid entries, the new mapping may not be observed by the page table walker and an invalidation may be needed. - if the uarch does not cache invalid entries, a reordered access could "miss" the new mapping and traps: in that case, we would actually only need to retry the access, no sfence.vma is required. So this patch removes those preventive sfence.vma and actually handles the possible (and unlikely) exceptions. And since the kernel stacks mappings lie in the vmalloc area, this handling must be done very early when the trap is taken, at the very beginning of handle_exception: this also rules out the vmalloc allocations in the fault path. Link: https://lore.kernel.org/linux-riscv/20230531093817.665799-1-bjorn@kernel.org/ [1] Link: https://lore.kernel.org/linux-riscv/20230801090927.2018653-1-dylan@andestech.com [2] Link: https://lore.kernel.org/linux-riscv/20230725132246.817726-1-alexghiti@rivosinc.com/ [3] Link: https://lore.kernel.org/lkml/20200508144043.13893-1-joro@8bytes.org/ [4] Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Yunhui Cui <cuiyunhui@bytedance.com> Link: https://lore.kernel.org/r/20240717060125.139416-4-alexghiti@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> Signed-off-by: Yanteng Si <si.yanteng@linux.dev> Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn> In 6.5, we removed the vmalloc fault path because that can't work (see [1] [2]). Then in order to make sure that new page table entries were seen by the page table walker, we had to preventively emit a sfence.vma on all harts [3] but this solution is very costly since it relies on IPI. And even there, we could end up in a loop of vmalloc faults if a vmalloc allocation is done in the IPI path (for example if it is traced, see [4]), which could result in a kernel stack overflow. Those preventive sfence.vma needed to be emitted because: - if the uarch caches invalid entries, the new mapping may not be observed by the page table walker and an invalidation may be needed. - if the uarch does not cache invalid entries, a reordered access could "miss" the new mapping and traps: in that case, we would actually only need to retry the access, no sfence.vma is required. So this patch removes those preventive sfence.vma and actually handles the possible (and unlikely) exceptions. And since the kernel stacks mappings lie in the vmalloc area, this handling must be done very early when the trap is taken, at the very beginning of handle_exception: this also rules out the vmalloc allocations in the fault path. Link: https://lore.kernel.org/linux-riscv/20230531093817.665799-1-bjorn@kernel.org/ [1] Link: https://lore.kernel.org/linux-riscv/20230801090927.2018653-1-dylan@andestech.com [2] Link: https://lore.kernel.org/linux-riscv/20230725132246.817726-1-alexghiti@rivosinc.com/ [3] Link: https://lore.kernel.org/lkml/20200508144043.13893-1-joro@8bytes.org/ [4] Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> Signed-off-by: Yanteng Si <si.yanteng@linux.dev> Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn> (cherry picked from commit 8f039db)
… with Svvptc mainline inclusion from mainline-v6.11 commit 7a21b2e370dab780ddb3aa80f2a4c8ff97bddccc category: feature bugzilla: RVCK-Project/rvck#104 -------------------------------- The preventive sfence.vma were emitted because new mappings must be made visible to the page table walker but Svvptc guarantees that it will happen within a bounded timeframe, so no need to sfence.vma for the uarchs that implement this extension, we will then take gratuitous (but very unlikely) page faults, similarly to x86 and arm64. This allows to drastically reduce the number of sfence.vma emitted: * Ubuntu boot to login: Before: ~630k sfence.vma After: ~200k sfence.vma * ltp - mmapstress01 Before: ~45k After: ~6.3k * lmbench - lat_pagefault Before: ~665k After: 832 (!) * lmbench - lat_mmap Before: ~546k After: 718 (!) Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> [Fix rvck conflicts: arch/riscv/mm/pgtable.c] Signed-off-by: Mingzheng Xing <xingmingzheng@iscas.ac.cn> (cherry picked from commit 32b3a64)
|
开始测试 log: https://github.com/RVCK-Project/rvck-olk/actions/runs/24122320451 参数解析结果
测试完成 详细结果:RVCK result
Kunit Test Result[07:17:29] Testing complete. Ran 454 tests: passed: 442, skipped: 12 Kernel Build ResultKernel build succeeded: RVCK-Project/rvck-olk/165__1/ facd9847cadf08a8a5274116284f633e /srv/guix_result/096585237e830d2722796b0cb85bdf49de9dcfcd/Image LAVA Checkargs:
result:Lava check done! lava log: https://lava.oerv.ac.cn/scheduler/job/1758 lava result count: [fail]: 20, [pass]: 1586, [skip]: 293 Check Patch Result
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync commits from rvck PR #105.
Source PR: RVCK-Project/rvck#105
Commits
7dc32f78220friscv: Only flush the mm icache when setting an exec pte8f039dbf211eriscv: Stop emitting preventive sfence.vma for new vmalloc mappings32b3a647c0a5riscv: Stop emitting preventive sfence.vma for new userspace mappings with Svvptc