perf: move EG() and CG() in ZTS builds into __thread storage

[henderkes]

So, I was chasing what caused the much higher instruction count on aarch64 for clang vs gcc and while figuring that out, got side tracked to figuring out that ZTS builds also have a ton of extra instructions compared to NTS builds. Turned out it was mostly just from constantly accessing executor and compiler globals when running phpstan. Each access became pseudo assembly:

mrs _tsrm_ls_cache
add cache offset low bits
add cache offset high bits
load relative base address
load accessed global

Here we're slashing one extra load and one add by moving EG and CG onto actual thread storage. Benchmarks are using phpstan on full laravel codebase (full symfony codebase showed the same, but I added the LE optimization after and don't want to rerun 6 hours of benchmarks) and phoronix-test-suite phpbench.

aarch64:

• ~80% fewer extra instructions
• Ampere A1 cloud machine, ~0.15% variance in wall clock between runs

build	PHPStan cyc	instr	wall	phpbench	phpbench JIT
gcc-nts	90.67B	108.22B	32.0s	715719	1164671
gcc-zts	94.44B	116.28B	33.2s	662475	1111162
gcc-zts-patch	91.61B	109.66B	32.3s	704490	1146721
clang-nts	95.20B	113.29B	33.4s	709077	1118672
clang-zts	97.92B	120.82B	34.4s	665074	1069905
clang-zts-patch	93.40B	114.72B	33.1s	696929	1101997

x86_64:

• ~35% fewer extra instructions
• no cycle count on x86_64 because lack of hardware counter support in WSL
• i7 12650h with locked cpu frequency, take the wall times with a grain of salt:

build	PHPStan cyc	instr	wall	phpbench	phpbench JIT
gcc-nts	n/a	11.01B	113.91s	1065687	1793058
gcc-zts	n/a	11.23B	115.47s	1033552	1767360
gcc-zts-patch	n/a	11.15B	116.34s	1005077	1713823
clang-nts	n/a	11.38B	112.92s	1125988	1795096
clang-zts	n/a	11.64B	114.67s	1087697	1757513
clang-zts-patch	n/a	11.53B	112.22s	1145744	1793294

LLM disclosure

claude

• adjusted the JIT
• transformed the raw bash results into these (hopefully beautiful) html tables
• edit: it failed at the tables, or I failed copy-pasting

[henderkes]

Meant to write comments explaining a few peculiar decisions... but got carried away in the evening. Will get to it and tag relevant reviewers in the morning.

[henderkes]

Here it's possible that a thread id was recycled, but the tls points to a now obsolete one

[henderkes]

we can't manually free __thread storage

[henderkes]

function largely copied from above, looking at it now I see that size_t should be printed as %zu.

[henderkes]

a PIE program can use local exec if it's the main executable. Only shared libraries (embed, extensions) need to fall back to initial-exed.

This alone would already be a small speedup (one fewer instruction per access)

[henderkes]

This would produce linker errors if tls size exceeded 4kb, but I did a test atatic build with 100 extensions statically compiled in (what a terrible idea) and tls size ended up at 3.7kb. I can't think of a way to test whether a link would succeed before compiling, so this is unconditional.

[henderkes]

@arnaud-lb I also had a different idea on achieving the same thing by pinning resolved EG and CG to a register on gcc. It worked well, but

• I wouldn't know how to guarantee the extra register doesn't regress performance in some scenarios
• libxml & other extensions clobbered any register I picked, so it took a large changelist
• is hybrid-vm only, for Clang I tried passing them as tailcall params, but it became very messy

Perhaps you have a different idea here?

[arnaud-lb]

This is a good idea, but you will need to check that it works in various configurations:

• Accessing TLS variables from other modules (shared extensions) used to be broken on some platforms (module wouldn't link or wouldn't load). I don't know if that's still the case, but that's one reason why extensions have their own _tsrm_ls_cache instead of accessing the main one.
• local-exec / initial-exec TLS when php is itself a shared library (apache module, embed) is allocated from the TLS surplus which is non portable and limited in size. The size of EG+CG might be larger than surplus. I don't remember why we specify the model explicitly but the loader patches the code to use more efficient models when applicable, so this may not be necessary.

Register pinning on the hybrid VM would work, but it may increase register pressure. I've tried adding new VM arguments recently for another reason, and it resulted in a performance regression. Global registers would work too, but only in VM code as the register can be clobbered when the variable is not in scope.

Some idea: EG()/CG() and a few others use ZEND_TSRMG_FAST(), which is _tsrm_ls_cache+some_offset. So EG(foo) is load(_tsrm_ls_cache)+load(some_offset)+offsetof(zend_executor_globals,foo). We could try to make the offset a const expr so that it folds to load(_tsrm_ls_cache)+const:

• Create a struct that contains all fast TLS entries (see php_reserve_tsrm_memory)
• Replace php_reserve_tsrm_memory() by tsrm_reserve(sizeof(the_struct)).
• #define EG(v) ZEND_TSRMG_FAST(offsetof(the_struct, eg), zend_executor_globals *, v)

[arnaud-lb]

Possibly these could be embeded in the main _tsrm_ls_cache (TSRMLS_MAIN_CACHE_DEFINE) :

ZEND_API TSRM_TLS TSRM_TLS_MODEL_ATTR struct {
    void *cache;
    zend_executor_globals eg;
    zend_compiler_globals cg;
} _tsrm_ls_cache;

This would simplify JIT changes as we can access eg/cg at an offset from jit->tls.

Also the compiler may generate better code when both EG and CG are used in the same function.