aarch64 msys2-runtime: fork() fails intermittently, roughly 1% of the time

[aswin-mcw]

Summary

fork() fails at random on the aarch64 msys2-runtime, at a rate around 1%. The
child dies with an access violation while the parent is copying the cygheap into
it:

bash: fork: retry: Resource temporarily unavailable
  0 [main] bash (16828) child_copy: cygheap read copy failed, 0x0..0x8000236E0,
           done 0, windows pid 16828, Win32 error 299
  0 [main] bash 3348 dofork: child -1 - forked process 16828 died unexpectedly,
           retry 0, exit code 0xC0000005, errno 11

Win32 error 299 is ERROR_PARTIAL_COPY; 0xC0000005 is an access violation.

This is not the same as the socket() crash reported in Multicorewareinc/MSYS2-packages#5 - fork() is
failing on its own, with both parent and child being ARM64 binaries in a
properly assembled ARM64 root.

Impact

Anything that spawns many processes fails at random: configure scripts, make,
makepkg, test suites. There is no error in most cases - the command simply
produces empty output and the script carries on with a wrong value.

It was found because three unrelated one-off failures turned up during testing
and none reproduced on retry:

• libtool's wrapper binary segfaulted once while building gpgme
  ($cwrapper --lt-dump-script), then never again;
• envsubst --version crashed once in a sweep of 165 executables, then passed
  20 consecutive runs;
• grep -c returned empty once in a functional test, then passed 300 runs.

All three are the same bug.

Reproduction

With an ARM64 bash as the parent, in an ARM64 root:

hits=0
for i in $(seq 1 400); do
  out=$(printf abc | sha256sum | cut -c1-8)
  [ "$out" != "ba7816bf" ] && hits=$((hits+1))
done
echo "failures: $hits/400"

Measured here:

what	failures
single grep -c spawn, x86_64 parent	1 / 300
three-process pipeline, x86_64 parent	6 / 300
three-process pipeline, ARM64 parent	5 / 400

So roughly 0.3% per spawn and 1-2% per pipeline - it scales with the number of
forks, as expected if each fork has an independent chance of failing.

The silent part is the dangerous part

When the parent is an x86_64 bash driving ARM64 children, no message is
printed at all - the command substitution simply yields an empty string. A
build that captures a version, a path or a checksum that way gets an empty value
and continues. Only with an ARM64 parent does bash report fork: retry and the
child_copy diagnostic.

Note

fork() and execvp() work correctly in the ordinary case - a standalone C
program doing fork+exec succeeds, and the whole 23-package chain builds. This is
a low-probability failure that only becomes visible at scale, which is exactly
why it is worth fixing before anyone runs a long build on this runtime.