[php-src] PR #22917: Improves test suite performance by 8.5-25x [experiment!]

Pull Request: https://github.com/php/php-src/pull/22917 Author: NickSdot While working on https://github.com/php/php-src/pull/22799 I was bothered by the test suite being so slow. CI needs ~45 minutes from push to finish; running tests locally also takes quite a while. I was wondering why that is and whether it must be like that. Hence, I decided to run an experiment to find out if we could make the developer experience notably better. > [!NOTE] > **This experiment was fully executed by Codex.** I was throwing some money on OpenAI to let > GPT 5.6 Sol pursue the /goal to make the test suite > "massively" faster. This PR is the result of Codex trying exactly that -- for 14 hours > straight. Consequently, I will not pretend that I understand the implications of every proposal -- > and I also did not review everything in detail yet. Though, I **did** spend plenty of time to > confirm the claims and reworking things. > > The result are interesting enough to open this **clearly flagged as LLM-executed experiment**, > hoping that it can serve as a discussion starter to find out whether anything can be picked up. > Would love to get feedback from people with more experience than me, to pin down which of these are > legit ideas and which are dead-ends right away. ## Results Values in seconds; measured on macOS. The full suite improvements are already awesome on their own. But targeted runs also show up to 25x faster results (-j10 vs. -j10), many **at least** 2x faster, and very few slightly slower. ### Full Suite | flag | master | here | gain | | |--------|---------:|--------:|-------------------:|--------------------------------------| | (none) | 2,223.224 | 69.833 | -2,153.39 (31.84x) | auto-detected concurrency by default | | -j1 | 2,224.902 | 443.214 | -1,781.69 (5.02x) | optionally opting out of concurrency | | -j10 | 594.445 | 69.768 | -524.68 (8.52x) | | ### Partial Runs | target | flag | master | here | gain | |---|---|---:|---:|---:| | ext/bcmath | (none) | 10.912 | 1.033 | -9.88s (10.56x) | | | -j1 | 10.912 | 2.038 | -8.87s (5.35x) | | | -j10 | 2.477 | 0.953 | -1.52s (2.60x) | | ext/bz2 | (none) | 2.537 | 0.596 | -1.94s (4.26x) | | | -j1 | 2.537 | 0.507 | -2.03s (5.00x) | | | -j10 | 0.925 | 0.663 | -0.26s (1.40x) | | ext/calendar | (none) | 3.896 | 0.757 | -3.14s (5.15x) | | | -j1 | 3.896 | 1.220 | -2.68s (3.19x) | | | -j10 | 1.048 | 0.702 | -0.35s (1.49x) | | ext/com_dotnet | (none) | 0.119 | 0.354 | +0.23s (0.34x) | | | -j1 | 0.119 | 0.179 | +0.06s (0.66x) | | | -j10 | 0.299 | 0.353 | +0.05s (0.85x) | | ext/ctype | (none) | 3.176 | 0.625 | -2.55s (5.08x) | | | -j1 | 3.176 | 0.676 | -2.50s (4.70x) | | | -j10 | 1.031 | 0.636 | -0.39s (1.62x) | | ext/curl | (none) | 34.101 | 6.639 | -27.46s (5.14x) | | | -j1 | 34.101 | 23.905 | -10.20s (1.43x) | | | -j10 | 35.727 | 6.655 | -29.07s (5.37x) | | ext/date | (none) | 44.032 | 2.711 | -41.32s (16.24x) | | | -j1 | 44.032 | 10.187 | -33.84s (4.32x) | | | -j10 | 9.308 | 2.487 | -6.82s (3.74x) | | ext/dba | (none) | 5.617 | 0.791 | -4.83s (7.10x) | | | -j1 | 5.617 | 0.910 | -4.71s (6.17x) | | | -j10 | 3.134 | 0.853 | -2.28s (3.67x) | | ext/dom | (none) | 54.535 | 1.915 | -52.62s (28.48x) | | | -j1 | 54.535 | 6.628 | -47.91s (8.23x) | | | -j10 | 10.087 | 1.826 | -8.26s (5.52x) | | ext/enchant | (none) | 0.122 | 0.304 | +0.18s (0.40x) | | | -j1 | 0.122 | 0.127 | +0.01s (0.96x) | | | -j10 | 0.308 | 0.314 | +0.01s (0.98x) | | ext/exif | (none) | 6.351 | 0.811 | -5.54s (7.83x) | | | -j1 | 6.351 | 1.196 | -5.16s (5.31x) | | | -j10 | 1.358 | 0.916 | -0.44s (1.48x) | | ext/ffi | (none) | 6.710 | 0.850 | -5.86s (7.89x) | | | -j1 | 6.710 | 1.202 | -5.51s (5.58x) | | | -j10 | 1.576 | 1.154 | -0.42s (1.37x) | | ext/fileinfo | (none) | 5.251 | 0.876 | -4.38s (5.99x) | | | -j1 | 5.251 | 1.696 | -3.56s (3.10x) | | | -j10 | 1.551 | 0.997 | -0.55s (1.56x) | | ext/filter | (none) | 7.543 | 0.915 | -6.63s (8.24x) | | | -j1 | 7.543 | 2.415 | -5.13s (3.12x) | | | -j10 | 1.926 | 1.080 | -0.85s (1.78x) | | ext/ftp | (none) | 7.842 | 1.682 | -6.16s (4.66x) | | | -j1 | 7.842 | 3.100 | -4.74s (2.53x) | | | -j10 | 2.288 | 1.557 | -0.73s (1.47x) | | ext/gd | (none) | 25.575 | 1.503 | -24.07s (17.02x) | | | -j1 | 25.575 | 3.430 | -22.14s (7.46x) | | | -j10 | 6.162 | 1.493 | -4.67s (4.13x) | | ext/gettext | (none) | 1.783 | 0.579 | -1.20s (3.08x) | | | -j1 | 1.783 | 0.511 | -1.27s (3.49x) | | | -j10 | 0.740 | 0.636 | -0.10s (1.16x) | | ext/gmp | (none) | 8.072 | 1.131 | -6.94s (7.14x) | | | -j1 | 8.072 | 2.373 | -5.70s (3.40x) | | | -j10 | 2.075 | 1.134 | -0.94s (1.83x) | | ext/hash | (none) | 4.881 | 0.611 | -4.27s (7.99x) | | | -j1 | 4.881 | 0.918 | -3.96s (5.32x) | | | -j10 | 1.100 | 0.493 | -0.61s (2.23x) | | ext/iconv | (none) | 5.313 | 0.953 | -4.36s (5.58x) | | | -j1 | 5.313 | 2.612 | -2.70s (2.03x) | | | -j10 | 1.420 | 1.023 | -0.40s (1.39x) | | ext/intl | (none) | 32.200 | 4.522 | -27.68s (7.12x) | | | -j1 | 32.200 | 7.158 | -25.04s (4.50x) | | | -j10 | 6.659 | 1.910 | -4.75s (3.49x) | | ext/json | (none) | 6.589 | 0.609 | -5.98s (10.82x) | | | -j1 | 6.589 | 0.966 | -5.62s (6.82x) | | | -j10 | 1.404 | 0.659 | -0.74s (2.13x) | | ext/ldap | (none) | 6.641 | 2.920 | -3.72s (2.27x) | | | -j1 | 6.641 | 2.937 | -3.70s (2.26x) | | | -j10 | 7.438 | 2.999 | -4.44s (2.48x) | | ext/libxml | (none) | 2.498 | 0.660 | -1.84s (3.78x) | | | -j1 | 2.498 | 0.669 | -1.83s (3.73x) | | | -j10 | 0.941 | 0.674 | -0.27s (1.40x) | | ext/mbstring | (none) | 30.374 | 2.481 | -27.89s (12.24x) | | | -j1 | 30.374 | 10.504 | -19.87s (2.89x) | | | -j10 | 7.703 | 2.473 | -5.23s (3.11x) | | ext/mysqli | (none) | 10.021 | 3.824 | -6.20s (2.62x) | | | -j1 | 10.021 | 3.684 | -6.34s (2.72x) | | | -j10 | 9.859 | 3.834 | -6.03s (2.57x) | | ext/odbc | (none) | 0.904 | 0.452 | -0.45s (2.00x) | | | -j1 | 0.904 | 0.352 | -0.55s (2.57x) | | | -j10 | 1.050 | 0.585 | -0.47s (1.79x) | | ext/opcache | (none) | 67.487 | 8.283 | -59.20s (8.15x) | | | -j1 | 67.487 | 54.224 | -13.26s (1.24x) | | | -j10 | 13.405 | 8.552 | -4.85s (1.57x) | | ext/openssl | (none) | 45.311 | 5.035 | -40.28s (9.00x) | | | -j1 | 45.311 | 28.753 | -16.56s (1.58x) | | | -j10 | 49.155 | 5.106 | -44.05s (9.63x) | | ext/pcntl | (none) | 6.540 | 2.506 | -4.03s (2.61x) | | | -j1 | 6.540 | 2.888 | -3.65s (2.26x) | | | -j10 | 3.305 | 2.554 | -0.75s (1.29x) | | ext/pcre | (none) | 12.600 | 1.716 | -10.88s (7.34x) | | | -j1 | 12.600 | 3.450 | -9.15s (3.65x) | | | -j10 | 4.478 | 1.811 | -2.67s (2.47x) | | ext/pdo | (none) | 1.933 | 0.628 | -1.31s (3.08x) | | | -j1 | 1.933 | 0.549 | -1.38s (3.52x) | | | -j10 | 0.971 | 0.679 | -0.29s (1.43x) | | ext/pdo_dblib | (none) | 237.447 | 8.270 | -229.18s (28.71x) | | | -j1 | 237.447 | 11.324 | -226.12s (20.97x) | | | -j10 | 200.863 | 8.234 | -192.63s (24.39x) | | ext/pdo_firebird | (none) | 0.124 | 0.249 | +0.12s (0.50x) | | | -j1 | 0.124 | 0.131 | +0.01s (0.95x) | | | -j10 | 0.289 | 0.269 | -0.02s (1.07x) | | ext/pdo_mysql | (none) | 3.330 | 2.324 | -1.01s (1.43x) | | | -j1 | 3.330 | 2.451 | -0.88s (1.36x) | | | -j10 | 3.807 | 2.287 | -1.52s (1.66x) | | ext/pdo_odbc | (none) | 2.126 | 2.233 | +0.11s (0.95x) | | | -j1 | 2.126 | 2.120 | -0.01s (1.00x) | | | -j10 | 2.467 | 2.160 | -0.31s (1.14x) | | ext/pdo_pgsql | (none) | 3.288 | 2.311 | -0.98s (1.42x) | | | -j1 | 3.288 | 2.564 | -0.72s (1.28x) | | | -j10 | 2.582 | 2.275 | -0.31s (1.13x) | | ext/pdo_sqlite | (none) | 20.236 | 14.875 | -5.36s (1.36x) | | | -j1 | 20.236 | 15.297 | -4.94s (1.32x) | | | -j10 | 16.838 | 14.561 | -2.28s (1.16x) | | ext/pgsql | (none) | 1.654 | 0.751 | -0.90s (2.20x) | | | -j1 | 1.654 | 0.626 | -1.03s (2.64x) | | | -j10 | 2.032 | 2.158 | +0.13s (0.94x) | | ext/phar | (none) | 56.768 | 5.441 | -51.33s (10.43x) | | | -j1 | 56.768 | 24.778 | -31.99s (2.29x) | | | -j10 | 11.601 | 7.420 | -4.18s (1.56x) | | ext/posix | (none) | 4.892 | 0.658 | -4.23s (7.43x) | | | -j1 | 4.892 | 0.796 | -4.10s (6.15x) | | | -j10 | 1.686 | 0.822 | -0.86s (2.05x) | | ext/random | (none) | 4.283 | 0.492 | -3.79s (8.71x) | | | -j1 | 4.283 | 0.774 | -3.51s (5.53x) | | | -j10 | 0.964 | 0.445 | -0.52s (2.17x) | | ext/readline | (none) | 2.292 | 0.676 | -1.62s (3.39x) | | | -j1 | 2.292 | 1.228 | -1.06s (1.87x) | | | -j10 | 1.065 | 0.640 | -0.42s (1.66x) | | ext/reflection | (none) | 31.491 | 1.242 | -30.25s (25.36x) | | | -j1 | 31.491 | 4.245 | -27.25s (7.42x) | | | -j10 | 6.614 | 1.262 | -5.35s (5.24x) | | ext/session | (none) | 17.882 | 2.428 | -15.45s (7.36x) | | | -j1 | 17.882 | 10.166 | -7.72s (1.76x) | | | -j10 | 3.771 | 2.731 | -1.04s (1.38x) | | ext/shmop | (none) | 0.474 | 0.331 | -0.14s (1.43x) | | | -j1 | 0.474 | 0.307 | -0.17s (1.54x) | | | -j10 | 0.336 | 0.341 | +0.01s (0.99x) | | ext/simplexml | (none) | 9.555 | 0.720 | -8.83s (13.27x) | | | -j1 | 9.555 | 1.109 | -8.45s (8.62x) | | | -j10 | 2.105 | 0.795 | -1.31s (2.65x) | | ext/skeleton | (none) | 0.117 | 0.196 | +0.08s (0.60x) | | | -j1 | 0.117 | 0.126 | +0.01s (0.93x) | | | -j10 | 0.194 | 0.193 | -0.00s (1.01x) | | ext/snmp | (none) | 43.267 | 6.695 | -36.57s (6.46x) | | | -j1 | 43.267 | 6.455 | -36.81s (6.70x) | | | -j10 | 43.186 | 6.596 | -36.59s (6.55x) | | ext/soap | (none) | 40.262 | 2.767 | -37.49s (14.55x) | | | -j1 | 40.262 | 10.148 | -30.11s (3.97x) | | | -j10 | 7.781 | 2.916 | -4.87s (2.67x) | | ext/sockets | (none) | 11.027 | 2.600 | -8.43s (4.24x) | | | -j1 | 11.027 | 3.256 | -7.77s (3.39x) | | | -j10 | 3.956 | 2.757 | -1.20s (1.43x) | | ext/sodium | (none) | 12.679 | 1.117 | -11.56s (11.35x) | | | -j1 | 12.679 | 2.257 | -10.42s (5.62x) | | | -j10 | 10.710 | 1.115 | -9.60s (9.61x) | | ext/spl | (none) | 51.527 | 2.119 | -49.41s (24.32x) | | | -j1 | 51.527 | 9.217 | -42.31s (5.59x) | | | -j10 | 9.515 | 2.597 | -6.92s (3.66x) | | ext/sqlite3 | (none) | 6.761 | 0.737 | -6.02s (9.17x) | | | -j1 | 6.761 | 1.203 | -5.56s (5.62x) | | | -j10 | 1.494 | 0.707 | -0.79s (2.11x) | | ext/standard | (none) | 322.044 | 14.536 | -307.51s (22.15x) | | | -j1 | 322.044 | 73.297 | -248.75s (4.39x) | | | -j10 | 56.019 | 14.828 | -41.19s (3.78x) | | ext/sysvmsg | (none) | 0.541 | 0.378 | -0.16s (1.43x) | | | -j1 | 0.541 | 0.211 | -0.33s (2.56x) | | | -j10 | 0.374 | 0.375 | +0.00s (1.00x) | | ext/sysvsem | (none) | 0.300 | 0.304 | +0.00s (0.99x) | | | -j1 | 0.300 | 0.242 | -0.06s (1.24x) | | | -j10 | 0.370 | 0.310 | -0.06s (1.19x) | | ext/sysvshm | (none) | 0.997 | 0.471 | -0.53s (2.12x) | | | -j1 | 0.997 | 0.380 | -0.62s (2.62x) | | | -j10 | 0.638 | 0.582 | -0.06s (1.10x) | | ext/tidy | (none) | 3.212 | 0.586 | -2.63s (5.48x) | | | -j1 | 3.212 | 0.706 | -2.51s (4.55x) | | | -j10 | 1.058 | 0.783 | -0.28s (1.35x) | | ext/tokenizer | (none) | 3.469 | 0.565 | -2.90s (6.14x) | | | -j1 | 3.469 | 0.693 | -2.78s (5.01x) | | | -j10 | 0.999 | 0.552 | -0.45s (1.81x) | | ext/uri | (none) | 22.419 | 0.779 | -21.64s (28.78x) | | | -j1 | 22.419 | 2.107 | -20.31s (10.64x) | | | -j10 | 5.517 | 0.730 | -4.79s (7.56x) | | ext/xml | (none) | 4.333 | 0.764 | -3.57s (5.67x) | | | -j1 | 4.333 | 0.809 | -3.52s (5.36x) | | | -j10 | 1.224 | 0.779 | -0.44s (1.57x) | | ext/xmlreader | (none) | 4.449 | 0.618 | -3.83s (7.20x) | | | -j1 | 4.449 | 0.697 | -3.75s (6.38x) | | | -j10 | 1.379 | 0.827 | -0.55s (1.67x) | | ext/xmlwriter | (none) | 3.234 | 0.618 | -2.62s (5.23x) | | | -j1 | 3.234 | 0.592 | -2.64s (5.46x) | | | -j10 | 0.925 | 0.553 | -0.37s (1.67x) | | ext/xsl | (none) | 0.129 | 0.416 | +0.29s (0.31x) | | | -j1 | 0.129 | 0.135 | +0.01s (0.96x) | | | -j10 | 0.340 | 0.342 | +0.00s (0.99x) | | ext/zend_test | (none) | 10.192 | 1.549 | -8.64s (6.58x) | | | -j1 | 10.192 | 5.229 | -4.96s (1.95x) | | | -j10 | 2.176 | 1.376 | -0.80s (1.58x) | | ext/zip | (none) | 11.918 | 1.040 | -10.88s (11.46x) | | | -j1 | 11.918 | 1.468 | -10.45s (8.12x) | | | -j10 | 2.609 | 0.904 | -1.71s (2.89x) | | ext/zlib | (none) | 11.232 | 1.199 | -10.03s (9.37x) | | | -j1 | 11.232 | 2.992 | -8.24s (3.75x) | | | -j10 | 2.196 | 1.025 | -1.17s (2.14x) | | sapi/cgi | (none) | 5.833 | 2.246 | -3.59s (2.60x) | | | -j1 | 5.833 | 5.047 | -0.79s (1.16x) | | | -j10 | 2.242 | 2.142 | -0.10s (1.05x) | | sapi/cli | (none) | 24.166 | 2.849 | -21.32s (8.48x) | | | -j1 | 24.166 | 16.504 | -7.66s (1.46x) | | | -j10 | 5.508 | 2.902 | -2.61s (1.90x) | | sapi/fpm | (none) | 1.595 | 0.421 | -1.17s (3.79x) | | | -j1 | 1.595 | 0.417 | -1.18s (3.82x) | | | -j10 | 1.491 | 0.410 | -1.08s (3.64x) | | sapi/phpdbg | (none) | n/a | n/a | n/a | | | -j1 | 246.564 | 6.091 | -240.47s (40.48x) | | | -j10 | n/a | n/a | n/a | | tests | (none) | 64.478 | 4.645 | -59.83s (13.88x) | | | -j1 | 64.478 | 20.204 | -44.27s (3.19x) | | | -j10 | 13.876 | 4.398 | -9.48s (3.16x) | | Zend | (none) | 373.757 | 15.621 | -358.14s (23.93x) | | | -j1 | 373.757 | 49.758 | -324.00s (7.51x) | | | -j10 | 75.593 | 13.010 | -62.58s (5.81x) | _The -j1 row compares the master default with the explicit sequential opt-out._ ## Commits | commit | change | type | before | after | saved | speedup | comment | | --- | --- | --- | ---: | ---: | ---: | ---: | --- | | 9b36b9145a0 | Run tests in parallel by default | (D) | 387.450 | 373.760 | 13.690 | 1.04× | Automatically selects up to 10 workers. Explicit -j1 remains the serial opt-out. Includes a regression proving the cap. | | 01dbbb753be | Select Windows test workers automatically | (W) | — | — | — | — | Non-ASAN Windows CI uses automatic detection; ASAN remains -j2. Requires Windows CI measurement. | | 90c943c22ed | Parallelize new_oom subprocesses | (D) | 363.220 | 283.265 | 79.955 | 1.28× | Uses bounded subprocess concurrency, ignores non-instantiable classes, and rejects subprocesses killed by signals. | | 6e0047855c0 | Reduce parallel batch size | (D) | 283.265 | 157.091 | 126.174 | 1.80× | Reduces the long tail caused by a few slow batches. | | f0f42a01620 | Reuse CLI startup with an isolated fork server | (F) | 0.679 | 0.251 | 0.428 | 2.71× | Readonly class/property subset, -j10, median of three runs, disabled versus enabled. Includes signal-safe child tracking, environment-aware caching and timeout/failure fallback coverage. | | dda593f40d4 | Cache and bound failed PDO connection probes | (C) | 200.863 | 8.234 | 192.629 | 24.39× | ext/pdo_dblib, -j10, final branch versus master. A one-second timeout applies only to the default availability probe. Configured connection attributes are preserved. | | 7d7ddc611b4 | Handle redirected tests in parallel runs | (V) | fatal / 3/2 | pass / 3/3 | — | — | Redirected child tests stay serial inside their already-parallel wrapper, while the parent receives the expanded test count. | | 2546b0748ce | Add bounded directory concurrency | (F) | 17.804 | 3.211 | 14.593 | 5.54× | Adds independent MAX_CONCURRENCY and SCOPED_CONFLICTS. Ordinary CONFLICTS remains strict. Includes regressions for both semantics. | | 1d5cdeb84f7 | Cache failed SNMP-agent probes | (C) | 43.186 | 6.596 | 36.590 | 6.55× | ext/snmp, -j10, final branch versus master. Only failed probes are cached. | | 7c3349b8a51 | Avoid external DNS in addrinfo test | (F) | 0.560 | 0.560 | 0.000 | 1.00× | No median gain. Retained as an offline/reliability improvement because the hostname is incidental to the error-code assertions. | | bba23c51046 | Run scandir overflow test on Windows only | (F) | 9.620 | 0.450 | 9.170 | 21.38× | Avoids creating 70,000 files on platforms unaffected by the Win32 DIR_W32.offset regression. | | c95d9c38a60 | Remove redundant filesystem-test delays | (F) | 15.540 | 3.410 | 12.130 | 4.56× | Replaces timing assumptions with explicit timestamps or removes waits after the required state is already established. | | 318646f54b0 | Reduce password-test work factors | (F) | 14.990 | 1.790 | 13.200 | 8.37× | Uses valid lower-cost bcrypt, Argon2, sodium, and extended-DES parameters while retaining the tested behavior. | | d3ec35d181f | Optimize Phar regression fixtures | (F) | 3.040 | 1.112 | 1.928 | 2.73× | Preserves bug #13727’s source-stream-close regression and uses a controlled iterator fixture. Median of three runs. | | 112f3ca117e | Avoid opcache revalidation delays | (F) | 4.650 | 0.630 | 4.020 | 7.38× | Advances the fixture timestamp explicitly instead of waiting for wall-clock time. | | 8779644b4b4 | Synchronize proc_open tests without fixed delays | (F) | 9.300 | 1.210 | 8.090 | 7.69× | Replaces sleeps with polling, socket handshakes, and observable process state. | | af1d402109d | Use a bounded passthru binary fixture | (F) | 0.880 | 0.700 | 0.180 | 1.26× | Preserves exact binary round-trip coverage with a deterministic 256 KiB all-byte pattern. | | 2bf072b1b00 | Parallelize OpenSSL tests safely | (C) | 49.155 | 5.106 | 44.049 | 9.63× | ext/openssl, -j10, final branch versus master. Uses isolated fixtures, dynamic ports, scoped server conflicts and a cap of 10. | | cf72013641f | Bypass the shell for test subprocesses | (F) | 12.120 | 10.690 | 1.430 | 1.13× | Structured proc_open() microbenchmark. Preserves environment isolation and falls back where shell semantics are required. | | b37c4620c4e | Run Curl tests with bounded concurrency | (C) | 35.727 | 6.655 | 29.072 | 5.37× | ext/curl, -j10, final branch versus master. Isolates duplicate fixtures and caps the directory at four workers. | | 559f5290780 | Isolate open_basedir fixtures | (F) | 1.372 | 0.667 | 0.705 | 2.06× | Uses independent fixture paths, scoped open_basedir conflicts, a cap of 10 and symlink-safe cleanup. | | 7246f618223 | Skip unavailable Unix datagram socket tests | (E) | 121.060 | 0.620 | 120.440 | 195.26× | Restricted environment only. Recognized unsupported/permission errors skip quickly; unexpected bind failures BORK. Capable hosts execute the original test body. | Legend: (D) development-step measurement; (C) cumulative module measurement; (F) focused local measurement; (E) environment-specific measurement; (V) correctness verification; (W) Windows CI pending. ## Misc - At first, I found it weird that it would propose adding test-only code to the SAPI. But then I learned that the already existing, internal --repeat flag does exactly that. - I am pretty sure that there are reasons for Windows in CI only running on -j2, and parallel not being the default. I, however, could not find out what these reasons are; hence, I kept changes Codex proposed. - not sure whether we fancy adding new phpt sections like MAX_CONCURRENCY, but it shows gains. - My repeated benchmark runs exposed existing bugs, fixed here in 9905f197bbe118843b1651cb42ab84bd5b7587b2 and 6be6a43ab1c93ff5c29b26eb07128e159ee3678f (also PR'ed separately in #22911), and f68eeaae2d32b847f61235f0626eacf18388a381 and a029596a5ca4251136ecc32d9777108a269edc3d (also PR'ed separately in #22912). - buggy concurrent execution CLI output (eg: TEST **22386**/22094 [2/10 concurrent test workers running]) was fixed. - Codex did *not* consider this done yet; so if there is general interest I will let it finish the job. - I know, another huge PR by me. Keen to split that up as we go.