[php-src] PR #24283: ext/standard: Add php_next_utf8_char_inline()

Pull Request: https://github.com/php/php-src/pull/24283 Author: ArtUkrainskiy json_encode(), utf8_decode() and the xml extension decode one character at a time with php_next_utf8_char(). This adds php_next_utf8_char_inline(), which handles the common UTF-8 cases at the call site and keeps the existing function as the fallback and the exported API. Non-ASCII text in json_encode() gets faster across all six compiler/CPU configurations tested: Cyrillic takes 0.73–0.80 of master's cycles and CJK 0.70–0.78. Some ASCII-heavy synthetic inputs show compiler- and code-layout-dependent regressions; the largest appear with GCC 13 and 14 on the Intel machine and are discussed under Trade-offs. **Before.** php_next_utf8_char() lives in html.c, so it cannot be inlined into ext/json. Inside it, get_next_char() handles 14 charsets and was not inlined either, so each character paid two calls and a switch on the charset, and the value, the new position and the status came back through memory: about 43 instructions per non-ASCII character in callgrind (36 in get_next_char(), 7 in the wrapper). **After.** php_next_utf8_char_inline() in ext/standard/html.h has the same contract as php_next_utf8_char(). It decodes ASCII and valid two- and three-byte sequences without a call; everything else (four-byte characters, every invalid or truncated sequence) goes to php_next_utf8_char(), which is unchanged and still exported. **Why the shortcut is exact.** A lead byte C2..DF followed by a continuation byte 80..BF is always a valid shortest-form character: C0 and C1 are the only overlong two-byte leads. For three bytes the inline path checks the same two conditions as get_next_char(): the result is at least U+0800 (not overlong) and outside U+D800..U+DFFF (not a surrogate). Any other input runs the old code, so which bytes are rejected or skipped and the returned status stay exactly as before. EXPECTED(c < 0xe0) keeps the two-byte case on the straight path. Without it GCC 13 placed that case behind two extra taken jumps: Cyrillic ran the same instructions but took 20% more cycles. ### Trade-offs - **Two inputs are slower with GCC 13 and 14 on the Intel machine:** URLs (1.10 and 1.14) and ASCII with an occasional é escaped as \u00e9 (1.12 and 1.13). The same regressions are absent or much smaller with clang 18/20 and on AMD: URLs are 1.01–1.06, the é input 0.90–0.96. The generated loop over plain bytes is identical to master's, at the same address (objdump, GCC 14); what changes is the layout GCC picks for the path that handles a special character: the flush of the pending plain run moved out of line, which adds a taken jump per special character. Hinting that branch in the source moves the slowdown to other inputs (a quote every 16 bytes 1.23 with GCC, a quote in every byte 1.33 with clang). Splitting the scan out of the loop and using a byte table removes it (see the appendix of the report). - **A newline every 40 bytes** is at 1.07 and 1.28 with GCC 13 and 14 in my builds, and at 0.97–1.02 when built with plain -O2 instead of the aligned functions and loops I use: code placement. - **An é exactly every 31 bytes** measures between 0.92 and 1.27 depending on compiler and, with clang 20, on the process, which is consistent with the branch predictor learning the periodic pattern. With the é at random positions and the same density it is 0.87–0.96 with JSON_UNESCAPED_UNICODE on all six builds. - **#24145** makes get_next_char() always inline, which also speeds up the out-of-line php_next_utf8_char(). On top of it this PR still removes 14–20% of the instructions on non-ASCII text, but with GCC 13 the cycles change less: Cyrillic 0.98, Cyrillic with JSON_UNESCAPED_UNICODE 0.79, CJK 0.88 of #24145's cycles. Both together: Cyrillic 0.80, CJK 0.78 of master. The two merge without conflicts. - **API.** php_next_utf8_char() and its exported symbol are unchanged; php_next_utf8_char_inline() is new (UPGRADING.INTERNALS). - **Other callers.** utf8_decode() (deprecated, still shipped) runs 2.2–2.8× fewer instructions on 4 KiB of Latin-1, Cyrillic or ASCII text. The xml extension's decoder uses the same path; not measured. - **Size.** +768 bytes of code with GCC 13: php_json_escape_string() +351, utf8_decode() +224, the xml decoder +249. ### Results Relative to master, lower is better. Release builds (-O2, x86-64, --enable-werror): GCC 13.3, GCC 14.2, clang 18.1 and clang 20.1 on an i7-13700H pinned to one P-core; GCC 15.2 and clang 21.1 on a Ryzen 9 5950X pinned to one core. Cycles and instructions per call from perf stat over N calls minus the same process with 0 calls; cycles are the median of 5–14 alternating runs per row, from one series per machine. Instructions are given as a range over the six builds. | input (4 KiB unless noted) | instructions | GCC 13.3 | GCC 14.2 | clang 18.1 | clang 20.1 | GCC 15.2 (AMD) | clang 21.1 (AMD) | |---|---:|---:|---:|---:|---:|---:|---:| | Cyrillic | 0.74–0.78 | 0.80 | 0.75 | 0.73 | 0.79 | 0.73 | 0.75 | | Cyrillic, JSON_UNESCAPED_UNICODE | 0.69–0.79 | 0.74 | 0.72 | 0.79 | 0.73 | 0.81 | 0.78 | | CJK | 0.76–0.81 | 0.78 | 0.78 | 0.73 | 0.75 | 0.70 | 0.75 | | CJK, JSON_UNESCAPED_UNICODE | 0.79–0.82 | 0.84 | 0.77 | 0.83 | 0.81 | 0.84 | 0.67 | | French | 0.83–0.86 | 0.96 | 0.90 | 0.83 | 0.80 | 0.82 | 0.85 | | French, JSON_UNESCAPED_UNICODE | 0.80–0.87 | 0.91 | 0.85 | 0.83 | 0.81 | 0.88 | 0.83 | | ASCII with an é every 31 bytes on average | 0.93–0.94 | 1.12 | 1.13 | 0.90 | 0.92 | 0.95 | 0.96 | | ASCII with an é every 31 bytes on average, JSON_UNESCAPED_UNICODE | 0.92–0.94 | 0.94 | 0.96 | 0.91 | 0.87 | 0.93 | 0.90 | | ASCII, 4 KiB | 1.00 | 0.99 | 1.00 | 0.96 | 1.00 | 1.00 | 1.00 | | HTML | 0.99–1.00 | 0.99 | 1.03 | 1.00 | 0.99 | 1.06 | 1.00 | | URLs | 0.99–1.00 | 1.10 | 1.14 | 1.01 | 1.01 | 1.01 | 1.06 | | a newline every 40 bytes | 1.00 | 1.07 | 1.28 | 1.00 | 1.00 | 1.04 | 1.00 | | a quote in every byte | 1.00–1.02 | 0.99 | 1.01 | 1.00 | 1.00 | 0.91 | 1.00 | | 1,000 arrays of 10 fields | 1.00 | 0.97 | 0.99 | 0.99 | 0.99 | 0.98 | 1.00 | | 1,000 objects of 10 declared properties | 1.00 | 0.97 | 1.00 | 0.99 | 1.01 | 0.99 | 1.00 | All 21 inputs, with the spread between runs, are in the report linked below. ### Verification Output and json_last_error() are identical to master with all six compiler and CPU combinations on: - every pair of bytes, alone and inside ASCII, and every three-byte sequence with an E0..EF lead byte, alone and inside ASCII, under six flag sets each; - every byte value at every position of ASCII strings 1–24 bytes long; - 1,000,000 random strings (5 seeds × 200,000) of 0–300 bytes, as values and as keys, mixing escapes, two-, three- and four-byte characters, U+2028/2029, overlong forms and invalid sequences, under 11 flag sets; - literal keys through arrays, stdClass and declared properties; - utf8_decode() on the same pairs and triples. ext/json, ext/xml and ext/standard/tests/strings pass with the four Intel builds and on a GCC debug build with ASan and UBSan, where the exhaustive checks above report nothing either. The new test json_encode_utf8_sequences.phpt covers the edges of the inline path and the fallback (lone continuation bytes, truncated and overlong sequences, surrogates, four-byte edges, F5) under the JSON_INVALID_UTF8_*, JSON_UNESCAPED_* and JSON_PARTIAL_OUTPUT_ON_ERROR flags; it fails if the overlong checks of the inline path are removed. Separately, the inline decoder was compared with get_next_char(cs_utf_8) on every three-byte prefix, buffer lengths 0–6 and start cursors 0 and 1 (285 million calls): no difference in value, cursor or status. **Not tested:** aarch64, 32-bit and MSVC builds (the change has no width-dependent arithmetic). Full tables, scripts, raw runs and the commands to reproduce them: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/json-encode-utf8-inline