[php-src] PR #25420: Make setlocale() thread-local in ZTS builds

Pull Request: https://github.com/php/php-src/pull/25420 Author: felixbuenemann In thread-safe builds, setlocale() changes the locale of the whole process: - a request uses the locale set by requests running in other threads; - the reset at the end of a request also resets the locale of all other requests; - concurrent calls can crash, because the C library's setlocale() isn't thread-safe. The manual documents the first point as a limitation. Windows doesn't have it: PHP keeps the locale per thread there since 5.3.6 (0778bedbe1f) and 5.6.20/7.0.5 (632fc51d984, bug #71596). This PR does the same for the other platforms, using newlocale() and uselocale(). It takes the approach of #20531, which was closed in favour of a broader ZTS rework. On internals ([externals.io/message/128991](https://externals.io/message/128991)), a pull request was suggested rather than an RFC. ## Changes - New zend_setlocale() in Zend/zend_locale.c. In ZTS builds with uselocale(), during a request it switches the current thread to a locale of its own. The locale-dependent functions of the C library (printf(), localeconv(), strftime(), strcoll(), nl_langinfo(), mblen()…) use that locale in place of the process locale. In other builds it's a macro for setlocale(). - PHP's own setlocale() calls go through it: the setlocale() function, the reset in PHP_RSHUTDOWN(basic), zend_reset_lc_ctype_locale(), zend_update_current_locale() and NumberFormatter::parse(). - configure.ac checks for uselocale() and xlocale.h. Behaviour that doesn't change: - The process locale is still set during startup, and it is the initial locale of every thread. Outside of a request (!EG(active)), zend_setlocale() changes the process locale as before. - A request that changed the locale still resets it to C at the end. That now only affects its own thread, as on Windows. - setlocale() returns the same names as in NTS builds, including the name of LC_ALL when the categories have different locales. That name has a different format in glibc, musl and macOS/BSD, so setlocale(LC_ALL, $saved) keeps working. - NTS builds and Windows are unaffected. ## Implementation notes - A locale object is created once for each combination of category locales and shared by all threads. Threads only switch between these objects with uselocale(). - The locale objects are never freed. Threads keep using them, a process only needs a few, and macOS crashes when newlocale() and freelocale() run concurrently. The number of objects is not bounded, though: it grows with the distinct combinations a process sets. - Lookup and creation happen under tsrm_env_lock(), because the "" locale reads the environment. - Requests that don't call setlocale() run no new code. - A thread that hasn't set a locale never calls setlocale(LC_ALL, NULL): on macOS that call writes to a static buffer, so concurrent calls return garbage. ## Tests - setlocale_variation6.phpt (new) covers restoring a saved name of LC_ALL when the categories have different locales. - setlocale_variation4.phpt and setlocale_variation5.phpt had a trailing semicolon in their --ENV-- section, so the locale was en_US.utf8;. glibc accepts that name in setlocale(), but not in newlocale(). The semicolon is removed. - phpt tests can't run threads. I tested the thread behaviour with the program below, which uses the embed SAPI: 9 threads run 2,000 requests each. A third of the threads switch to German, a third to French, and a third never set a locale. Each request checks its initial locale and its results. | | macOS 27 | Debian 13 (glibc 2.41) | Alpine 3.24 (musl) | |---|---|---|---| | This PR | passes (3 of 3 runs) | passes (3 of 3 runs) | passes (3 of 3 runs) | | master | aborts (3 of 3 runs) | wrong locales (3 of 3 runs) | wrong locales (3 of 3 runs) | - On the same three systems, I compared the output of a ZTS build with an NTS build for 61 setlocale() edge cases (invalid names, '0', '', null, arrays, POSIX, wrong types…), 7 environments (LANG, LC_ALL, LC_NUMERIC, none…) and 15 names of LC_ALL with missing, extra or empty parts. The output is identical. - The tests in ext/standard/tests/strings, ext/standard/tests/general_functions, ext/ctype, ext/pcre and tests/lang give the same results in ZTS and NTS builds on all three systems. The NumberFormatter tests pass on Debian and Alpine. - Not tested: FreeBSD, Solaris and Windows. ## Limitations - C code that calls the C library's setlocale() directly, in an extension or a library, still changes the process locale, and doesn't see the locale of the thread. Extensions should call zend_setlocale() instead (see UPGRADING.INTERNALS). - On systems where LC_ALL is negative (AIX, as far as I know), the category table in zend_locale.c won't compile in ZTS builds. - On other C libraries than glibc, musl and those of macOS and the BSDs, the name of LC_ALL for different category locales uses the musl format. It still round-trips through setlocale(). <details> <summary>Thread test program (embed SAPI)</summary> ```c /* Runs PHP requests in several threads with the embed SAPI. A third of the * threads switch to German, a third to French and a third never set a locale. * Each request checks that it starts with the initial locale and that requests * in other threads don't change its locale. * Usage: threads [threads] [requests per thread] */ #include <sapi/embed/php_embed.h> #include <locale.h> #include <pthread.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #define KINDS 3 static int requests = 2000; static const char *locales[KINDS] = {"de_DE.UTF-8", "fr_FR.UTF-8", ""}; static char codes[KINDS][2048]; static char expected[KINDS][1024]; /* Returns "<initial locale>|<locale>|<decimal point>|<number>|<month>" */ static const char *script = "(function () {" " $locale = '%s';" " $initial = setlocale(LC_ALL, 0);" " $set = $locale === '' ? $initial : setlocale(LC_ALL, $locale);" " usleep(50);" " $result = implode('|', [$initial, $set, setlocale(LC_ALL, 0)," " localeconv()['decimal_point'], sprintf('%%.1f', 1.5), nl_langinfo(MON_1)]);" " if (%d) { setlocale(LC_ALL, $initial); }" " return $result;" "})()"; static bool eval(const char *code, char *result, size_t size) { zval retval; if (zend_eval_string(code, &retval, "request") != SUCCESS || Z_TYPE(retval) != IS_STRING) { return false; } snprintf(result, size, "%s", Z_STRVAL(retval)); zval_ptr_dtor(&retval); return true; } static void *run(void *arg) { long id = (long) arg, wrong = 0; int kind = id % KINDS; char result[1024], example[1024] = ""; (void) ts_resource(0); for (int i = 0; i < requests; i++) { php_request_startup(); if (!eval(codes[kind], result, sizeof(result)) || strcmp(result, expected[kind]) != 0) { if (!wrong++) { snprintf(example, sizeof(example), "%s", result); } } php_request_shutdown(NULL); } printf("thread %2ld (%-11s): %ld of %d requests wrong%s%s\n", id, locales[kind][0] ? locales[kind] : "no locale", wrong, requests, wrong ? ", e.g. " : "", example); ts_free_thread(); return (void *) wrong; } int main(int argc, char **argv) { int count = argc > 1 ? atoi(argv[1]) : 9; long failed = 0; char initial[512], code[2048]; pthread_t threads[64]; if (argc > 2) { requests = atoi(argv[2]); } php_embed_init(1, argv); /* The expected results, from a single thread */ for (int kind = 0; kind < KINDS; kind++) { snprintf(code, sizeof(code), script, locales[kind], 1); snprintf(codes[kind], sizeof(codes[kind]), script, locales[kind], 0); if (!eval(code, expected[kind], sizeof(expected[kind]))) { return 2; } printf("expected (%-11s): %s\n", locales[kind][0] ? locales[kind] : "no locale", expected[kind]); } snprintf(initial, sizeof(initial), "%s", setlocale(LC_ALL, NULL)); for (long i = 0; i < count; i++) { pthread_create(&threads[i], NULL, run, (void *) i); } for (int i = 0; i < count; i++) { void *result; pthread_join(threads[i], &result); failed += (long) result; } if (strcmp(initial, setlocale(LC_ALL, NULL)) != 0) { printf("process locale changed from %s to %s\n", initial, setlocale(LC_ALL, NULL)); failed++; } printf(failed ? "FAILED\n" : "OK\n"); php_embed_shutdown(); return failed != 0; } ``` Built with --enable-zts --enable-embed, and linked against libphp. </details>