Bug #73265 [Opn]: Loading browscap.ini at startup causes high memory usage

Edit report at https://bugs.php.net/bug.php?id=73265&edit=1 ID: 73265 Updated by: nikic@php.net Reported by: spam2 at rhsoft dot net Summary: Loading browscap.ini at startup causes high memory usage Status: Open Type: Bug Package: Performance problem Operating System: Linux PHP Version: 7.0.14 Block user comment: N Private report: N New Comment: get_browser() is probably slower in PHP 7 because of the PCRE JIT. Most of the time in get_browser() is spent is compiling regular expressions, and the JIT requires more complication time. I have spent some time yesterday implementing many memory usage and performance optimizations for browscap, I'll probably put up a PR today. However, the changes are very extensive, so not sure if we target PHP 7.0 for that. Previous Comments: ------------------------------------------------------------------------ [2016-12-16 16:38:58] spam2 at rhsoft dot net so - now it is proven that get_browser() with PHP7 is magnitudes slower than with PHP5, i have stored the binaries and extensions of the two versions running before and after the upgrade and gave them identical configurations PHP 5.6.26 (cli) (built: Oct 3 2016 12:45:59) PHP 7.0.11 (cli) (built: Oct 5 2016 22:28:49) ______________________________ 10 x get_browser('Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0'); [harry@srv-rhsoft:/data/scripts/php5-versus-7]$ ./test.sh PHP5 real 0m5.876s user 0m5.799s sys 0m0.034s PHP7 real 0m14.396s user 0m14.216s sys 0m0.085s ______________________________ 100 x get_browser('Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0'); [harry@srv-rhsoft:/data/scripts/php5-versus-7]$ ./test.sh PHP5 real 0m57.583s user 0m57.180s sys 0m0.052s PHP7 real 2m23.615s user 2m22.030s sys 0m0.691s ______________________________ that's factor 2.4 and so in fact the asnwer to my initial question below is "get_browser()" which was used until yesterday on every inital request with no active session which is always true for "ab"-benchmarks so one thing is the dramatical memory usage with 7.0.14 but in general the function got extremely slow with the same "browscap.ini" _________________________ we have two different inhouse cms-systems, while one is 46% faster with PHP7 the other one sucks terrible - are there things known by developers which are internally slower now while most other become faster and should be avoided? ------------------------------------------------------------------------ [2016-12-15 16:02:32] spam2 at rhsoft dot net > From a quick test, performance of loading browscap.ini > did not significantly change between 5.6 and 7.0. > When you compared to PHP 5.6, were you also loading > this browscap.ini? *yes* - i synced the "php.spec" for 5.6 and 7.0 days before the upgrade so that it's drop in and only the loadmodule line in httpd needs to be changed what we did was * restart webserver * benchmark cms 1 * restart webserver * benchmark cms 2 * note numbers "dnf -y upgrade" * restart webserver * benchmark cms 1 * restart webserver * benchmark cms 2 * note numbers drop from 100 to 50 requests per second, well, that application is *using* get_browser() within a if(function_exists('get_browser')) - hence i was able to add it to 'disabled_functions' - so it's not only about loading browscap and a likely regression in 7.0.14 but also about get_browser() with the same "browscap.ini" became noticeable slower PHP 5.6.26: Requests per second: 107.24 [#/sec] (mean) Time per request: 279.741 [ms] (mean) Time per request: 9.325 [ms] (mean, across all concurrent requests) Transfer rate: 1531.03 [Kbytes/sec] received PHP 7.0.11 PGO: Requests per second: 56.30 [#/sec] (mean) Time per request: 532.864 [ms] (mean) Time per request: 17.762 [ms] (mean, across all concurrent requests) Transfer rate: 803.75 [Kbytes/sec] received PHP 7.0.14 NON-BROWSCAP: Requests per second: 327.45 [#/sec] (mean) Time per request: 61.077 [ms] (mean) Time per request: 3.054 [ms] (mean, across all concurrent requests) Transfer rate: 4754.80 [Kbytes/sec] received ------------------------------------------------------------------------ [2016-12-15 13:27:36] spam2 at rhsoft dot net i have updated it on 2016-11-19 * Do Dez 08 2016 Reindl Harald <h.reindl@thelounge.net> - update to PHP 7.0.14 so i can't say it for 100% sure but i am very certain that 7.0.14 made this part much worser - what is *not* normal is zend_parse_ini_file() calling that often within a while(true) loop -> http://access.thelounge.net/harry/massif.txt since that was a cli script with a loop i would expect that only happen *once* at startup and your comment (This could also explain why you're seeing a difference in 7.0.14, because there were some changes to the ini parser) could explain some parts of it ------------------------------------------------------------------------ [2016-12-15 12:57:59] nikic@php.net Okay, not terribly surprised that loading an 8MB ini file is going to use lots of memory. From a quick test, performance of loading browscap.ini did not significantly change between 5.6 and 7.0. When you compared to PHP 5.6, were you also loading this browscap.ini? There are some optimization opportunities here: a) We shouldn't be loading the browscap.ini unless it is actually necessary. In fact, browscap.ini is already loaded lazily by get_browser() if the browscap ini option is only available during activation (but not startup). However, this mode has the disadvantage that the data is loaded per-thread and per-request. If the get_browser() function is actually used on every request, this is going to be (significantly) more expensive than loading it once. Ideally, we'd have lazy loading that still shares across requests and threads. The latter would require manual synchronization, but I think we could implement the former easily and then drop the startup loading. Threads are not relevant in most PHP deployments. b) We can optimize the loading and in-memory representation of browscap.ini. As we use a generic ini parser that is certainly not optimized for this use-case the former may be hard, but there are probably some cheap wins for the memory representation (e.g. we could intern ini keys, which for browscap are repeating). ------------------------------------------------------------------------ [2016-12-15 12:55:13] spam2 at rhsoft dot net in fact browscap is the reason for the whole bugreport, obviously with or without the memory overhead bug get_browser() which is used by the CMS of my co-developer which dropped from 100 to 50 requests per second with PHP7 luckily he is using it within function_exists() and add it to disabled_functions boosted his system from 100 requests per second with PHP5.6 to 320 with PHP 7.0.14 Requests per second: 319.89 [#/sec] (mean) Time per request: 62.522 [ms] (mean) Time per request: 3.126 [ms] (mean, across all concurrent requests) Transfer rate: 4644.90 [Kbytes/sec] received ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=73265 -- Edit this bug report at https://bugs.php.net/bug.php?id=73265&edit=1