Bug #73807 [Ana]: Performance problem with processing post request over 2000000 chars

Edit report at https://bugs.php.net/bug.php?id=73807&edit=1 ID: 73807 Updated by: nikic@php.net Reported by: pparadowski at media4u dot pl Summary: Performance problem with processing post request over 2000000 chars Status: Analyzed Type: Bug Package: Performance problem Operating System: FreeBSD 9.4-11.0 PHP Version: 5.6.29, 7.0,7.1 Block user comment: N Private report: N New Comment: Just tried this on Ubuntu. I measured using a stop-watch (REQUEST_TIME_FLOAT does not seem to give a useful value in cli-server), so rough numbers are: 8M: ~0s 16M: ~2s 32M: ~7s 64M: ~25s Clearly the increase isn't linear, so this isn't just a FreeBSD problem. It might be that FreeBSD suffers more because it uses a smaller BUFSIZ. Previous Comments: ------------------------------------------------------------------------ [2017-02-02 13:29:33] nikic@php.net @pstef: Thanks! That was just a wild guess, based on usage of smart_str API, which was previously reported to be slow on FreeBSD due to slow realloc(). (Thinking again that can't be, as it's used with non-persistent strings and as such uses our own allocator anyway.) I just took a look at the php_std_post_handler() code: https://github.com/php/php-src/blob/7cba31535cbf24c0b8a24ae094afd9ed670435b0/main/php_variables.c#L339 I think the problem is that, in the case where POST form-urlencoded key-value pairs are significantly larger than the stream buffer size, the current implementation will keep appending to the buffer and rescan it in its entirety every time. This results in quadratic time complexity. (It does not rescan parts where it already detected a complete key-value pair, only if there's an unfinished one.) This is probably caused by this commit: https://github.com/php/php-src/commit/2438490addfbfba51e12246a74588b2382caa08a#diff-28ccb3aa37e01a68f5510ad6de4ab738L231 Looks like this changed the code from buffering everything upfront to using a stream. As such, this issue could not occur previously. What leaves me stumped here is why this issue would only occur on FreeBSD, rather than on all platforms. ------------------------------------------------------------------------ [2017-02-02 11:50:37] pstef at freebsd dot org This has nothing to do with realloc(). php_std_post_handler() spends 92% in memchr() and 7% in memmove(). ------------------------------------------------------------------------ [2017-01-31 10:43:01] nikic@php.net Another cause might be FreeBSD's *extremely* slow realloc() implementation. I remember this causing performance issues in other places. ------------------------------------------------------------------------ [2017-01-31 07:12:09] rasmus@php.net Ok, so it isn't accf_http doing it. It still feels like something along those lines. Like a socket buffer being too small and the resulting context switching causing it to slow down. Perhaps try playing around with pmcstat or dtrace and see if you can get a picture of what one of these slow requests is doing compared to a smaller fast one. ------------------------------------------------------------------------ [2017-01-31 07:00:22] pparadowski at media4u dot pl While testing accf_http module is disabled ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=73807 -- Edit this bug report at https://bugs.php.net/bug.php?id=73807&edit=1