Bug #73412 [Ana->Nab]: url-wrappers in PHP7 terrible slow

Edit report at https://bugs.php.net/bug.php?id=73412&edit=1 ID: 73412 Updated by: rasmus@php.net Reported by: spam2 at rhsoft dot net Summary: url-wrappers in PHP7 terrible slow -Status: Analyzed +Status: Not a bug Type: Bug Package: Performance problem Operating System: Linux PHP Version: 7.0.12 Block user comment: N Private report: N New Comment: No, I test clean PHP binaries. No 3rd-party hacks. And like I showed with my example, there is no major performance difference between url wrappers and curl. Previous Comments: ------------------------------------------------------------------------ [2016-10-31 05:21:24] spam2 at rhsoft dot net Hi thanks for feedback - please dont miss the part under the line - mabye that's the reason we talk about 0.647s versus 36s for jsut 50 requests here while the libcurl variant takes 29s for FIVE THOUSAND requests and that's hardly a difference between GET/HEAD (BTW: why in the world does get_headers() make GET requests while http://php.net/manual/de/function.get-headers.php don't claim that as also common sense says GET by what it does and the fact that even with <Limit GET POST> you can't disable HEAD-Requests - point is it feels like doing 1 request per second which is not far from teh truth and i have a script doing around 100000 requests which was fine with 5.x and pure file_get_contents() ____________________________________ is your PHP build a proper hardened build? i saw as example when Fedora 23 switched the whole distribution https://fedoraproject.org/wiki/Changes/Harden_All_Packages grep go magnitued slower and now with F24 it's as fast as before and still a hardened build, so i guess there was some bug in the code which made it make that way [root@srv-rhsoft:~]$ /usr/bin/hardening-check /usr/bin/php /usr/bin/php: Position Independent Executable: yes Stack protected: yes Fortify Source functions: yes (some protected functions found) Read-only relocations: yes Immediate binding: yes if that's the reason may i suggest developers only test with hardened builds because no one should run these days any non-PIE/non-FULLRELRO code except broken one which don't build proper and can't yet replaced ------------------------------------------------------------------------ [2016-10-31 01:52:19] rasmus@php.net I see a couple of issues with your sample code here. The biggest one being that you are not comparing the same things at all. get_headers(), as per the docs, uses a GET request by default. But your curl example uses a HEAD request. Obviously a HEAD request is going to be much quicker. Your php get_headers() vs. wget timing isn't valid either since obviously php cli has way more startup work to do than wget. I just wrote a quick test and I don't see a drastic difference between curl and the url wrappers on Linux. I got this output: 4.2943890094757 5.9716069698334 So yes, get_headers() is a bit slower, but certainly not orders of magnitude like you are claiming. This is my test script: <?php function curl_headers($url) { $ch = curl_init($url); curl_setopt($ch, CURLOPT_NOBODY, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_HEADER, true); $r = curl_exec($ch); curl_close($ch); return explode("\n", trim($r)); } $i=0; $start = microtime(true); while($i++<1000) { $headers = curl_headers('http://htpc/index.html'); } echo (microtime(true) - $start)."\n"; $i=0; stream_context_set_default(['http' => ['method' => 'HEAD']]); $start = microtime(true); while($i++<1000) { $headers = get_headers('http://htpc/index.html'); } echo (microtime(true) - $start)."\n"; ------------------------------------------------------------------------ [2016-10-30 18:56:01] spam2 at rhsoft dot net just switch between curl and pure PHP and you have factor 57 by *just* 57 requests, that's ridiculous, especially since the curl stuff uses much more function calls and wiithin 36 secodns every sane software can make thousands of http requests _______________________________ $use_url = true; real 0m0.647s user 0m0.057s sys 0m0.041s _______________________________ $use_url = false; real 0m36.959s user 0m0.047s sys 0m0.051s _______________________________ #!/usr/bin/php <?php if(PHP_SAPI == 'cli') { $use_url = false; $dynamic = array(); $static = array(); $staticp = array(); $requests = 50; for($count=1; $count<=$requests; $count++) { $dynamic[] = get_repsonse_time('http://corecms/index.php'); $static[] = get_repsonse_time('http://corecms/static.htm'); $staticp[] = get_repsonse_time('http://corecms/static.php'); } $dynamic_sum = array_sum($dynamic); $static_sum = array_sum($static); $staticp_sum = array_sum($staticp); $dynamic_avg = $dynamic_sum / count($dynamic); $static_avg = $static_sum / count($static); $staticp_avg = $staticp_sum / count($staticp); echo 'COUNT: ' . count($dynamic) . "\n"; echo 'DYNAMIC CMS: ' . round($dynamic_avg) . " us\n"; echo 'STATIC HTM: ' . round($static_avg) . " us\n"; echo 'STATIC PHP: ' . round($staticp_avg) . " us\n"; echo 'FACTOR D/S: ' . round($dynamic_avg / $static_avg, 1) . "\n"; echo 'FACTOR P/S: ' . round($staticp_avg / $static_avg, 1) . "\n"; echo 'FACTOR D/P: ' . round($dynamic_avg / $staticp_avg, 1) . "\n"; } function get_repsonse_time($url) { global $use_url; switch($use_url) { case false: $response = get_headers($url, 1); $response = explode('D=', $response['X-Response-Time']); return (int)trim(str_replace(' us', '', $response[1])); break; case true: $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5); curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'HEAD'); $response = curl_exec($ch); curl_close($ch); $response = explode("\n", $response); foreach($response as $response_line) { if(strpos($response_line, 'X-Response-Time') !== false) { $response = explode('D=', $response_line); return (int)trim(str_replace(' us', '', $response[1])); break; } } break; } } ?> ------------------------------------------------------------------------ [2016-10-30 18:33:09] spam2 at rhsoft dot net changed subject ------------------------------------------------------------------------ [2016-10-30 18:31:27] spam2 at rhsoft dot net the same for file_get_contents, HEAD Requests or GET, the same with the curl extension is *magnitudes* faster and in fact this is a PHP/ regression because i just needed to rewrite some fuzzy-request scripts which would run the whole day long and originally had no problem with PHP5.x in short: url-wrappers in PHP7 are broken and unuseable in case you have a noticebale amount of requests triggering them $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_URL, $dst_url); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5); curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0'); curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'HEAD'); $buffer = curl_exec($ch); curl_close($ch); ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=73412 -- Edit this bug report at https://bugs.php.net/bug.php?id=73412&edit=1