Bug #72736 [Ver->Csd]: Slow performance when fetching large dataset with mysqli / PDO
Edit report at https://bugs.php.net/bug.php?id=72736&edit=1
ID: 72736
Updated by: dmitry@php.net
Reported by: petr dot hrabal at gmail dot com
Summary: Slow performance when fetching large dataset with
mysqli / PDO
-Status: Verified
+Status: Closed
Type: Bug
Package: *Database Functions
Operating System: Debian 8
PHP Version: 7.0.9, 7.1.0-dev
Assigned To: dmitry
Block user comment: N
Private report: N
New Comment:
The fix for this bug has been committed.
Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
For Windows:
http://windows.php.net/snapshots/
Thank you for the report, and for helping us make PHP better.
Previous Comments:
------------------------------------------------------------------------
[2016-10-24 08:56:01] dmitry@php.net
The attached patch should fix the problem in PHP-7.0 (without BC breaks).
------------------------------------------------------------------------
[2016-10-24 08:53:03] dmitry@php.net
The following patch has been added/updated:
Patch Name: mm-01.diff
Revision: 1477299181
URL: https://bugs.php.net/patch-display.php?bug=72736&patch=mm-01.diff&revision=1477299181
------------------------------------------------------------------------
[2016-10-21 23:21:15] nikic@php.net
Could have sworn I posted a more detailed analysis here... I no longer remember the specifics. The
issue is that we're doing a naive linear search on the chunk list until we find a chunk with
enough free pages. This ends up causing quadratic complexity. I checked how jemalloc handles this,
they use an RB-tree to find an appropriate chunk. We should probably do something similar as well --
though it might already help if we just move completely full chunks into a separate list,
there's really no point scanning them.
------------------------------------------------------------------------
[2016-10-21 18:31:54] jim dot hofer at gmail dot com
Just wanted to add this bug also affects Centos 6 & 7 and still present in PHP 7.0.12.
USE_ZEND_ALLOC=0 appears to be a good temporary alternative, but also leads to other weird behavior.
Like this bug https://bugs.php.net/bug.php?id=73370
------------------------------------------------------------------------
[2016-08-06 15:54:37] nikic@php.net
It's not really necessary to bring mysql into this. Just allocating a bunch of strings without
deallocating them in between is enough:
<?php
$n = 50000000;
$s = "aaaaaaaaaaaaaaaaaaaaa";
$a = [];
$t = microtime(true);
for ($i = 0; $i < $n; $i++) {
$s++;
$a[] = $s;
}
var_dump(microtime(true) - $t);
This gives me the following results with and without ZMM:
nikic@saturn:~/php-src-fast$ sapi/cli/php -d memory_limit=-1 t043.php
float(30.064018011093)
nikic@saturn:~/php-src-fast$ USE_ZEND_ALLOC=0 sapi/cli/php -d memory_limit=-1 t043.php
float(3.7853488922119)
Generally the ZMM is somewhat faster than the system allocator, not 20x slower.
Assigning to dmitry...
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=72736
--
Edit this bug report at https://bugs.php.net/bug.php?id=72736&edit=1
Thread (9 messages)