Bug #70984 [Ana]: Script extreme slow compared to 5.6, MAP_HUGETLB problem?

Edit report at https://bugs.php.net/bug.php?id=70984&edit=1 ID: 70984 Updated by: yohgaki@php.net Reported by: arjen at react dot com Summary: Script extreme slow compared to 5.6, MAP_HUGETLB problem? Status: Analyzed Type: Bug Package: Scripting Engine problem Operating System: Linux PHP Version: 7.0.0RC8 Block user comment: N Private report: N New Comment: @sjon Your guess seems right. I rebooted my system with 1 huge page and I only get slower PHP 7.0 execution only at the first time. I should have gotten strace output when my system was performing poorly. Anyway, PHP may handle mmap error more gracefully. Previous Comments: ------------------------------------------------------------------------ [2015-12-08 09:28:24] yohgaki@php.net Huge page setting affects more for 7.0. I observed extremely slow execution (60 secs or more) on occasion when HugePages_Total is 1. The reporter's system uses no huge page and experiences slow execution. My Fedora 22 seems performing better without huge page. It seems we are better to document huge page setting some where in the manual. ------------------------------------------------------------------------ [2015-12-08 09:20:44] sjon at hortensius dot net Adding vm.nr_hugepages will claim a few pages as huge; making the allocation work. @yohgaki: it will probably be the reboot that 'fixes' this for you, like I already posted. However, PHP should fail better when no hugepages can be allocated. Imo the allocator should store a HUGETLB failure for x number of runs / time instead of keep trying (which seems to have a significant impact on performance). Increasing nr_hugepages is a workaround; and potentially claims memory that can not be used by applications not using HUGETLB allocations. ------------------------------------------------------------------------ [2015-12-08 09:15:26] yohgaki@php.net It appears vm.nr_hugepages=0 helped also. PHP 7.0 debug build [yohgaki@dev PHP-7.0]$ time ./php-bin t.php real 0m0.839s user 0m0.812s sys 0m0.026s [yohgaki@dev PHP-7.0]$ time ./php-bin t.php real 0m0.847s user 0m0.819s sys 0m0.027s PHP 5.6 Fedora 22 rpm package [yohgaki@dev PHP-7.0]$ time php t.php real 0m1.166s user 0m0.805s sys 0m0.195s [yohgaki@dev PHP-7.0]$ time php t.php real 0m1.012s user 0m0.816s sys 0m0.205s ------------------------------------------------------------------------ [2015-12-08 09:03:10] yohgaki@php.net My system is Fedora 22 with 32GB memory. 'sysctl -a | grep huge' showed only 1 huge page. PHP 7 was about 100% or more slower than PHP 5.6 with the test script. I set vm.nr_hugepages=512 in /etc/sysctl.conf and rebooted the system as it seemed rebooting is required. PHP 7.0 (debug build) $ time ./php-bin t.php real 0m0.938s user 0m0.910s sys 0m0.027s PHP 5.6 (Fedora22) $ time php t.php real 0m1.099s user 0m0.882s sys 0m0.226s It appears vm.nr_hugepages=512 helped a lot. Thanks for the tip, Rasmus. vm.nr_hugepages=0 may help, but I didn't test this. (yet) ------------------------------------------------------------------------ [2015-11-29 17:01:25] sjon at hortensius dot net Not sure if it'll help; but here are the first few lines from perf-report: 33.41% php-7.0.0RC8 [kernel.vmlinux] [k] pageblock_pfn_to_page 31.77% php-7.0.0RC8 [kernel.vmlinux] [k] isolate_freepages_block 3.78% php-7.0.0RC8 [kernel.vmlinux] [k] get_pfnblock_flags_mask 2.96% php-7.0.0RC8 libc-2.22.so [.] __memcpy_avx_unaligned 2.51% php-7.0.0RC8 [kernel.vmlinux] [k] memcmp 2.23% php-7.0.0RC8 [kernel.vmlinux] [k] compaction_alloc valgrind-cachegrind tells me most instruction cost (56%) goes towards __memcpy_avx_unaligned, then 34% into unknown and the rest is < 4% each. Strace shows the same as reported. If you are unable to reproduce or need anything else, I'm happy to help! ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=70984 -- Edit this bug report at https://bugs.php.net/bug.php?id=70984&edit=1