Bug #73428 [Opn]: php-fpm hangs

Edit report at https://bugs.php.net/bug.php?id=73428&edit=1 ID: 73428 User updated by: chris at cretaforce dot gr Reported by: chris at cretaforce dot gr Summary: php-fpm hangs Status: Open Type: Bug Package: *General Issues Operating System: FreeBSD 10.3-STABLE PHP Version: 5.6.27 Block user comment: N Private report: N New Comment: gdb /usr/local/sbin/php-fpm 60638 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Attaching to program: /usr/local/sbin/php-fpm, process 60638 Reading symbols from /lib/libcrypt.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypt.so.5 Reading symbols from /usr/local/lib/libpcre.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libpcre.so.1 Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.5 Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done. [New Thread 802806400 (LWP 101470/php-fpm)] Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/local/lib/libxml2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libxml2.so.2 Reading symbols from /lib/libz.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.6 Reading symbols from /usr/lib/liblzma.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/liblzma.so.5 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/php/20131226/ioncube/ioncube_loader.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/ioncube/ioncube_loader.so Reading symbols from /usr/local/lib/php/20131226/opcache.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/opcache.so Reading symbols from /usr/local/lib/php/20131226/session.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/session.so Reading symbols from /usr/local/lib/php/20131226/bcmath.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/bcmath.so Reading symbols from /usr/local/lib/php/20131226/bz2.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/bz2.so Reading symbols from /usr/lib/libbz2.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/local/lib/php/20131226/calendar.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/calendar.so Reading symbols from /usr/local/lib/php/20131226/ctype.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/ctype.so Reading symbols from /usr/local/lib/php/20131226/curl.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/curl.so Reading symbols from /usr/local/lib/libcurl.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libcurl.so.4 Reading symbols from /usr/local/lib/libssl.so.9...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libssl.so.9 Reading symbols from /usr/local/lib/libcrypto.so.9...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libcrypto.so.9 Reading symbols from /usr/local/lib/php/20131226/dom.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/dom.so Reading symbols from /usr/local/lib/php/20131226/exif.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/exif.so Reading symbols from /usr/local/lib/php/20131226/fileinfo.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/fileinfo.so Reading symbols from /usr/local/lib/php/20131226/filter.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/filter.so Reading symbols from /usr/local/lib/php/20131226/ftp.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/ftp.so Reading symbols from /usr/local/lib/php/20131226/gd.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/gd.so Reading symbols from /usr/local/lib/libt1.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libt1.so.5 Reading symbols from /usr/local/lib/libpng16.so.16...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libpng16.so.16 Reading symbols from /usr/local/lib/libjpeg.so.8...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libjpeg.so.8 Reading symbols from /usr/local/lib/libfreetype.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libfreetype.so.6 Reading symbols from /usr/local/lib/php/20131226/gettext.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/gettext.so Reading symbols from /usr/local/lib/libintl.so.8...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libintl.so.8 Reading symbols from /usr/local/lib/php/20131226/hash.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/hash.so Reading symbols from /usr/local/lib/php/20131226/iconv.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/iconv.so Reading symbols from /usr/local/lib/libiconv.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libiconv.so.2 Reading symbols from /usr/local/lib/php/20131226/imap.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/imap.so Reading symbols from /usr/local/lib/libc-client4.so.9...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libc-client4.so.9 Reading symbols from /usr/lib/libpam.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libpam.so.5 Reading symbols from /usr/local/lib/php/20131226/intl.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/intl.so Reading symbols from /usr/local/lib/libicui18n.so.57...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libicui18n.so.57 Reading symbols from /usr/local/lib/libicuuc.so.57...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libicuuc.so.57 Reading symbols from /usr/local/lib/libicudata.so.57... warning: Lowest section in /usr/local/lib/libicudata.so.57 is .hash at 0000000000000120 (no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libicudata.so.57 Reading symbols from /usr/local/lib/libicuio.so.57...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libicuio.so.57 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /usr/lib/libc++.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libc++.so.1 Reading symbols from /lib/libcxxrt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libcxxrt.so.1 Reading symbols from /usr/local/lib/php/20131226/json.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/json.so Reading symbols from /usr/local/lib/php/20131226/mbstring.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/mbstring.so Reading symbols from /usr/local/lib/libonig.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libonig.so.1 Reading symbols from /usr/local/lib/php/20131226/mcrypt.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/mcrypt.so Reading symbols from /usr/local/lib/libmcrypt.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libmcrypt.so.4 Reading symbols from /usr/local/lib/libltdl.so.7...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libltdl.so.7 Reading symbols from /usr/local/lib/php/20131226/mysql.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/mysql.so Reading symbols from /usr/local/lib/php/20131226/mysqli.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/mysqli.so Reading symbols from /usr/local/lib/php/20131226/openssl.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/openssl.so Reading symbols from /usr/local/lib/php/20131226/pdo.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/pdo.so Reading symbols from /usr/local/lib/php/20131226/posix.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/posix.so Reading symbols from /usr/local/lib/php/20131226/simplexml.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/simplexml.so Reading symbols from /usr/local/lib/php/20131226/soap.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/soap.so Reading symbols from /usr/local/lib/php/20131226/sockets.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/sockets.so Reading symbols from /usr/local/lib/php/20131226/sqlite3.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/sqlite3.so Reading symbols from /usr/local/lib/libsqlite3.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libsqlite3.so.0 Reading symbols from /usr/local/lib/php/20131226/tidy.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/tidy.so Reading symbols from /usr/local/lib/libtidy-0.99.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libtidy-0.99.so.0 Reading symbols from /usr/local/lib/php/20131226/tokenizer.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/tokenizer.so Reading symbols from /usr/local/lib/php/20131226/xml.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/xml.so Reading symbols from /usr/local/lib/php/20131226/xmlwriter.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/xmlwriter.so Reading symbols from /usr/local/lib/php/20131226/zip.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/zip.so Reading symbols from /usr/local/lib/libzip.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libzip.so.4 Reading symbols from /usr/local/lib/php/20131226/zlib.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/zlib.so Reading symbols from /usr/local/lib/php/20131226/pdf.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/pdf.so Reading symbols from /usr/local/lib/libpdf.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libpdf.so.6 Reading symbols from /usr/local/lib/php/20131226/pdo_mysql.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/pdo_mysql.so Reading symbols from /usr/local/lib/php/20131226/pdo_sqlite.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/pdo_sqlite.so Reading symbols from /usr/local/lib/php/20131226/phar.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/phar.so Reading symbols from /usr/local/lib/php/20131226/xmlreader.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/xmlreader.so Reading symbols from /usr/local/lib/php/20131226/xmlrpc.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/xmlrpc.so Reading symbols from /usr/local/lib/php/20131226/xsl.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/php/20131226/xsl.so Reading symbols from /usr/local/lib/libexslt.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libexslt.so.0 Reading symbols from /usr/local/lib/libxslt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libxslt.so.1 Reading symbols from /usr/local/lib/libgcrypt.so.20...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libgcrypt.so.20 Reading symbols from /usr/local/lib/libgpg-error.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libgpg-error.so.0 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 [Switching to Thread 802806400 (LWP 101470/php-fpm)] 0x0000000801dda4ba in _fcntl () from /lib/libc.so.7 (gdb) (gdb) bt #0 0x0000000801dda4ba in _fcntl () from /lib/libc.so.7 #1 0x0000000801d88506 in __fcntl_compat () from /lib/libc.so.7 #2 0x00000008012b7a78 in pthread_suspend_all_np () from /lib/libthr.so.3 #3 0x0000000801d885c2 in fcntl () from /lib/libc.so.7 #4 0x0000000802c0989b in accel_shutdown () from /usr/local/lib/php/20131226/opcache.so #5 0x00000000005810c7 in zend_llist_apply () #6 0x000000000057d641 in init_executor () #7 0x000000000058bae3 in zend_activate () #8 0x00000000005319d3 in php_request_startup () #9 0x0000000000617531 in main () (gdb) Previous Comments: ------------------------------------------------------------------------ [2016-10-31 09:00:24] chris at cretaforce dot gr Description: ------------ There is a problem with PHP 5.6.27 (maybe in previous version too) or Freebsd 10.3-STABLE (Wed Oct 26 20:16:09 EEST 2016) and randomly php-fpm (with pm=dynamic) hangs. It doesn't process requests for whole server. top shows 1 or 2 processes that use 100% CPU. truss -p 32701 kill(18810,SIGKILL) ERR#3 'No such process' fcntl(3,F_GETLK,0x7fffffffc5c0) = 0 (0x0) kill(18810,SIGKILL) ERR#3 'No such process' fcntl(3,F_GETLK,0x7fffffffc5c0) = 0 (0x0) kill(18810,SIGKILL) ERR#3 'No such process' fcntl(3,F_GETLK,0x7fffffffc5c0) = 0 (0x0) kill(18810,SIGKILL) ERR#3 'No such process' fcntl(3,F_GETLK,0x7fffffffc5c0) = 0 (0x0) fstat -p 32701 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W gioannou php-fpm 32701 text /usr 162750 -rwxr-xr-x 4471845 r gioannou php-fpm 32701 wd / 321033 drwxr-xr-x 512 r gioannou php-fpm 32701 root / 2 drwxr-xr-x 512 r gioannou php-fpm 32701 0* local stream fffff8032888e4b0 gioannou php-fpm 32701 1 /dev 13 crw-rw-rw- null rw gioannou php-fpm 32701 2 /dev 13 crw-rw-rw- null rw gioannou php-fpm 32701 3 - 6084 -rw-rw-rw- 0 rw gioannou php-fpm 32701 4* local stream fffff80457b1aa50 procstat -kk 32701 PID TID COMM TDNAME KSTACK 32701 101779 php-fpm - <running> ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=73428&edit=1