Bug #66694 [Com]: php-fpm is crashing when using eventport

Edit report at https://bugs.php.net/bug.php?id=66694&edit=1 ID: 66694 Comment by: d dot v dot taylor at leedsmet dot ac dot uk Reported by: eugene at zhegan dot in Summary: php-fpm is crashing when using eventport Status: Open Type: Bug Package: FPM related Operating System: Solaris 11.1 PHP Version: 5.4.25 Block user comment: N Private report: N New Comment: I've hit this too on 5.5.11, but with a Bus Error rather than a SEGV (which will be a SPARC vs x86 thing). It particularly happens if I kill -QUIT FPM before it's handled *any* requests (and so hasn't received any events). This is the backtrace (using mdb): fpm_event_fire+0x1c(2e70687035002e, ffffffff, 1, ffffffff7fffd470, ffffffff7fffd460, ffffffffffffffff) fpm_event_port_wait+0x1ec(1014a3c50, 3e8, 2, 100efe858, 1768, 0) fpm_event_loop+0x3f0(0, 0, 0, 0, 0, 0) fpm_run+0xa8(ffffffff7ffff928, 0, a, 100effc80, ffffffff7df02a40, ffffffff7df02a40) main+0x12bc(1, ffffffffffffffff, ffffffff7ffffaa8, ffffffff7df001c0, 2880, 2800) _start+0x12c(0, 0, 0, 0, 0, 0) This is what I think is going on in my case (maybe yours too?) - long story short it's a memory reuse problem: fpm_event_port_init() malloc()s a chunk of memory to store events retrieved from the port, and doesn't initialise that memory at all. In my case it happened to have previously stored a pointer to a string used in processing php-fpm.conf. fpm_event_port_wait() uses port_getn() to look for events, with a parameter nget - the intended number of events - set to 1. In normal use that nget parameter is updated to the number of events actually retrieved, but if port_getn is interrupted by a signal and no events have been sent to the port, nget is left at 1 (see http://marc.info/?l=opensolaris-networking-discuss&m=125441701003618) The code below the port_getn() call sees nget > 0, assumes an event has been returned, and tries to process it. Because the memory assigned to store events hasn't been zeroed out, it gets through the brief sanity check and passes a junk pointer to fpm_event_fire - which then reads the random string (in my case ".php5\0.") as itself being a pointer (0x2e70687035002e), tries to dereference it and falls in a heap. I'm not sure if it's a safe assumption that if port_getn() was interrupted by a signal and we're only asking for one event, then there *definitely* won't have been any events retrieved - I suspect not. I also don't know if just zeroing out the memory in fpm_event_port_init() is enough, or if that only works because I'm only hitting it immediately after startup. Previous Comments: ------------------------------------------------------------------------ [2014-02-11 08:57:37] eugene at zhegan dot in Description: ------------ php-fpm is crashing when 'events.mechanism' is set to 'port'. It doesn't crash on first request, it can process some, however, the crash is imminent and my installation dumps cores like 1 per minute (this server is used by ~50 developers). When set to 'events.mechanism = /dev/poll' php-fpm stop to crash. Backtrace (they all do look similar): # /usr/local/solarisstudio12.3/bin/dbx /usr/local/php-5.4.25/sbin/php-fpm core_hyperion_php-fpm_0_%c_5046 For information about new features see `help changes' To remove this message, put `dbxenv suppress_startup_message 7.9' in your .dbxrc Reading php-fpm core file header read successfully Reading ld.so.1 Reading preloadable_libiconv.so Reading libcrypt.so.1 Reading libresolv.so.2 Reading librt.so.1 Reading libmysqlclient.so.18.0.0 Reading libsybdb.so.5.0.0 Reading libnsl.so.1 Reading libsocket.so.1 Reading libpthread.so.1 Reading libldap.so.5 Reading liblber-2.4.so.2.8.3 Reading libpng15.so.15.13.0 Reading libz.so.1 Reading libm.so.2 Reading libjpeg.so.62.0.0 Reading libcrypto.so.1.0.0 Reading libssl.so.1.0.0 Reading libxml2.so.2 Reading libfreetype.so.6 Reading libc.so.1 Reading libgcc_s.so.1 Reading libdl.so.1 Reading libnspr4.so Reading libmd.so.1 Reading libmp.so.2 Reading libsasl.so.1 Reading libplc4.so Reading libnss3.so Reading libnssutil3.so Reading libplds4.so Reading libssl3.so Reading libthread.so.1 Reading libbz2.so.1 Reading libcryptoutil.so.1 Reading ru_RU.UTF-8.so.3 Reading methods_unicode.so.3 Reading xdebug.so Reading bz2.so Reading curl.so Reading libcurl.so.4.2.0 Reading imap.so Reading libpam.so.1 Reading memcache.so Reading mcrypt.so Reading libmcrypt.so.4.4.8 Reading libltdl.so.7.3.0 Reading oci8.so Reading libclntsh.so.11.1 dbx: internal warning: "(null)"::srcname(InfoOrigin_CMDLINE): value "ntcontab.c" being set to "nt.c" Reading libnnz11.so Reading libkstat.so.1 Reading libgen.so.1 Reading libsched.so.1 Reading libaio.so.1 Reading libpool.so.1 Reading openssl.so Reading pinba.so Reading redis.so Reading xmlrpc.so Reading xsl.so Reading libexslt.so.0 Reading libxslt.so.1 Reading nss_files.so.1 Reading nss_ldap.so.1 Reading libsldap.so.1 Reading libscf.so.1 t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address) 0x00000000008f75d7: fpm_event_fire+0x0009: movq 0x0000000000000028(%rdi),%rax (dbx) where current thread: t@1 =>[1] fpm_event_fire(0x75723d63642c61, 0x26, 0x0, 0x1, 0x4, 0xffff80ffbfffd0b0), at 0x8f75d7 [2] fpm_event_port_wait(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x13b6, 0x7d141, 0xe18ed, 0x1, 0x0, 0xffff80ffbffffb3c, 0xffff80ffbffffc18, 0x1, 0xd55d0c, 0xffff80ffbfffd160, 0x8f2925, 0xe23c20, 0x0), at 0x902998 [3] fpm_event_loop(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f7927 [4] fpm_run(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f2925 [5] main(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8fa704 (dbx) Another one: # /usr/local/solarisstudio12.3/bin/dbx /usr/local/php-5.4.25/sbin/php-fpm core_hyperion_php-fpm_0_%c_4964 For information about new features see `help changes' To remove this message, put `dbxenv suppress_startup_message 7.9' in your .dbxrc Reading php-fpm core file header read successfully Reading ld.so.1 Reading preloadable_libiconv.so Reading libcrypt.so.1 Reading libresolv.so.2 Reading librt.so.1 Reading libmysqlclient.so.18.0.0 Reading libsybdb.so.5.0.0 Reading libnsl.so.1 Reading libsocket.so.1 Reading libpthread.so.1 Reading libldap.so.5 Reading liblber-2.4.so.2.8.3 Reading libpng15.so.15.13.0 Reading libz.so.1 Reading libm.so.2 Reading libjpeg.so.62.0.0 Reading libcrypto.so.1.0.0 Reading libssl.so.1.0.0 Reading libxml2.so.2 Reading libfreetype.so.6 Reading libc.so.1 Reading libgcc_s.so.1 Reading libdl.so.1 Reading libnspr4.so Reading libmd.so.1 Reading libmp.so.2 Reading libsasl.so.1 Reading libplc4.so Reading libnss3.so Reading libnssutil3.so Reading libplds4.so Reading libssl3.so Reading libthread.so.1 Reading libbz2.so.1 Reading libcryptoutil.so.1 Reading ru_RU.UTF-8.so.3 Reading methods_unicode.so.3 Reading xdebug.so Reading bz2.so Reading curl.so Reading libcurl.so.4.2.0 Reading imap.so Reading libpam.so.1 Reading memcache.so Reading mcrypt.so Reading libmcrypt.so.4.4.8 Reading libltdl.so.7.3.0 Reading oci8.so Reading libclntsh.so.11.1 dbx: internal warning: "(null)"::srcname(InfoOrigin_CMDLINE): value "ntcontab.c" being set to "nt.c" Reading libnnz11.so Reading libkstat.so.1 Reading libgen.so.1 Reading libsched.so.1 Reading libaio.so.1 Reading libpool.so.1 Reading openssl.so Reading pinba.so Reading redis.so Reading xmlrpc.so Reading xsl.so Reading libexslt.so.0 Reading libxslt.so.1 Reading nss_files.so.1 Reading nss_ldap.so.1 Reading libsldap.so.1 Reading libscf.so.1 t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address) 0x00000000008f75d7: fpm_event_fire+0x0009: movq 0x0000000000000028(%rdi),%rax (dbx) where current thread: t@1 =>[1] fpm_event_fire(0x75723d63642c61, 0x26, 0x0, 0x1, 0x4, 0xffff80ffbfffd0b0), at 0x8f75d7 [2] fpm_event_port_wait(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x1364, 0x7d0da, 0xb7ce3, 0x1, 0x0, 0xffff80ffbffffb3c, 0xffff80ffbffffc18, 0x1, 0xd55d0c, 0xffff80ffbfffd160, 0x8f2925, 0xe23c20, 0x0), at 0x902998 [3] fpm_event_loop(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f7927 [4] fpm_run(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8f2925 [5] main(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x8fa704 (dbx) ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=66694&edit=1