php/zend and apache segmentation faults

> Hi, > > I am new to the group here and would quickly like to understand if this is > the right place to share questions on php/zend ? Just to give an idea we at > info edge ltd run most of the web properties on LAMP stack and use various > versions of PHP(5.2 to latest 5.4.x) with Zend Engine . > > Evidently we get a lot of segmentation faults as logged by apache in error > logs and when we try to debug them they more or less point issues to zend . > Although I am not a programmer and probably do not understand the exact > cause of segfaults but I am willing to believe this is something to do with > PHP . we have eliminated all doubts on hardware/apache file system and so > forth . Here is a quick output of gdb backtrace on one of the cores that > apache wrote on a segfault . > > This GDB was configured as "i386-redhat-linux-gnu"...Using host > libthread_db library "/lib/tls/libthread_db.so.1". > > Core was generated by `/usr/local/apache/bin/httpd -k start'. > Program terminated with signal 11, Segmentation fault. > Reading symbols from /usr/lib/libz.so.1...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libssl.so.4...done. > Loaded symbols for /lib/libssl.so.4 > Reading symbols from /lib/libcrypto.so.4...done. > Loaded symbols for /lib/libcrypto.so.4 > Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. > Loaded symbols for /usr/lib/libgssapi_krb5.so.2 > Reading symbols from /usr/lib/libkrb5.so.3...done. > Loaded symbols for /usr/lib/libkrb5.so.3 > Reading symbols from /lib/libcom_err.so.2...done. > Loaded symbols for /lib/libcom_err.so.2 > Reading symbols from /usr/lib/libk5crypto.so.3...done. > Loaded symbols for /usr/lib/libk5crypto.so.3 > Reading symbols from /lib/libresolv.so.2...done. > Loaded symbols for /lib/libresolv.so.2 > Reading symbols from /usr/local/apache/lib/libaprutil-0.so.0...done. > Loaded symbols for /usr/local/apache/lib/libaprutil-0.so.0 > Reading symbols from /usr/lib/libgdbm.so.2...done. > Loaded symbols for /usr/lib/libgdbm.so.2 > Reading symbols from /usr/lib/tls/i686/libdb-4.2.so...done. > Loaded symbols for /usr/lib/tls/i686/libdb-4.2.so > Reading symbols from /usr/lib/libexpat.so.0...done. > Loaded symbols for /usr/lib/libexpat.so.0 > Reading symbols from /usr/local/apache/lib/libapr-0.so.0...done. > Loaded symbols for /usr/local/apache/lib/libapr-0.so.0 > Reading symbols from /lib/tls/librt.so.1...done. > Loaded symbols for /lib/tls/librt.so.1 > Reading symbols from /lib/tls/libm.so.6...done. > Loaded symbols for /lib/tls/libm.so.6 > Reading symbols from /lib/libcrypt.so.1...done. > Loaded symbols for /lib/libcrypt.so.1 > Reading symbols from /lib/libnsl.so.1...done. > Loaded symbols for /lib/libnsl.so.1 > Reading symbols from /lib/tls/libpthread.so.0...done. > Loaded symbols for /lib/tls/libpthread.so.0 > Reading symbols from /lib/libdl.so.2...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib/tls/libc.so.6...done. > Loaded symbols for /lib/tls/libc.so.6 > Reading symbols from /lib/ld-linux.so.2...done. > Loaded symbols for /lib/ld-linux.so.2 > Reading symbols from /lib/libnss_files.so.2...done. > Loaded symbols for /lib/libnss_files.so.2 > Reading symbols from /usr/local/apache/modules/libphp5.so...done. > Loaded symbols for /usr/local/apache/modules/libphp5.so > Reading symbols from > /usr/local/mysql/lib/mysql/libmysqlclient.so.15...done. > Loaded symbols for /usr/local/mysql/lib/mysql/libmysqlclient.so.15 > Reading symbols from /usr/lib/libpng12.so.0...done. > Loaded symbols for /usr/lib/libpng12.so.0 > Reading symbols from /usr/lib/libjpeg.so.62...done. > Loaded symbols for /usr/lib/libjpeg.so.62 > Reading symbols from /usr/lib/libcurl.so.3...done. > Loaded symbols for /usr/lib/libcurl.so.3 > Reading symbols from /usr/lib/libidn.so.11...done. > Loaded symbols for /usr/lib/libidn.so.11 > Reading symbols from /usr/lib/libxml2.so.2...done. > Loaded symbols for /usr/lib/libxml2.so.2 > Reading symbols from /usr/local/apache/modules/mod_expires.so...done. > Loaded symbols for /usr/local/apache/modules/mod_expires.so > Reading symbols from /usr/local/apache/modules/mod_headers.so...done. > Loaded symbols for /usr/local/apache/modules/mod_headers.so > Reading symbols from /usr/local/apache/modules/mod_rpaf-2.0.so...done. > Loaded symbols for /usr/local/apache/modules/mod_rpaf-2.0.so > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/xcache.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/xcache.so > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/pdo.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/pdo.so > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/pdo_mysql.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/pdo_mysql.so > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/memcache.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/memcache.so > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/wordmapping.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/wordmapping.so > Reading symbols from /usr/lib/libstdc++.so.6...done. > Loaded symbols for /usr/lib/libstdc++.so.6 > Reading symbols from /lib/libgcc_s.so.1...done. > Loaded symbols for /lib/libgcc_s.so.1 > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/oauth.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/oauth.so > Reading symbols from > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/mcrypt.so...done. > Loaded symbols for > /usr/local/php/lib/php/extensions/no-debug-non-zts-20060613/mcrypt.so > Reading symbols from /usr/local/lib/libmcrypt.so.4...done. > Loaded symbols for /usr/local/lib/libmcrypt.so.4 > Reading symbols from /usr/lib/libltdl.so.3...done. > Loaded symbols for /usr/lib/libltdl.so.3 > Reading symbols from /lib/libnss_dns.so.2...done. > Loaded symbols for /lib/libnss_dns.so.2 > #0 0x00574e4d in _zend_mm_free_int (heap=0x8c89600, p=Variable "p" is not > available. > ) at /opt/php-5.2.6/Zend/zend_alloc.c:832 > 832 if (UNEXPECTED(prev->next_free_block != mm_block) || > UNEXPECTED(next->prev_free_block != mm_block)) { > (gdb) > if (UNEXPECTED(prev->next_free_block != mm_block) || > UNEXPECTED(next->prev_free_block != mm_block)) { > > > > ###########analyse this one from bottom to top > > > #0 0x00574e4d in _zend_mm_free_int (heap=0x8c89600, p=Variable "p" is not > available. > ) at /opt/php-5.2.6/Zend/zend_alloc.c:832 > #1 0x00582ea1 in destroy_op_array (op_array=0xafbfbf04) at > /opt/php-5.2.6/Zend/zend_opcode.c:234 > #2 0x005954c4 in zend_hash_destroy (ht=0xafc7408c) at > /opt/php-5.2.6/Zend/zend_hash.c:526 > #3 0x00583162 in destroy_zend_class (pce=0x8e1fbec) at > /opt/php-5.2.6/Zend/zend_opcode.c:186 > #4 0x00595698 in zend_hash_apply_deleter (ht=0x8c89850, p=0x8e1fbe0) at > /opt/php-5.2.6/Zend/zend_hash.c:611 > #5 0x00595947 in zend_hash_reverse_apply (ht=0x8c89850, > apply_func=0x57f260 <clean_non_persistent_class>) > at /opt/php-5.2.6/Zend/zend_hash.c:760 > #6 0x0057f8fd in shutdown_executor () at > /opt/php-5.2.6/Zend/zend_execute_API.c:291 > #7 0x0058c0f8 in zend_deactivate () at /opt/php-5.2.6/Zend/zend.c:860 > #8 0x0055454a in php_request_shutdown (dummy=0x0) at > /opt/php-5.2.6/main/main.c:1486 > #9 0x006089de in php_handler (r=0x8df6230) at > /opt/php-5.2.6/sapi/apache2handler/sapi_apache2.c:469 > #10 0x0809b54e in ap_run_handler (r=0x8df6230) at config.c:152 > #11 0x0809b919 in ap_invoke_handler (r=0x8df6230) at config.c:364 > #12 0x0808468d in ap_process_request (r=0x8df6230) at http_request.c:249 > #13 0x080801d9 in ap_process_http_connection (c=0x8debff8) at > http_core.c:251 > #14 0x080a4ae6 in ap_run_process_connection (c=0x8debff8) at > connection.c:43 > #15 0x08099d29 in child_main (child_num_arg=Variable "child_num_arg" is > not available. > ) at prefork.c:610 > #16 0x08099f53 in make_child (s=Variable "s" is not available. > ) at prefork.c:704 > #17 0x0809a8b9 in ap_mpm_run (_pconf=0x8c2f0a8, plog=0x8c67188, > s=0x8c34618) at prefork.c:839 > #18 0x0809f887 in main (argc=3, argv=0xbfe03244) at main.c:618 > > (gdb) dump_bt executor_globals.current_execute_data > (gdb) bt > #0 0x00574e4d in _zend_mm_free_int (heap=0x8c89600, p=Variable "p" is not > available. > ) at /opt/php-5.2.6/Zend/zend_alloc.c:832 > #1 0x00582ea1 in destroy_op_array (op_array=0xafbfbf04) at > /opt/php-5.2.6/Zend/zend_opcode.c:234 > #2 0x005954c4 in zend_hash_destroy (ht=0xafc7408c) at > /opt/php-5.2.6/Zend/zend_hash.c:526 > #3 0x00583162 in destroy_zend_class (pce=0x8e1fbec) at > /opt/php-5.2.6/Zend/zend_opcode.c:186 > #4 0x00595698 in zend_hash_apply_deleter (ht=0x8c89850, p=0x8e1fbe0) at > /opt/php-5.2.6/Zend/zend_hash.c:611 > #5 0x00595947 in zend_hash_reverse_apply (ht=0x8c89850, > apply_func=0x57f260 <clean_non_persistent_class>) > at /opt/php-5.2.6/Zend/zend_hash.c:760 > #6 0x0057f8fd in shutdown_executor () at > /opt/php-5.2.6/Zend/zend_execute_API.c:291 > #7 0x0058c0f8 in zend_deactivate () at /opt/php-5.2.6/Zend/zend.c:860 > #8 0x0055454a in php_request_shutdown (dummy=0x0) at > /opt/php-5.2.6/main/main.c:1486 > #9 0x006089de in php_handler (r=0x8df6230) at > /opt/php-5.2.6/sapi/apache2handler/sapi_apache2.c:469 > #10 0x0809b54e in ap_run_handler (r=0x8df6230) at config.c:152 > #11 0x0809b919 in ap_invoke_handler (r=0x8df6230) at config.c:364 > #12 0x0808468d in ap_process_request (r=0x8df6230) at http_request.c:249 > #13 0x080801d9 in ap_process_http_connection (c=0x8debff8) at > http_core.c:251 > #14 0x080a4ae6 in ap_run_process_connection (c=0x8debff8) at > connection.c:43 > #15 0x08099d29 in child_main (child_num_arg=Variable "child_num_arg" is > not available. > ) at prefork.c:610 > #16 0x08099f53 in make_child (s=Variable "s" is not available. > ) at prefork.c:704 > #17 0x0809a8b9 in ap_mpm_run (_pconf=0x8c2f0a8, plog=0x8c67188, > s=0x8c34618) at prefork.c:839 > #18 0x0809f887 in main (argc=3, argv=0xbfe03244) at main.c:618 > > > (gdb) bt full > #0 0x00574e4d in _zend_mm_free_int (heap=0x8c89600, p=Variable "p" is not > available. > ) at /opt/php-5.2.6/Zend/zend_alloc.c:832 > prev = Variable "prev" is not available. > > (gdb) frame > #0 0x00574e4d in _zend_mm_free_int (heap=0x8c89600, p=Variable "p" is not > available. > ) at /opt/php-5.2.6/Zend/zend_alloc.c:832 > 832 if (UNEXPECTED(prev->next_free_block != mm_block) || > UNEXPECTED(next->prev_free_block != mm_block)) { > (gdb) frame 2 > #2 0x005954c4 in zend_hash_destroy (ht=0xafc7408c) at > /opt/php-5.2.6/Zend/zend_hash.c:526 > 526 ht->pDestructor(q->pData); > (gdb) > > (gdb) info locals > p = (Bucket *) 0xafbfbf98 > q = (Bucket *) 0xafbfbecc > (gdb) info args > ht = (HashTable *) 0xafc7408c > (gdb) > > (gdb) bt full > #0 0x00574e4d in _zend_mm_free_int (heap=0x8c89600, p=Variable "p" is not > available. > ) at /opt/php-5.2.6/Zend/zend_alloc.c:832 > prev = Variable "prev" is not available > > > > > # /opt/php-5.2.6/Zend/zend_alloc.c > > current_conn = ap_run_create_connection(ptrans, ap_server_conf, csd, > my_child_num, sbh, bucket_alloc); > if (current_conn) { > ap_process_connection(current_conn, csd); > > apr_signal(AP_SIG_GRACEFUL, SIG_IGN); > child_main(slot); > > make_child(ap_server_conf, free_slots[i]); > > > > > > #main.c > > if (ap_mpm_run(pconf, plog, server_conf)) > break; > > > #/opt/php-5.2.6/Zend/zend_hash.c > p = ht->pListHead; > while (p != NULL) { > q = p; > p = p->pListNext; > if (ht->pDestructor) { > ht->pDestructor(q->pData); > } > if (q->pData != &q->pDataPtr) { > pefree(q->pData, ht->persistent); > } > pefree(q, ht->persistent); > } > > > > > -- > *Tariq Wali.* > > -- *Tariq Wali.*