#43610 [Fbk->Opn]: fastcgi socket dies on high concurrency
ID: 43610
User updated by: oliver at realtsp dot com
Reported By: oliver at realtsp dot com
-Status: Feedback
+Status: Open
Bug Type: CGI related
Operating System: FreeBSD 6.2
PHP Version: 5.2.5
New Comment:
We have tried with
http://snaps.php.net/php5.2-latest.tar.gz
Result is unchanged.
NOTE that the php workers and parent processes are still showing on ps
after the crash (same as before the crash). But lightly cannot get a
sensible response from them.
[root@muriwai /usr/ports/lang/php5]# pstree
...
|-+- 25262 www /usr/local/sbin/lighttpd -f
/usr/local/etc/lighttpd.conf
| \-+= 25263 www /usr/local/bin/php-cgi
| |--- 25264 www /usr/local/bin/php-cgi
| |--- 25265 www /usr/local/bin/php-cgi
| |--- 25266 www /usr/local/bin/php-cgi
| |--- 25267 www /usr/local/bin/php-cgi
| |--- 25268 www /usr/local/bin/php-cgi
| |--- 25269 www /usr/local/bin/php-cgi
| |--- 25270 www /usr/local/bin/php-cgi
| |--- 25271 www /usr/local/bin/php-cgi
| |--- 25272 www /usr/local/bin/php-cgi
| |--- 25273 www /usr/local/bin/php-cgi
| |--- 25274 www /usr/local/bin/php-cgi
| |--- 25275 www /usr/local/bin/php-cgi
| |--- 25276 www /usr/local/bin/php-cgi
| |--- 25277 www /usr/local/bin/php-cgi
| |--- 25278 www /usr/local/bin/php-cgi
| \--- 25279 www /usr/local/bin/php-cgi
....
Previous Comments:
------------------------------------------------------------------------
[2007-12-17 09:17:30] sniper@php.net
Please try using this CVS snapshot:
http://snaps.php.net/php5.2-latest.tar.gz
For Windows (zip):
http://snaps.php.net/win32/php5.2-win32-latest.zip
For Windows (installer):
http://snaps.php.net/win32/php5.2-win32-installer-latest.msi
------------------------------------------------------------------------
[2007-12-16 21:55:00] oliver at realtsp dot com
Description:
------------
Version information below.
When I load the server with siege, once the fastcgi-php parent process
reaches a load of ~200 concurrent requests the process appears to crash
and refuses to accept further connections, even after the load is
removed again. Only way to recover is to restart lighttpd and thereby
the fastcgi-php server (and its children).
Clearly a load of 200+ is probably overloaded not sustainable. However
a non-recoverable crash means that even when a temporary load drops away
(produced by some aggressive robot on our production setup for example)
the server remains unusable and returns 500 responses.
php version as below, but patched with this:
http://cvs.php.net/viewvc.cgi/php-src/main/SAPI.c?r1=1.202.2.7.2.15&r2=1.202.2.7.2.16&pathrev=PHP_5_2&diff_format=u
because of this bug:
http://bugs.php.net/bug.php?id=43295
That patch removes the errors on
/root/php-5.2.5/main/SAPI.c(445)
but the "overload crash remains".
[root@muriwai /usr/ports/lang/php5]# lighttpd -v
lighttpd-1.4.18 (ssl) - a light and fast webserver
Build-Date: Dec 5 2007 18:23:49
fastcgi.server = ( ".php" =>
( "localhost" =>
(
"socket" =>
"/var/run/lighttpd/php-fastcgi.socket",
"bin-path" =>
"/usr/local/bin/php-cgi",
"max-procs" => 1,
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "500"
),
"broken-scriptfilename" => "enable"
)
)
)
[root@muriwai /usr/ports/lang/php5]# php-cgi -v
PHP 5.2.5 (cgi-fcgi) (built: Dec 16 2007 20:47:09) (DEBUG)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
[root@muriwai /usr/ports/lang/php5]# php-cgi -m
[PHP Modules]
cgi-fcgi
date
libxml
Reflection
standard
[Zend Modules]
NOTE: no opcode cache or third party extensions
php.ini parsed is "none" (ie all defaults)
FreeBSD muriwai.realtsp.com 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Fri Jan
12 08:43:30 UTC 2007
root@portnoy.cse.buffalo.edu:/usr/obj/usr/src/sys/SMP amd64
Reproduce code:
---------------
a trivial script will do:
<?php
sleep(1);
phpinfo();
with .siegerc
#
# Default number of simulated concurrent users
# ex: concurrent = 25
#
concurrent = 250
Expected result:
----------------
php fastcgi parent process (and children) remaining stable. maybe an
"overloaded" or even temporary 500 type response would be acceptable.
but crashing and then being completely unreponsive is a bit of an
issue.
Actual result:
--------------
lighttpd.error.log reports this
2007-12-16 21:19:22: (mod_fastcgi.c.1731) connect failed: Connection
refused on unix:/var/run/lighttpd/php-fastcgi.socket-87058-0
2007-12-16 21:19:22: (mod_fastcgi.c.2885) backend died; we'll disable
it for 5 seconds and send the request to another backend instead:
reconnects: 0 load: 210
2007-12-16 21:19:22: (mod_fastcgi.c.3496) all handlers for /index.php
on .php are down.
i haven't managed a backtrace yet, because this is not that easy with a
fastcgi process, but i am working on it.
NOTE: despite what lighty says above..it does not restart the php
parent process. I am not sure why, but this is a separate issue i
believe.
------------------------------------------------------------------------
--
Edit this bug report at http://bugs.php.net/?id=43610&edit=1
Thread (8 messages)