Bug #52948 [Opn->Bgs]: Improper connection closing logic leads to TIME_WAIT sockets on server

Edit report at http://bugs.php.net/bug.php?id=52948&edit=1 ID: 52948 Updated by: cataphract@php.net Reported by: gilles dot rayrat at continuent dot com Summary: Improper connection closing logic leads to TIME_WAIT sockets on server -Status: Open +Status: Bogus Type: Bug Package: MySQLi related Operating System: Any PHP Version: 5.2.14 Block user comment: N New Comment: If you're not using mysqlnd, it's not under our control how the connection is closed. We just rely on the mysql library that's installed in the system; whatever socket is used, it's opaque to us. I have serious doubts that calling shutdown() would make any difference whatsoever. See http://lxr.free-electrons.com/source/net/ipv4/tcp.c#L1871 and compare with tcp_shutdown. In any case, I'm closing as bogus as this is not about mysql's C client library, not mysqlnd. Previous Comments: ------------------------------------------------------------------------ [2010-10-01 18:58:40] gilles dot rayrat at continuent dot com So I'm not using mysqlnd at all. This behavior is not predictable and probably depends on the link and on the velocity of both ends. The shutdown calls allow better control over time_wait tcp connection state, so why not introducing an option so that the user can choose the behavior he wants. Note that there are some threads where sys admins complain about these numerous time_wait sockets and they might appreciate the trick. For the record, here is the mysql (non i) code: <?php # Simple script to demonstrate the TIME_WAIT issue # Connects to the database, nothing more $host = "u2"; # Host where the Tungsten Connector is running $port = 3306; # Using the Tungsten Connector port $username = "tungsten"; $password = "secret"; $dbname = "test"; echo "Connecting to $host $username $password $dbname $port\n"; # Make the connection $connection = mysql_connect($host, $username, $password); mysql_close($connection); ?> ------------------------------------------------------------------------ [2010-10-01 18:12:22] cataphract@php.net You can check if you're using mysqlnd with phpinfo() or php -i, if in the CLI. Example: php -i | grep mysqlnd Client API library version => mysqlnd 5.0.7-dev - 091210 - $Revision: 300533 $ mysqlnd mysqlnd => enabled Version => mysqlnd 5.0.7-dev - 091210 - $Revision: 300533 $ If you're not using mysqlnd, then it's not our problem to fix. See: http://www.php.net/manual/en/mysqlnd.overview.php As to the shutdown implementation in Linux, it doesn't seem to do a whole lot with SHUT_RD (in terms of packets sent): http://lxr.free-electrons.com/source/net/ipv4/af_inet.c#L768 (note the how++ on line #776) http://lxr.free-electrons.com/source/net/ipv4/tcp.c#L1852 SHUT_RW would force a FIN, but my tests show a FIN is always being sent by the client. The only variable is if the FIN reaches the server before it sends its own FIN. If it does reach it before, then we have a TIME_WAIT on the client; otherwise we have a simultaneous termination and TIME_WAIT on both. There's no way we would be able to prevent this. Web servers typically wait a few seconds to allow the client to disconnect and avoid the TIME_WAIT, maybe the mysql server could do the same. In any case, it's out of our hands. ------------------------------------------------------------------------ [2010-10-01 17:26:22] gilles dot rayrat at continuent dot com Re-reading my previous comment, I realized that the 1st paragraph can be misunderstood. No offense, I just need you to tell me how to provide more accurate info Cheers, Gilles. ------------------------------------------------------------------------ [2010-10-01 17:18:55] gilles dot rayrat at continuent dot com I'm not sure what I'm using, I am not a php user. I run ubuntu 9.10 and got the package "php5-mysql". If you need further info, please advise on how to collect it. I also encountered the case of double time_wait, I guess that the behavior is not fully predictable, and depends on whom between server and client starts the connection closing first. Using, on client, shutdown(sock, SHUT_RD) before sending the quit command, then running shutdown(sock, SHUT_WR) will give more predictable (and, at least to me, appropriate) results ------------------------------------------------------------------------ [2010-10-01 03:10:15] cataphract@php.net I can't reproduce this. Are you using mysqlnd? With the mysql client (package for Ubuntu 10.4), I sometimes get a simultaneous shutdown (which means TIME_WAIT on both sides): (look only at the source, 192.168.1.102 is the client and 192.168.1.2 is the server; the varying destination is because of DNAT) 02:20:31.109223 IP (tos 0x8, ttl 64, id 30843, offset 0, flags [DF], proto TCP (6), length 57) 192.168.1.102.50375 > 79.168.249.157.3306: P 451:456(5) ack 5105 win 23584 <nop,nop,timestamp 549252121 211008142> 02:20:31.109535 IP (tos 0x8, ttl 64, id 30844, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.102.50375 > 79.168.249.157.3306: F, cksum 0xe3b8 (correct), 456:456(0) ack 5105 win 23584 <nop,nop,timestamp 549252121 211008142> 02:20:31.109766 IP (tos 0x0, ttl 128, id 7011, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.2.3306 > 192.168.1.1.50375: F, cksum 0xc83d (correct), 5105:5105(0) ack 456 win 64705 <nop,nop,timestamp 211008872 549252121> 02:20:31.110026 IP (tos 0x0, ttl 128, id 7012, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.2.3306 > 192.168.1.1.50375: ., cksum 0xc83c (correct), ack 457 win 64705 <nop,nop,timestamp 211008872 549252121> 02:20:31.112839 IP (tos 0x8, ttl 64, id 30845, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.102.50375 > 79.168.249.157.3306: ., cksum 0xe0dd (correct), ack 5106 win 23584 <nop,nop,timestamp 549252121 211008872> Other times I get the same I get with your example script, a client initiated shutdown, with TIME_WAIT only on the client): 02:24:51.798486 IP (tos 0x0, ttl 64, id 18397, offset 0, flags [DF], proto TCP (6), length 57) 192.168.1.102.55529 > 79.168.249.157.3306: P 70:75(5) ack 79 win 5840 <nop,nop,timestamp 549278188 211034639> 02:24:51.798790 IP (tos 0x0, ttl 64, id 18398, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.102.55529 > 79.168.249.157.3306: F, cksum 0x3dd4 (correct), 75:75(0) ack 79 win 5840 <nop,nop,timestamp 549278188 211034639> 02:24:51.800286 IP (tos 0x0, ttl 128, id 8297, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.2.3306 > 192.168.1.1.55529: ., cksum 0xdd38 (correct), ack 76 win 65086 <nop,nop,timestamp 211034940 549278188> 02:24:51.800583 IP (tos 0x0, ttl 128, id 8298, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.2.3306 > 192.168.1.1.55529: F, cksum 0xdd37 (correct), 79:79(0) ack 76 win 65086 <nop,nop,timestamp 211034940 549278188> 02:24:51.802275 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52) 192.168.1.102.55529 > 79.168.249.157.3306: ., cksum 0x3ca6 (correct), ack 80 win 5840 <nop,nop,timestamp 549278188 211034940> Looking at the source code for mysqlnd, it just calls the PHP stream closing function, which, on Linux, is just a call to close(). I'm by no means a socket expert, but from what I read, unless someone messed with the linger options, close() will send the FIN as soon as it's emptied the send buffer. I don't see how calling shutdown explicitly could possibly help. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/bug.php?id=52948 -- Edit this bug report at http://bugs.php.net/bug.php?id=52948&edit=1