Bug #80830 [Nab->Opn]: Running out of local ports when using curl

Edit report at https://bugs.php.net/bug.php?id=80830&edit=1 ID: 80830 Updated by: cmb@php.net Reported by: external dot maik dot dieterich at bosch dot com Summary: Running out of local ports when using curl -Status: Not a bug +Status: Open Type: Bug Package: Network related Operating System: Windows Server 2019 PHP Version: 7.4.15 Assigned To: cmb Block user comment: N Private report: N New Comment: To clarify: you are not initializing multiple curl handles in a single request, but rather for different requests, and you are using the apache2handler SAPI? In this case there may indeed be a problem, since the SAPI intializes libcurl (curl_global_init()) at its startup and shuts it down (curl_global_cleanup()) at its shutdown, so libcurl might keep connections open, but may not reuse them. > It's causing localhost to localhost TIME_WAITs. Frankly, I have no idea why that happens. The curl extension is relatively thin wrapper around libcurl, so I wonder whether we set up or use libcurl inappropriately, or whether Apache or the SAPI interferes. Previous Comments: ------------------------------------------------------------------------ [2021-03-09 08:23:25] michael dot mckeever at bosch dot com Thank you @danack and @cmb for the quick response. Your suggestions with reusing the the curl handle is correct if you're using a cli application and you're in control of the environment. The problem described by Maik is that if you're using apache you can't reuse curl handles. @rtrtrtrtrt, your comment is unfortunately not helping in this topic ;-) The key question is, why is the mass usage of curl.exe not causing TIME_WAITs and the php libcurl functions causing TIME_WAITs. Furthermore the php functinos are not causing TIME_WAITs to the called server as one would expect, but it's causing TIME_WAITs for some kind of local redirects. It's causing localhost to localhost TIME_WAITs. ------------------------------------------------------------------------ [2021-03-04 20:07:01] rtrtrtrtrt at dfdfdfdf dot dfd it's just how TCP works https://superuser.com/questions/173535/what-are-close-wait-and-time-wait-states ------------------------------------------------------------------------ [2021-03-04 18:32:27] external dot maik dot dieterich at bosch dot com First thanks for the quick response. What is not clear for me, is why the connection I open to the external server can be closed immediately and why this local local port connections, which i do not have if i use curl from command line, are still open even if the external connection I open is closed. Because each connection is opened in a different thread i think I can not reuse them. ------------------------------------------------------------------------ [2021-03-04 17:19:18] cmb@php.net I think @danack is spot on. But instead of fiddling with these Windows settings, I suggest to re-use the same curl resource (object as of PHP 8.0.0), like described by Dan, to avoid calling curl_easy_cleanup() under the hood, because[1] | re-using handles is a key to good performance with libcurl Anyway, this is not a PHP issue. [1] <https://curl.se/libcurl/c/curl_easy_cleanup.html> ------------------------------------------------------------------------ [2021-03-04 15:51:58] danack@php.net This is a feature of Windows. It keeps network connections open for a long time by default. You could either work around this by "Adjust the MaxUserPort and TcpTimedWaitDelay settings" - https://docs.microsoft.com/en-us/biztalk/technical-guides/settings-that-can-be-modified-to-improve-network-performance Or you can re-use the same Curl object, which will almost certainly re-use the underlying network connection if possible, ``` $ch = curl_init($url); for($i = 0; $i < 5000; $i++) { curl_setopt($ch, CURLOPT_URL, $url); $response = curl_exec($ch); } curl_close($ch); ``` I'll leave it open, so that someone who is more sure that there is nothing to be done in PHP, can close it. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=80830 -- Edit this bug report at https://bugs.php.net/bug.php?id=80830&edit=1