Bug #78485 [Com]: Random "couldn't resolve host" failures in curl_multi

Edit report at https://bugs.php.net/bug.php?id=78485&edit=1 ID: 78485 Comment by: sinus at sinpi dot net Reported by: sinus at sinpi dot net Summary: Random "couldn't resolve host" failures in curl_multi Status: Open Type: Bug Package: HTTP related Operating System: CloudLinux PHP Version: 7.2.22 Block user comment: N Private report: N New Comment: The issue can be seen at http://djab.eu/isuvu/bug.php with source viewable at http://djab.eu/isuvu/bug.php.txt . Previous Comments: ------------------------------------------------------------------------ [2019-09-02 20:35:04] nikic@php.net Is there any publicly accessible host for which this might be reproduced? ------------------------------------------------------------------------ [2019-09-02 20:33:25] sinus at sinpi dot net I finally managed to make a workaround, the hard way: by stashing the failed handles and starting a whole new curl_multi_init session with just the failures. The original problem remains, though: why would n+1 handles immediately fail to resolve. ------------------------------------------------------------------------ [2019-09-02 19:39:20] sinus at sinpi dot net Trying another workaround, I built another retry mechanism, storing the failed downloads for later and only re-queuing them once the "good" downloads complete (getting a result==0 in curl_multi_info_read). However, it would seem that curl_multi does not, in fact, like having its list modified when it's running: my newly created curl_init handles, added anew, simply got ignored: curl_multi_exec's $still_running return silently dropped by one on the next round, as if it merely removed the new handle from its list. So another question arises - CAN curl_multi actually handle additions or removals of its child handles while in a curl_multi_exec session..? ------------------------------------------------------------------------ [2019-09-02 17:55:02] sinus at sinpi dot net To try and work around the issue, I wrote a retry mechanism - if a CURLE_COULDNT_RESOLVE_HOST message is detected, I curl_multi_remove_handle the handle, curl_close it, make a new curl_init and curl_multi_add_handle again, and return to calling curl_multi_exec. The docs don't state whether adding or removing handles during a multi session is permitted, so why not give it a try. The result, however, was exactly the same. All of the newly re-added child handles immediately got CURLE_COULDNT_RESOLVE_HOST results. ------------------------------------------------------------------------ [2019-09-02 17:02:32] sinus at sinpi dot net Description: ------------ I'm experiencing a pretty random failure of curl_multi to grab some of the handles added to it. I've already reported it to devs of libcurl, but they told me to toss this hot potato back here, as apparently curl_multi_exec is php7-curl's wrapper around libcurl's curl_multi_perform and any faults must lie in there. So, the matter at hand. I create 12 handles for http:// URLs, all using the same target machine, just different files. I add them to curl_multi. And as soon as I curl_multi_exec, randomly sometimes they all work, and sometimes only exactly first 8 get fetched, others immediately get a 6=="couldn't resolve host name" message queued in curl_multi_info_read. It's always the first handles that succeed, and the final handles that fail - n first handles work, n+1 till the end fail to resolve, with n sometimes being all 12, sometimes 8, a few times 4, but I also caught it on 11. This is in cURL 7.62 under PHP 7.2.21. Test script: --------------- $cm = curl_multi_init(); for ($i=1;i<=12;i++) { $handles[$i] = curl_init ("http://the.same.host.com/file" . $i); curl_multi_add_handle ( $cm, $handles[$i]); } $status = curl_multi_exec ($cm, $still_running); // $status gets 0, $still_running gets 8. $info = curl_multi_info_read ($cm, $queue); // $info['result']==6 "couldn't resolve host" for handle 9, $queue==3 as there are also failures queued for handles 10, 11 and 12 Expected result: ---------------- All 12 URLs retrieved, naturally. Actual result: -------------- Usually only 8 retrieved, or 12 as intended, or 4, or other numbers. It's always the last handles that fail. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=78485&edit=1