Req #80956 [Opn->Csd]: Forward PHP streams as extra_fds to curl_multi_wait in curl_multi_select
Edit report at https://bugs.php.net/bug.php?id=80956&edit=1
ID: 80956
User updated by: asmqb7 at gmail dot com
Reported by: asmqb7 at gmail dot com
Summary: Forward PHP streams as extra_fds to curl_multi_wait
in curl_multi_select
-Status: Open
+Status: Closed
Type: Feature/Change Request
Package: cURL related
Operating System: Linux
PHP Version: 8.0.3
Block user comment: N
Private report: N
New Comment:
CLOSED INVALID - the idea in this bug remains interesting, unresolved and relevant, but I've
discovered the implementation approach suggested here is unnecessary and so boneheaded I'm
nuking this ticket from orbit and starting again.
Please see new bug #81075 which contains a rehash of the context and a *slightly* saner approach to
a solution :)
I also make some pragmatic suggestions regarding the fd_set issue.
Previous Comments:
------------------------------------------------------------------------
[2021-04-16 06:03:43] twosee@php.net
This can really improve the convenience of development and I am also interested to implement this.
Perhaps the reason why it has not been implemented before is that the implementation of a data
structure to represent pollfd/curl_waitfd (or use arrays to represent them) is a bit troublesome.
But, this may also add some workload to Swoole/Swow (extensions that replaced the implementation of
cURL).
In addition, it seems that the implementation of select in PHP made some patches on the Windows
platform to support non-socket handles.
And I took a quick look at the select() implementation in cURL[1], not only may it be different from
PHP's select implementation, but it may also use poll. I donât know what effect this will
have if we use cURL's select to replace PHP's.
[1] <https://github.com/curl/curl/blob/master/lib/select.c>
------------------------------------------------------------------------
[2021-04-15 14:07:50] asmqb7 at gmail dot com
Description:
------------
It's currently not possible to efficiently run a curl_multi request while simultaneously
selecting on other file descriptors. At first glance, this appears to be trivially easy to
completely fix. (I'd implement it myself but I'm still getting familiar with PHP's
internals.)
According to the cURL documentation at https://curl.se/libcurl/c/curl_multi_wait.html,
----
CURLMcode curl_multi_wait(CURLM *multi_handle,
struct curl_waitfd extra_fds[],
unsigned int extra_nfds,
int timeout_ms,
int *numfds);
(...Synopsis...)
The calling application may pass additional curl_waitfd structures which are similar to
poll(2)'s
pollfd structure to be waited on in the same call.
----
It would seem fairly straightforward to simply introduce a 3rd parameter to curl_multi_select():
curl_multi_select (CurlMultiHandle $multi_handle, float $timeout = 1.0, array $extra_fds = []) :
int
The additional streams would then be inserted into extra_fds.
** Main question: are there scenarios in PHP (core) where a Stream-like-object doesn't boil
down to a fd? **
FWIW, curl_multi_select() is currently not using extra_fds for anything else, so implementing this
shouldn't step on anything's toes. From https://github.com/php/php-src/blob/5b01c4863fe9e4bc2702b2bbf66d292d23001a18/ext/curl/multi.c#L186:
186 error = curl_multi_wait(mh->multi, /*extra_fds*/NULL, /*extra_nfds*/0,
(unsigned long) (timeout * 1000.0), &numfds);
My general use case for this is to be able to achieve proper blocking polling on a bunch of streams,
when one of those streams is a [set of] cURL requests, for example in the context of async or event
loop frameworks.
IMO such libraries can easily bear the load of intelligently handling the "stream_select() vs
curl_multi_select()?" question - this problem is endemic to cURL design idiosyncrasies, and
cURL's efficiency (and HTTP/3 support, and general batteries-included behavior, etc etc) make
it definitively worth it to plumb through the support described here so the PHP userspace can be
enabled to fully adapt to cURL's way of doing things.
In my small event loop framework, I've basically had to implement recurring ticks that
ultimately make stream_select() timeout every 100000us so I can then curl_multi_select($mh,
/*timeout*/0). This is not at all particularly efficient, hence this little feature request.
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=80956&edit=1
Thread (3 messages)