Edit report at https://bugs.php.net/bug.php?id=55438&edit=1
ID: 55438
Comment by: phpnet at lostreality dot org
Reported by: xuefer at gmail dot com
Summary: race condition: curlwapper is not sending http
header randomly
Status: Assigned
Type: Bug
Package: cURL related
Operating System: gentoo
PHP Version: 5.3.6
Assigned To: pierrick
Block user comment: N
Private report: N
New Comment:
I have curl-7.15.5-15.el5 according to rpm -q, but I can only locate /usr/lib/libcurl.so.3.0.0 and
/usr/lib64/libcurl.so.3.0.0 on my machine I'm testing on (CentOS 5.8). The binary says:
/usr/bin/curl -V
curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Normally, I use the stock RPMs for PHP, but a recent project I was working on failed to run properly
on another machine, where the owners also use CentOS 5.8, but use CPanel/WHM instead of the CentOS
RPMs for PHP. CPanel uses the --with-curlwrappers option, where as the stock CentOS and RHEL RPMs
have never used that option on any of their builds. It took a lot of digging before I realized that
it was the --with-curlwrappers option that caused the scripts to fail on that machine while working
perfectly on mine.
To verify if the headers were actually sent, I used: tcpdump -i eth1 -Als0 host www.example.com
I had two PuTTY windows open, one with tcpdump, the other running the test script I mentioned before
with: ./php-5.4.9/sapi/cli/php ./test.php
It was pretty clear to me that the headers were never sent before the patch, and always sent after
the patch.
Previous Comments:
------------------------------------------------------------------------
[2012-12-19 05:50:16] pierrick@php.net
I tried to reproduce this bug but wasn't able to do it.
Could you give me more details on the libcurl version used by your PHP instance ?
And also, how do you make sure that the headers are not properly sent ?
------------------------------------------------------------------------
[2012-12-19 04:33:11] phpnet at lostreality dot org
I submitted a patch that moves the slist from a local variable in php_curl_stream_opener() into the
php_curl_stream struct. The headers are no longer cleared and freed at the end of
php_curl_stream_opener(). The code to free the slist is moved into php_curl_stream_close() instead.
I'm not sure if this is the best approach, but it clearly gives me a 100% success rate with
having headers get sent, where as I had a literal 0% success rate before (not sure if there is
really a race condition or not, just that the headers get cleared and the slist freed before they
get used.)
The test code I used was as follows (Actual cookie and URL redacted)
<?php
$opt = array('http' => array('method' => 'GET',
'header' => 'Cookie: foo=bar'));
$ctx = stream_context_create($opt);
$f = fopen('http://www.example.com/',
'r', false, $ctx);
fread($f, 1); //work-around curl-wrappers bug where meta_data doesn't exist until the stream is
read
$data = stream_get_meta_data($f);
fclose($f);
var_dump($data);
?>
I compiled PHP with the following flags (not that I think anything matters to this bug other than
--with-curlwrappers):
--enable-static --with-mcrypt --with-ldap --with-iconv --enable-mbstring --with-gd --enable-mbregex
--with-zlib --with-imap --enable-ftp --with-gettext --enable-sockets --with-mysql=/usr --enable-cgi
--with-imap-ssl --enable-sockets --with-pdo-mysql --with-openssl --with-kerberos --with-curl
--with-curlwrappers --with-tidy --with-pcre-regex --with-bz2 --enable-zip --with-libdir=/lib64
------------------------------------------------------------------------
[2012-12-18 19:29:56] phpnet at lostreality dot org
I think I am seeing this same problem too (On 5.3.10, but nothing has changed in the source in 5.4.9
either).
Can you explain how this is happening, or suggest a work-around? I was digging into the PHP source,
expecting that the --with-curlwrappers option was basically broken and incomplete. I was surprised
to find the line:
curl_easy_setopt(curlstream->curl, CURLOPT_HTTPHEADER, slist);
Because that code all seems to indicate that the headers should be sent, but no matter what I try,
nothing I put in headers ever appears in the actual request. I keep running tcpdump but I never see
the headers I put in http->header.
------------------------------------------------------------------------
[2011-08-17 11:45:54] xuefer at gmail dot com
sorry for the mismatch http url string. i was trying to remove some string for
privacy
------------------------------------------------------------------------
[2011-08-17 11:38:50] xuefer at gmail dot com
Description:
------------
background: php is configured with curl wrapper, which make file_get_contents
use curl. i haven't tested with calling curl functions directly
php unset curl header too soon before curl make the request
expected order: set header, build and send request. unset header
actual order 1: set header, build and send request. unset header (good)
actual order 2: set header, unset header, build and send request (bad)
"send request" comes after php "unset header"
curl behavior randomly, by sending request before or after php unset the header
Test script:
---------------
#!/usr/lib/php5.3/bin/php
<?php
for (;;) {
$username = 'test1';
$password = mt_rand(0, 99999) .
'....................................................................................';
$authUrl = "http://localhost/";
$context = stream_context_create(array(
'http' => array(
'header' => "Authorization: Basic "
. base64_encode("$username:$password")
)
));
$http_response_header = array();
$data = file_get_contents($authUrl, false, $context);
sleep(1);
}
?>
tcpdump -nilo dst port 80 -w- -s0
Expected result:
----------------
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6MTQzNjEuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6NTY3MDQuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6MTQzNjEuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6NTY3MDQuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
Actual result:
--------------
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6MTQzNjEuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
Authorization: Basic
dGVzdDE6NTY3MDQuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u
Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4=
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
GET / HTTP/1.1
User-Agent: PHP/5.3.6-pl0-gentoo
Host: localhost
Accept: */*
gdb --args /usr/lib/php5.3/bin/php.debug test.php
(gdb) l /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:367
360 slist = curl_slist_append(slist,
trimmed);
361 efree(trimmed);
362 p = php_strtok_r(NULL, "\r\n",
&token);
363 }
364 efree(copy_ctx_opt);
365 }
366 if (slist) {
367 curl_easy_setopt(curlstream->curl,
CURLOPT_HTTPHEADER, slist);
368 }
369 }
370 if (SUCCESS == php_stream_context_get_option(context,
"http", "method", &ctx_opt) && Z_TYPE_PP(ctx_opt) == IS_STRING) {
371 if (strcasecmp(Z_STRVAL_PP(ctx_opt), "get")) {
372 if (!strcasecmp(Z_STRVAL_PP(ctx_opt),
"head")) {
373 curl_easy_setopt(curlstream-
>curl, CURLOPT_NOBODY, 1);
374 } else {
(gdb) l /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:501
494 if (msg_found) {
495 goto exit_fail;
496 }
497 }
498
499 /* context headers are not needed anymore */
500 if (slist) {
501 curl_easy_setopt(curlstream->curl, CURLOPT_HTTPHEADER,
NULL);
502 curl_slist_free_all(slist);
503 }
504 return stream;
505
506 exit_fail:
507 php_stream_close(stream);
508 if (slist) {
(gdb) br 367
Breakpoint 1 at 0x1c59c9: file /usr/src/debug/dev-lang/php-5.3.6/sapis-
build/cli/ext/curl/streams.c, line 367.
(gdb) br 501
Breakpoint 2 at 0x1c61bc: file /usr/src/debug/dev-lang/php-5.3.6/sapis-
build/cli/ext/curl/streams.c, line 501.
(gdb) br Curl_send_plain
Breakpoint 4 at 0xb7698ca0: file sendf.c, line 279.
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/lib/php5.3/bin/php ./test
[Thread debugging using libthread_db enabled]
warning: Lowest section in /usr/lib/libicudata.so.48 is .hash at 000000f4
[New Thread 0xb55e76f0 (LWP 31857)]
[Switching to Thread 0xb55e76f0 (LWP 31857)]
Breakpoint 1, php_curl_stream_opener (wrapper=0x80aacd44, filename=0x80c45c24
"http://localhost/vbb/authapi.php",
mode=0x80a2a01b "rb", options=4, opened_path=0x0, context=0x80c47224)
at /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:367
367 curl_easy_setopt(curlstream->curl,
CURLOPT_HTTPHEADER, slist);
(gdb) c
Continuing.
Breakpoint 2, php_curl_stream_opener (wrapper=0x80aacd44, filename=0x80c45c24
"http://localhost/",
mode=0x80a2a01b "rb", options=4, opened_path=0x0, context=0x80c47224)
at /usr/src/debug/dev-lang/php-5.3.6/sapis-build/cli/ext/curl/streams.c:501
501 curl_easy_setopt(curlstream->curl, CURLOPT_HTTPHEADER,
NULL);
(gdb) c
Continuing.
Breakpoint 4, Curl_send_plain (conn=0x80cd0628, num=0, mem=0x80cd1120, len=82,
code=0xbff162f4) at sendf.c:279
279 {
(gdb) p ((char*)mem)[0]@len
$1 = "GET /vbb/authapi.php HTTP/1.1\r\nUser-Agent: PHP/5.3.6-pl0-gentoo\r\nHost:
localhost\r\nAccept: */*\r\n\r\n"
Authorization header is not there
===========================
to reproduce this bug, curl is built with ares and without threads
(in another case http request is not even send, when curl is built with threads
and without ares, was a curl issue i guess)
# equery uses net-misc/curl
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for net-misc/curl-7.21.4:
U I
+ + ares : Enabled c-ares dns support
+ + gnutls : Prefer gnutls over nss and openssl as the crypto engine
- - idn : Enable support for Internationalized Domain Names
+ + ipv6 : Adds support for IP version 6
+ + kerberos : Adds kerberos support
- - ldap : Adds LDAP support (Lightweight Directory Access Protocol)
- - libssh2 : Enabled SSH urls in curl using libssh2
- - nss : Prefer NSS over openssl as the crypto engine
+ + ssl : Enable crypto engine support (via openssl if USE='-gnutls -
nss')
- - static-libs : Build static libraries
- - test : Workaround to pull in packages needed to run with
FEATURES=test. Portage-2.1.2 handles this internally,
so don't set it in make.conf/package.use anymore
- - threads : Adds threads support for various packages. Usually pthreads
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=55438&edit=1