Edit report at https://bugs.php.net/bug.php?id=64187&edit=1
ID: 64187
Updated by: mike@php.net
Reported by: nachms+php at gmail dot com
Summary: CGI/FastCGI truncates input to modulo 4GB
Status: Assigned
Type: Bug
Package: Streams related
Operating System: Linux
PHP Version: 5.6
Assigned To: mike
Block user comment: N
Private report: N
New Comment:
Sorry for the confusion. INI(enable_post_data_reading) only affects POSTs not PUTs. Anything
non-POST is *not* automatically read into the temp stream.
Previous Comments:
------------------------------------------------------------------------
[2014-07-17 11:19:13] nachms+php at gmail dot com
> Yes, the php://input stream is creating a temporary file in INI(upload_tmp_dir).
This cannot be disabled ATM.
Based on some superficial testing, it appears that only POST is obeying that variable, but not PUT,
but we'll do more testing to be certain.
It may also make sense to have two separate variables to control PUT and POST.
> If you set INI(enable_post_data_reading) to "Off", it will defer creation until you
> actually read it, i.e. it will write to the temp file as you read from the stream.
We need to do more testing here too, but after talking to our security team about this change to
PHP, they seem to think there's a good chance PHP 5.6 can be DoS'd by PUTing a huge amount
of data to any script, at least when automatic file creation is enabled.
Hopefully, we'll have more testing done later today.
------------------------------------------------------------------------
[2014-07-17 06:26:00] mike@php.net
Thank you for testing!
Yes, the php://input stream is creating a temporary file in INI(upload_tmp_dir).
This cannot be disabled ATM.
If you set INI(enable_post_data_reading) to "Off", it will defer creation until you
actually read it, i.e. it will write to the temp file as you read from the stream.
If INI(enable_post_data_reading) is "Off" and php://input is not used, the input data will
be discarded at the end of the request.
------------------------------------------------------------------------
[2014-07-16 21:20:05] nachms+php at gmail dot com
I played more with 5.6, finding it odd that we're being limited to ~1.7GB (which seems to have
no mathematical significance), whereas in PHP 5.3-5.5, we were seeing the size upload capped to
Content-Length%4GB.
Then I noticed that's how much free space is available in /tmp on our test server.
In PHP < 5.6, uploads via PUT were streamed, so the PHP script could save wherever it wanted to
as it was being uploaded, or stream the data into a database or remote server, or just run some
formulas on the data without storing any of raw PUT data.
In PHP 5.6, it seems the truncation bug has been corrected, but in the process the data is no longer
streamable, making it difficult to effectively deal with huge files, introducing issues into past
cases that worked (because they were <4GB).
Is there a way to tell PHP 5.6 to not buffer/save uploads via PUT? PUT, unlike POST doesn't
need PHP to parse the data to put into the various superglobals.
------------------------------------------------------------------------
[2014-07-16 19:57:59] mike@php.net
Please try with a source build.
I've no problems POSTing or PUTting files of size 10G.
â mike@smugmug:~/tmp$ l -hl 10G
-rw-r--r-- 1 mike users 9.8G 1. Jul 21:44 10G
â mike@smugmug:~/tmp$ cat /srv/http/put.php
<?php
$tmp = tempnam("/var/tmp", "put");
var_dump(file_put_contents($tmp, fopen("php://input","r")), filesize($tmp),
unlink($tmp));
â mike@smugmug:~/build/php-5.6-dbg$ ./sapi/cgi/php-cgi -b 0:9999 -d post_max_size=99G -d
upload_max_filesize=99G
â mike@smugmug:~/tmp$ curl -v --upload-file ~/tmp/10G http://localhost:88/put.php
* Hostname was NOT found in DNS cache
* Trying ::1...
* connect to ::1 port 88 failed: Connection refused
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 88 (#0)
> PUT /put.php HTTP/1.1
> User-Agent: curl/7.37.0
> Host: localhost:88
> Accept: */*
> Content-Length: 10485760000
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
* Server nginx/1.6.0 is not blacklisted
< Server: nginx/1.6.0
< Date: Wed, 16 Jul 2014 19:47:33 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Powered-By: PHP/5.6.0-dev
<
int(10485760000)
int(10485760000)
bool(true)
* Connection #0 to host localhost left intact
------------------------------------------------------------------------
[2014-07-15 22:06:37] nachms+php at gmail dot com
Yes, as I said in my previous post, it fails in php5-cgi_5.6.0~rc2+dfsg-3_amd64.deb, which as far as
I know is "5.6 or later".
Amount Read: 1374904320 (Should be Amount Read: 4296015872)
Was only able to send 1374978048 bytes. (Should not get this error)
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=64187
--
Edit this bug report at https://bugs.php.net/bug.php?id=64187&edit=1