Bug #75829 [Com]: cURL and file_get_contents does not work with URL containing special character

Edit report at https://bugs.php.net/bug.php?id=75829&edit=1 ID: 75829 Comment by: spam2 at rhsoft dot net Reported by: marco dot marsala at live dot it Summary: cURL and file_get_contents does not work with URL containing special character Status: Open Type: Bug Package: URL related Operating System: both Ubuntu and Windows PHP Version: 7.2.1 Block user comment: N Private report: N New Comment: the problem lis NOT in DOMDocument->loadHTMLFile() too, it is that you don't escape your input proper which is your job as programmer be it for SQL, URL's or whatever and the idea that this is a bug in a programming language is wrong from the start frankly you even refer to https://bugs.php.net/bug.php?id=48550 [2009-06-15 17:51 UTC] jani@php.net Well, browsers do a lot of thing automatically you need to do yourself when you do it yourself. This is obviously not any bug. (tested with curl on command line with same result..) Previous Comments: ------------------------------------------------------------------------ [2018-01-17 08:09:15] marco dot marsala at live dot it I agree with you that is a bad practice to have such characters in URL, but sometimes you have to interact with external websites that use such characters in their URLs (for example when writing a scraper). Btw, the problem is on DOMDocument->loadHTMLFile() too ------------------------------------------------------------------------ [2018-01-17 08:00:04] spam2 at rhsoft dot net https://en.wikipedia.org/wiki/Garbage_in,_garbage_out PHP is a programming language and not a artificial intelligence hence you are responsible as programmer to encode input as needed and have such chars in a URL/filename is bad practice anyways ------------------------------------------------------------------------ [2018-01-17 07:59:09] marco dot marsala at live dot it I don't think is a duplicate of #48550 but a much broader vision of the issue. ------------------------------------------------------------------------ [2018-01-17 07:52:57] marco dot marsala at live dot it Description: ------------ cURL and file_get_contents does not work with URL contanining special characters (for example: accented letters, quotes). The URL sent to the remote server is badly encoded; special characters should be encoded as urlencode() would, like any modern web browser does. Test script: --------------- $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "https://www.astegiudiziarie.it/vendita-asta-appartamento-genova-via-san-giovanni-d'acri-14-1360824"); echo curl_exec($ch); // got server error page echo file_get_contents("https://www.astegiudiziarie.it/vendita-asta-appartamento-genova-via-san-giovanni-d'acri-14-1360824"); // got server error page // my function to fix encoding function urlencode_parts($url) { $parts = parse_url($url); $parts['path'] = implode('/', array_map('urlencode', explode('/', $parts['path']))); $url = new \http\Url($parts); return $url->toString(); } $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, urlencode_parts("https://www.astegiudiziarie.it/vendita-asta-appartamento-genova-via-san-giovanni-d'acri-14-1360824")); echo curl_exec($ch); // now ok echo file_get_contents(urlencode_parts("https://www.astegiudiziarie.it/vendita-asta-appartamento-genova-via-san-giovanni-d'acri-14-1360824")); // now ok Expected result: ---------------- The script should download the same page you can see with a web browser. Actual result: -------------- The server replies with a server error page. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=75829&edit=1