Req #72392 [Opn->Wfx]: array_diff not working as expected due to unintuitive behavior in file()

Edit report at https://bugs.php.net/bug.php?id=72392&edit=1 ID: 72392 Updated by: requinix@php.net Reported by: sky at skyboydston dot com Summary: array_diff not working as expected due to unintuitive behavior in file() -Status: Open +Status: Wont fix Type: Feature/Change Request Package: Arrays related Operating System: Windows 8 PHP Version: 7.0.7 Block user comment: N Private report: N New Comment: The newline has been present for years and its presence is clearly noted with a Note in the Return Values section of the function's documentation. Removing the newline would break backwards compatibility for all the people who use it while expecting them; for example, using it to copy one file to another while making some content changes along the way (which I myself have done before). Those who don't want them are already using FILE_IGNORE_NEW_LINES - or perhaps r/trim if they weren't aware of the flag. One way or another some group of people would need to use a flag and others would not. However changing the behavior adds the BC break and makes it noticeably more awkward to write code targeting multiple versions of PHP. With that said, if you feel strongly about this then you should bring it up for discussion on the internals mailing list. Previous Comments: ------------------------------------------------------------------------ [2016-06-12 20:28:21] sky at skyboydston dot com Description: ------------ This is part of a mini-application that takes inputs of arrays of email addresses, then compares the lists to create a final list of only valid addresses. One text file contains all of the addresses while the other two lists contain addresses that are invalid, either because their users have unsubscribed or because emails to their addresses have been undeliverable. The problem is that the array_diff() function is not working as expected. You can see from the output (shown below) that the reason is related to the output of the file() function, which will take into account newlines in .txt files by default, making array values unequal if they happen to fall on the last line in one file but on a preceding line in another file. It's necessary to actively suppress the newlines by using the FILE_IGNORE_NEW_LINES flag on file(). My suggestion is that this behavior is unintuitive and that PHP would be more user friendly if the suppression of these newlines was the default for file(). Not setting it up like that seems to be creating a situation that's fairly difficult to debug. In order for a developer to see that this is the cause of their problem in array_diff(), it's necessary to add <pre></pre> tags so that the newlines are apparent, and even then, not glaringly so as would be best. In my case, I simply happened to insert the <pre></pre> tags because I was preparing the code for other's review here and on stackoverflow.com. I believe the following post was created by others having difficulties related to this issue: http://stackoverflow.com/questions/7348280/array-diff-not-working-as-expected-what-could-be-the-reason Contents of the three files: all.txt: one@one.com two@two.com three@three.net four@four.org five@five.com unsub.txt: four@four.org two@two.com bounced.txt: one@one.com Incorrect output: Array ( [0] => one@one.com [1] => two@two.com [2] => three@three.net [3] => four@four.org [4] => five@five.com ) Array ( [0] => four@four.org [1] => two@two.com ) Array ( [0] => one@one.com ) one@one.com two@two.com three@three.net five@five.com 7.0.6 Correct output: Array ( [0] => one@one.com [1] => two@two.com [2] => three@three.net [3] => four@four.org [4] => five@five.com ) Array ( [0] => four@four.org [1] => two@two.com ) Array ( [0] => one@one.com ) three@three.net five@five.com 7.0.6 Test script: --------------- $all = array(); // Initialize the arrays we'll need $unsub = array(); $bounced = array(); $valid = array(); // Populate them and display their contents to make sure we're getting what we want. $all = file('all.txt', FILE_IGNORE_NEW_LINES); // <-- This flag can be removed to show the code which does not behave as expected. Make sure to remove it for all three files. echo '<pre>'; print_r($all); echo '</pre><br>'; $unsub = file('unsub.txt', FILE_IGNORE_NEW_LINES); echo '<pre>'; print_r($unsub); echo '</pre><br>'; $bounced = file('bounced.txt', FILE_IGNORE_NEW_LINES); echo '<pre>'; print_r($bounced); echo '</pre><br>'; $valid = array_diff($all, $unsub, $bounced); // Here's where we originally seemed to be having a problem foreach ($valid as $value) { print($value . '<br>'); } echo (PHP_VERSION); // This behavior has also been tested and is the same in version 5.6.19 ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=72392&edit=1