Re: 5.4 again

On May 11, 2011, at 3:56 PM, Rasmus Lerdorf wrote: > On 05/11/2011 02:10 PM, Philip Olson wrote: > >> So, that's the concern there. But if the alpha is simply a trick to convince people to >> test out a specific PHP 5.4 snapshot, and feel 5.4 is real, then do it. ;) > > There are still quite a few test failures in trunk. Some of them are also in the 5_3 branch. In > some cases the tests are simply bad. In a few the test case contains binary data that got mangled in > the move to Subversion. It would be nice if just 1 in 10 people reading the list here would grab > both trunk and 5_3 and run "make test" in each tree and then fix at least 1 test each. We > would have no test failures by the end of the day other than a few tricky ones. If an alpha release > will encourage this, great. If we could get people to just do it on their own without the alpha, > even better. > > And yes, I know the tests take forever to run. Get yourself a fast machine with an SSD, and > remember you can run partial tests using: > > make test TESTS=ext/hash > > for example to just run the tests for the hash extension. > > Also, when a test fails, cd into the ext/hash/tests directory and you will see .out, .exp, > .diff and .php files for the failed test. That is, the output, the expected output, the diff between > them and the php script itself extracted from the .phpt file containing the failed test case. > > And if you can't figure out how to fix a test, post the details here. I'd love to > point some of the obvious talents and energy of this list towards the code. If you don't have > an svn account for committing your fixed test, go to > http://www.php.net/svn-php.php and fill in the little form at the > bottom there and put in the test that you fixed and a 1-liner about how you fixed it and I will set > you up with an account right away. Info on how to check out the code from svn is here: > https://wiki.php.net/vcs/svnfaq This is a great challenge. Long ago I decided that failed tests are expected in PHP, because out of the ~100 or so times I've run it these past n years, it has always (afair) failed. Changing this fact by having it pass would change peoples (at least my) perception that running 'make test' is a waste of time, as is (was) looking into why they failed. For example, just now trunk gives me 55 failed tests and 17 expected fails with a vanilla build. We once had a matrix showing test results per setup (OS, phpversion, per configure switches) but it was someones pet project and the code has long since been lost (he looked years ago). Maybe such a beast would be useful. Also, how about we change[1] it so php.qa.reports[2] stops receiving 5_2 results, and always receives snapshot results? For example, snapshot test results (5_3 and trunk) are configured to _not_ go there today. Regards, Philip [1] http://svn.php.net/viewvc/web/qa/trunk/include/release-qa.php [2] http://news.php.net/php.qa.reports