PEAR-wide unit test suite

Hello all, I've been working quite some time on our pear-wide unit test suite and like to share some results. With version 3.1.4, phpunit got the ability to run .phpt tests. Things like code coverage and meaningful error messages have been missing, but I flooded Sebastian with patches - with all phpt patches applied to phpunit, our test suite basically runs every single .phpt test beside the normal phpunit AllTests.php files that other packages use. I ran the whole suite on my development system here at home and have the following results: Tests: 2432, Failures: 282, Errors: 1, Incomplete: 9, Skipped: 824. Among "normal" failures due to bugs, some packages using .phpt fail all tests. I analyzed some of them: 'Net_Monitor',//takes long time, but fails 'PEAR_Command_Packaging',//PEAR_Frontend not found 'PHP_Parser',//many undefined variables or strings used without quotes 'PHP_Parser_DocblockParser',//undefined function docblock_tokenize() 'Services_Delicious',//messages changed 'Services_Hatena',//access to undefined properties and array keys, symlink problem 'Text_Statistics',//invalid phpt section 'Text_Wiki',//simply fail because of code changes 'Validate',//some code failes because someone renamed a file, and normal failures 'Services_Amazon',//symlinking include problem 'XML_Parser',//symlinking include problem The code that creates symlinks and changes the include path is not really perfect; any package using XML_Parser currently fails because of include problems. I'll fix this when I find a good solution. The test log can be found on http://tmp.cweiske.de/pear-alltests.log.bz2 I needed to bzip it since it's uncompressed 5.7MiB. It becomes more and more apparent that unit tests are really useful - people begin to judge code by their test coverage, and with our suite, we will stay competetive. So everybody: Fix failing tests and write new test - nobody likes code that is tested but fails 50%. -- Regards/Mit freundlichen Grüßen Christian Weiske