Re: run-tests.php results

(for further replies, perhaps should we move to php-qa@ ?)

When I launch run-tests.php, I get pretty many failed tests. Is it normal

for a release to have some 35% of tests to fail or do we have a lot of bugs or do we have buggy tests?

The PEAR tests should all pass, the other tests have a few unresolved issues still. Could you post your "make test" output here?

Here's "make test" results on 3 different platforms (Linux RedHat 6.2 i386, OpenBSD2.6 i386 and NetBSD1.4.2 i386) with the same ./configure line: ./configure --enable-discard-path --with-mysql --with-dbase --enable-yp --enable-ftp --enable-trans-sid --enable-track-vars --enable-inline-optimization Linux and OpenBSD output the same errors: TEST RESULT SUMMARY ============================= Number of tests: 100

Tests skipped:      18 ( 18%)
Tests failed:        8 (9.8%)
Tests passed:       74 ( 90%)

[...] Running tests in /home/cahagn_o/php/php-4.0.4RC5/pear/DB/tests ==============================================================

DB_Error/DB_Warning test (db_error.phpt)                             ... failed
DB configurable error handler test (db_error2.phpt)                  ... failed
DB::factory test                                                     ... passed
DB::parseDSN test                                                    ... passed

Running tests in /home/cahagn_o/php/php-4.0.4RC5/pear/DB/tests/mysql ==================================================================== (all skipped)%) Running tests in /home/cahagn_o/php/php-4.0.4RC5/pear/XML/tests =============================================================== (all skipped)%) Running tests in /home/cahagn_o/php/php-4.0.4RC5/pear/tests ===========================================================

PEAR constructor/destructor test                                     ... passed
PEAR_Error test (pear_error.phpt)                                    ... failed
PEAR_Error in die mode                                               ... passed

Running tests in /home/cahagn_o/php/php-4.0.4RC5/tests ======================================================

dirname test (dirname.phpt)                                          ... failed

Running tests in /home/cahagn_o/php/php-4.0.4RC5/tests/basic ============================================================

Trivial "Hello World" test                                           ... passed
Simple POST Method test (002.phpt)                                   ... failed
GET and POST Method combined (003.phpt)                              ... failed
Two variables in POST data (004.phpt)                                ... failed
Three variables in POST data (005.phpt)                              ... failed
Add 3 variables together and print result                            ... passed
Multiply 3 variables and print result                                ... passed
Divide 3 variables and print result                                  ... passed
Subtract 3 variables and print result                                ... passed
Testing | and & operators                                            ... passed
Testing $argc and $argv handling                                     ... passed

[...] NetBSD even has more errors (btw, the termcaps seem to be less friendly :) TEST RESULT SUMMARY ============================= Number of tests: 100

Tests skipped:      18 ( 18%)
Tests failed:       12 ( 15%)
Tests passed:       70 ( 85%)

[...] ESC[1mRunning tests in /nfs/s5/space/users/guest/www/compil/php-4.0.4RC5/pear/DB/testsESC[m ================================================================================ ESC[1mDB_Error/DB_Warning test (db_error.phpt) ... failedESC[m ESC[1mDB configurable error handler test (db_error2.phpt) ... failedESC[m ESC[1mDB::factory test (db_factory.phpt) ... failedESC[m ESC[1mDB::parseDSN test (db_parsedsn.phpt) ... failedESC[m ESC[1mRunning tests in /nfs/s5/space/users/guest/www/compil/php-4.0.4RC5/pear/DB/tests/mysqlESC[m ====================================================================================== (all skipped) ESC[1mRunning tests in /nfs/s5/space/users/guest/www/compil/php-4.0.4RC5/pear/XML/testsESC[m ================================================================================= (all skipped) ESC[1mRunning tests in /nfs/s5/space/users/guest/www/compil/php-4.0.4RC5/pear/testsESC[m ============================================================================= ESC[1mPEAR constructor/destructor test (pear1.phpt) ... failedESC[m ESC[1mPEAR_Error test (pear_error.phpt) ... failedESC[m ESC[1mPEAR_Error in die mode (pear_error2.phpt) ... failedESC[m ESC[1mRunning tests in /nfs/s5/space/users/guest/www/compil/php-4.0.4RC5/testsESC[m ======================================================================== ESC[1mdirname test (dirname.phpt) ... failedESC[m ESC[1mRunning tests in /nfs/s5/space/users/guest/www/compil/php-4.0.4RC5/tests/basicESC[m ==============================================================================

Trivial "Hello World" test                                           ... passed
ESC[1mSimple POST Method test (002.phpt)    ... failedESC[m
ESC[1mGET and POST Method combined (003.phpt)    ... failedESC[m
ESC[1mTwo variables in POST data (004.phpt)    ... failedESC[m
ESC[1mThree variables in POST data (005.phpt)    ... failedESC[m
Add 3 variables together and print result                            ... passed
Multiply 3 variables and print result                                ... passed
Divide 3 variables and print result                                  ... passed
Subtract 3 variables and print result                                ... passed
Testing | and & operators                                            ... passed
Testing $argc and $argv handling                                     ... passed

-- Olivier.Cahagne@epita.fr