1) What good is a regression testing framework when the developers
clearly ignore it?
Example: the exit() behavior change breaking tests left everyone
without worrying.
I believe that exit() was behaving in a very non-Un*x way, and they fixed it, and it was bad of the regression tests to rely on the former (undocumented?) behaviour...
That's my impression...
2) What good is a regression testing framework when the tests themselves
are broken?
Example: any .phpt file that leaves the evaluation to quicktester.inc
is broken: or should I expect include("tests/quicktester.inc") work
when WC-root/tests is not in include_path? The php invoked by make
test gets "-c php.ini-dist" on the command line. The commandlines
created by run-tests.php do not include the -c switch, leaving the
door open to potentially bogus failures.
Bogus failures, if they are investigated and cleared are annoying, but not evil.
I wonder why even pretend regression tests are used?
They actually *ARE* being used now -- Or at least I think that's what is generating all the "Test results" traffic...
Now for something completely related. run-tests.php is IMO needlessly
big. I'll even call it bloated. Someone was clearly enjoying typing.
This makes it unnecesarily hard to read.
Submit a patch to fix it. :-)
--
Got Music? http://l-i-e.com/artists.htm