Re: run-tests.php

Hello Zoe, why is the line number important here? If you include the exact error message, that is including the function name red and green you would still get the error. This is actually an issue I see frequently. We have to put as much as possible but nothing that could change because of test layout or some weird settings. And in fact line numbers can change when a test has to be touched for whatever reason (for example adding ini settings using ini_set() or because of haveing to define a variable or adding a new method to a class or whatever). Thus filename and line numbers were catched be %s and %d but we did not catch the ' on line ' or actual test script name. marcus Friday, September 7, 2007, 10:38:11 AM, you wrote: > Hi there > I just fell into a small trap with php-runtests.php. This test case > shows it: > --TEST-- > Testing regex matching in run-tests.phpt > --FILE-- > <?php > echo "Warning: something wrong in function red at line 10\n"; > echo "Warning: something wrong in function green at line 13\n"; > echo "Write whatever you like, it will be swallowed at line 16\n"; ?>> > --EXPECTF-- > Warning: something wrong in function %s at line %d > Warning: something wrong in function %s at line %d > This is a silly test but it passes. To make it fail properly it's > necessary to put the line numbers in the expected output and not use > with %d. We've used %d this way in quite a few tests :-( and I tripped > over it an an error case test that has lots of similar lines of output. > %s is substituted by .+? in php-runtests.php and I think the regex > matching (preg_match) is working correctly. I've found one file system > test which is passing (and shouldn't be) because of this - I'll fix it > later today. > If I'm correct in thinking that php-runtest.php is working as designed > I think it might be a good idea to put something on the QA web pages > (I'm happy to write it) to suggest that recommeded practice is to use > exact line numbers in the expected output instead of %d. Now that I'm > writing this it seems almost too obvious to state - but it didn't seem > so obvious earlier in the year. > Comments? > Zoe Slattery > IBM Best regards, Marcus