Re: New run-tests.php feature

Hi, On Thu, 2011-09-01 at 13:35 +0200, Hannes Magnusson wrote: > Throw qa and internals@ into the loop to. > > I'd also like to move the XFAIL section (printed out in the end) above > the FAIL section. > We have bucketloads of xfailed tests, I actually have to scroll up to > see the failed tests which is really annoying and silly. Good thing to do! johannes > Anyone have strong opinion on the printout order? > > -Hannes > > > ---------- Forwarded message ---------- > From: Hannes Magnusson <bjori@php.net> > Date: Thu, Sep 1, 2011 at 13:32 > Subject: Re: [PHP-CVS] svn: /php/php-src/ > branches/PHP_5_3/run-tests.php branches/PHP_5_4/run-tests.php > trunk/run-tests.php > To: Pierre Joye <pierre.php@gmail.com> > Cc: php-cvs@lists.php.net > > > I did want to use -v at first, but that has a completely different > meaning already. > Its really only useful when looking at what is going on in a single > testcase to debug run-tests :] > > -Hannes > > On Thu, Sep 1, 2011 at 13:26, Pierre Joye <pierre.php@gmail.com> wrote: > > ah right, I miss this part of the patch. I would suggest -v instead > > and add a level of verbosity (less means no output but failed tests > > for example). > > > > On Thu, Sep 1, 2011 at 1:13 PM, Hannes Magnusson <bjori@php.net> wrote: > >> It is, -g > >> > >> so, you can either do $ SHOW_ONLY_GROUPS=FAIL,WARN make test > >> or via run-tets, $ TEST_PHP_EXECUTABLE=auto php run-tests.php -g XFAIL,SKIP > >> > >> -Hannes > >> > >> > >> On Thu, Sep 1, 2011 at 13:06, Pierre Joye <pierre.php@gmail.com> wrote: > >>> good idea! > >>> > >>> please add it as a command line option as well, maybe using a verbose option > >>> > >>> On Thu, Sep 1, 2011 at 1:00 PM, Hannes Magnusson <bjori@php.net> wrote: > >>>> bjori Thu, 01 Sep 2011 11:00:51 +0000 > >>>> > >>>> Revision: > >>>> http://svn.php.net/viewvc?view=revision&revision=315967 > >>>> > >>>> Log: > >>>> Seeing thousands of PASS tests flying by is meaningless. > >>>> Add an option to only print certain result groups. > >>>> > >>>> Changed paths: > >>>> U php/php-src/branches/PHP_5_3/run-tests.php > >>>> U php/php-src/branches/PHP_5_4/run-tests.php > >>>> U php/php-src/trunk/run-tests.php > >>>> > >>>> Modified: php/php-src/branches/PHP_5_3/run-tests.php > >>>> =================================================================== > >>>> --- php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 10:16:21 UTC (rev > >>>> 315966) > >>>> +++ php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 11:00:51 UTC (rev > >>>> 315967) > >>>> @@ -207,6 +207,12 @@ > >>>> $DETAILED = 0; > >>>> } > >>>> > >>>> +if (getenv('SHOW_ONLY_GROUPS')) { > >>>> + $SHOW_ONLY_GROUPS = explode(",", > >>>> getenv('SHOW_ONLY_GROUPS')); > >>>> +} else { > >>>> + $SHOW_ONLY_GROUPS = array(); > >>>> +} > >>>> + > >>>> // Check whether user test dirs are requested. > >>>> if (getenv('TEST_PHP_USER')) { > >>>> $user_tests = explode (',', > >>>> getenv('TEST_PHP_USER')); > >>>> @@ -560,6 +566,9 @@ > >>>> case 'd': > >>>> $ini_overwrites[] = $argv[++$i]; > >>>> break; > >>>> + case 'g': > >>>> + $SHOW_ONLY_GROUPS = > >>>> explode(",", $argv[++$i]);; > >>>> + break; > >>>> //case 'h' > >>>> case '--keep-all': > >>>> foreach($cfgfiles as $file) { > >>>> @@ -675,6 +684,9 @@ > >>>> -d foo=bar Pass -d option to the php binary (Define INI entry foo > >>>> with value 'bar'). > >>>> > >>>> + -g Comma seperated list of groups to show during test run > >>>> + (e.x. FAIL,SKIP). > >>>> + > >>>> -m Test for memory leaks with Valgrind. > >>>> > >>>> -p <php> Specify PHP executable to run. > >>>> @@ -2426,16 +2438,24 @@ > >>>> function show_test($test_idx, $shortname) > >>>> { > >>>> global $test_cnt; > >>>> + global $line_length; > >>>> > >>>> - echo "TEST $test_idx/$test_cnt [$shortname]\r"; > >>>> + $str = "TEST $test_idx/$test_cnt [$shortname]\r"; > >>>> + $line_length = strlen($str); > >>>> + echo $str; > >>>> flush(); > >>>> } > >>>> > >>>> function show_result($result, $tested, $tested_file, $extra = '', > >>>> $temp_filenames = null) > >>>> { > >>>> - global $html_output, $html_file, $temp_target, $temp_urlbase; > >>>> + global $html_output, $html_file, $temp_target, $temp_urlbase, > >>>> $line_length, $SHOW_ONLY_GROUPS; > >>>> > >>>> - echo "$result $tested [$tested_file] $extra\n"; > >>>> + if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) { > >>>> + echo "$result $tested [$tested_file] $extra\n"; > >>>> + } else { > >>>> + // Write over the last line to avoid random trailing chars on > >>>> next echo > >>>> + echo str_repeat(" ", $line_length), "\r"; > >>>> + } > >>>> > >>>> if ($html_output) { > >>>> > >>>> > >>>> Modified: php/php-src/branches/PHP_5_4/run-tests.php > >>>> =================================================================== > >>>> --- php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 10:16:21 UTC (rev > >>>> 315966) > >>>> +++ php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 11:00:51 UTC (rev > >>>> 315967) > >>>> @@ -207,6 +207,12 @@ > >>>> $DETAILED = 0; > >>>> } > >>>> > >>>> +if (getenv('SHOW_ONLY_GROUPS')) { > >>>> + $SHOW_ONLY_GROUPS = explode(",", > >>>> getenv('SHOW_ONLY_GROUPS')); > >>>> +} else { > >>>> + $SHOW_ONLY_GROUPS = array(); > >>>> +} > >>>> + > >>>> // Check whether user test dirs are requested. > >>>> if (getenv('TEST_PHP_USER')) { > >>>> $user_tests = explode (',', > >>>> getenv('TEST_PHP_USER')); > >>>> @@ -560,6 +566,9 @@ > >>>> case 'd': > >>>> $ini_overwrites[] = $argv[++$i]; > >>>> break; > >>>> + case 'g': > >>>> + $SHOW_ONLY_GROUPS = > >>>> explode(",", $argv[++$i]);; > >>>> + break; > >>>> //case 'h' > >>>> case '--keep-all': > >>>> foreach($cfgfiles as $file) { > >>>> @@ -675,6 +684,9 @@ > >>>> -d foo=bar Pass -d option to the php binary (Define INI entry foo > >>>> with value 'bar'). > >>>> > >>>> + -g Comma seperated list of groups to show during test run > >>>> + (e.x. FAIL,SKIP). > >>>> + > >>>> -m Test for memory leaks with Valgrind. > >>>> > >>>> -p <php> Specify PHP executable to run. > >>>> @@ -2426,16 +2438,24 @@ > >>>> function show_test($test_idx, $shortname) > >>>> { > >>>> global $test_cnt; > >>>> + global $line_length; > >>>> > >>>> - echo "TEST $test_idx/$test_cnt [$shortname]\r"; > >>>> + $str = "TEST $test_idx/$test_cnt [$shortname]\r"; > >>>> + $line_length = strlen($str); > >>>> + echo $str; > >>>> flush(); > >>>> } > >>>> > >>>> function show_result($result, $tested, $tested_file, $extra = '', > >>>> $temp_filenames = null) > >>>> { > >>>> - global $html_output, $html_file, $temp_target, $temp_urlbase; > >>>> + global $html_output, $html_file, $temp_target, $temp_urlbase, > >>>> $line_length, $SHOW_ONLY_GROUPS; > >>>> > >>>> - echo "$result $tested [$tested_file] $extra\n"; > >>>> + if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) { > >>>> + echo "$result $tested [$tested_file] $extra\n"; > >>>> + } else { > >>>> + // Write over the last line to avoid random trailing chars on > >>>> next echo > >>>> + echo str_repeat(" ", $line_length), "\r"; > >>>> + } > >>>> > >>>> if ($html_output) { > >>>> > >>>> > >>>> Modified: php/php-src/trunk/run-tests.php > >>>> =================================================================== > >>>> --- php/php-src/trunk/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966) > >>>> +++ php/php-src/trunk/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967) > >>>> @@ -207,6 +207,12 @@ > >>>> $DETAILED = 0; > >>>> } > >>>> > >>>> +if (getenv('SHOW_ONLY_GROUPS')) { > >>>> + $SHOW_ONLY_GROUPS = explode(",", > >>>> getenv('SHOW_ONLY_GROUPS')); > >>>> +} else { > >>>> + $SHOW_ONLY_GROUPS = array(); > >>>> +} > >>>> + > >>>> // Check whether user test dirs are requested. > >>>> if (getenv('TEST_PHP_USER')) { > >>>> $user_tests = explode (',', > >>>> getenv('TEST_PHP_USER')); > >>>> @@ -560,6 +566,9 @@ > >>>> case 'd': > >>>> $ini_overwrites[] = $argv[++$i]; > >>>> break; > >>>> + case 'g': > >>>> + $SHOW_ONLY_GROUPS = > >>>> explode(",", $argv[++$i]);; > >>>> + break; > >>>> //case 'h' > >>>> case '--keep-all': > >>>> foreach($cfgfiles as $file) { > >>>> @@ -675,6 +684,9 @@ > >>>> -d foo=bar Pass -d option to the php binary (Define INI entry foo > >>>> with value 'bar'). > >>>> > >>>> + -g Comma seperated list of groups to show during test run > >>>> + (e.x. FAIL,SKIP). > >>>> + > >>>> -m Test for memory leaks with Valgrind. > >>>> > >>>> -p <php> Specify PHP executable to run. > >>>> @@ -2426,16 +2438,24 @@ > >>>> function show_test($test_idx, $shortname) > >>>> { > >>>> global $test_cnt; > >>>> + global $line_length; > >>>> > >>>> - echo "TEST $test_idx/$test_cnt [$shortname]\r"; > >>>> + $str = "TEST $test_idx/$test_cnt [$shortname]\r"; > >>>> + $line_length = strlen($str); > >>>> + echo $str; > >>>> flush(); > >>>> } > >>>> > >>>> function show_result($result, $tested, $tested_file, $extra = '', > >>>> $temp_filenames = null) > >>>> { > >>>> - global $html_output, $html_file, $temp_target, $temp_urlbase; > >>>> + global $html_output, $html_file, $temp_target, $temp_urlbase, > >>>> $line_length, $SHOW_ONLY_GROUPS; > >>>> > >>>> - echo "$result $tested [$tested_file] $extra\n"; > >>>> + if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) { > >>>> + echo "$result $tested [$tested_file] $extra\n"; > >>>> + } else { > >>>> + // Write over the last line to avoid random trailing chars on > >>>> next echo > >>>> + echo str_repeat(" ", $line_length), "\r"; > >>>> + } > >>>> > >>>> if ($html_output) { > >>>> > >>>> > >>>> > >>>> -- > >>>> PHP CVS Mailing List (http://www.php.net/) > >>>> To unsubscribe, visit: > >>>> http://www.php.net/unsub.php > >>>> > >>> > >>> > >>> > >>> -- > >>> Pierre > >>> > >>> @pierrejoye | http://blog.thepimp.net | > >>> http://www.libgd.org > >>> > >> > > > > > > > > -- > > Pierre > > > > @pierrejoye | http://blog.thepimp.net | > > http://www.libgd.org > > > > -- > > PHP CVS Mailing List (http://www.php.net/) > > To unsubscribe, visit: http://www.php.net/unsub.php > > > > >