New run-tests.php feature
| From: | Hannes Magnusson | Date: | Thu, 01 Sep 2011 11:35:17 +0000 |
| Subject: | New run-tests.php feature | ||
| Groups: | php.internals php.qa | ||
| Request: | Send a blank email to internals+get-55054@lists.php.net to get a copy of this message | ||
Throw qa and internals@ into the loop to.
I'd also like to move the XFAIL section (printed out in the end) above
the FAIL section.
We have bucketloads of xfailed tests, I actually have to scroll up to
see the failed tests which is really annoying and silly.
Anyone have strong opinion on the printout order?
-Hannes
---------- Forwarded message ----------
From: Hannes Magnusson <bjori@php.net>
Date: Thu, Sep 1, 2011 at 13:32
Subject: Re: [PHP-CVS] svn: /php/php-src/
branches/PHP_5_3/run-tests.php branches/PHP_5_4/run-tests.php
trunk/run-tests.php
To: Pierre Joye <pierre.php@gmail.com>
Cc: php-cvs@lists.php.net
I did want to use -v at first, but that has a completely different
meaning already.
Its really only useful when looking at what is going on in a single
testcase to debug run-tests :]
-Hannes
On Thu, Sep 1, 2011 at 13:26, Pierre Joye <pierre.php@gmail.com> wrote:
> ah right, I miss this part of the patch. I would suggest -v instead
> and add a level of verbosity (less means no output but failed tests
> for example).
>
> On Thu, Sep 1, 2011 at 1:13 PM, Hannes Magnusson <bjori@php.net> wrote:
>> It is, -g
>>
>> so, you can either do $ SHOW_ONLY_GROUPS=FAIL,WARN make test
>> or via run-tets, $ TEST_PHP_EXECUTABLE=auto php run-tests.php -g XFAIL,SKIP
>>
>> -Hannes
>>
>>
>> On Thu, Sep 1, 2011 at 13:06, Pierre Joye <pierre.php@gmail.com> wrote:
>>> good idea!
>>>
>>> please add it as a command line option as well, maybe using a verbose option
>>>
>>> On Thu, Sep 1, 2011 at 1:00 PM, Hannes Magnusson <bjori@php.net> wrote:
>>>> bjori Thu, 01 Sep 2011
>>>> 11:00:51 +0000
>>>>
>>>> Revision:
>>>> http://svn.php.net/viewvc?view=revision&revision=315967
>>>>
>>>> Log:
>>>> Seeing thousands of PASS tests flying by is meaningless.
>>>> Add an option to only print certain result groups.
>>>>
>>>> Changed paths:
>>>> U php/php-src/branches/PHP_5_3/run-tests.php
>>>> U php/php-src/branches/PHP_5_4/run-tests.php
>>>> U php/php-src/trunk/run-tests.php
>>>>
>>>> Modified: php/php-src/branches/PHP_5_3/run-tests.php
>>>> ===================================================================
>>>> --- php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 10:16:21 UTC (rev
>>>> 315966)
>>>> +++ php/php-src/branches/PHP_5_3/run-tests.php 2011-09-01 11:00:51 UTC (rev
>>>> 315967)
>>>> @@ -207,6 +207,12 @@
>>>> $DETAILED = 0;
>>>> }
>>>>
>>>> +if (getenv('SHOW_ONLY_GROUPS')) {
>>>> + $SHOW_ONLY_GROUPS = explode(",",
>>>> getenv('SHOW_ONLY_GROUPS'));
>>>> +} else {
>>>> + $SHOW_ONLY_GROUPS = array();
>>>> +}
>>>> +
>>>> // Check whether user test dirs are requested.
>>>> if (getenv('TEST_PHP_USER')) {
>>>> $user_tests = explode (',',
>>>> getenv('TEST_PHP_USER'));
>>>> @@ -560,6 +566,9 @@
>>>> case 'd':
>>>> $ini_overwrites[] =
>>>> $argv[++$i];
>>>> break;
>>>> + case 'g':
>>>> + $SHOW_ONLY_GROUPS =
>>>> explode(",", $argv[++$i]);;
>>>> + break;
>>>> //case 'h'
>>>> case '--keep-all':
>>>> foreach($cfgfiles as
>>>> $file) {
>>>> @@ -675,6 +684,9 @@
>>>> -d foo=bar Pass -d option to the php binary (Define INI entry foo
>>>> with value 'bar').
>>>>
>>>> + -g Comma seperated list of groups to show during test run
>>>> + (e.x. FAIL,SKIP).
>>>> +
>>>> -m Test for memory leaks with Valgrind.
>>>>
>>>> -p <php> Specify PHP executable to run.
>>>> @@ -2426,16 +2438,24 @@
>>>> function show_test($test_idx, $shortname)
>>>> {
>>>> global $test_cnt;
>>>> + global $line_length;
>>>>
>>>> - echo "TEST $test_idx/$test_cnt [$shortname]\r";
>>>> + $str = "TEST $test_idx/$test_cnt [$shortname]\r";
>>>> + $line_length = strlen($str);
>>>> + echo $str;
>>>> flush();
>>>> }
>>>>
>>>> function show_result($result, $tested, $tested_file, $extra = '',
>>>> $temp_filenames = null)
>>>> {
>>>> - global $html_output, $html_file, $temp_target, $temp_urlbase;
>>>> + global $html_output, $html_file, $temp_target, $temp_urlbase,
>>>> $line_length, $SHOW_ONLY_GROUPS;
>>>>
>>>> - echo "$result $tested [$tested_file] $extra\n";
>>>> + if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {
>>>> + echo "$result $tested [$tested_file] $extra\n";
>>>> + } else {
>>>> + // Write over the last line to avoid random trailing chars
>>>> on next echo
>>>> + echo str_repeat(" ", $line_length),
>>>> "\r";
>>>> + }
>>>>
>>>> if ($html_output) {
>>>>
>>>>
>>>> Modified: php/php-src/branches/PHP_5_4/run-tests.php
>>>> ===================================================================
>>>> --- php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 10:16:21 UTC (rev
>>>> 315966)
>>>> +++ php/php-src/branches/PHP_5_4/run-tests.php 2011-09-01 11:00:51 UTC (rev
>>>> 315967)
>>>> @@ -207,6 +207,12 @@
>>>> $DETAILED = 0;
>>>> }
>>>>
>>>> +if (getenv('SHOW_ONLY_GROUPS')) {
>>>> + $SHOW_ONLY_GROUPS = explode(",",
>>>> getenv('SHOW_ONLY_GROUPS'));
>>>> +} else {
>>>> + $SHOW_ONLY_GROUPS = array();
>>>> +}
>>>> +
>>>> // Check whether user test dirs are requested.
>>>> if (getenv('TEST_PHP_USER')) {
>>>> $user_tests = explode (',',
>>>> getenv('TEST_PHP_USER'));
>>>> @@ -560,6 +566,9 @@
>>>> case 'd':
>>>> $ini_overwrites[] =
>>>> $argv[++$i];
>>>> break;
>>>> + case 'g':
>>>> + $SHOW_ONLY_GROUPS =
>>>> explode(",", $argv[++$i]);;
>>>> + break;
>>>> //case 'h'
>>>> case '--keep-all':
>>>> foreach($cfgfiles as
>>>> $file) {
>>>> @@ -675,6 +684,9 @@
>>>> -d foo=bar Pass -d option to the php binary (Define INI entry foo
>>>> with value 'bar').
>>>>
>>>> + -g Comma seperated list of groups to show during test run
>>>> + (e.x. FAIL,SKIP).
>>>> +
>>>> -m Test for memory leaks with Valgrind.
>>>>
>>>> -p <php> Specify PHP executable to run.
>>>> @@ -2426,16 +2438,24 @@
>>>> function show_test($test_idx, $shortname)
>>>> {
>>>> global $test_cnt;
>>>> + global $line_length;
>>>>
>>>> - echo "TEST $test_idx/$test_cnt [$shortname]\r";
>>>> + $str = "TEST $test_idx/$test_cnt [$shortname]\r";
>>>> + $line_length = strlen($str);
>>>> + echo $str;
>>>> flush();
>>>> }
>>>>
>>>> function show_result($result, $tested, $tested_file, $extra = '',
>>>> $temp_filenames = null)
>>>> {
>>>> - global $html_output, $html_file, $temp_target, $temp_urlbase;
>>>> + global $html_output, $html_file, $temp_target, $temp_urlbase,
>>>> $line_length, $SHOW_ONLY_GROUPS;
>>>>
>>>> - echo "$result $tested [$tested_file] $extra\n";
>>>> + if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {
>>>> + echo "$result $tested [$tested_file] $extra\n";
>>>> + } else {
>>>> + // Write over the last line to avoid random trailing chars
>>>> on next echo
>>>> + echo str_repeat(" ", $line_length),
>>>> "\r";
>>>> + }
>>>>
>>>> if ($html_output) {
>>>>
>>>>
>>>> Modified: php/php-src/trunk/run-tests.php
>>>> ===================================================================
>>>> --- php/php-src/trunk/run-tests.php 2011-09-01 10:16:21 UTC (rev 315966)
>>>> +++ php/php-src/trunk/run-tests.php 2011-09-01 11:00:51 UTC (rev 315967)
>>>> @@ -207,6 +207,12 @@
>>>> $DETAILED = 0;
>>>> }
>>>>
>>>> +if (getenv('SHOW_ONLY_GROUPS')) {
>>>> + $SHOW_ONLY_GROUPS = explode(",",
>>>> getenv('SHOW_ONLY_GROUPS'));
>>>> +} else {
>>>> + $SHOW_ONLY_GROUPS = array();
>>>> +}
>>>> +
>>>> // Check whether user test dirs are requested.
>>>> if (getenv('TEST_PHP_USER')) {
>>>> $user_tests = explode (',',
>>>> getenv('TEST_PHP_USER'));
>>>> @@ -560,6 +566,9 @@
>>>> case 'd':
>>>> $ini_overwrites[] =
>>>> $argv[++$i];
>>>> break;
>>>> + case 'g':
>>>> + $SHOW_ONLY_GROUPS =
>>>> explode(",", $argv[++$i]);;
>>>> + break;
>>>> //case 'h'
>>>> case '--keep-all':
>>>> foreach($cfgfiles as
>>>> $file) {
>>>> @@ -675,6 +684,9 @@
>>>> -d foo=bar Pass -d option to the php binary (Define INI entry foo
>>>> with value 'bar').
>>>>
>>>> + -g Comma seperated list of groups to show during test run
>>>> + (e.x. FAIL,SKIP).
>>>> +
>>>> -m Test for memory leaks with Valgrind.
>>>>
>>>> -p <php> Specify PHP executable to run.
>>>> @@ -2426,16 +2438,24 @@
>>>> function show_test($test_idx, $shortname)
>>>> {
>>>> global $test_cnt;
>>>> + global $line_length;
>>>>
>>>> - echo "TEST $test_idx/$test_cnt [$shortname]\r";
>>>> + $str = "TEST $test_idx/$test_cnt [$shortname]\r";
>>>> + $line_length = strlen($str);
>>>> + echo $str;
>>>> flush();
>>>> }
>>>>
>>>> function show_result($result, $tested, $tested_file, $extra = '',
>>>> $temp_filenames = null)
>>>> {
>>>> - global $html_output, $html_file, $temp_target, $temp_urlbase;
>>>> + global $html_output, $html_file, $temp_target, $temp_urlbase,
>>>> $line_length, $SHOW_ONLY_GROUPS;
>>>>
>>>> - echo "$result $tested [$tested_file] $extra\n";
>>>> + if (!$SHOW_ONLY_GROUPS || in_array($result, $SHOW_ONLY_GROUPS)) {
>>>> + echo "$result $tested [$tested_file] $extra\n";
>>>> + } else {
>>>> + // Write over the last line to avoid random trailing chars
>>>> on next echo
>>>> + echo str_repeat(" ", $line_length),
>>>> "\r";
>>>> + }
>>>>
>>>> if ($html_output) {
>>>>
>>>>
>>>>
>>>> --
>>>> PHP CVS Mailing List (http://www.php.net/)
>>>> To unsubscribe, visit: http://www.php.net/unsub.php
>>>>
>>>
>>>
>>>
>>> --
>>> Pierre
>>>
>>> @pierrejoye | http://blog.thepimp.net |
>>> http://www.libgd.org
>>>
>>
>
>
>
> --
> Pierre
>
> @pierrejoye | http://blog.thepimp.net |
> http://www.libgd.org
>
> --
> PHP CVS Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>