Re: run-tests re-write

Hi Nuno

So I have some questions: - does it support valgrind? this is really a must have. Not yet. But part of the reason I wrote the note was to flush out the 'must haves'. So that's good to know.

- how much time does your implementation takes? I see you have many little files and a lot of code, so I wonder if it can compete with the current implementation. (by looking to gsoc.php.net you can see the process takes a loong time, so we cannot accept big slowdowns). I've run some comparisons with run-tests.php running all the tests under ext/standard/tests and get very similar timings. If you look in the phpruntests/documentation directory there is a file which says what the times were. I looked at the profile with Xdebug yesterday as well - it was OK but I'm sure we can improve it.

- what the gsoc project is about? It seems the thing is done, or am I missing something? It's the one about running tests in parallel. That was one of the key initial requirements for this project - others are also described in the phpruntests/documentation directory.

By the way, the code is now in PHP CVS, under a project called 'phpruntests'. Now - my turn for questions :-). When I started doing this I ran tests in parallel using the existing code and pcntl() to spawn processes. The code was a lash-up (it's in the prototype directory), but *for the groups of tests that I ran*, it demonstrated that there were no interdependencies between tests in different directories. There is one caveat to that - I know there is nothing that I ran that used --REDIRECTTEST--, so I know this is something we still have to deal with. Anyway - my question for you, what dependencies do you know about between tests in different directories? Or in fact any other interdependencies? Zoe

Nuno ----- Original Message ----- From: "zoe" <zoe.slattery@googlemail.com> To: <php-qa@lists.php.net>; "Stefan Priebsch" <stefan.priebsch@e-novative.de> Sent: Wednesday, April 15, 2009 11:45 AM Subject: [PHP-QA] run-tests re-write

Hi all Stefan Priebsch and I have been working on a re-write of the run-tests code for about 6 months, on and off :-). The code is experimental, it's not complete, it's only ever been tested on Linux...so there is still a long way to go. However, it has got to the stage where it runs all the CLI tests under ext/standard/tests and gives the same results (with one exception) at run-tests.php. I think it's about in the state where implementing most of the rest of the sections will not be too hard. As we submitted a GSOC project to work on aspects of this code it seems sensible to make the code more publicly available. At the moment it's in Google's SVN and the source, docs, tests etc can be browsed here: http://code.google.com/p/phpruntests/source/browse I haven't wanted to put this into PHP because it was so experimental, but it may make sense to find a home for it in CVS in the near future, especially if we have a GSOC student working on it. Any feedback on what's there now would be welcome. Zoe PS - there are quite a few of the exiting features of run-tests that I think we should not re-implement, I suspect they are not used. PPS - The other thing I found when running this is that there are 40 tests with broken --CLEAN-- sections. The current implementation doesn't check that --CLEAN-- has worked, but mine does. See this for an example: ext/standard/tests/general_functions/is_array.phpt. I'll work through cleaning these up over the next few days.