Re: Parallel run-tests

On 21/05/2012 06:45, Nuno Lopes wrote:

Hi Zoe, Thanks for undertaking this project! I just have a few concerns about this: - The speedup seems a bit low to me. Maybe with a higher number of extensions enabled it will improve?.. I don't think it will improve with a higher number of extensions. It _may_ improve as a result of someone doing some work on the tests are scheduled - but I wouldn't count on it. Looking at improving performance is important and I wanted to get some confirmation that it is faster, I think we have that level of confirmation now. However, if I'm the only person working on this, further performance work will come after complete implementation + debugging.

- Is there any developer documentation available? If I wanted to do a change or a bug fix today, how could I do it? Lots - https://wiki.php.net/qa/runtests. I have just updated the "Development" section which has instructions on how to build and test the code. The 'Documentation' section has not been updated recently but I think it is still valid.

- Can it be packaged as a single drop-in file replacement for run-tests.php? The deployment is very important to me. IMHO, the optimal solution would be to have a drop-in replacement for the current script, so that most developers wouldn't even notice the difference. Yes - totally agree. I think I experimented with packaging it as a .phar (so it would be called run-tests.phar, not run-tests.php), but it's so long ago that I can't remember. I have added it to the to-do list (https://wiki.php.net/qa/runtests/todos).

- From previous emails exchanged in this thread, it seems that this new version requires a few extensions to run (gzip, soap??). This is not acceptable. The script must be able to run with --disable-all. Of course in that case the parallel version will fail. No, it doesn't. That was a stupid bug and I fixed it last night :-)

Zoe

Nuno -----Original Message----- From: zoe slattery Sent: Thursday, May 17, 2012 1:00 PM To: internals@lists.php.net ; PHP QA Subject: [PHP-QA] Parallel run-tests Hi Over the past couple of weeks I have updated the parallel run-tests (fixed a couple of minor bugs in the PHP code and the build.xml), it's now almost at the point where I could go ahead and implement the last pieces. Here is a summary and a few questions: 1. In rebasing the code the the dev't stream I found a number of tests with non-standard sections. My code checks test case structure and objects to anything non-standard, the current run-tests.php mainly ignores this kind of thing. I fixed up about 15 of these tests (see #62022) already - I'll fix the rest if there are no objections - I will open another bug report first. 2. If there is agreement to use this code it would make sense to replace the existing run-tests code with it, or rather, it would make no sense to try and maintain both versions. The new code is OO PHP, it's in http://git.php.net/repository/phpruntests.git, is there any problem with it staying there long term and maybe copying a run-tests.phar into the PHP source directory? I have no idea what the right answer is, suggestions welcome. 3. I ran a couple of small tests on my dual core Mac yesterday. For a standard set of tests, the parallel code ran in 207 seconds, sequential in 293 seconds and the standard run-tests.php took 298 seconds. This is an improvement but I suspect we could still do better by looking at the scheduling algorithm. At the moment it's very simple, we just assemble a list of directories with tests in and hand them out to processors till everything is done. Being able to handle tests that must be run in sequence (mysql, mysqli) will mean making some changes to this. So, perhaps we give an explicit list to p1 and let the scheduler distribute the rest of the tests? Or maybe we should have a 'process map' for all tests for extensions that are build by default? Again, suggestions welcome. 4. REDIRECTTEST still needs to be implemented, I understand how it works and this isn't (afaict) a major issue. 5. Testing. I'm able to do basic testing on Mac OSX and Linux. I really need access to an 8 way Linux system, or someone who has access and would be interested in looking at performance? Any volunteers? This is probably the most interesting part of the project :-) 6. Windows. I'm not in a position to do anything much with Windows except some very basic checks to make sure that the sequential version runs. The parallel code won't work because we used pcntl(), however I know that Stefan and George were keen to design the code so that a Windows solution could be implemented if anyone thought of one. If anyone wants to pick up this aspect I'd be happy to get them started. Zoe