Re: Concurrency support for run-tests

Hi! > I added a CONCURRENCY_GROUP section, see > > https://github.com/m6w6/php-src/compare/parallel-run-tests?expand=1 > > If the next test to execute is in the same CONCURRENCY_GROUP as one > which is already/still running, it will be pushed back. > > So tests with f.e. listening servers could use "port:9999" as > CONCURRENCY_GROUP or mysql tests could use "mysql:dbname", etc. As > example, I added the open_basedir tests to one group because they all > tamper around with the same subdirectories. The problem there is that for a lot of tests the dependency is in common include file (like server.inc or connect.inc) and extracting it into the main file may be a bit time-consuming and also means we'd have to keep the information in sync in many places at once. > I also added a shuffle($test_files) if $concurrency is greater than 2 > because that helps running the test suite concurrently actually. > Anybody got an idea why the tests are sorted at all? To have a predictable order and reproducible test runs? Imagine we have an inadvertent test dependency, in which test X fails after test Y is run. If the order is stable, it is relatively easy to discover. If it is random, reproducing the exact test alignment which caused the failure is next to impossible. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/