Re: Continuous Integration Servers for php-src

Hi Isreal, Hi Zoe, I'm not aware of the current status of the new test infrastructure and all the requirements. If it is possible to integrate the new test runner we should do this. It will significantly reduce the testing time on some machines (particular SPARC). I would also prefer that we are _not_ testing after every commit. A full compilation can take very long depending on the system. If we want to run tests on older systems (e.g. an old SPARC with Solaris 9) it will take ages and the queue will just be filled. It would suggest that we build max. 4 times a day to make sure that the queue can be processed. As soon as we have a working configuration for slaves I'll setup a slave on the OpenSolaris SPARC machine that php.net can use for testing purposes. David On 08.10.09 09:10, zoe wrote:

Hi Israel - this looks like a good initiative :-) Well done. I have a suggestion about the PHPT tests. We have been working on a new test runner, it has major advantages over the existing test runner. It runs tests in parallel - so depending on the numbers of processors you have it can be much faster, the output is XML and would be easier for you to handle than the existing test runner output. I would really like to see the new test runner integrated in what you are doing. There is one problem, the implementation of REDIRECTTEST is not done yet and I have no time to do it, however the code does handle the fact that it's missing - the result is just that some (not a huge number) of the tests are skipped. My only problem is that I have very little time currently to help with this, I'm wondering if anyone on the wider QA team would be interested? I can certainly help with answering any questions on how the run-tests.php code works? Zoe

Hi everyone, My name is Israel Ekpo <iekpo@php.net <mailto:iekpo@php.net>> I am currently working on the UNIX buildbot system on the Debian server (80.82.121.134). Please review this implementation set up and let me know what you think. I just want to maintain transparent communication with all parties involved so that I don't end up straying too far from what is expected. After this continuous integration server is completed and finalized, it will be replicated across the other UNIX servers in a slave-master set up. For now, each slave will be doing pretty much what the master is doing (testing php 5.2, 5.3 and 6.0) However, the results will only be accessible and stored on the master server. Here is the rough draft for the implementation details: ==================================================================== 0. Target Subversion URLs ==================================================================== The priority as stated by Hannes Magnusson, is to set it up for PHP 5.3 first then PHP 5.2 and 6.0+ So the build bot will only be monitoring commits to the following subversion URLS : PHP 5.3: http://svn.php.net/repository/php/php-src/branches/PHP_5_3 PHP 5.2: http://svn.php.net/repository/php/php-src/branches/PHP_5_2 PHP 6: http://svn.php.net/repository/php/php-src/trunk A separate build daemon will be set up for each branch on each UNIX server. ==================================================================== 1. Notification of Build System ==================================================================== A post commit hook will notify the build system that a commit has occurred (I will need the assistance of systems guys set up this post-commit hook). A simple web service will be set up at the master to receive these notifications. The revision number will be entered into a queue and then a queue processing system will take it from there. The notification will be sent to the master and the other slaves will be notified after that. I am planning for the master to be the Debian server and the other servers will be the slaves. ===================================================================== 2. Processing items in the queue ===================================================================== The build will occur in very distinct phases/states. Each stage/state will be fine-tuned or broken down if needed in the future. We should be able to find out what state a particular job is at any give instant. Here are the states : 2A : The source code will be checked out of SVN. 2B : Then the buildconf command will be run. 2C : Then the configure script will be run will all the extensions enabled. 2D : Then make 2E : Then make install If any of the above steps fail, this particular job is put on HOLD and an email notification will be sent out. 2F : Then make test (or any other simpler alternative) 2G: The results will then be agregated an analyzed. This will be explained in 4 below However, we have to take care of the following cases/scenarios to avoid false alarms. 1. Removal of phpt scripts (a phpt script that was in the previous revision no longer exists) 2. Addition of new phpt scripts (a phpt script that never existed is now introduced) ===================================================================== 3. Handling Errors and Build-Failures ===================================================================== Whenever a particular build fails (steps 2A-E), human intervention may be needed because the reason could be a missing dependency such as libraries, tools etc. As for steps 2G, someone will have to fix the source code to make it PASS the test. If the problem is not in the source code, then the phpt script may have to be adjusted to make it pass the test. ===================================================================== 4. Intepretation of Build results ===================================================================== 2G1 : Existing phpt files. If the result changes from PASS to FAIL, a FAILURE notice will be sent out (RED) If the result changes from FAIL to PASS, a RESTORED notice will be sent out (GREEN) If the result remains at PASS, no notice will be sent out. If the result remains at FAIL, no notice will be sent out. 2G1 : Removed phpt files. A REMOVED notice will be sent out (ORANGE) 2G1 : Newly introduced phpt files. If the result is FAIL, a FAILURE notice will be sent out (RED) If the result is PASS, a PASS notice will be sent out (GREEN) ===================================================================== 4. Managing Results Data ===================================================================== The results will be stored in a MySQL database that will be archived from time to time. There will be a web interface to view the results of each test for each server and SVN URL ===================================================================== References : ===================================================================== http://wiki.php.net/internals/buildbot#operating_systems_available http://www.php.net/svn.php -- "Good Enough" is not good enough. To give anything less than your best is to sacrifice the gift. Quality First. Measure Twice. Cut Once.

-- David Soria Parra - Software Engineer Sun Microsystems GmbH Sonnenallee 1 85551 Heimstetten Germany www.sun.de mailto: david.soriaparra@sun.com Amtsgericht Muenchen: HRB 161028 Geschäftsführer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin Haering