Re: [Gallery-devel] [Fwd: PHP 5.2.2RC1 testing]

Hi Marcus, Marcus Boerger wrote: > it might be a good idea to put stuff like those unit tests under the gcov > machine. Maybe it is enough if we have such unit tests run without valgrind. I am running a very similar approach since ~ a year now using CruiseControl and various popular PHP Open Source software on the test side (I mailed you personally about this some months ago). So I have some experience and thoughts to share with you: - Clean the Tests up to not rely on external resources but be completely self contained. The Zend Framework's Unit Tests for example have a good chance to fail because Website xyz is not available etc. - Don't rely only on Unit Tests (I set up a Selenium & VMWare infrastructure, running some GUI acceptance tests). This way the different web server SAPIs can be tested, too. - Make it as easy as possible to install and use for anyone. This way chances are high, people install and run it and we get a lot of results from a lot of different platforms, configure options etc. People can also run their own closed source software unit tests against CVS versions of PHP. A yet unresolved issue in my CruiseControl infrastructure is a way to collect the data from different CruiseControl instances to set up some nice GUI to represent the results and find correlations between failing tests and platforms for example. Pikefarm has something very similar and I think MySQL too (Brian Aker blogged about that some time ago if I remember right). The University of Maryland also has some nice approach on distributed continuous quality assurance, I'll post details on that later. I met Prof. Adam Porter, leader of that project some time ago at a conference and he was interested in integration PHP into his build farm. - Sebastian