Re: QA Reports aggregator

On Fri, May 20, 2011 at 15:08, Pierre Joye <pierre.php@gmail.com> wrote: > Hi Olivier, > > Something along this line is what we have been working on for some > time already with the rmtools (/rmtools in SVN). > > rmtools include tools to build php and report the log and to run the > tests. For the tests we have implemented it for our custom tests > (which run through web servers and cli, and are slightly more complex > than simple phpt) and include performance regressions. > > Maybe we could share our work and integrate what you have done so far > into rmtools? Doing so will have the great advantage to make it work > immediately for any new branch and later for pecl as well. > > The goal is to have one place where RMs and developers can quickly > check that everything is fine in a current branch, on all platforms. The key difference is that the rmtools build process tests different architectures and in a controlled environment. The 'make tests' user submitted results are in an uncontrolled environment, on all sorts of possible wacky configuration and platforms. I think these should be kept separate to begin with, and then some day later down the road all this could be then centrally aggregated (and maybe even gcov thrown into the mix). -Hannes