Re: QA Reports aggregator

On Fri, May 20, 2011 at 3:37 PM, Hannes Magnusson <hannes.magnusson@gmail.com> wrote: > On Fri, May 20, 2011 at 15:08, Pierre Joye <pierre.php@gmail.com> wrote: >> Hi Olivier, >> >> Something along this line is what we have been working on for some >> time already with the rmtools (/rmtools in SVN). >> >> rmtools include tools to build php and report the log and to run the >> tests. For the tests we have implemented it for our custom tests >> (which run through web servers and cli, and are slightly more complex >> than simple phpt) and include performance regressions. >> >> Maybe we could share our work and integrate what you have done so far >> into rmtools? Doing so will have the great advantage to make it work >> immediately for any new branch and later for pecl as well. >> >> The goal is to have one place where RMs and developers can quickly >> check that everything is fine in a current branch, on all platforms. > > > The key difference is that the rmtools build process tests different > architectures and in a controlled environment. > The 'make tests' user submitted results are in an uncontrolled > environment, on all sorts of possible wacky configuration and > platforms. There is no difference in how they work. One single place to control everything is way more easier to work with and to maintain. It is critical as well to integrate revisions in the results and the delta between them. > I think these should be kept separate to begin with, and then some day > later down the road all this could be then centrally aggregated (and > maybe even gcov thrown into the mix). Not later, now. And that's something we are working on. -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org