Re: Use JUNIT XML to build QA reports

On Sat, Feb 25, 2012 at 3:27 PM, Olivier Doucet <webmaster@ajeux.com> wrote: > Hello everyone, > > By reading run-tests.php source code, I can see that a JUNIT XML file can > be easily built in 'make test'. > This contains all succeeded, failed and skipped tests with diffs. > Yep, we even have an improved version (running on ci.qa.php.net currently) thanks to Shein ( https://github.com/Tyrael/ci.qa.php.net/commits/master/run-tests.php) which still needs to be merged to the svn. > > I wondered if we can modify run-tests.php to automatically build this file > each time, and when the user is asked to send report, send this XML report > as well to QA report interface. This would give us the following : > - list of succeeded tests (to know if one test is 100% failing or not) > - better diff parsing (some glitches in current implementation due to email > format when you send a qa-report). > we could, but I think that would require a clear consensus from the qa/internals group. I remember we also had some problems with the diffs (having whitespace chars not allowed in CDATA, or something like that) > > > I also would like to add a special env variable to be able to send the QA > report automatically : we would be able to have more QA reports this way > (and also add ci.qa.php.net reports too in qa.php.net/reports/ interface > easily). > this would be also easy(setting the email sending on by default), but I remember somebody mentioning that we don't want to be spammed with the reports, and it would create some problem, as the current report contains sensitive system specific informations. ps: you can currently grab the junit.xml's from ci.qa.php.net if you want ( http://ci.qa.php.net/job/php-src-5.3-matrix-build/344/architecture=x86,os=bsd-freebsd-8.2/artifact/junit.xml for example) -- Ferenc Kovács @Tyr43l - http://tyrael.hu