Re: Benchmark Suite

On 05.06.2007 11:25, Sebastian Bergmann wrote:

Antony Dovgal wrote:

Hello Sebastian.

Hello Antony,

What's the status of this? Do you have anything working?

back in February/March when I first posted about this, I discussed this with Marcus. The conclusion was that we wanted to rewrite run-tests.php based on the Iterator approach I used for the benchmark system's prototype.

Require SPL for a test suite? Doesn't seem to be good idea..

Benchmarks would then be .phpb in addition to .phpt for tests. I did some work on this but stopped when I got negative feedback (on IRC) towards the idea of rewriting "something that works" (run-tests.php).

Well, I personally think we'll have to do it sooner or later. The current version run-tests.php is really hard to maintain and improve. We can have both for some time and drop the old one when we're sure that the new one is stable. It's a developers tool anyway..

I see several options to continue: - Add .phpb style support for benchmarks to the current run-tests.php. - Improve run-tests.php to also collect Peak Memory Usage, store the

     data in SQLite, and have a test (optionally) fail when it takes more
     memory than it did in the past (one of the ideas that I had for the
     rewrite).

Oh, that would be too much IMO..

- Do the rewrite. Opinions?

I'm for the rewrite and I'm ready to help. -- Wbr, Antony Dovgal