Re: Re: [RFC] Better benchmarks for PHP

On Fri, Feb 20, 2009 at 12:24 AM, Ólafur Waage <olafurw@gmail.com> wrote: > Also what would be the next benchmark to port in your opinion. I suggested Richards since it was shortest, while actually doing something. I would do them in order of how easy they are. I took a quick glance, at think perhaps crypto or deltablue. The other v8 ones look hard or weird. From the sunspider ones, 3d-raytrace.js looks nice, and 3d-cube.js looks ok. We might be out of useful tests after that, as either too hard to port, or as not great benchmarks. Paul > On Thu, Feb 19, 2009 at 10:19 PM, Paul Biggar <paul.biggar@gmail.com> wrote: > >> On Thu, Feb 19, 2009 at 9:24 AM, Ólafur Waage <olafurw@gmail.com> wrote: >> > Thank you, >> >> No problem. >> >> > "- Packet::__construct inits the array to 4, rather than use DATA_SIZE.. >> > I suspect this could be omited entirely? I think I saw this somewhere >> > else as well..." >> > >> > in PHP >> > array(DATA_SIZE) // Array ( [0] => 4 ) >> > >> > in JS >> > new Array(DATA_SIZE) // Array ( [0] => , [1] => , [2] => , [3] => ) >> >> I realized that. I just meant that DATA_SIZE is not used for its >> original purpose. But I think its no harm. >> >> >> >> I've been looking at this for a while, and done a fair amount of >> debugging and testing, but I can't find the bug. Are you having any >> success yourself? Once the bug is fixed, I think you're good to >> commit. Nuno has, to my knowledge, sorted karma etc, so ping him once >> you're done. >> >> >> Looking forward to reviewing the next benchmark. >> >> Paul >> >> >> > Olafur >> > >> > On Thu, Feb 19, 2009 at 3:48 AM, Paul Biggar <paul.biggar@gmail.com> >> wrote: >> >> >> >> On Thu, Feb 19, 2009 at 1:30 AM, Ólafur Waage <olafurw@gmail.com> >> wrote: >> >> > I have done a line by line convertion of richards.js from the V8 >> google >> >> > test >> >> > but i am having some issues with inheritance in it. >> >> > >> >> > http://cznp.com/php/benchmark/ >> >> >> >> OK, I've taken a look. Overall, this looks almost perfect, so the >> >> following comments are very minor. >> >> >> >> - dont take out the copyright notice >> >> - I would recommend copying the comments from richards.js verbatim. >> >> - You might point to y«‘7 >> >> S™e Ó&è}http://www.cl.cam.ac.uk/~mr10/Bench.html<http://www.cl.cam.ac.uk/%7Emr10/Bench.html>. >> Note >> >> that in his code package, there is a README, and a python >> >> implementation. That might help a little. >> >> - In TaskControlBlock::run, $packet does not need to be declared. >> >> - TaskControlBlock::toString is missing. >> >> - IdleTask::toString is missing. >> >> - DeviceTask::toString is missing. >> >> - WorkerTask::toString is missing. >> >> - HandlerTask::toString is missing. >> >> - Packet::toString is missing. >> >> - Packet::__construct inits the array to 4, rather than use DATA_SIZE.. >> >> I suspect this could be omited entirely? I think I saw this somewhere >> >> else as well... >> >> - Is "var peek, next = queue;" the same as " $peek = $queue; >> >> $next = $queue;"? >> >> - Hmmm, none of the toStrings are used, which I presume is why you >> >> omitted them. Oh well. >> >> >> >> >> >> I wonder if its worth keeping the structure of the file as close to >> >> the original as possible? Just a thought. No doubt there was a reason >> >> you rearranged it. >> >> >> >> My javascript is poor, but I would think your code looks perfect. Yet >> >> it fails. I've confirms richards.js doesnt fail (at least in Rhino), >> >> but I cant seem to find your bug. There is a null dereference of >> >> $packet->id in Scheduler::queue(), but I think that's by design, so >> >> probably not it. >> >> >> >> Great work. >> >> >> >> Paul >> >> >> >> >> >> > If someone can take a look at it, since i am still figuring out what >> >> > exactly >> >> > the code is doing, but meanwhile i did a convertion of one SunSpider >> >> > test, >> >> > the nsieve.js one. Its in the same folder and it can be added if you >> >> > guys >> >> > want. >> >> > >> >> > I am going to look at more SunSpider tests while trying to fix the >> >> > richards >> >> > test. >> >> > >> >> > Thanks, >> >> > Ólafur Waage >> >> > olafurw@gmail.com >> >> > >> >> > On Mon, Feb 16, 2009 at 10:07 PM, Nuno Lopes <nlopess@php.net> wrote: >> >> > >> >> >> I think that can be viewed as part of the project. Compiling a >> (little) >> >> >> list of interesting applications is a challenging task. >> >> >> For example, we can tweak say phpbb 3 to have it deployed >> automatically >> >> >> and >> >> >> then use some real world data to build a set of requests that you >> want >> >> >> to >> >> >> perform and benchmark. I actually run a phpbb forum with a little >> more >> >> >> than >> >> >> 1 million visitors/month, so I can provide this data. >> >> >> >> >> >> You can also think about e.g. mediawiki. it should be easy to >> >> >> automatize, >> >> >> and we could then ask for stats from the wikipedia guys. >> >> >> >> >> >> Nuno >> >> >> >> >> >> >> >> >> ----- Original Message ----- >> >> >> Since we are talking about benchmarking real world applications, do >> we >> >> >> have >> >> >> a list of applications we wish to run the benchmarks on? >> >> >> >> >> >> On Sun, Feb 15, 2009 at 5:02 PM, Paul Biggar >> >> >> <paul.biggar@gmail.com> >> >> >> wrote: >> >> >> >> >> >> On Sun, Feb 15, 2009 at 4:39 PM, Ólafur Waage >> >> >> <olafurw@gmail..com> >> >> >> wrote: >> >> >>> > I have not delved deep into the tests that are referenced in the >> rfc >> >> >>> > but >> >> >>> do >> >> >>> > they include comparison data? >> >> >>> >> >> >>> Most have very little. Typically, they are built into the benchmark. >> >> >>> I've updated the wiki page with everything I know about the data >> sets. >> >> >>> >> >> >>> >> >> >>> Paul >> >> >>> >> >> >>> > >> >> >>> > Thanks, >> >> >>> > Olafur >> >> >>> > >> >> >>> > On Sun, Feb 15, 2009 at 4:05 PM, Pierre Joye < >> pierre.php@gmail.com> >> >> >>> wrote: >> >> >>> > >> >> >>> >> hi, >> >> >>> >> >> >> >>> >> On Sun, Feb 15, 2009 at 8:18 AM, Sebastian Bergmann >> >> >>> >> <sb@sebastian-bergmann.de> wrote: >> >> >>> >> > Nuno Lopes schrieb: >> >> >>> >> >> Comments, criticisms and volunteers especially >> >> >>> >> >> welcome, >> >> >>> >> > >> >> >>> >> > When you do this, think about splitting each benchmark >> >> >>> >> > into a >> >> >>> separate >> >> >>> >> > file and have it run by a separate PHP process, see >> >> >>> >> > [1]. >> >> >>> >> >> >> >>> >> To split them into separate files/cases is definitively a >> >> >>> >> good >> >> >>> >> thing. >> >> >>> >> >> >> >>> >> I'm not sure we should run them in parallel. As it >> >> >>> >> makes sense >> for >> >> >>> >> the >> >> >>> >> phpt, it may have counter effects for benchmark tests. >> >> >>> >> Concurrent >> >> >>> >> executions of a given feature/extension could help (for apc >> >> >>> >> and >> >> >>> >> stat >> >> >>> >> cache for example), but maybe not for every part of php, or >> >> >>> >> not >> by >> >> >>> >> default. >> >> >>> >> >> >> >>> >> Cheers, >> >> >>> >> -- >> >> >>> >> Pierre >> >> >>> >> >> >> >>> >> http://blog.thepimp.net | >> >> >>> >> http://www.libgd.org >> >> >>> >> >> >> >>> >> -- >> >> >>> >> PHP Quality Assurance Mailing List >> >> >>> >> <http://www.php.net/> >> >> >>> >> To unsubscribe, visit: >> >> >>> >> http://www.php.net/unsub.php >> >> >>> >> >> >> >>> >> >> >> >>> > >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Paul Biggar >> >> >>> paul.biggar@gmail.com >> >> >>> >> >> >> >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> Paul Biggar >> >> paul.biggar@gmail.com >> > >> > >> >> >> >> -- >> Paul Biggar >> paul.biggar@gmail.com >> > -- Paul Biggar paul.biggar@gmail.com