Re: cvs: php-benchmarks / deltablue.php

On Mon, May 4, 2009 at 5:51 PM, Alexander Hjalmarsson <hjalle@sgh.se> wrote: > About the looping part, 10 seconds sounds good as a direction. What do you mean with > considering memory usage? Currently, (on PHP 5.2.9) the benchmark uses over 100M, causing it to crash. We might need to consider the memory limit when choosing the parameter, if we perceived 100M to be too much memory, or something like that. On the other hand, we could just add a call to set_memory_limit. > This benchmark (deltablue) includes some lines in test functions that tests the > "output". In the projectionTest there are lines like: > [] if ($dests->at($i)->value != $i * 5 + 1000) > [] echo ("Projection test 3 failed!"); > > If those values are not the same, it will echo this message and the test has failed. It's > kind of the same for the chainTest as well. Those lines exist in the SunSpider test suite as well. > > I don't know if this satisfies the testing part, but I have not found any real test code > for the deltablue algorithm. Its not great, but its not awful. It would be nice if there was something to compare the output to that of sunspider, using Rhino or something. But this is very-very-very-low-priority. What's there is probably fine. Thanks, Paul > -----Original Message----- > From: Paul Biggar [mailto:paul.biggar@gmail.com] > Sent: den 4 maj 2009 17:46 > To: Alexander Hjalmarsson > Cc: php-qa@lists.php.net > Subject: Re: [PHP-QA] cvs: php-benchmarks / deltablue.php > > Hi Alexander, > > On Mon, May 4, 2009 at 4:24 PM, Alexander Hjalmarsson <hjalle@sgh.se> wrote: > >> The grammar in the comments are a result of a bad copy-paste usage from the java-source. I >> guess he (Mario Wolzcko I believe) had a habit of doing so or thought it was the best way. I could >> change it if that’s needed. > > If those were the comments in the Java source, I dont think there is > any harm in leaving them. > > >> I took the loop-figure from the SunSpider-test >> (http://svn.webkit.org/repository/webkit/trunk/SunSpider/tests/v8-deltablue.js) and I have no idea >> where they got that figure from. I guess it can be a good idea to lower it in the beginning at least >> so it doesn't time-out like Ólafur said. > > I would guess that they decided on a number that made their tests run > for some period of time. However, looping like that is cheating in > benchmarking, so it would be better if we could eliminate it. Can we > play with the parameters to chainTest and projectionTest to make it > run longer (assuming the loop is removed)? I think a good number here > is whatever makes it run for a few (10?) seconds. The other benchmarks > should be modified to run for about the same amount of time. We also > need to consider memory usage here. > > > Is the benchmark tested? The raytracer code came with some reasonable > tests to see that it worked - it just checked it got the same result > as the javascript tests. Is anything like that practical for > deltablue? > > > Thanks, > Paul > > > > -- > Paul Biggar > paul.biggar@gmail.com > > -- > PHP Quality Assurance Mailing List <http://www.php.net/> > To unsubscribe, visit: http://www.php.net/unsub.php > > > -- > PHP Quality Assurance Mailing List <http://www.php.net/> > To unsubscribe, visit: http://www.php.net/unsub.php > > -- Paul Biggar paul.biggar@gmail.com