Re: [benchmarks] Server-side PHP

Hi Mich, On Thu, Apr 23, 2009 at 4:19 AM, Michiaki Tatsubori <mich@acm.org> wrote: > Hi, > > Let me just break some discussion towards better PHP benchmarks. > For benchmarks considering server-side usage of PHP, we need to take > following two issues into account: > * Measureing per-request processing > * Allowing warming-up for other types of implementation of PHP runtime > > PER-REQUEST PROCESSING > When a server-side PHP application runs, it consumes significant > amount of time for preparing constants and functions.  For example, it > needs to load all the application-specific libraries which are > explicitly loaded through include/require instructions.  We cannot > measure these potions through a command-line benchmark as we cannot > redefine constants and functions in a loop. > > WARMING-UP > It is better if other PHP runtime implementation can benefit from the > benchmark we are creating.  For instance, APC (PECL) code caching, > on-demand compilation by Phalanger (.Net), and Java JIT compilation > for Quercus (Resin) and P8 (WebSphere sMash) needs some warming up > before reaching their typical performance.  Maybe it is a must-support > feature.  To do this, giving some flexibility for warming the measured > runtime is mandatory. Are both of these handled by counting requests-per-second after x amount of time (as well as the steady state)? Or do we want to try to measure warm-up on each request? > An idea in my mind is to develop "purified" version of realistic Web > applications, like SPECweb, but provide load generating scenarios that > never put overwhelming load on Web servers, by omitting static file > requests. That's a nice idea. I dont know how realistic SPECweb is though. SpecJVM was shown to be very unrealisitic compared to real programs. It also suffers from being (very) closed source, making it difficult for many people here to use it. I believe you were working on something with SugarCRM? To be honest, I have a hard time seeing how this should end up. Could you give a quick overview of what the results would look like, and what work a benchmarker would have to do to get it working? For example, for the RUBBoS benchmark (if it can be coaxed to work) requires machines for the DB, for PHP and for the simulated clients, requires them to provide shell accounts for the 'master' benchmark instance, and measures memory usage as well as requests-per-second. Is this the sort of setup you had in mind? Thanks, Paul -- Paul Biggar paul.biggar@gmail.com