Re: benchmark tests on mdb + others

Hello, On 01/24/2003 07:36 PM, Lukas Smith wrote:

Anyways don't overestimate this benchmark. 1) it is really simplistic in that it just fetches some data from a single table a lot of times 2) it uses MDB::fetchInto() which evades all the type conversion if statements that are found in the [fetch|query|get][One|Row|Col|All] methods. If you use these things will slow down. If you actually use the type conversions things will slow down even more. 3) it's a benchmark! It's a lying piece of *"$%&"!

Now you have said it all. Your benchmark code contains even more flaws than the original from ADODB author. So, it can't be considered by any serious developers. Like you said it is a lie. The problem is that if you repeat a lie too many times it becomes the truth. Just by fixing and tweaking a little the Metabase benchmark I managed to make it run marginally faster than MDB. If you want I can send you a fixed version so you understand why keep spreading these benchmarks are only fooling people. Just to summarize the flaws of the benchmark let me point what I fixed in Metabase: - The test functions QueryOnce and QueryOnce2 were switched. On one you would query all the results at once and in the other you fetched each result row individually. The problem is that what the functions did in MDB test were not the same as in Metabase test. So you were comparing apples with oranges. No wonder why MDB performed much faster. - Metabase intentionally only establishes a database connection when the first query is run. Since the first query of the test was made part of the benchmark loop, the database connection overhead was included in the benchmark timing. The difference would be diluted after 400 queries, but in real world scripts that are made of just a few queries, the connection overhead is much more noticeable, especially in high-end databases like Oracle. - Even knowing that Metabase provides a direct to driver object interface, like ADODb author you still insist on using Metabase global functions that you know very well add some overhead to pass the driver object from a global array. - Like ADODB author you keep fetching result column rows with individual functions calls passing the column by name, when you know that Metabase has a function to fetch individual rows that put column values in arrays indexed by number like you use in MDB and of course is much much faster because. - The result retrieving loop was full of unnecessary function calls and other expensive instructions that translate into more expensive Zend opcodes and so it made PHP run slower, which is not Metabase fault. What I fixed made quiet a difference in the number of expensive Zend opcodes. - Anyway, what really made the whole difference was to omit database result data type converstion. This the most important detail here. Without database result data type conversion you are not really talking about portable database programming. That is why I even did not bother to compare the results of PEAR-DB and ADODB because none of these provide true database portability. In the end you may still need to handle database specific details in your application making it not portable at all. Bottom line is that database abstraction layer benchmarks are useless because thay are misleading and pointless. They only were started because ADODB author had a commercial product to sell and so it was convinient to lock his users in his database abstraction package where there would be no competitor to his product. Anyway, if anybody is concerned about speed, forget about using database abstraction layers. Use PHP functions directly and forget portability. Portability compromises speed and vice-versa. Either you use slower but portable code or you use the fastest but not portable code. Any mid-term solution will only lead to solutions that are not portable nor are the fastest. Most people will only realize why PEAR-DB and ADODB based applications are not portable and are slower when they the try an application with several types of databases that uses more than just text and integer fields. Until then true portability and speed are just an illusion. On a side note, for me, in PHP, database abstraction layers are conceptually dead. Software development techniques must progress to overcome current limitations. I am working on Metastorage that provides a much better abstraction that is closer to the application level. Currently it generates Metabase based code but soon it will start generating code that uses PHP database specific API function calls. This result in much less code because it will not use any abstraction layer and it will run at top speed because it just uses direct PHP database API calls. Possibly, in a future version, client side SQL queries will be replaced by server side stored procedures to match all the potential of the underlying database that support them. Until then, for those that are interested, you may find more information on Metastorage here: http://www.meta-language.net/news-2002-12-09-metastorage.html http://www.meta-language.net/metastorage.html -- Regards, Manuel Lemos