Re: [metabase-dev] pearifying metabase: phase 1

Hello, Lukas Smith wrote: > Minor: > - fetchmode (minor) This can easily be emulated because Metabase supports random access to database rows. > - DSN support (minor) This was planned to be added to Metabase similarly to JDBC and it seems to PEAR-DB. I just disagree that the database name be required in the DSN as it seems that PEAR-DB requires. Metabase lets you change the database to each you want to connect in the with the same driver object instance. Actually you may perform queries to a server that has no databases installed. So what PEAR-DB does? I haven't checked, but does it fails to connect because you did not specify a database name? > - Missing Drivers > Frontbase (minor?: will the frontbase guys write one/modify the current > for us?), > LDAP (minor: I want it myself, but the driver is fairly new and not that > "feature rich" yet, so we should be able to port it) Yes, this is a long shot that in the end you may realize that doesn't quite fit in a database abstraction layer that is supposed to execute SQL queries. > - tableInfo (minor: metabase allready has some of those features) Actually Metabase does not have this yet. I want to add it because I want Metabase manager to be able to reverse engineer schemas of applications already installed by traditional means, to encourage people to migrate to Metabase and use the database independent schema definition. > - flexible default behavior (minor: shouldn't be too big of a deal to > implement) What does this mean? > Small: > - Error support (small?: maybe not a big deal through the use if > customer error handler feature of metabase) Yes, Metabase error handler callback should be sufficient to implement whatever error handling methods provides. Metabase passes more information to the error handler callback than just the error string returned by the Error() driver method. > - transaction (small?: I haven't look at either metabases' nor pear db's > implementation) I don't see a problem here. Metabase eventually will implement more transaction related things soon. > - DB_result object (minor) I think you have all in Metabase that you need to emulate PEAR-DB and probably more. > - error messages (small?) Error messages is not a problem. What Metabase does not provide is error number abstraction. The way I see it, errors are fatal conditions and so in each situation they should be handled the same way regardless of the type of error that happened. So, I never implemented error abstraction because I don't see much point in having a way to distinguish error types programatically. Usually an error string is enough for logging so the programmer can read the exact error message that the database back end returned along with some context information provided by Metabase but all in text because that is what is useful for humans (programmers). Also, it seems to me that error abstraction is quite utopic because an operation can fail in thousands of ways and a database can return different error codes for each so it will be an hell to categorize all the codes even if the driver can antecipate the meaning of all of them. Another thing that I disagree in PEAR-DB design is that certain conditions that are not errors but just status information is abstracted as errors. For those conditions there should be functions to retrieve them. For instance, querying affected rows or if a driver supports a certain capacity. If a programmer calls a function that is not implemented for a certain driver, that is a programming error, not a run-time error. The program should handle that as a fatal error with any need to distinguish what type of error was that. The right way for programmers to handle the (lack of ) ability of a driver to implement a certain capacity is to query the driver before trying to use it. So there is not need for abstracting this type error either. > - getListOf (small?: Thomas was not sure about the status inside of PEAR > of these features) I don't know what is this. > Large: > - LOB support (large?: I haven't look at either metabases' nor pear db's > implementation) AFAIK, PEAR-DB doesn't quite handle LOBs. It simply treats them as normal text fields and so it retrieves data all at once. This is inadequate because if you very large fields you will blow your computer memory trying to store or retrieve all at once. The proper way to handle LOBs is with a streaming interface. PHP lacks of streams abstraction, I mean you sure have streams for files, networking connections, but you can't redefine those streams in a way that you can use the same interface functions to store or retrieve data to them. This is something that is very well done in Java. Anyway, Metabase just provides a set a separate functions to abstract LOB streams. For instance you can execute a SELECT query that returns a LOB field and connect it to an INSERT or UPDATE prepared query that inserts the data in another LOB field. > - API changess like function naming, parameter orders, coding style > stuff (large: a lot of work but nothing too complicated) Forget it for now. Keep in mind that at this point you should be concentrated on making the proof of concept which is wrapping PEAR-DB API around Metabase API and demonstrate that it works as needed so nobody developing PEAR-DB applications need to change their code and the same for Metabase based applications. > Unsure: > - storage.php (?: is this used a lot? Will this work out of the box once > we get API compatibility with PEAR DB? This could basically be > understood as another level of abstraction . maybe this is where the > LDAP handler should actually "attack" rather than as it is now in the > level of PEAR DB core that was meant for relational rdbms) Yes, that seems to be an higher level abstraction that just sits on PEAR-DB API. There is nothing to do about it. > - different ways of how data is stored in the DB for emulated features: > sequences, NULL ..? (?: this is actually one of those most difficult > areas .. hopefully a simply convert script could take care of it . but > its definitely ugly if certain features have been emulated differently > in the db) Yes, personally this is another thing that I quite don't agree on some PEAR-DB implementations. PEAR-DB should not be executing implicit schema management functions on demand because for security reasons it may not be desired that the current application user have privileges to do it. For instance, it is wrong to create sequence emulation tables on demand because as I said the application user may not have privileges to create tables. The proper way to do it is to have separate schema management functions like Metabase provides for things creating/dropping/altering tables/fields/indexes/sequences. Of course PEAR-DB does those things like on demand schema management because it lacks of explicit functions to do so by design. Honestly I don't know what to do about it because there is probably a reasonable number of PEAR-DB users relying on that behaviour. I think that PEAR-DB functions that rely on that behaviour should be deprecated but I'm afraid the problem won't go away doing just that. > Approach: > Dunno if this is crazy, but I would much rather not start off building > wrappers, but to actually work on a merged version that is easy to make > backward compatible with both abstraction layers (again as a warning > this is my first shot at merging two fairly large and estabhlished > projects - especially with opensource you gotta please the public or you > will face forks with would defeat the purpose of this merger - so I am a > bit nervous). I think that is unrealistic. You may waste very easily many months rewriting everything and in the end realize that you may still not make the proof of concept of the merget. I advise that you just stick to the minimum necessary to make the proof of concept. > Probably I will attempt at getting Pears DB.php to play nice with the > metabase_database.php :-) That is more realistic. > PS: For anyone hoping for a faster pace with this development: > unfortunately I do have very limited time to work on this. So any help > is greatly appreciated. That may also be a problem. If you are not able to make the proof of concept soon, the actual merger never start and more releases of PEAR-DB and Metabase are made with new and incompatible feature implementations making the goal even harder. Just try to no wonder too much on things for which you not get quick answers. Regards, Manuel Lemos