Re: We Want Quality

At 01:44 PM 6/29/00 -0500, Andrei Zmievski wrote:

On Thu, 29 Jun 2000, rasmus@lerdorf.on.ca wrote: I don't think shutting down CVS for 5 days is an option. There are too many components at too many different stages of development to make this work without annoying the hell out of the developers. Perhaps, "shutting down CVS" was a bit strongly worded. I think it would also be fine if the maintainers of non-essential extensions kept working on them (whlie it still would not reflect well on PHP if it was released with broken domxml or other extension). But, during RC cycle we should limit commits only to serious bug fixes and then start another RC after a few of them.

Again, having a feature freeze is good, but the discard_path fix I put into the CVS was actually a serious bug fix and needed to go in. We posted RC's and builds but not enough people were testing them. If we can get an organized test group that would be good.

I do think that mainline changes (stuff that is critical for PHP to run at all) just before a release need some serious review and should probably cause the release date to be pushed back until the new RC candidate has been out there for a couple of days. Agreed. The real issue here is that the release candidates are not tested on enough platforms and enough different environments. I would much rather start a process by which we never release something until the release candidate has passed a checklist. Something like:

Apache/Linux/Mysql                    y y y
Apache/Linux/pgsql                    y y y
Apache/Linux/Oracle                   y y y
Apache/FreeBSD/MySQL                  y y y
Apache/FreeBSD/pgsql                  y y y
IIS-CGI/Win32/MSSQL                   y y y
IIS-CGI/Win32/Oracle                  y y y
IIS-ISAPI/Win32/MSSQL                 y y y
IIS-ISAPI/Win32/Oracle                y y y

I realize it may not be feasible to get 3 people for each of the Windows configurations at this point, but if we went through a checklist like this on a release candidate and we get 3 people that check off each of these environments I think it would at least eliminate the really silly mistakes. Any code changes would require the RC to be rebuilt and for this checklist to be cleared. The release should then be generated from the RC CVS tag so the release is exactly like the RC that passed. This would be good too. And we could do it for more modules than you listed if we got more volunteers. We could even have a chart of which modules are currently stable and which are not (sort of like Tinderbox in mozilla.org). Regression test suite is sorely missed too.

Setting up people for testing is a good thing too but to be quite honest what concerns me more is the bugs in the bugs database. We don't have enough people capable of closing bugs. Testing is just the first step, someone actually needs to close bug reports. Most CVS commiters add new stuff and very few actually deal with bug reports. Let's also try and address this issue while we're at it although I think it's going to be hard to force volunteers to work on the bug database :) Andi --- Andi Gutmans <andi@zend.com> http://www.zend.com/