Re: An alternate view of PEAR QA

From: Date: Sun, 18 Jan 2004 22:54:29 +0000
Subject: Re: An alternate view of PEAR QA
References: 1 2  Groups: php.pear.qa 
Request: Send a blank email to pear-qa+get-370@lists.php.net to get a copy of this message
On Sun, Jan 18, 2004 at 11:26:33PM +0100, Klaus Guenther wrote: > > To that end, the QA team's job starts just after a package release is > > submitted by a developer. The release is marked as "pending approval > > by QA" and won't become a public release until the QA team blesses it. > > The QA team has the ability to fail the release, essentially sending > > it back to the developers for improvement. > > This is very fine, but is based on the assumption that we'll have a large QA > team that can act fast. Based on the current number of releases, this would > probably hamper PEAR growth. I'd hate to get into a situation where a > maintainer would have to wait a month before his/her package is accepted. I see your point, but I still think this is the correct approach. A lot of the proposed QA process could be automated to make it more manageable. > Of course, there could be a PEPr-like system which would set a > one-week limit and if there was at least one positive vote, it would > pass. If no one votes, the system would send hourly reminders to the > QA core group (individual email addresses, of course, not the list) > :-D I think we can explore these kinds of changes if we find problems in this area. Your suggestion sounds reasonable. > > Developers will be able to mark special "checkpoint" releases that are > > sent directly to the QA team for testing and feedback. These releases > > will never become public. > > I don't like this. QA shouldn't take over the function of the pear-dev list. I suppose I never considered testing one of the functions of the pear-dev list. Up to this point, mosts of the QA-oriented functions have been happening there, but it's my hope that this (pear-qa) list will assume more of that role so that pear-dev can focus on package development (and make the posting volume more manageable). I'm not wed to the "checkpoint" release idea, so lets strike it from the proposal. I merely included it because it seemed like a nice side effect of my proposed organization. > > - Its ability to pass its tests. If the QA team finds the tests' > > coverage lacking, they may submit new tests to the developer for > > the inclusion in the package. > > LOL! I'd love to have someone do that for my packages. If someone out there > has time, it would be a great thing. However, don't count on QA for writing > those things -- that's up to the package devs and volunteers on pear-dev. Fair enough, although I don't think I was clear. I wasn't expecting the QA team to write tests. I was merely suggesting that they should have the ability to improve existing tests or submit news tests back to the developments. I wouldn't consider authoring tests scripts as a function of our QA group (although it is a core function of many QA groups). > > - Its adherence to the established coding standards. > > - Its backward compatibility. > > - The quality of its documentation. > > Why not create a PEPr-like system for documenting these things for each > release? The developer would check all boxes that apply (for example, check > the box if unit tests are missing, etc.). That's basically what I'm suggesting, although I don't see the point in submitting releases that are missing unit tests, for example. I believe everything I listed above are official requirements for packages these days. If we were to follow the letter of the law, no package would be released without tests and documentation. > > This proposed system aims to prevent "bad" packages from ever being > > released into the wild. Unfortunately, mistakes do happen, and we'll > > probably have to adopt most of Lukas' proposals [1] to deal with those > > kinds of situations when they occur. > > But what about the dev releases? It would be very sad (and make a lot of devs > mad) if dev releases couldn't be made (I'm referring to all releases with a > status below "stable"). That's a good point. I only intended my proposal to apply to stable packages. The complement is that a package couldn't become "stable" until it passes through the QA process. > Speaking of which, it would be possible to apply your approval idea to stable > packages, allowing all packages below that to pass through unhindered. This > would prevent incidents like Console_Getopts a little while back. -- Jon Parise (jon@php.net) :: The PHP Project (http://www.php.net/)

« previous php.pear.qa (#370) next »