Re: An alternate view of PEAR QA
| From: | Jon Parise | Date: | Sun, 18 Jan 2004 22:54:29 +0000 |
| Subject: | Re: An alternate view of PEAR QA | ||
| References: | 1 2 | Groups: | php.pear.qa |
| Request: | Send a blank email to pear-qa+get-370@lists.php.net to get a copy of this message | ||
On Sun, Jan 18, 2004 at 11:26:33PM +0100, Klaus Guenther wrote:
> > To that end, the QA team's job starts just after a package release is
> > submitted by a developer. The release is marked as "pending approval
> > by QA" and won't become a public release until the QA team blesses it.
> > The QA team has the ability to fail the release, essentially sending
> > it back to the developers for improvement.
>
> This is very fine, but is based on the assumption that we'll have a large QA
> team that can act fast. Based on the current number of releases, this would
> probably hamper PEAR growth. I'd hate to get into a situation where a
> maintainer would have to wait a month before his/her package is accepted.
I see your point, but I still think this is the correct approach. A
lot of the proposed QA process could be automated to make it more
manageable.
> Of course, there could be a PEPr-like system which would set a
> one-week limit and if there was at least one positive vote, it would
> pass. If no one votes, the system would send hourly reminders to the
> QA core group (individual email addresses, of course, not the list)
> :-D
I think we can explore these kinds of changes if we find problems in
this area. Your suggestion sounds reasonable.
> > Developers will be able to mark special "checkpoint" releases that are
> > sent directly to the QA team for testing and feedback. These releases
> > will never become public.
>
> I don't like this. QA shouldn't take over the function of the pear-dev list.
I suppose I never considered testing one of the functions of the
pear-dev list. Up to this point, mosts of the QA-oriented functions
have been happening there, but it's my hope that this (pear-qa) list
will assume more of that role so that pear-dev can focus on package
development (and make the posting volume more manageable).
I'm not wed to the "checkpoint" release idea, so lets strike it from
the proposal. I merely included it because it seemed like a nice side
effect of my proposed organization.
> > - Its ability to pass its tests. If the QA team finds the tests'
> > coverage lacking, they may submit new tests to the developer for
> > the inclusion in the package.
>
> LOL! I'd love to have someone do that for my packages. If someone out there
> has time, it would be a great thing. However, don't count on QA for writing
> those things -- that's up to the package devs and volunteers on pear-dev.
Fair enough, although I don't think I was clear. I wasn't expecting
the QA team to write tests. I was merely suggesting that they should
have the ability to improve existing tests or submit news tests back
to the developments. I wouldn't consider authoring tests scripts as a
function of our QA group (although it is a core function of many QA
groups).
> > - Its adherence to the established coding standards.
> > - Its backward compatibility.
> > - The quality of its documentation.
>
> Why not create a PEPr-like system for documenting these things for each
> release? The developer would check all boxes that apply (for example, check
> the box if unit tests are missing, etc.).
That's basically what I'm suggesting, although I don't see the point
in submitting releases that are missing unit tests, for example. I
believe everything I listed above are official requirements for
packages these days. If we were to follow the letter of the law, no
package would be released without tests and documentation.
> > This proposed system aims to prevent "bad" packages from ever being
> > released into the wild. Unfortunately, mistakes do happen, and we'll
> > probably have to adopt most of Lukas' proposals [1] to deal with those
> > kinds of situations when they occur.
>
> But what about the dev releases? It would be very sad (and make a lot of devs
> mad) if dev releases couldn't be made (I'm referring to all releases with a
> status below "stable").
That's a good point. I only intended my proposal to apply to stable
packages. The complement is that a package couldn't become "stable"
until it passes through the QA process.
> Speaking of which, it would be possible to apply your approval idea to stable
> packages, allowing all packages below that to pass through unhindered. This
> would prevent incidents like Console_Getopts a little while back.
--
Jon Parise (jon@php.net) :: The PHP Project (http://www.php.net/)