Re: An alternate view of PEAR QA
| From: | Lukas Smith | Date: | Mon, 19 Jan 2004 20:25:28 +0000 |
| Subject: | Re: An alternate view of PEAR QA | ||
| References: | 1 2 3 | Groups: | php.pear.qa |
| Request: | Send a blank email to pear-qa+get-380@lists.php.net to get a copy of this message | ||
Jon Parise wrote:
On Mon, Jan 19, 2004 at 10:03:38AM +0100, Lukas Smith wrote:Well no CVS means you dont have access to the history which makes the job harder and relegates QA to a pure "nagging" job. It also makes it harder to handle orphaned packages, but thats another story.The reason why I proposed a different system was not because your proposal is not what we ideally want, but that its simply unrealistic with the means we have today. We dont have enough people that will dedicate their time to PEAR QA. Furthermore packages dont have test suites (and we havent made that a requirement either). Packages dont even have to reside in any CVS atm. Thats why I dont see that we can really monitor the changes, test the changes and provide feedback within a reasonable timeframe.My proposed system intentionally has no connection to CVS. That's why I emphasized the fact that the QA process is release-based. From my original email:In order to achieve the best results, quality assurance should operate on a per-release. QA members should not be concerned with development-specific systems such as CVS. Instead, their view into the package world should focus on pearweb's interface.(http://news.php.net/article.php?group=php.pear.qa&article=368)
The system I proposed is more geared to be realistic solution and not so much the ideal. However we shouldnt give up that ideal and maybe one by one we will be able to increase our standards to also require test suites, everything in (atleast synced into) PEAR CVS and tons of willing and able people that will make sure that this QA process doesnt turn into a hassle but more into a great source of people making sure that your package works as intended.
I'm not trying to be unrealistic, but I don't think there's another way to perform proper quality assurance.Well I fear you are. However I wish you are not :-) I dont know if trying either system will hurt in the long run. Like trying my proposal will make moving to the other system harder (eventhough its the ideal) or if your system will turn into a fiasco that will hurt PEAR and the general interest in PEAR QA. Maybe its also not that big a drama either way.
For reference, here are the goals (http://qa.php.net/projects.php) of
the PHP QA team:
- Release Candidate Build
- Automated Smoke Tests
- Automated QA Checklists
- Automated Regression Tests
Well php has a lot of differences.
#1 they are much more closed in terms of adding new functionality
#2 they are more open in regards to other people touching other peoples code
#3 they have fairly long release cycles
#4 they are totaly undermanned
#5 their tests hardly cover all the parts of php
If we're really just talking about forming a group whose job is to be on alert for potential problems that occur in released packages, it's no longer a quality assurance group; instead, it's a live maintainance group.true .. QA sounds better though :-) maybe we should look at the stats of how many releases we are seeing today and how many releases we see in the future. also we should check how many packages have test suites (and how much they actually cover of the API). finally we should consider how long QA'ing a release should take. maybe we need a survey that all devs need to fill out. regards, Lukas Smith smith@backendmedia.com _______________________________ BackendMedia www.backendmedia.com berlin@backendmedia.com Linn Zwoch Smith GbR Pariser Str. 44 D-10707 Berlin Tel +49 30 83 22 50 00 Fax +49 30 83 22 50 07