Re: [RFC] - How QA Team will work.

Hi Helgi, Thanks for getting this QA thing rolling :-) I've commented the proposal where I feel it needs ammendment, clarification or simple spelling fixes. So here goes... From: "Helgi Þormar" <helgi@trance.is> > QA Group Membership: > Have a limited number (no more than 5) core QA people with extended > QA-karma I fully agree that QA needs karma, and that it should be available to a limited number of people, but 5 seems to be a bit small -- even the PEAR Group has 7 members... How about 10 people for QA core, to be extended as needed (max 15 with core karma)? My reasoning: A small QA core team will be unable to appropriately respond 24/7 to issues such as pulling a borked release. Therefore, it is desirable to have core members around the globe. Less than one person per continent seems to me to be too great a limitation. I fear that if the group is too small, there will only be 1 or 2 people initially on the team, because there would need to be room left for members in other areas of the world. Instead, I would suggest 10 or 15 people, of which, perhaps, the largest group will be located in Europe (probably 50%). > New members must be voted into the QA team and don't get any > special karma, but the mailing list will remain open of course. What happens when someone leaves the QA team? The current document does not provide for someone taking their place (and getting appropriate karma). > New member as to get 2/3 or 3/4 of the votes in his favor to get in > the team, this is to prevent that the member will have anyone > against him and making it a possible flame war or any such with in > the group. 2/3 of voting members (abstainees don't count). The vote needs to have a time frame. We could use the same time regulations as the PEAR Group (Five days -- see http://pear.php.net/group/docs/20040322-vm.php ). > Decisions effecting a specific package which requires the karma > level of the core team can also be voted on by the entire team, but > ultimate decision is up to the QA core team. s/effecting/affecting/ This assumes that we are talking about something like modifying a package for which we have no permission. However, removing a release is time sensitive and cannot wait for a vote. > If any major QA things are found in any package and the QA team > doesn't have permission to change that package with out contacting > the lead first then the QA team will file a bug report. > Any major issues can stay open for 1 months (2+1+1 weeks) until QA > may fix it themselves. This assumes that the affected release has been pulled. That needs to be mentioned somewhere in this process. > QA has to keep track of those QA issues that are in bug report some > how, with a wiki of some kind for example. > This would only be for having overview over which package > maintainers need to be email for reminders and such things. Eventually this can be automated using PEPr. There could be flags for QA bugs, etc. with automatic reminders (based on severity) and an email to the QA group once that time has passed. > Also QA must get the permission to delete any release that might > have any issues that have been found after the release to reduce > the effects of the problem. s/must get/has/ > QA will be given the right to make release for packages that have > gotten QA fixes in CVS but haven't released anything month from the > time the fixes were added. > Package maintainers can also grant QA the rights to make a release > if necessary. "but haven't had a release in one month after the date of the last QA CVS commit"? > Summarize of the solution s/Summarize/Summary/ > QA core group access right: s/right/rights/ > * Max 5 core members that get the right karma. > * QA will get more permission to modify packages that haven't > given QA permission to do such things, the modify rights are > granted based on if the package leads don't react with the s/don't/doesn't/ > problem with in specific time frame > * QA core gets the rights to delete and make release of > packages if needed s/make release/roll releases/ > QA team in general: > * Rest of the QA team will identify QA problems, help writing > QA related documents, helping people resolve their QA > problems and such things and same goes with QA core members. s/goes with/goes for/ > * Forming of the QA team, voting which persons should be in > the core of the QA team. > * Giving the core dudes full CVS karma on the pear CVS module, > as well as full access rights to package management part of > pear web? All authors already get access to /pear and /pear-doc repositories. The QA core team could additionally have access to /php-src/pear just in case something needs to be modified there, but I'd not suggest doing anything about that, because it could become very hairy. Better let the people who know what they are doing commit the patches there. Additionally, the core should have /pearweb access to the pear.php.net/qa section. > * Getting list of who want to be in the team and vote which of > them will be in the team. This will be a difficult point, especially since the seats in the core team are limited. Perhaps the method for selecting the initial members should be more precise. If there are more applicants than seats, there should be an open election, where everyone may cast as many votes as there are seats (only 1 vote may be cast per candidate). Of course, if there are less people than seats, all may be accepted and the remaining seats can be filled by QA group vote. > * Finding out which packages lead allow QA to help keeping > their package QA approved Lukas already started such a list. > * Better coordination of the work which is done, who is going > to do what. Know before hand who is going to write some more > entries to the manual and so on. s/before hand/beforehand/ (or more simply: in advance) > * Make a wiki page of some sort for the info tracking > regarding QA editing package that QA doesn't have permission > to edit. Sounds sort of like a changelog to me. Perhaps it would be a simple database table, where the QA team can add a comment that would be impossible to correct. However, I think it is unnecessary (and possibly unwise) not to trust the QA team with editing such a list, when they have such great powers. For example, what if a typo is in it, or what about someone who speaks english natively correcting it so that it is understandable. (While almost all PEAR developers speak English well enough to communicate intellegently, there are occasions when even native speakers don't express themselves intellegently -- if that is left unfixed, it makes PEAR QA look unprofessional.) Klaus