Re: Extracts from the 'We Want Quality' thread

At 02:17 PM 7/1/00 -0500, Richard Lynch wrote: >> 4) Discuss strategies to improve response to trouble >> reports on the list(s). The current method of "pick >> what interests you" seems pretty haphazard. Perhaps >> organizing the incoming stuff by platform/OS and >> dividing them up chronologically? > >Actually, this "haphazard" strategy is one of the strengths of OpenSource -- >People are far more committed and concientious working on what they enjoy >rather than what is assigned. Good point, but what if what people like doing is mostly beauracratic meddling - there should be a place for people like this in open source projects... ;) > [Think not? How often have you blown off >work in favor of doing some hacking not all that different, but more fun? >Thought so. :-)] > >Of course, there's always the stuff left over that doesn't interest anybody. >[EG Anything involving Windows :-)] > >This is why the Core Developers generally deserve to be the "benevolent >junta" that they are, since they pick up the pieces nobody else wants. > >For a QA team, I would suggest a synthesis of the two strategies -- Let >people pick what they want, and then start assigning the "boring" stuff >based on skills/experience/whatever. Otherwise, you are sapping the >strength of OpenSource with over-management. This sounds like a good approach. >The other problem with having a single person responsible (primary contact) >for a given OS/platform/whatever is that they're bound to be too busy, or on >vacation, or ill, or... *SOMETHING* always happens to a single point of >failure to make it fail. A setup like this gives us an awful lot of single >points of failure, and then PHP can't be released until something is done >about it when (not if) it happens. Not an ideal setup. True enough - I was going to be working on the application to track developers this weekend, but Canada Day festivities and one or two little surprises go in the way. Several people have suggested that we have as many testers as possible - a few extra pairs of eyeballs certainly couldn't hurt the testing for a platform. >>2) The contact person should be committed to fully >>installing the new release on her/his platform. >>She/he should find a user of the same platform who >>is *new* to the installation (or relatively so, >>after all we want everyone to be using it 8^). You >>learn *important* things from a pair of fresh eyes. >>The newbie should also be committed to installing >>and testing, collaborating with the primary contact >>as required, who can shuttle information back to the >>development team. > >You needn't always have somebody "new" looking at something. No matter how >long I've been using PHP, I'm still a pretty good idiot at installing :-) >Alas, I don't want to commit to installing frequently, when I know I won't >have the time nor inclination... At any rate, there are people who have a >flair for looking at things as if they had not ever seen them before. I would think that as long as the testers keep track of any extra, undocumented steps or inconsistencies in the install, we should be fine. >It might even make more sense to *not* have a person assigned to a given >platform: Let people pick what they want -- But keep a record of what's not >picked, and have that be what's displayed as "Needs Doing". Hopefully the tester tracking app. should mostly take care of that. We will be able to see who is available and what platform(s) they have access to. >Some of the *best* people for a "fresh" set of eyes are ones who install >frequently on some *other* platform. They're the ones who will spot >inconsistencies and lame design the fastest. > >> (Rasmus) >> I would much rather start a process by which we never >> release something until the release candidate has passed >> a checklist. Something like: > >The chart thing is a no-brainer: Except that it become geometrically, >recursively complex with each wrinkle: >o each php.ini setting >o every compilation switch (worse for multi-value switches) >o every module (MySQL et al) > . every module configuration/ini setting > . every module compilation switch > . every module sub-component that affects it (SSL & MMAP is an example, >I think) > every sub-component configuration/ini setting > every sub-component compilation switch > every module sub-sub-component > . > . > . > >Obviously we can't follow this down forever, but there are definitely bug >reports I'm facing that knowledgable, educated, we-had-to-patch-PHP users >are reporting that should have been caught. (EG: --enabled-memory-limit) > >I think one of our first tasks should be to try to enumerate the entire >space, and what sub-set of it is realistic to handle, and what sub-set is >most critical. Hopefully the most critical and the realistic will more or >less match up... This is a good idea - perhaps we should just try to clear out as many existing bugs reports as possible first. I would think that this is the easiest path to take to start with. >I suspect these knowledgable users posted their problem and solution all on >PHP3 (nka PHP-General): While we all know they should have put it at >bugs.php.net, this probably indicates that we need somebody monitoring >PHP-General (et al) who has a good over-all knowledge of what's in the bugs >database, and can pull from PHP-General and push into the database -- or >contact "violaters" to gently persuade them to do so. I try to do this, to >some extent, but am not knowledgable enough to do it really well. > >> (Zeev) >> Zeev points out that QA is tedious, thankless and >> unispiring work... >> -- perhaps because of this, we should make sure that the >> QA team understands how important this kind of work is! > >Ideally, OpenSource means that people who at least have some interest are >doing this, rather than people who hate it... Or at least, even if they >don't care for the work, they are already convinced of its importance. > >I guess my basic thesis throughout is that it's more important that we focus >on publically documenting what's done and not done and needs to be done, >than worrying about who will do it. The nature of OpenSource will take care >of the rest. I think that this is what we are trying - however, - as predicted - interest seems minimal. >OTOH, I'm mostly kibbitzing here, and will probably end up doing little of >the actual work: Feel free to ignore me :-) - Zak