Re: Quality control of PECL modules
| From: | Wez Furlong | Date: | Thu, 23 Feb 2006 03:57:47 +0000 |
| Subject: | Re: Quality control of PECL modules | ||
| References: | 1 2 3 | Groups: | php.pecl.dev |
| Request: | Send a blank email to pecl-dev+get-3529@lists.php.net to get a copy of this message | ||
Maturity:
Maturity doesn't necessarily tell you anything about the code, except
for its age. For instance, AT&T libc is mature, but still has the
fopen() bug when your process opens a large number of file
descriptors. For me, maturity is implied by how long the code-base
has remained largely unchanged and the size and frequency of bug fix
commits.
I think maturity is hard to measure and hard to interpret, and that it
might be best left to the user to look at the history of package
releases. To help them decide this, it might be nice to tie the bug
tracker back to a package and display the number of bugs filed, fixed
and bogus'd against a given release. It would also be nice to
summarize what percentage of the codebase changed versus the prior
release; I don't know if there is a convenient tool for that.
Stability:
We could follow the lead from the Sun folks; they make a point of
marking up the documentation of their various APIs and tools with
attributes defined in the
attributes(5) man page
(http://docs.sun.com/app/docs/doc/816-3328/6m9k8pg0c?a=view)
One of these attributes is "Interface Stability", which clearly
defines levels of stability. We could adopt this nomenclature in our
documentation and package releases.
Test Suite:
I believe that Marcus and John's work on code-coverage builds and runs
the tests for as many extensions as he could get running. If this
includes the pecl extensions too, it would be nice to capture the
results and summarize them; we'd see the pass/fail rate along with the
code-coverage of the test suite. This will allow people to make an
informed decision about the test pass rate--extensions with a simple
hello world test will have 100% pass rate, but less than 1% coverage,
which tells you that the test suite on its own is no indication of the
quality of the extension.
Active Maintainer:
We already track the maintainers on the pecl package pages. We
distinguish between "lead" and "regular" developers; "leads" are
effectively the active maintainers. I think we just need to make it
clearer what that means.
In General:
These things sound really nice to have, but do raise the amount of
development overhead; I'm not sure we have the resources to properly
maintain them. Whatever we decide needs to be balanced against the
amount of effort it takes to implement them.
--Wez.
On 2/22/06, Andi Gutmans <andi@zend.com> wrote:
> Hey,
>
> I got lots of question about which PECL modules are stable and which aren't.
> Any interest of coming up with some system where we give official
> marks to these modules as to their stability?
> We could come up with criteria which include:
> a) Maturity
> b) API stability
> c) Test suite
> d) Active maintainer
> etc...
> I haven't thought it through too much but I think we could create
> some mechanism which would work. Having users vote would be another
> valuable input but I don't think it'd be enough. We might want to
> have 3 or four different ratings such as "very stable", "pretty
> good", "not thoroughly tested", "crap/unfinished" :)
>
> Any thoughts?
> Andi
>
> --
> PECL development discussion Mailing List (http://pecl.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>