Unit tests for QuickForm (was: Re: [PEAR-DEV] [PEPr] Comment on RFC::How the QA Team will work.)

Hi! Greg Beaver wrote:

What method do you use to regression test?

None. :[ We rely on our users' eyes to find bugs. *chuckle*

Larger projects outside of PEAR have no trouble writing unit tests, look at WACT. In addition, it is no more difficult to unit test than it is to write documentation, which both you and Alexey have shown great ability to do for HTML_Quickform.

Well, *the* person to praise here is you. Most of the stuff there is generated by phpDocumentor.

I know it is hard to write unit tests for a large pre-existing package, I've been writing them for phpDocumentor and PHP_Parser. I think everyone would be willing to see incremental additions of unit testing to each release for large pre-existing projects that would be delayed months by unit testing requirements.

I thought about doing unit tests for QF but was scared by the size of the needed work. 1) Element classes. These ones are pretty straightforward, but there is a lot of 'em. There are also elements that rely on JavaScript... 2) Base class. Lots of methods, but still doable. It is easy to fake form submit by assigning to $_GET / $_POST. 3) Renderers. I am not sure how to approach these: keep the files with "expected" output and compare them with the actual? 4) Rules. Easy to test the server-side, but impossible to do JavaScript. I don't actually have much experience with unit testing --- I did the tests for Sigma, but it is *much* smaller than QF. Do you have any specific suggestions?

One of the two serious problems that QA has been formed to deal with are BC breaks/broken releases. API unit tests will kill this one cold, and need not be complex. The verification that method parameters/names haven't changed from release to release is a crucial part of the release process, and would be a lot simpler with both interfaces and unit tests for the interfaces. The other issue is orphaned packages and is not relevant.

I think lack of tests is *the* problem with the proposed QA group. I don't think they are going to write tests --- this is too boring. They will monkey around "fixing bugs". And as there is nothing to test the behaviour, will introduce *more* bugs to the packages: they most certainly know less about the package than its original developer(s).