Re: Unit tests for QuickForm

Alexey Borzov wrote:

1) Element classes. These ones are pretty straightforward, but there is a lot of 'em. There are also elements that rely on JavaScript... 2) Base class. Lots of methods, but still doable. It is easy to fake form submit by assigning to $_GET / $_POST. 3) Renderers. I am not sure how to approach these: keep the files with "expected" output and compare them with the actual? 4) Rules. Easy to test the server-side, but impossible to do JavaScript. I don't actually have much experience with unit testing --- I did the tests for Sigma, but it is *much* smaller than QF. Do you have any specific suggestions?

In the case of javascript, I don't see any immediate solution - this is an example of an external requirement that should not be ignored when talking about unit testing. The javascript is a small portion of the package, fortunately, so the best you can do is to verify that the javascript created is what you expect it to be, in other words that the PHP code that generates the javascript generates what you expect. This will at least catch any mistakes in the generation, if not in the actual javascript. I would suggest getting an easy to use testing interface. I have stolen Lorenzo's testing framework based on PHPUnit from HTML_CSS for the unit tests in PEAR_PackageFileManager, PHP_Parser (cvs) and Games_Chess. I was not familiar with SimpleTest when I started unit testing, and you might want to go with that route first, as it will be easier with your complex external requirements, especially the mock objects. If you do go with the pre-made PHPUnit stuff, I would recommend taking a look at Games_Chess and PHP_Parser as these are newer versions, and I modified the output of the decorator class so that it is easier to view large serialized arrays that differ in a web browser. I coded a lot of helper methods that are useful for catching PHP errors, PEAR Errors, and you'll find yourself coding specific methods over and over again in your testing, like populating the $_GET/$_POST or simulating some other aspect of the environment. I think with renderers, you would probably benefit from using a mock object idea. In other words, create an object that extends the renderer, so that internal state can be examined properly. class test_thismethod_default_renderer {

    function _test_getState()
    {
        return $this->whatever_you_need;
    }

} in the test case, you would do something like this sequence: function test_thismethod() {

    $testrenderer = new test_thismethod_default_renderer([params]);
    $this->assertEquals(expected value of internal state pre-method, $testrenderer->_test_getState(), 'is the initial state right?');
    $testrenderer->thismethod([params]);
    $this->assertEquals(expected value of internal state, $testrenderer->_test_getState(), 'did the method affect the state as expected?');

} In any case, the most important thing you can do to begin is to get a testing framework that involves 1 click, or 1 command-line run. I prefer a click because I tend to be more visual-oriented, but you might prefer something else. Incidentally, "testsuite.php" is the main testing file, I have been too lazy to rename it index.php or something more easily identifiable. The second most important thing to do is start with the most atomic element and build complexity outwards. Test the methods/classes that have the fewest internal/external dependencies first, otherwise you'll probably go insane :).

I think lack of tests is *the* problem with the proposed QA group. I don't think they are going to write tests --- this is too boring. They will monkey around "fixing bugs". And as there is nothing to test the behaviour, will introduce *more* bugs to the packages: they most certainly know less about the package than its original developer(s).

I agree, also because writing tests requires 1) an intimate knowledge of the API 2) an intimate knowledge of the package 3) karma to commit to the tests directory, and permission to fix bugs found. This last one makes it especially difficult, as you might have a bug in the *test* instead of the code :)