Re: testcase repository

Hi I was thinking about the testing of PHP, automating it and such. Here are some ideas and such, all feedback welcome! I based my ideas on an article in the June 97 issue of Dr Dobbs, by Fred Wild. He describes a structure and set of scripts that allow one to drop in scripts to be automatically tested. Have a look at it for further details. The directory structure contains, essentially, four main directories, bin, input, ideal and actual. (see the article for more details). First create a script, that optionally takes some input (maybe a file to read). Put it into the 'bin' directory. Input file into the 'input' directory. Create a file that gives the expected output of the script. Put it into the 'ideal' directory. The script file, input file, ideal file are called by the same name. Now run the main script, which loops through all the scripts in the 'bin' directory and runs them, putting the output of each script into the 'actual' directory. Then run the compare script, which loops through each file in the 'ideal' directory and compares it with the same file in the 'actual' directory. Any differences are then written to another file. There is a bit more to it than that, but that is the main concept. Now, there are some unsolved riddles in this setup... 1. What sort of script to run? At the moment I have been testing by running the php.exe under Windows and passing it the file name. Seems to work fine, but it may not be ideal. What I may do is create a small test harness, that does an HTTP GET to the webserver, passing it the script and then collection the returned page, something like that. Then we are actually testing the server setup. 2. This idea works well for the 'built in' parts of PHP (i.e. testing 'foreach' statement, or reading files etc) but is not practical for testing the modules, as it will need the correct module installation, and MySQL installation on any testing computer. Not an insurmountable problem, but needs thinking about. 3. There are two different sorts of things to test, one is the standard behaviour of the system (i.e. sorting the same array should always give the same result) and also scripts created to resolve bugs. Whether they should be in the same system or not.... OK, thats where I'm at so far. I've got a system running this setup with a few (kinda toy) scripts for testing and it works good. Feedback anyone? -- jalal -- On Mon, 17 Jul 2000 13:28:11 +0300 (IDT), Stanislav Malyshev wrote: >Is somebody going to make testcase repository for regression tests? I >mean, let's asy, one has fixed bug #1234 and has a testcase script for it >(one should, or how else one makes sure it's fixed?). What should be done >with this script? >Just for now I run it, see it works and forget. I'd be very good if >somebody (not me - not time :) would organize some place where such >things could be put and then used for regression tests. > >-- >Stanislav Malyshev stas@zend.com >+972-3-6139665 > > >-- >PHP Quality Assurance Mailing List <http://www.php.net/> >To unsubscribe, e-mail: php-qa-unsubscribe@lists.php.net >For additional commands, e-mail: php-qa-help@lists.php.net >To contact the list administrators, e-mail: php-list-admin@lists.php.net