Re: rewriting run-tests.php

Ilia A. wrote:

On October 29, 2002 05:22 pm, Shane Caraveo wrote:

2. tests via http. rather than executing php a http request is made.

An interesting idea, certainly would allow us to test the request processing code and so on. However, I am not certain we need to run all the tests from the webserver sapi. IMHO the webserver testing should be limited to output control tests as well as input (FILES/GET/POST) parsing. There is one potential problem that I see with web server tests, this problem being the fact that the PHP's output could be skewed/altered by other modules/extensions running on the server. Making the output of the test unreliable, so you may need to have the test output the results to a file on the harddrive and compare that to the expected data.

Yeah, The scripts could write a file, and echo the filename as a response to the request, then the test system can grab the results from the file. No, not all tests are necessary to run through the web server, unless we're testing for more that just successful test completion.

3. multithreaded testing.

Majority of the problems with ZTS builds are the results of various ZTS wrappers having bugs in them, 99% of those can be resolved by simply adding the --enable-experimental-zts flag and compiling the PHP and then running the test suit on the code.

I don't know why it's still labeled experimental, I guess because it's on linux. This is how it's been compiled on windows since the start of php4. You essentially have to have that flag to have a multithreaded PHP at all. If you don't use it and try entering php with multiple threads, bang! So it's not so much an issue with the wrappers unless they are buggy to begin with, but most of the aparent and bad stuff should already have been caught with the windows builds. The ultimate issues with thread safety, other than misc. bugs in php's core, are third party libraries that may or may not be thread safe, or design issues with specific features in PHP. They need to be identified so that it can fixed or be documented. A thread based test system should go some ways in helping to do that.

As for testing PHP & thread-safety another possibility is to use the embed 'sapi' in combination with a simple c program that would receive a sample test to run as well as how many threads to run the test in. This is probably better then using a PECL module, since it does not add additional dependencies.

I did a similar thing a long time ago, thus the sapi/isapi/stresstest stuff. Shane