Re: PHP Data Mining/Data Scraping

THanks, I will have far to much data to append to a GET request so a POST is the best option I think. "Myron Turner" <turnermm02@shaw.ca> wrote in message news:4653133E.3000201@shaw.ca... > >> On Sat, May 19, 2007 10:22 pm, Shannon Whitty wrote: >> >>> I'm looking for a piece of software or coding that will let me post a >>> form >>> to another URL, accept the response, search it for a specific >>> "success" >>> string and then let me continue processing the rest of my program. >>> >> >> http://php.net/curl >> >> >>> I want to accept queries on behalf of my supplier, forward it to them >>> behind >>> the scenes, accept their response and display it within my website. >>> >>> Has anyone had any experience with this? Is there a simple, basic >>> utility >>> to let me do this? >>> >>> I was kind of hoping I could avoid developing it myself. >>> > As I understand this, you want to create a web page of your own which > accepts requests for customers who are going to order products from your > supplier. You want to have a form on your page which accepts their > requests, then forward the form data on to your supplier's web site, where > presumably it will be processed. Then you want to retrieve the response > from your supplier's page, and display the result on your own web page. > You suggest that the response string for "success" is relatively stable > and that this string is this what you want to search for in the response. > > This doesn't sound like a very complicated problem. You can do this > either using Ajax or not. The basic solution is the same. You have a > script on the server which accepts the form data from your page and > re-sends it to the supplier's site. If your supplier's site accepts form > data using GET, then you can simply create a url with the form data > attached in a query string: > > http://my.supplier.com?fdata_1=data1&fdata_2=data2 > > Send this url to your suppler using file_get_contents: > > $return_string = > > file_get_contents("http://my.supplier.com?fdata_1=data1&fdata_2=data2"); > > This will return the html file as a string which you can then parse with > preg_match() for the 'success' string. > The problem is more involved if your supplier doesn't accept GET but only > accepts POST. Then you have to use either curl or fsockopen to post your > data. I've tested the following fockopen script and it worked for me: > > <?php > $fp = fsockopen("my.supplier.com", 80, $errno, $errstr, 30); > if (!$fp) { > echo "$errstr ($errno)<br />\n"; > } else { > $out = "POST http://my.supplier.com/form_page.html / > HTTP/1.1\r\n"; > $out .= "Host: my.supplier.com\r\n"; > > $post = "form_data_1=data_1&formdata_2=data_2"; > $len = strlen($post); > $post .= "\r\n"; > $out .="Content-Length: $len\r\n"; $out .= "Connection: > Close\r\n\r\n"; > > $out .= $post; > > fwrite($fp, $out); > $result= ""; > while (!feof($fp)) { > $result .= fgets($fp, 128); > } > fclose($fp); > echo $result; > > > } > ?> > > You have to adhere to the above sequence. The posted data comes last and > it is preceded by a content-length header which tells the receiving server > how long the posted data is. The returned result is the html page > returned from your posted request. > > -- > > _____________________ > Myron Turner > http://www.room535.org > http://www.bstatzero.org > http://www.mturner.org/XML_PullParser/