AW: HTML_DocumentDependencies / Net_URL problem

Hello Laurent, thanks for your answer... > I'm late to post a comment on this proposal (Status: Called for Votes), > but i think, even if this package is a good idea, its too much > restrictive. > > Try to parse a document that is a dynamic html page (php script) > > ... > <a href="<?php echo $faq; ?>">FAQ</a> > ... You'r right, that would not work ;-) Here's what I thought: - In most cases PHP and HTML are separated, e.g. using e template engine. In this case it's easy to write a script that checks the file(s). HTML_DocumentDependencies would use the html string as input. - There's normally not need to check a document on each request, so performance is unimportant and if there's no better way we can open the file using a HTTP request. - Often (e.g. when using a template engine) there're images in the templates html source and in the content that's handled by our application, so the only way to check all dependencies is to read the output. Here're some use cases: - Check complete site via http. That should be quite easy, because you can get a list of all links and follow them. - Check the complete output of an application or check just the HTML source of your templates (assuming that "interal" Links will be handled an other way). - Read data and write to a database so it's fast to check if e.g. an images is required somewhere before deleting it. > This package does not check validity of such sources(IN), and produced > un-expected results. > Do you have think of that ?! Ok, but I think this class wouldn't be the right place to check validity? It's just intended to read information from something that you would normally send to a browser.. (and imho this should be valid ;-) //host1/parser.php: <?php [..] $uri = 'http://host2/text.html'; $req = &new HTTP_Request($uri); [..] [foreach a tag found in html] { $dest = $tag->getAttribute('href'); $url =& new Net_URL($dest); print_r($url); } ?> > and print_r($url) will give something with > host = host1 > while you open doc(text.html) on host2 Exactly that's my problem. "http://host2/example/text.html" contains e.g. <a href="./anotherfile.html">text</a> This Link refers to http://host2/example/anotherfile.html and not to something relative to the script that's currently running. It's relative to the opened file (using HTTP_Request($uri) and not to the PHP Script that has opened it. Another example: When I open the website of company1 and this document contains an image to <img src="/images/logo.jpg"/> this means this company's logo and not some logo.jpg on my local server from where I started the script. ... and that's my problem: It seems not possible to tell Net_URL something like: "When I initialize you with a relative/incomplete URL, this URL is relative to XY and NOT to this Script". Hoping I'm not simply too stupid... ;-) Chris -----Ursprüngliche Nachricht----- Von: Laurent Laville [mailto:pear@laurent-laville.org] Gesendet: Mittwoch, 25. Januar 2006 16:03 An: Christoph Lütjen - webEdition Software GmbH Cc: pear-dev@lists.php.net Betreff: Re: HTML_DocumentDependencies / Net_URL problem Christoph Lütjen - webEdition Software GmbH a écrit : > Hi! > Hello Christoph, > > > Short time ago I started writing the HTML_DocumentDependencies class which > is intended to have an easy way to parse an HTML file and to get a list of > links, images,.. which could be stored in a database or wherever. > (http://pear.php.net/pepr/pepr-proposal-show.php?id=338) (and to learn how > to add classes to PEAR ;-) > I'm late to post a comment on this proposal (Status: Called for Votes), but i think, even if this package is a good idea, its too much restrictive. Try to parse a document that is a dynamic html page (php script) ... <a href="<?php echo $faq; ?>">FAQ</a> ... and you may have results such as : destination = <?php echo $faq; ?> type = link Another example : Try to parse an Office HTML Word document (its impossible); btw it could be very could. This package does not check validity of such sources(IN), and produced un-expected results. Do you have think of that ?! > > > Now Justin told me to use the Net_URL class to represent urls found in > document. Makes sense, but now I’ve a problem with the Net_URL class… It > seems that it’s not possible to set the “context” of the url, meaning that I > cannot define to which url a relative url is relative. (“./file.html” makes > no sense if I don’t know where “.” is. ) > Justin has right to suggest you to use Net_URL > > > I described this behaviour in a feature request: > http://pear.php.net/bugs/bug.php?id=6541 which was closed as > “Bogus”. So now > I’m a little confused because currently I see no other way than parsing the > (part of an) url and create a full uri if I want to have the right data in > Net_URL object. … Would be great if someone could take a look at it and give > me a hint if ‘v overlooked something… > For your problem, i think it's easy (or i don't really understand your need). Related to Bug#6541 description: Example: http://host1/parser.php opens http://host2/text.html and reads a list of links in text.html. Returns list as array of Net_URL objects. If text.html contains an url like "/link.html" the url object will look like this: Net_URL object [host] => 'host1' ... Should be [host] => 'host2' If you use HTTP_Request, solution is <?php require_once 'Net/URL.php'; require_once 'HTTP/Request.php'; $uri = 'http://host2/text.html'; $req = &new HTTP_Request($uri); $req->setMethod(HTTP_REQUEST_METHOD_GET); $req->sendRequest(); $response1 = $req->getResponseBody(); $dom = new DOMDocument(); @$dom->loadHTML($response1); $tags = $dom->getElementsByTagName('a'); foreach ($tags as $tag) { $dest = $tag->getAttribute('href'); $url =& new Net_URL($dest); print_r($url); } ?> and print_r($url) will give something with host = host1 while you open doc(text.html) on host2 > > > Thanks. > > Chris > > Hope it will help Laurent