Re: Finding out when a Web page has changed

>> I need to write an application that accepts a list of URLs and checks them >> on a daily basis (via cron) to see if the pages have changed in the past day. >> >> I need some help with this. Does anyone know the most optimal way to find >> out when a particular Web page has been modified? I am thinking about using >> the Last-Modified: HTTP header - however, all servers do not return this >> header - any ideas on what the fallback should be? > >You could calculate and store the MD5 hash of the page. If the hash is >different the next day, you know the page has been modified. Does this mean that I need to read the entire contents of the HTTP stream into a variable, calculate the hash and store it for comparison? Or is there an easier way to get the MD5 hash? Basically, I'm wondering if there is a way to do this without having to read the entire URL contents via HTTP. Vikram