Re: Generated pages date
| From: | (Stig Sæther Bakken) | Date: | Mon, 14 May 2001 23:03:08 +0000 |
| Subject: | Re: Generated pages date | ||
| References: | 1 2 3 | Groups: | php.mirrors |
| Request: | Send a blank email to php-mirrors+get-4995@lists.php.net to get a copy of this message | ||
Hi,
You should tell the url-minder people that their page comparison is
broken. There's just too many sites out there that have tiny little
differences on each reload, and there are much better ways than doing
a plain binary compare, or having everyone add custom tags to their
HTML.
One suggestion is using a partial checksum algorithm: you split the
document into N parts and make a checksum on each part. If at least
N-M checksums are the same, two documents are considered equal. Try
starting off with N=9 and M=1. It can be tuned by adjusting N and M
or splitting the document in different ways (per-byte, per-line or
looking at HTML structure).
- Stig
[jhlavon@relaxtime.cz]
> Hi,
> oop, sorry, you don't understood me - problem isn't in words
> "generated/changed"
> but in date value after it. Search robot read every night your page, calculate
> sum, and because you generate for every call new date/time on www-page, it
> thinks that page was changed and in "goodwill" it send me message about change -
> every night about 3 a.m. :((
>
> Please, generate what date/time what you will, but use tags "<url-minder-ignore>
> and </url-minder-ignore>"
> or "<mind-it-ignore> a </mind-it-ignore>". This search robot is very good
> service, I will use it also for your webs, alike for ten another ...
>
> Thanks, Jiri
>
> Colin Viebrock wrote:
>
> > Okay, this is done on PHP.net ... hopefully it doesn't break anything, but
> > if it does we'll have to take it out again.
> >
> > Let me know if it works for you.
> >
> > - Colin
> >
> > ----- Original Message -----
> > From: <jhlavon@relaxtime.cz>
> > To: <webmaster@mysql.com>; <webmaster@php.net>
> > Sent: Friday, May 11, 2001 8:29 AM
> > Subject: Generated pages date
> >
> > > Hi, I am using 3rd-party watching service to watch changes on your
> > www-pages.
> > > Because you have on your pages entry similar of "Page generated:
> > 2001/05/07
> > > 16:38:54", I receive change-message every day.
> > >
> > > Please, please, include code for generated date in group of tags
> > >
> > > "<url-minder-ignore> and </url-minder-ignore>"
> > > or
> > > "<mind-it-ignore> a </mind-it-ignore>",
> > >
> > > that robots use for this events.
> > >
> > > Thans you very much.
> > >
> > > Best regards
> > > Jiri Hlavon
> > >
>
--
Stig Sæther Bakken <ssb@fast.no>
Fast Search & Transfer ASA, Trondheim, Norway