Re: cvs: docweb /scripts checkent.php

Nuno Lopes wrote:

I think that sleeping 7 seconds is too much. The script will take about 1 hour to run. Do we need to sleep() so many time?

Ah, now that you point that out (and I think of it, more) I agree. Additionally, most links are not on the same server, correct? Therefore different servers wouldn't even know about our sleep.

The user agent is necessary because I've seen one server rejecting the request because of the non-existence of the user agent (!).

Right. Omitting UA is bad. Sending our own UA is good -- a webserver that has a whitelist of UAs is going to deny a significant part of the internet population, anyway.

I think we don't need to over-complicate things, requesting the robots.txt file, etc... Using a generic UA is enough.

Requesting (and honoring) robots.txt, IMO, is necessary. We're running a non-human user agent, here. But I don't see any hard rule about this on the W3C site, so.. <shrug> (-:

This script probably doesn't have interest for translations teams. Maybe we could restrict it to 'all' and 'en' languages and leave it as-is.

You're right. "all" is "en", BTW.. unless the default language ever changes (which I doubt). ... Vincent's reply came in at the end of me writing this, but his points were very similar. S