Re: Suggestion: The mirroring array

From: Date: Fri, 08 Aug 2003 10:13:22 +0000
Subject: Re: Suggestion: The mirroring array
References: 1  Groups: php.mirrors 
Request: Send a blank email to php-mirrors+get-19978@lists.php.net to get a copy of this message
As of today, mirrors are given a specific subdomain to host (such as ar.php.net). The problem with this concept is that quite often mirrors get either outdated, dead, unreachable, and so on, and people who are "aware of the mirror", either by directly accessing to it, or people linking to the mirrors directly, instead of the main website, get simply nothing when such thing happens.
If a mirror have serious problems, our system disables it right away, and we see those problematic mirrors. If you visit the mirror management page, you can see that we have very few mirrors with long lasting problems. Most of the problems are only affecting mirror acess for a few minutes/hours.
Disables it = "removes it from mirrors.php". I believe I was talking about the fact that EXISTING links to the mirrors, or people who use the mirrors directly, will get a connection refused/bad mirror. What the script currently disable *does not* stop this.
I have tried to write it down here, but it seems it did not got through... The automatic redirection is not done to disabled mirorrs, and other mirror problems are only there for a few minutes/hours. I have recevied hourly reports of mirror checks for some time (now I have set it to send it me daily), and I have always seen the same mirrors having problems...
Instead of having each mirror under a specific "number" in an array of a country, install all the mirrors as "countrycode.php.net", and use a smarter system than the current one, that will look up the IP of the CNAME that people give every once in a while (a day, for instance), and then try to contact all the mirrors according to the IP against the countrycode name. Given the "the mirror on this IP is working" status code, generate the subdomain of php.net with that IP, too. Given *no* IPs are return to have positive responses, point the subdomain to the PHP.net master mirror (or any other US mirror, whatever is decided).
Do you mean this on the DNS level? That was discussed heavily before...
Hmm yeah, I meant that in the DNS level... given the fact that since master.php.net is already messing with the zone file by itself, I don't see the problem in the mirrors script updating the zone...
IMHO James or Jim came up with this idea before, and it turned out, that DNS caches are a problem in this case. If we set the DNS data to expire in a short time, then our DNS server will be overwhelmed. Maybe they will come in and add their findings here... If there would be a searchable archive of the ML, I would suggest you search there. Maybe download the messages with nntp, and search in them... You'll find this thread.
This new concept also helps spreading the load on countries who has numerous mirrors (i have a reason to believe that currently only the first mirror is utilized).
One mirorr is selected randomly from the same country, see the code yourself (include/loadavg.inc)...
Again, I was not talking about people being directed from the main PHP site.
You said that you have reason to beleive that currently only the first mirror is utilized. Do you think that people use the first mirrors primarily, they go directly there?
I believe that the target should be to use *as less as possible* the www.php.net server, up to reaching up a point in the future when people from outside USA will not use it at all (as is done in lots of big projects, one of them being the well famous Akamai.net...)
Well, this would be nice. I am not a system admin expert though, so I can only provide opinions. I have no shell access to the machines, nor do I would like to get one ;) Goba

« previous php.mirrors (#19978) next »