RE: Manuals and Spiders

From: Date: Wed, 23 Apr 2003 17:41:48 +0000
Subject: RE: Manuals and Spiders
References: 1  Groups: php.mirrors 
Request: Send a blank email to php-mirrors+get-16957@lists.php.net to get a copy of this message
> I think it would be doable to request the database from your > server from > time to time with a cronned PHP script. Then it can do any > "fixing" in > case you won't provide us exactly what is suitable... I think a cron is a good idea - at least to start off this project. I can do the fixing at our end, you just need to define the exact format that you want. What we will do, is give you a url like http://www.directi.com/iptocountry.php?site=php.net&dbformat=1&lastupdat e=unixtimestamp to update the db. This will give you the entire db in the exact format that you want. This way the fixing code does not need to be there on every mirror. The db is fixed as per php.net's requirements on our server itself. Also, if we ever update the db in a different format - in case we start giving more details than what exists in the current db - we would just need to update the script at a single location. This will ensure that php and all its related sites would always get the db in the correct format. > Every piece of our current system works with three letter > country codes > for example (mirror images, mirror country specs). So IMHO it > would be > the best to have this list with three letter codes, instead of two > letter ones. > > Staying with the textfile based model, it would probably be > the best to > have fixed width fields (IP numbers padded with zeros at the > left), so > we can build up a small index to help seek in the file. Then > there won't > be any need to have more then one data file. The index would help us > seek near to the place where the IP info is stored, so we > only need to > check some IP ranges there, near to the IP... > > This may lead to a significantly bigger file because of the padding > zeros and the three letter country codes, but we can leave out the > commas in this case (and the country name is definitely not > needed as we > have a mapping array for this: > http://www.php.net/include/countries.inc ) > Just give me a sample of the format that you want - by changing 2-3 lines of the current db as per the new format and sending it across to me / on the list. Once I have this, we can easily write a script to convert the current db format to the one you want. We will then provide you with the url that you can put in the cron to get the db automatically. The params to be included in the update url are as below. Let me know if any one can think of any more essential params, or if anyone thinks a particular param should not exist: http://www.directi.com/iptocountry.php?site=php.net&dbformat=1&lastupdat e=unixtimestamp Site - would tell us which group of sites is requesting this data. So all php related websites would send this as php.net Dbformat - in case we start giving additional params that you want to use, we just need to update the version number. This will allow us to maintain a single script for all formats. We will provide the update as per the format requested. Lastupdate - we can return a "no update", in case there is no updated db since the last update date. The cron on the server should call a local php script which will make a call to this URL. In case php decides to change the dbformat then this local script is updated using rsync. The cron does not need to change for ever. - Divyank

« previous php.mirrors (#16957) next »