Re: Big Site !!!!

Hi folks Mark Charette wrote: > > and I have been told that many search engines do not look at PHP code. > ... I have no problems with search engines crawling > the site (other than the fact that some days there's 4 or 5 robots from > different places on the site simultaneously). I parse all .htm and .html > files - that way the search engines don't know that some of the pages are > dynamically generated. > From what I understand, most of the big search engines will not index anything that looks like a dynamic page. This is from AltaVista: >>>>>>>>>>>>>>>>>>>>>>>> Dynamic pages also block Web crawlers. While it's great to give visitors unique experiences, tailored to their needs, the techniques you use to do that could stop search engines from indexing your content and hence could greatly reduce your potential traffic. Dynamically generated pages are created on the fly from a variety of elements held in databases. Typically such pages have a question mark (?) in the URL. When a search engine crawler arrives at such a page, it captures the content but halts immediately, and will not follow the links, because it sees ahead of it an infinite number of pages -- a black hole that would bring it to a crash. Active Server Pages (.asp) with question marks in their URLs (indicating that the page is a script for the construction of a page, rather than just static content) are not indexed. >>>>>>>>>>>>>>>>>>>>>>>>> So if you want to be indexed, you need to use a extension such as .html on your pages as Mark suggests, but you also need to avoid URLs that contain the tell-tale ?foo=bar&boo=bust value pairings. Geoff Caplan. British Goldsmiths