How search engines - Web Crawler
Better to know how these search engines actually work and how the information on clients to launch a search.
There are basically two types of search engines. The first is covered by crawlers or Spider.
The spiders are using search engines to index websites. If the pages on your site a search engine after the close of their deposit necessary, the search engine indexes Spider your entire website. A “Spider” is an automated program, is the search engine. Spider visited a Web site, please read the actual content at the meta-tags of the site and follow the links that the website links. The spider, then all this information are back to a central depository of securities, where the data indexed. It is go to every link on your site and content of these sites. Some spiders indicated that a number of pages of your website, that this is not to create a site of 500 pages!
The spider is at regular intervals on the sites to ensure that all information, has changed. The frequency with which this happens, by moderators of the search engine.
A spider is almost like a book, which contains the table of contents, content and links and references to all sites during their search, and it can go up to a million pages per day.
For example, Excite, Lycos, Altavista and Google.
If you use a search engine for finding information, it is actually the search index where he established and no real research on the Internet. The various classifications of different search engines, because not every search engine uses the same search algorithm by the index.
One of the things, a search algorithm for scans is the frequency and location of the keywords on a Web page, but it can also identify, by keyword or filling artificial spamdexing. Then algorithms to analyse the nature and the way the pages of links to other pages on the Internet. Through the audit, such as pages to connect, a motor can also determine what constitutes a page if the keywords pages are similar to those keywords on the page.