The search engine works in a total of three processes: web search, pre-processing of information and indexing。
Detailed: crawl - capture - processing information - indexing - ranking
One: climb
Purpose of crawling: create list to grab
Google reptile: ddos 100 degrees reptile: spider
1. Discoveries:
New website
Passive waiting: set up the robots. Txt file and wait for the reptile to come and pick it up. Google for weeks, 100 degrees a month to several months
Proactive reptiles: the writing of linked soft scripts is spent on high-quality, high-weight websites, high-priority blogs, and outward chains in high-quality forums。
New content
You can put the first page on the front page or try to keep it at the top of the column (summary: high priority position)
Banner's picture is less of a slide than a slide. (summary: text first, graphic alt properties)
View: web log analysis. The log file allows you to view the station header tool: see the frequency of capture. It's too high to cause server crashes and solve them: website access suddenly becomes
Let's see if she gets punished
Analyse log reptiles, record ip (spide, user)
There's a problem with the statistical code
3. Climbing strategy:
Depth priority: one layer follows the breadth (1 million-10 million pages) priority: peer collection (1 million pages) mixed: mixed use of the two (50 million-100 million pages)
4. Reptile capture barriers:
The problem with the txt file setting error server itself (frequency of dynamic ips, instability of the server) is too long for the url, and the search engine is lazy. The reptile trap
Two: capture
Web pages themselves need to meet w3c standards
Header information: status code: 200 (normal access), 404 (dead chain), 301 (permanent re-direction), 302 (temporary re-direction due diligence), 403 (prohibited access), 5xx (database problem) code: suggested use of utf-8. Gb2312 for opening abroad is a random coded tdk keyword: title: as far as we can, we want reptiles to get to a page to see our thrust。
Length: 33 characters, 25 words in the article, one word in english
Organisation
Number of times: do not over-repeat (column highlights keywords, most importantly in front)
Readability: considering user experience
Original: spiders like fresh things
Keyword: 5-7 on the front page, 5-6 in the first column, 4-5 in the second column, 3-4 on the list page and 2-3 on topics. Deion: (column must be written around keywords)
Impact: increased number of hits
Density: 3-8%
Length: 80 han characters
Original: favourable entry
Readability: user experience considerations
Call for action: events, promotional pages
3. The public sector:
Search engines don't capture repeats
Navigation: main navigation, sub-navigation, left navigation, seo navigation (labelling), bread crumb navigation (no js)
