Changes

Jump to: navigation, search

Robots Are Our Friends

148 bytes added, 05:00, 7 November 2012
no edit summary
== Typical Reasons for not allowing Robots ==
 
* While indexing a dynamic site, robots can put an extra strain on the server, causing a slow response, or in some cases, pegging the CPU at 100%.
== Throttling ==
MrDys
19
edits
Retrieved from "https://wiki.code4lib.org/Special:MobileDiff/28188"

Navigation menu