Changes

Robots Are Our Friends

148 bytes added, 05:00, 7 November 2012
no edit summary
== Typical Reasons for not allowing Robots ==
 
* While indexing a dynamic site, robots can put an extra strain on the server, causing a slow response, or in some cases, pegging the CPU at 100%.
== Throttling ==
19
edits