Changes

Robots Are Our Friends

148 bytes added, 05:00, 7 November 2012

no edit summary

== Typical Reasons for not allowing Robots ==

* While indexing a dynamic site, robots can put an extra strain on the server, causing a slow response, or in some cases, pegging the CPU at 100%.

== Throttling ==

edits