Page history

Robots Are Our Friends

17 November 2012

Edsu
→‎Typical Reasons for not allowing Robots
12:37
+247
Edsu
no edit summary
12:30
+1
Edsu
no edit summary
12:29
+4
Edsu
no edit summary
12:26
+2
Edsu
no edit summary
12:26
-2
Edsu
no edit summary
12:25
-1

7 November 2012

Bibliotechy
no edit summary
18:15
+487
MrDys
no edit summary
16:01
+34
Bibliotechy
no edit summary
15:57
+506
Bibliotechy
no edit summary
15:36
+526
Edsu
no edit summary
m
15:31
-1
Edsu
added details about crawl-delay
15:30
+352
MrDys
→‎Throttling
15:08
+433
MrDys
→‎Throttling
14:53
+520
MrDys
no edit summary
05:00
+148

6 November 2012

Edsu
no edit summary
23:01
+299
Edsu
RobotsAreOurFriends moved to Robots Are Our Friends
m
22:46
Edsu
→‎Robots Are Our Friends
22:46
-28
Edsu
→‎Robots Are Our Friends
22:39
+4
Edsu
→‎Robots Are Our Friends
22:39
-4
Edsu
New page: = Robots Are Our Friends = For a variety of reasons cultural heritage organizations can often have [http://www.robotstxt.org/ robots.txt] pages that restrict what web crawlers (aka robots...
22:38
+951

Code4Lib
- Mobile
- Desktop

Privacy