Commit Graph

5 Commits (19cd3c1faedb2c7dc2c8c2e6ea39f90c194c8be3)

Author SHA1 Message Date
John Benediktsson fd853f9958 Cleanup some lint warnings. 2011-10-15 19:19:44 -07:00
Doug Coleman b2bf680959 allow robot-identifiers to be set for robots.txt 2009-06-19 14:41:48 -05:00
Doug Coleman 65802b6aaa fix unit test in robots 2009-04-07 10:08:01 -05:00
Doug Coleman b6064813c9 construct a glob-matching quote for robots.txt files 2009-04-06 17:39:44 -05:00
Doug Coleman c626cab383 a robots.txt parser 2009-03-20 21:56:48 -05:00