{ $description "Creates a new web spider with a given base url." } ;
HELP:run-spider
{ $values
{ "spider" spider } }
{ $description "Runs a spider until completion. See the " { $subsection "spider-tutorial" } " for a complete description of the tuple slots that affect how thet spider works." } ;
ARTICLE: "spider-tutorial""Spider tutorial"
"To create a new spider, call the " { $link <spider> } " word with a link to the site you wish to spider."
"Since we happen to know that not all pages of a wiki are suitable for spidering, we will spider only the wiki view pages, not the edit or revisions pages. To do this, we add a filter through which new links are tested; links that pass the filter are added to the todo queue, while links that do not are discarded. You can add several filters to the filter array, but we'll just add a single one for now."