steve
/
factor
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

spider: reuse url ref inside fill-spidered-result.

db4
John Benediktsson 2013-12-08 09:15:42 -08:00
parent b4037edf48
commit c68db22281
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
extra/spider/spider.factor
View File

@ -90,8 +90,9 @@ fetched-in parsed-html links processed-in fetched-at ;
path>> file-extension { ".htm" ".html" f } member? ; path>> file-extension { ".htm" ".html" f } member? ;
:: fill-spidered-result ( spider spider-result -- ) :: fill-spidered-result ( spider spider-result -- )
f spider-result url>> dup :> url spider spidered>> set-at spider-result url>> :> url
[ spider-result url>> http-get ] benchmark :> ( headers html fetched-in ) f url spider spidered>> set-at
[ url http-get ] benchmark :> ( headers html fetched-in )
[ [
url url-html? [ url url-html? [
html parse-html html parse-html