steve
/
factor
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

lexer: Add #hashtag support.

modern-harvey3-triple
Doug Coleman 2018-08-10 10:42:00 -05:00
parent 9c7fdc3a34
commit 24d266c1f1
1 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
core/lexer/lexer.factor
View File

@ -134,16 +134,22 @@ DEFER: parse-token
nip nip
] if ; ] if ;
! TODO: combinations of prefix: \$#_:
! suffix: \_:
: unescape-token ( string -- string' ) : unescape-token ( string -- string' )
dup length 1 = [ "\\" ?head drop ] unless ; dup length 1 = [ "\\" ?head drop ] unless ;
: unhashtag-token ( string -- string' )
dup length 1 = [ "#" ?head [ drop f ] when ] unless ;
: unescape-tokens ( seq -- seq' ) : unescape-tokens ( seq -- seq' )
[ unescape-token ] map ; [ unescape-token ] map ;
: parse-token ( lexer -- str/f ) : parse-token ( lexer -- str/f )
dup parse-raw [ skip-comments ] [ drop f ] if* ; dup parse-raw [ skip-comments ] [ drop f ] if* ;
: ?scan-token ( -- str/f ) lexer get parse-token unescape-token ; : ?scan-token ( -- str/f ) lexer get parse-token unescape-token unhashtag-token ;
PREDICATE: unexpected-eof < unexpected got>> not ; PREDICATE: unexpected-eof < unexpected got>> not ;