modern: Save whitespace...seems to just work?

modern-harvey3-triple
Doug Coleman 2018-08-03 11:31:07 -04:00
parent 46e930874a
commit 947793673a
2 changed files with 23 additions and 2 deletions

View File

@ -358,10 +358,16 @@ DEFER: lex-factor-top*
! Advance the index and read again!
: read-token-or-whitespace-top ( n string slice -- n' string slice/f )
dup length 0 = [ [ 1 + ] 2dip drop lex-factor-top ] when ;
dup length 0 = [
! [ 1 + ] 2dip drop lex-factor-top
merge-slice-til-non-whitespace
] when ;
: read-token-or-whitespace-nested ( n string slice -- n' string slice/f )
dup length 0 = [ [ 1 + ] 2dip drop lex-factor-nested ] when ;
dup length 0 = [
! [ 1 + ] 2dip drop lex-factor-nested
merge-slice-til-non-whitespace
] when ;
: lex-factor-fallthrough ( n/f string slice/f ch/f -- n'/f string literal )
{

View File

@ -82,6 +82,16 @@ ERROR: unexpected-end n string ;
f string f f
] if ; inline
:: slice-til-non-whitespace ( n string -- n' string slice/f ch/f )
n [
n string [ "\s\r\n" member? not ] find-from :> ( n' ch )
n' string
n n' string ?<slice>
ch
] [
f string f f
] if ; inline
:: (slice-until) ( n string quot -- n' string slice/f ch/f )
n string quot find-from :> ( n' ch )
n' string
@ -135,6 +145,11 @@ ERROR: unexpected-end n string ;
[ slice-til-whitespace drop ] dip merge-slices
] when ;
: merge-slice-til-non-whitespace ( n string slice -- n' string slice' )
pick [
[ slice-til-non-whitespace drop ] dip merge-slices
] when ;
: merge-slice-til-eol ( n string slice -- n' string slice' )
[ slice-til-eol drop ] dip merge-slices ;