steve
/
factor
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

io: more hot-rodding for #376 

* Change the interface of read-into to return a slice and an eof boolean separately so the compiler can optimize the slice.
* Add an each-stream-block-slice combinator that behaves like each-block but reuses a preallocated buffer for every iteration.
* Pull some strings in the stream-read-into implementation to further improve type propagation and bounds check elimination.
db4
Joe Groff 2011-11-09 17:21:44 -08:00
parent a67931b7c5
commit 4f4bbd3304
4 changed files with 37 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
core/io/io-docs.factor
View File

@ -51,13 +51,13 @@ HELP: stream-read-unsafe
$io-error ;
HELP: read-into
{ $values { "buf" { $or byte-array specialized-array string } } { "buf-slice/f" { $or slice f } } }
{ $contract "Reads from the current " { $link input-stream } " into the sequence " { $snippet "buf" } ", until either the length of " { $snippet "buf" } " is reached or the stream is exhausted. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, or " { $link f } " if the stream was exhausted." }
{ $values { "buf" { $or byte-array specialized-array string } } { "buf-slice" slice } { "more?" boolean } }
{ $contract "Reads from the current " { $link input-stream } " into the sequence " { $snippet "buf" } ", until either the length of " { $snippet "buf" } " is reached or the stream is exhausted. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, and a boolean value that will be " { $link f } " if the stream was exhausted." }
$io-error ;
HELP: stream-read-into
{ $values { "buf" { $or byte-array specialized-array string } } { "stream" "an input stream" } { "buf-slice/f" { $or slice f } } }
{ $contract "Reads from the stream into the sequence " { $snippet "buf" } ", until either the length of " { $snippet "buf" } " is reached or the stream is exhausted. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, or " { $link f } " if the stream was exhausted." }
{ $values { "buf" { $or byte-array specialized-array string } } { "stream" "an input stream" } { "buf-slice" slice } { "more?" boolean } }
{ $contract "Reads from the stream into the sequence " { $snippet "buf" } ", until either the length of " { $snippet "buf" } " is reached or the stream is exhausted. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, and a boolean value that will be " { $link f } " if the stream was exhausted." }
{ $notes "Most code only works on one stream at a time and should instead use " { $link read-into } "; see " { $link "stdio" } "." }
$io-error ;
@ -80,13 +80,13 @@ HELP: stream-read-partial-unsafe
$io-error ;
HELP: read-partial-into
{ $values { "buf" { $or byte-array specialized-array string } } { "buf-slice/f" { $or slice f } } }
{ $contract "Reads available data from the current " { $link input-stream } " into the sequence " { $snippet "buf" } " without blocking until all immediately available data is read or the length of " { $snippet "buf" } " is reached. If no data is immediately available, blocks until data is available. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, or " { $link f } " if the stream was exhausted." }
{ $values { "buf" { $or byte-array specialized-array string } } { "buf-slice" slice } { "more?" boolean } }
{ $contract "Reads available data from the current " { $link input-stream } " into the sequence " { $snippet "buf" } " without blocking until all immediately available data is read or the length of " { $snippet "buf" } " is reached. If no data is immediately available, blocks until data is available. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, and a boolean that will be " { $link f } " if the stream was exhausted." }
$io-error ;
HELP: stream-read-partial-into
{ $values { "buf" { $or byte-array specialized-array string } } { "stream" "an input stream" } { "buf-slice/f" { $or slice f } } }
{ $contract "Reads available data from the stream into the sequence " { $snippet "buf" } " without blocking until all immediately available data is read or the length of " { $snippet "buf" } " is reached. If no data is immediately available, blocks until data is available. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, or " { $link f } " if the stream was exhausted." }
{ $values { "buf" { $or byte-array specialized-array string } } { "stream" "an input stream" } { "buf-slice" slice } { "more?" boolean } }
{ $contract "Reads available data from the stream into the sequence " { $snippet "buf" } " without blocking until all immediately available data is read or the length of " { $snippet "buf" } " is reached. If no data is immediately available, blocks until data is available. Returns a " { $link slice } " over the part of " { $snippet "buf" } " that was written to, and a boolean that will be " { $link f } " if the stream was exhausted." }
{ $notes "Most code only works on one stream at a time and should instead use " { $link read-partial-into } "; see " { $link "stdio" } "." }
$io-error ;

6
core/io/io-tests.factor
View File

@ -27,9 +27,9 @@ M: up-to-13-reader stream-read1
[ up-to-13-reader new [ 20 swap stream-read ] [ 20 swap stream-read ] bi ] unit-test
{
T{ slice f 0 8 B{ 0 1 2 3 4 5 6 7 } }
T{ slice f 0 5 B{ 8 9 10 11 12 205 206 207 } }
f
T{ slice f 0 8 B{ 0 1 2 3 4 5 6 7 } } t
T{ slice f 0 5 B{ 8 9 10 11 12 205 206 207 } } t
T{ slice f 0 0 B{ 8 9 10 11 12 205 206 207 } } f
} [
up-to-13-reader new
[ B{ 200 201 202 203 204 205 206 207 } swap stream-read-into ]

26
core/io/io.factor
View File

@ -129,7 +129,10 @@ SYMBOL: error-stream
: (read-into) ( buf stream quot -- buf-slice/f )
[ dup length over ] 2dip call
[ drop f ] [ head-slice ] if-zero ; inline
[ (head) <slice-unsafe> ] [ zero? not ] bi ; inline
: fast>fixnum ( n -- n' )
dup fixnum? [ >fixnum ] unless ; inline
PRIVATE>
@ -141,20 +144,21 @@ PRIVATE>
ERROR: invalid-read-buffer buf stream ;
: stream-read-into ( buf stream -- buf-slice/f )
[ stream-read-unsafe ] (read-into) ; inline
USE: kernel.private
: stream-read-into ( buf stream -- buf-slice more? )
[ stream-read-unsafe { fixnum } declare ] (read-into) ; inline
: stream-read-partial-into ( buf stream -- buf-slice/f )
[ stream-read-partial-unsafe ] (read-into) ; inline
: stream-read-partial-into ( buf stream -- buf-slice more? )
[ stream-read-partial-unsafe { fixnum } declare ] (read-into) ; inline
: read ( n -- seq ) input-stream get stream-read ; inline
: read-partial ( n -- seq ) input-stream get stream-read-partial ; inline
: read-into ( buf -- buf-slice/f )
: read-into ( buf -- buf-slice more? )
input-stream get stream-read-into ; inline
: read-partial-into ( buf -- buf-slice/f )
: read-partial-into ( buf -- buf-slice more? )
input-stream get stream-read-partial-into ; inline
: each-stream-line ( ... stream quot: ( ... line -- ... ) -- ... )
@ -169,9 +173,17 @@ ERROR: invalid-read-buffer buf stream ;
: lines ( -- seq )
input-stream get stream-lines ; inline
: each-stream-block-slice ( ... stream quot: ( ... block-slice -- ... ) -- ... )
[ drop ] prepose
swap [ 65536 swap (new-sequence-for-stream) ] keep
[ stream-read-partial-into ] 2curry each-morsel drop ; inline
: each-stream-block ( ... stream quot: ( ... block -- ... ) -- ... )
swap [ 65536 swap stream-read-partial ] curry each-morsel ; inline
: each-block-slice ( ... quot: ( ... block -- ... ) -- ... )
input-stream get swap each-stream-block ; inline
: each-block ( ... quot: ( ... block -- ... ) -- ... )
input-stream get swap each-stream-block ; inline

7
core/sequences/sequences.factor
View File

@ -224,6 +224,13 @@ TUPLE: slice-error from to seq reason ;
[ drop > "start > end" slice-error ]
3tri ; inline
<PRIVATE
: <slice-unsafe> ( from to seq -- slice )
slice boa ; inline
PRIVATE>
: <slice> ( from to seq -- slice )
check-slice
dup slice? [ collapse-slice ] when