mapped-file no longer supports sequence protocol. Instead, io.mmap.functor generates mapped arrays for all primitive C types

db4
Slava Pestov 2008-12-02 21:50:34 -06:00
parent f683a76a5e
commit aa3e1cdb6c
19 changed files with 113 additions and 37 deletions

View File

@ -205,10 +205,10 @@ ARTICLE: "cookbook-io" "Input and output cookbook"
} }
"Convert a file of 4-byte cells from little to big endian or vice versa, by directly mapping it into memory and operating on it with sequence words:" "Convert a file of 4-byte cells from little to big endian or vice versa, by directly mapping it into memory and operating on it with sequence words:"
{ $code { $code
"USING: accessors grouping io.files io.mmap kernel sequences ;" "USING: accessors grouping io.files io.mmap.char kernel sequences ;"
"\"mydata.dat\" dup file-info size>> [" "\"mydata.dat\" ["
" 4 <sliced-groups> [ reverse-here ] change-each" " 4 <sliced-groups> [ reverse-here ] change-each"
"] with-mapped-file" "] with-mapped-char-file"
} }
"Send some bytes to a remote host:" "Send some bytes to a remote host:"
{ $code { $code

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.alien ;
IN: io.mmap.alien
<< "void*" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.bool ;
IN: io.mmap.bool
<< "bool" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.char ;
IN: io.mmap.char
<< "char" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.double ;
IN: io.mmap.double
<< "double" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.float ;
IN: io.mmap.float
<< "float" define-mapped-array >>

View File

@ -1,6 +1,7 @@
! Copyright (C) 2008 Slava Pestov. ! Copyright (C) 2008 Slava Pestov.
! See http://factorcode.org/license.txt for BSD license. ! See http://factorcode.org/license.txt for BSD license.
USING: functors accessors alien.c-types math kernel words ; USING: io.mmap functors accessors alien.c-types math kernel
words fry ;
IN: io.mmap.functor IN: io.mmap.functor
SLOT: address SLOT: address
@ -10,13 +11,18 @@ SLOT: length
[ [ address>> ] [ length>> ] bi ] dip [ [ address>> ] [ length>> ] bi ] dip
heap-size [ 1- + ] keep /i ; heap-size [ 1- + ] keep /i ;
FUNCTOR: mapped-array-functor ( T -- ) FUNCTOR: define-mapped-array ( T -- )
C DEFINES <mapped-${T}-array> <mapped-A> DEFINES <mapped-${T}-array>
<A> IS <direct-${T}-array> <A> IS <direct-${T}-array>
with-mapped-A-file DEFINES with-mapped-${T}-file
WHERE WHERE
: C mapped-file>direct <A> execute ; inline : <mapped-A> ( mapped-file -- direct-array )
T mapped-file>direct <A> execute ; inline
: with-mapped-A-file ( path length quot -- )
'[ <mapped-A> execute @ ] with-mapped-file ; inline
;FUNCTOR ;FUNCTOR

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.int ;
IN: io.mmap.int
<< "int" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.long ;
IN: io.mmap.long
<< "long" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.longlong ;
IN: io.mmap.longlong
<< "longlong" define-mapped-array >>

View File

@ -11,9 +11,9 @@ HELP: mapped-file
} ; } ;
HELP: <mapped-file> HELP: <mapped-file>
{ $values { "path" "a pathname string" } { "length" integer } { "mmap" mapped-file } } { $values { "path" "a pathname string" } { "mmap" mapped-file } }
{ $contract "Opens a file and maps the first " { $snippet "length" } " bytes into memory. The length is permitted to exceed the length of the file on disk, in which case the remaining space is padded with zero bytes." } { $contract "Opens a file and maps its contents into memory. The length is permitted to exceed the length of the file on disk, in which case the remaining space is padded with zero bytes." }
{ $notes "You must call " { $link close-mapped-file } " when you are finished working with the returned object, to reclaim resources. The " { $link with-mapped-file } " provides an abstraction which can close the mapped file for you." } { $notes "You must call " { $link dispose } " when you are finished working with the returned object, to reclaim resources. The " { $link with-mapped-file } " provides an abstraction which can close the mapped file for you." }
{ $errors "Throws an error if a memory mapping could not be established." } ; { $errors "Throws an error if a memory mapping could not be established." } ;
HELP: with-mapped-file HELP: with-mapped-file
@ -26,6 +26,33 @@ HELP: close-mapped-file
{ $contract "Releases system resources associated with the mapped file. This word should not be called by user code; use " { $link dispose } " instead." } { $contract "Releases system resources associated with the mapped file. This word should not be called by user code; use " { $link dispose } " instead." }
{ $errors "Throws an error if a memory mapping could not be established." } ; { $errors "Throws an error if a memory mapping could not be established." } ;
ARTICLE: "io.mmap.arrays" "Memory-mapped arrays"
"Mapped file can be viewed as a sequence using the words in sub-vocabularies of " { $vocab-link "io.mmap" } ". For each primitive C type " { $snippet "T" } ", a set of words are defined in the vocabulary named " { $snippet "io.mmap.T" } ":"
{ $table
{ { $snippet "<mapped-T-array>" } { "Wraps a " { $link mapped-file } " in a sequence; stack effect " { $snippet "( mapped-file -- direct-array )" } } }
{ { $snippet "with-mapped-T-file" } { "Maps a file into memory and wraps it in a sequence by combining " { $link with-mapped-file } " and " { $snippet "<mapped-T-array>" } "; stack effect " { $snippet "( path quot -- )" } } }
}
"The primitive C types for which mapped arrays exist:"
{ $list
{ $snippet "char" }
{ $snippet "uchar" }
{ $snippet "short" }
{ $snippet "ushort" }
{ $snippet "int" }
{ $snippet "uint" }
{ $snippet "long" }
{ $snippet "ulong" }
{ $snippet "longlong" }
{ $snippet "ulonglong" }
{ $snippet "float" }
{ $snippet "double" }
{ $snippet "void*" }
{ $snippet "bool" }
} ;
ARTICLE: "io.mmap.low-level" "Reading and writing mapped files directly"
"Data can be read and written from the " { $link mapped-file } " by applying low-level alien words to the " { $slot "address" } " slot. See " { $link "reading-writing-memory" } "." ;
ARTICLE: "io.mmap" "Memory-mapped files" ARTICLE: "io.mmap" "Memory-mapped files"
"The " { $vocab-link "io.mmap" } " vocabulary implements support for memory-mapped files." "The " { $vocab-link "io.mmap" } " vocabulary implements support for memory-mapped files."
{ $subsection <mapped-file> } { $subsection <mapped-file> }
@ -33,7 +60,8 @@ ARTICLE: "io.mmap" "Memory-mapped files"
$nl $nl
"A utility combinator which wraps the above:" "A utility combinator which wraps the above:"
{ $subsection with-mapped-file } { $subsection with-mapped-file }
"Memory mapped files implement the " { $link "sequence-protocol" } " and present themselves as a sequence of bytes. The underlying memory area can also be accessed directly with the " { $snippet "address" } " slot." $nl "Instances of " { $link mapped-file } " don't support any interesting operations in themselves. There are two facilities for accessing their contents:"
"Data can be read and written from the memory area using alien words. See " { $link "reading-writing-memory" } "." ; { $subsection "io.mmap.arrays" }
{ $subsection "io.mmap.low-level" } ;
ABOUT: "io.mmap" ABOUT: "io.mmap"

View File

@ -1,10 +1,10 @@
USING: io io.mmap io.files kernel tools.test continuations USING: io io.mmap io.mmap.char io.files kernel tools.test
sequences io.encodings.ascii accessors ; continuations sequences io.encodings.ascii accessors ;
IN: io.mmap.tests IN: io.mmap.tests
[ "mmap-test-file.txt" temp-file delete-file ] ignore-errors [ "mmap-test-file.txt" temp-file delete-file ] ignore-errors
[ ] [ "12345" "mmap-test-file.txt" temp-file ascii set-file-contents ] unit-test [ ] [ "12345" "mmap-test-file.txt" temp-file ascii set-file-contents ] unit-test
[ ] [ "mmap-test-file.txt" temp-file dup file-info size>> [ CHAR: 2 0 pick set-nth drop ] with-mapped-file ] unit-test [ ] [ "mmap-test-file.txt" temp-file [ CHAR: 2 0 pick set-nth drop ] with-mapped-char-file ] unit-test
[ 5 ] [ "mmap-test-file.txt" temp-file dup file-info size>> [ length ] with-mapped-file ] unit-test [ 5 ] [ "mmap-test-file.txt" temp-file [ length ] with-mapped-char-file ] unit-test
[ "22345" ] [ "mmap-test-file.txt" temp-file ascii file-contents ] unit-test [ "22345" ] [ "mmap-test-file.txt" temp-file ascii file-contents ] unit-test
[ "mmap-test-file.txt" temp-file delete-file ] ignore-errors [ "mmap-test-file.txt" temp-file delete-file ] ignore-errors

View File

@ -1,39 +1,25 @@
! Copyright (C) 2007, 2008 Doug Coleman, Slava Pestov. ! Copyright (C) 2007, 2008 Doug Coleman, Slava Pestov.
! See http://factorcode.org/license.txt for BSD license. ! See http://factorcode.org/license.txt for BSD license.
USING: continuations destructors io.backend kernel quotations USING: continuations destructors io.files io.backend kernel
sequences system alien alien.accessors accessors quotations system alien alien.accessors accessors system
sequences.private system vocabs.loader combinators vocabs.loader combinators alien.c-types ;
specialized-arrays.direct functors alien.c-types
io.mmap.functor ;
IN: io.mmap IN: io.mmap
TUPLE: mapped-file address handle length disposed ; TUPLE: mapped-file address handle length disposed ;
M: mapped-file length dup check-disposed length>> ;
M: mapped-file nth-unsafe
dup check-disposed address>> swap alien-unsigned-1 ;
M: mapped-file set-nth-unsafe
dup check-disposed address>> swap set-alien-unsigned-1 ;
INSTANCE: mapped-file sequence
HOOK: (mapped-file) io-backend ( path length -- address handle ) HOOK: (mapped-file) io-backend ( path length -- address handle )
: <mapped-file> ( path length -- mmap ) : <mapped-file> ( path -- mmap )
[ [ normalize-path ] dip (mapped-file) ] keep [ normalize-path ] [ file-info size>> ] bi [ (mapped-file) ] keep
f mapped-file boa ; f mapped-file boa ;
HOOK: close-mapped-file io-backend ( mmap -- ) HOOK: close-mapped-file io-backend ( mmap -- )
M: mapped-file dispose* ( mmap -- ) close-mapped-file ; M: mapped-file dispose* ( mmap -- ) close-mapped-file ;
: with-mapped-file ( path length quot -- ) : with-mapped-file ( path quot -- )
[ <mapped-file> ] dip with-disposal ; inline [ <mapped-file> ] dip with-disposal ; inline
APPLY: mapped-array-functor primitive-types
{ {
{ [ os unix? ] [ "io.unix.mmap" require ] } { [ os unix? ] [ "io.unix.mmap" require ] }
{ [ os winnt? ] [ "io.windows.mmap" require ] } { [ os winnt? ] [ "io.windows.mmap" require ] }

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.short ;
IN: io.mmap.short
<< "short" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.uchar ;
IN: io.mmap.uchar
<< "uchar" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.uint ;
IN: io.mmap.uint
<< "uint" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.ulong ;
IN: io.mmap.ulong
<< "ulong" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USING: io.mmap.functor specialized-arrays.direct.ulonglong ;
IN: io.mmap.ulonglong
<< "ulonglong" define-mapped-array >>

View File

@ -0,0 +1,4 @@
USE: specialized-arrays.functor
IN: specialized-arrays.ushort
<< "ushort" define-array >>