changes in io.encodings.string

db4
Daniel Ehrenberg 2008-03-05 19:12:40 -06:00
parent 1abb7c643e
commit 946d3e7414
11 changed files with 42 additions and 19 deletions

View File

@ -7,7 +7,6 @@ ARTICLE: "io.encodings" "I/O encodings"
"Many streams deal with bytes, rather than Unicode code points, at some level. The translation between these two things is specified by an encoding. To abstract this away from the programmer, Factor provides a system where these streams are associated with an encoding which is always used when the stream is read from or written to. For most purposes, an encoding descriptor consisting of a symbol is all that is needed when initializing a stream." "Many streams deal with bytes, rather than Unicode code points, at some level. The translation between these two things is specified by an encoding. To abstract this away from the programmer, Factor provides a system where these streams are associated with an encoding which is always used when the stream is read from or written to. For most purposes, an encoding descriptor consisting of a symbol is all that is needed when initializing a stream."
{ $subsection "encodings-constructors" } { $subsection "encodings-constructors" }
{ $subsection "encodings-descriptors" } { $subsection "encodings-descriptors" }
{ $subsection "encodings-string" }
{ $subsection "encodings-protocol" } ; { $subsection "encodings-protocol" } ;
ARTICLE: "encodings-constructors" "Constructing an encoded stream" ARTICLE: "encodings-constructors" "Constructing an encoded stream"
@ -49,10 +48,6 @@ ARTICLE: "encodings-protocol" "Encoding protocol"
{ $subsection init-decoder } { $subsection init-decoder }
{ $subsection encode-string } ; { $subsection encode-string } ;
ARTICLE: "encodings-string" "Encoding and decoding strings"
"Strings can be encoded and decoded with the following words:"
{ $subsection encode-string } ;
HELP: decode-step ( buf char encoding -- ) HELP: decode-step ( buf char encoding -- )
{ $values { "buf" "A string buffer which characters can be pushed to" } { $values { "buf" "A string buffer which characters can be pushed to" }
{ "char" "An octet which is read from a stream" } { "char" "An octet which is read from a stream" }

View File

@ -0,0 +1 @@
Daniel Ehrenberg

View File

@ -0,0 +1,16 @@
USING: help.markup help.syntax byte-arrays strings ;
IN: io.encodings.string
ARTICLE: "io.encodings.string" "Encoding and decoding strings"
"Strings can be encoded or decoded to and from byte arrays through an encoding with the following words:"
{ $subsection encode }
{ $subsection decode } ;
HELP: decode
{ $values { "byte-array" byte-array } { "encoding" "an encoding descriptor" }
{ "string" string } }
{ $description "Decodes the byte array using the given encoding, outputting a string" } ;
HELP: encode
{ $values { "string" string } { "encoding" "an encoding descriptor" } { "byte-array" byte-array } }
{ $description "Encodes the given string into a byte array with the given encoding." } ;

View File

@ -1,5 +1,9 @@
USING: io.encodings.utf8 io.encodings.utf16 io.encodings.string tools.test ; USING: strings io.encodings.utf8 io.encodings.utf16
io.encodings.string tools.test ;
IN: io.encodings.string.tests IN: io.encodings.string.tests
[ "hello" ] [ "hello" utf8 decode-string ] unit-test [ "hello" ] [ "hello" utf8 decode ] unit-test
[ "he" ] [ "\0h\0e" utf16be decode-string ] unit-test [ "he" ] [ "\0h\0e" utf16be decode ] unit-test
[ "hello" ] [ "hello" utf8 encode >string ] unit-test
[ "\0h\0e" ] [ "he" utf16be encode >string ] unit-test

View File

@ -1,5 +1,8 @@
USING: io.encodings io io.streams.byte-array ; USING: io io.streams.byte-array ;
IN: io.encodings.string IN: io.encodings.string
: decode-string ( byte-array encoding -- string ) : decode ( byte-array encoding -- string )
<byte-reader> contents ; <byte-reader> contents ;
: encode ( string encoding -- byte-array )
[ write ] with-byte-writer ;

View File

@ -0,0 +1 @@
Encoding and decoding strings

View File

@ -0,0 +1 @@
text

View File

@ -2,7 +2,7 @@ USING: help help.markup help.syntax help.definitions help.topics
namespaces words sequences classes assocs vocabs kernel arrays namespaces words sequences classes assocs vocabs kernel arrays
prettyprint.backend kernel.private io generic math system prettyprint.backend kernel.private io generic math system
strings sbufs vectors byte-arrays bit-arrays float-arrays strings sbufs vectors byte-arrays bit-arrays float-arrays
quotations io.streams.byte-array ; quotations io.streams.byte-array io.encodings.string ;
IN: help.handbook IN: help.handbook
ARTICLE: "conventions" "Conventions" ARTICLE: "conventions" "Conventions"
@ -186,8 +186,10 @@ ARTICLE: "io" "Input and output"
{ $subsection "io.files" } { $subsection "io.files" }
{ $subsection "io.mmap" } { $subsection "io.mmap" }
{ $subsection "io.monitors" } { $subsection "io.monitors" }
{ $heading "Other features" } { $heading "Encodings" }
{ $subsection "io.encodings" } { $subsection "io.encodings" }
{ $subsection "io.encodings.string" }
{ $heading "Other features" }
{ $subsection "network-streams" } { $subsection "network-streams" }
{ $subsection "io.launcher" } { $subsection "io.launcher" }
{ $subsection "io.timeouts" } ; { $subsection "io.timeouts" } ;

View File

@ -2,7 +2,7 @@
! See http://factorcode.org/license.txt for BSD license. ! See http://factorcode.org/license.txt for BSD license.
USING: hashtables io io.streams.string kernel math namespaces USING: hashtables io io.streams.string kernel math namespaces
math.parser assocs sequences strings splitting ascii math.parser assocs sequences strings splitting ascii
io.encodings.utf8 io.encodings io.encodings.string namespaces io.encodings.utf8 io.encodings.string namespaces
unicode.case combinators vectors sorting new-slots accessors unicode.case combinators vectors sorting new-slots accessors
calendar calendar.format quotations arrays ; calendar calendar.format quotations arrays ;
IN: http IN: http
@ -18,7 +18,7 @@ IN: http
swap "/_-." member? or ; foldable swap "/_-." member? or ; foldable
: push-utf8 ( ch -- ) : push-utf8 ( ch -- )
1string utf8 encode-string [ CHAR: % , >hex 2 CHAR: 0 pad-left % ] each ; 1string utf8 encode [ CHAR: % , >hex 2 CHAR: 0 pad-left % ] each ;
: url-encode ( str -- str ) : url-encode ( str -- str )
[ [ [ [
@ -50,7 +50,7 @@ IN: http
] if ; ] if ;
: url-decode ( str -- str ) : url-decode ( str -- str )
[ 0 swap url-decode-iter ] "" make utf8 decode-string ; [ 0 swap url-decode-iter ] "" make utf8 decode ;
: crlf "\r\n" write ; : crlf "\r\n" write ;

View File

@ -3,7 +3,7 @@
USING: alien alien.c-types arrays ui ui.gadgets ui.gestures USING: alien alien.c-types arrays ui ui.gadgets ui.gestures
ui.backend ui.clipboards ui.gadgets.worlds assocs kernel math ui.backend ui.clipboards ui.gadgets.worlds assocs kernel math
namespaces opengl sequences strings x11.xlib x11.events x11.xim namespaces opengl sequences strings x11.xlib x11.events x11.xim
x11.glx x11.clipboard x11.constants x11.windows io.encodings x11.glx x11.clipboard x11.constants x11.windows io.encodings.string
io.encodings.utf8 combinators debugger system command-line io.encodings.utf8 combinators debugger system command-line
ui.render math.vectors tuples opengl.gl threads ; ui.render math.vectors tuples opengl.gl threads ;
IN: ui.x11 IN: ui.x11
@ -137,7 +137,7 @@ M: world selection-notify-event
: encode-clipboard ( string type -- bytes ) : encode-clipboard ( string type -- bytes )
XSelectionRequestEvent-target XA_UTF8_STRING = XSelectionRequestEvent-target XA_UTF8_STRING =
[ utf8 encode-string ] [ string>char-alien ] if ; [ utf8 encode ] [ string>char-alien ] if ;
: set-selection-prop ( evt -- ) : set-selection-prop ( evt -- )
dpy get swap dpy get swap
@ -212,7 +212,7 @@ M: x-clipboard paste-clipboard
: set-title-new ( dpy window string -- ) : set-title-new ( dpy window string -- )
>r >r
XA_NET_WM_NAME XA_UTF8_STRING 8 PropModeReplace XA_NET_WM_NAME XA_UTF8_STRING 8 PropModeReplace
r> utf8 encode-string dup length XChangeProperty drop ; r> utf8 encode dup length XChangeProperty drop ;
M: x11-ui-backend set-title ( string world -- ) M: x11-ui-backend set-title ( string world -- )
world-handle x11-handle-window swap dpy get -rot world-handle x11-handle-window swap dpy get -rot

View File

@ -36,7 +36,7 @@ TUPLE: x-clipboard atom contents ;
>r XSelectionEvent-property zero? [ >r XSelectionEvent-property zero? [
r> drop f r> drop f
] [ ] [
r> selection-property 1 window-property utf8 decode-string r> selection-property 1 window-property utf8 decode
] if ; ] if ;
: own-selection ( prop win -- ) : own-selection ( prop win -- )