steve
/
factor
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Add more unit tests for code-point-length

db4
Doug Coleman 2009-03-19 17:36:49 -05:00
parent ebaf05f26b
commit 92d58b1160
2 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
core/io/encodings/utf8/utf8-tests.factor
View File

@ -1,5 +1,5 @@
USING: io.encodings.utf8 tools.test io.encodings.string strings arrays
bootstrap.unicode ;
bootstrap.unicode kernel sequences ;
IN: io.encodings.utf8.tests
: decode-utf8-w/stream ( array -- newarray )
@ -25,3 +25,7 @@ IN: io.encodings.utf8.tests
[ 3 ] [ 1 "日本語" >utf8-index ] unit-test
[ 3 ] [ 9 "日本語" utf8-index> ] unit-test
[ 3 ] [ 2 "lápis" >utf8-index ] unit-test
[ V{ } ] [ 100000 [ [ code-point-length ] [ 1string utf8 encode length ] bi = not ] filter ] unit-test

4
core/io/encodings/utf8/utf8.factor
View File

@ -73,7 +73,7 @@ M: utf8 encode-char
PRIVATE>
: code-point-length ( n -- x )
log2 {
next-power-of-2 log2 {
{ [ dup 0 7 between? ] [ 1 ] }
{ [ dup 8 11 between? ] [ 2 ] }
{ [ dup 12 16 between? ] [ 3 ] }
@ -87,4 +87,4 @@ PRIVATE>
code-point-offsets [ <= ] with find drop ;
: >utf8-index ( n string -- n' )
code-point-offsets nth ;
code-point-offsets nth ;