steve
/
factor
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

io.encodings.detect: detect empty files as utf8 rather than raise a sequence error

db4
Joe Groff 2010-06-10 17:32:56 -07:00 committed by Doug Coleman
parent 0add13c50c
commit 6c1bf4e0eb
2 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
extra/io/encodings/detect/detect-tests.factor
View File

@ -30,6 +30,7 @@ unit-test
unit-test
! Default to utf8 if decoding succeeds and there are no nulls
[ utf8 ] [ HEX{ } detect-byte-array ] unit-test
[ utf8 ] [ HEX{ 31 32 33 } detect-byte-array ] unit-test
[ utf8 ] [ HEX{ 31 32 C2 A0 33 } detect-byte-array ] unit-test
[ latin1 ] [ HEX{ 31 32 A0 33 } detect-byte-array ] unit-test
@ -41,3 +42,4 @@ unit-test
[ binary ] [ HEX{ 31 32 33 C2 A0 00 } detect-byte-array ] unit-test
[ binary ] [ HEX{ 31 32 33 C2 A0 00 30 } detect-byte-array ] unit-test

1
extra/io/encodings/detect/detect.factor
View File

@ -38,6 +38,7 @@ PRIVATE>
{ [ dup HEX{ EF BB BF } head? ] [ drop utf8 ] }
{ [ dup $[ "<?xml" >byte-array ] head? ] [ detect-xml-prolog ] }
{ [ 0 over member? ] [ drop binary ] }
{ [ dup empty? ] [ drop utf8 ] }
{ [ dup valid-utf8? ] [ drop utf8 ] }
[ drop default-8bit-encoding get ]
} cond ;