steve
/
factor
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
factor/extra/io/encodings/detect/detect-tests.factor

45 lines
1.7 KiB
Factor
Raw Normal View History

core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
! (c)2010 Joe Groff bsd license
byte-arrays.hex: removing HEX{ ab cd ef } in favor of B{ 0xab 0xcd 0xef }.
2016-11-09 00:10:58 -05:00
USING: byte-arrays io.encodings.8-bit.koi8-r
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
io.encodings.8-bit.latin1 io.encodings.binary
io.encodings.detect io.encodings.utf16 io.encodings.utf32
io.encodings.utf8 namespaces tools.test ;
IN: io.encodings.detect.tests
! UTF encodings with BOMs
byte-arrays.hex: removing HEX{ ab cd ef } in favor of B{ 0xab 0xcd 0xef }.
2016-11-09 00:10:58 -05:00
{ utf16be } [ B{ 0xFE 0xFF 0x00 0x31 0x00 0x32 0x00 0x33 } detect-byte-array ] unit-test
{ utf16le } [ B{ 0xFF 0xFE 0x31 0x00 0x32 0x00 0x33 0x00 } detect-byte-array ] unit-test
{ utf32be } [ B{ 0x00 0x00 0xFE 0xFF 0x00 0x00 0x00 0x31 0x00 0x00 0x00 0x32 0x00 0x00 0x00 0x33 } detect-byte-array ] unit-test
{ utf32le } [ B{ 0xFF 0xFE 0x00 0x00 0x31 0x00 0x00 0x00 0x32 0x00 0x00 0x00 0x33 0x00 0x00 0x00 } detect-byte-array ] unit-test
{ utf8 } [ B{ 0xEF 0xBB 0xBF 0x31 0x32 0x33 } detect-byte-array ] unit-test
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
! XML prolog
factor: second stab at [ ] [ ] unit-test -> { } [ ] unit-test
2015-07-03 12:39:59 -04:00
{ utf8 }
factor: remove """ string syntax for now. there are HEREDOC:, STRING:, escaping \", and something else soon.
2015-07-26 01:59:56 -04:00
[ "<?xml version=\"1.0\"?>" >byte-array detect-byte-array ]
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
unit-test
factor: second stab at [ ] [ ] unit-test -> { } [ ] unit-test
2015-07-03 12:39:59 -04:00
{ utf8 }
factor: remove """ string syntax for now. there are HEREDOC:, STRING:, escaping \", and something else soon.
2015-07-26 01:59:56 -04:00
[ "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" >byte-array detect-byte-array ]
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
unit-test
factor: second stab at [ ] [ ] unit-test -> { } [ ] unit-test
2015-07-03 12:39:59 -04:00
{ latin1 }
factor: remove """ string syntax for now. there are HEREDOC:, STRING:, escaping \", and something else soon.
2015-07-26 01:59:56 -04:00
[ "<?xml version='1.0' encoding='ISO-8859-1'?>" >byte-array detect-byte-array ]
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
unit-test
factor: second stab at [ ] [ ] unit-test -> { } [ ] unit-test
2015-07-03 12:39:59 -04:00
{ latin1 }
factor: remove """ string syntax for now. there are HEREDOC:, STRING:, escaping \", and something else soon.
2015-07-26 01:59:56 -04:00
[ "<?xml version='1.0' encoding=\"ISO-8859-1\" " >byte-array detect-byte-array ]
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
unit-test
! Default to utf8 if decoding succeeds and there are no nulls
byte-arrays.hex: removing HEX{ ab cd ef } in favor of B{ 0xab 0xcd 0xef }.
2016-11-09 00:10:58 -05:00
{ utf8 } [ B{ } detect-byte-array ] unit-test
{ utf8 } [ B{ 0x31 0x32 0x33 } detect-byte-array ] unit-test
{ utf8 } [ B{ 0x31 0x32 0xC2 0xA0 0x33 } detect-byte-array ] unit-test
{ latin1 } [ B{ 0x31 0x32 0xA0 0x33 } detect-byte-array ] unit-test
factor: second stab at [ ] [ ] unit-test -> { } [ ] unit-test
2015-07-03 12:39:59 -04:00
{ koi8-r } [
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
koi8-r default-8bit-encoding [
byte-arrays.hex: removing HEX{ ab cd ef } in favor of B{ 0xab 0xcd 0xef }.
2016-11-09 00:10:58 -05:00
B{ 0x31 0x32 0xA0 0x33 } detect-byte-array
core, basis, extra: Remove DOS line endings from files. Remove whitespace from end of lines. Add a newline to the end of each file.
2015-06-29 19:43:15 -04:00
] with-variable
] unit-test
byte-arrays.hex: removing HEX{ ab cd ef } in favor of B{ 0xab 0xcd 0xef }.
2016-11-09 00:10:58 -05:00
{ binary } [ B{ 0x31 0x32 0x33 0xC2 0xA0 0x00 } detect-byte-array ] unit-test
{ binary } [ B{ 0x31 0x32 0x33 0xC2 0xA0 0x00 0x30 } detect-byte-array ] unit-test