checksums.multi: new vocab

windows-high-dpi
Alexander Iljin 2018-02-25 02:32:54 +01:00 committed by John Benediktsson
parent 1386b556af
commit 156f69b3a3
5 changed files with 121 additions and 0 deletions

View File

@ -0,0 +1 @@
Alexander Ilin

View File

@ -0,0 +1,62 @@
! Copyright (C) 2018 Alexander Ilin.
! See http://factorcode.org/license.txt for BSD license.
USING: checksums checksums.common checksums.md5 checksums.sha
destructors help.markup help.syntax kernel sequences ;
IN: checksums.multi
ABOUT: "checksums.multi"
ARTICLE: "checksums.multi" "Checksumming with Multiple Algorithms"
"The " { $vocab-link "checksums.multi" } " vocabulary makes it possible to calculate multiple checksums with just one pass over a data stream."
$nl
"The " { $link multi-checksum } " tuple holds a sequence of " { $link block-checksum } " instances, such as " { $link md5 } " or " { $link sha1 } ". When the " { $link initialize-checksum-state } " method is called on it, a new instance of " { $link block-checksum-state } " is created for all the " { $slot "checksums" } ", and returned as a new " { $link multi-state } " instance. You can then use " { $link add-checksum-bytes } " to stream data to it. When done, call " { $link get-checksum } " to finalize the process, read the resulting checksums and dispose of the tuple in one step. If you want to cancel the work without calling " { $link get-checksum } ", you must " { $link dispose } " of the tuple so that all implementation-specific resources are released."
$nl
"The " { $link checksum-bytes } " and the " { $link checksum-stream } " methods encapsulate the above protocol, including instantiation and disposal of the " { $link multi-state } " tuple."
{ $examples
{ $unchecked-example "USING: byte-arrays checksums checksums.md5 checksums.sha ;"
"\"test\" >byte-array { md5 sha1 } <multi-checksum> checksum-bytes ."
"{
B{
9 143 107 205 70 33 211 115 202 222 78 131 38 39 180 246
}
B{
169 74 143 229 204 177 155 166 28 76 8 115 211 145 233
135 152 47 187 211
}
}" }
$nl
{ $unchecked-example "USING: checksums checksums.common checksums.md5 checksums.sha"
"io io.encodings.binary namespaces ;"
"\"LICENSE.txt\" binary ["
" input-stream get { md5 sha1 } <multi-checksum> checksum-stream"
"] with-file-reader ."
"{
B{
220 158 207 218 50 163 198 36 234 90 122 65 197 14 224
16
}
B{
132 132 148 224 101 202 198 114 38 53 127 18 70 170 108
53 25 255 174 207
}
}" }
} ;
HELP: <multi-checksum>
{ $values
{ "checksums" sequence }
{ "multi-checksum" multi-checksum }
}
{ $description "This is a simple constructor for the " { $link multi-checksum } " tuple. The " { $snippet "checksums" } " must be a sequence of " { $link block-checksum } " instances." } ;
HELP: multi-checksum
{ $class-description "This is an instance of the " { $link block-checksum } " mixin. It calculates multiple checksums by sequentially passing the data it receives to all the checksums in its " { $slot "checksums" } " slot. This way, even though the individual checksums are not calculated in parallel, you still can have the performance benefits of only reading a disk file once, or not having to temporarily store the data streamed from a network." } ;
HELP: multi-state
{ $class-description "This class represents the current state of a " { $link multi-checksum } " checksum calculation. It has an array of associated checksum states until it is disposed. You may call " { $link add-checksum-bytes } " multiple times to pipe data to all the checksums in the " { $slot "checksums" } " slot. When finished, call " { $link get-checksum } " to receive the results and release implementation-specific resources, or " { $link dispose } " to release the resources and discard the result. After the first " { $link get-checksum } " call the returned value is stored in the " { $slot "results" } " slot, and subsequent calls return the same value." }
{ $notes "It is not possible to add more data to the checksum after the first " { $link get-checksum } " call."
$nl
"Most code should use " { $link with-checksum-state } " to make sure the resources are properly disposed of. Higher level words like " { $link checksum-bytes } " and " { $link checksum-stream } " use it to perform the disposal." } ;
{ multi-checksum multi-state } related-words

View File

@ -0,0 +1,31 @@
! Copyright (C) 2018 Alexander Ilin.
! See http://factorcode.org/license.txt for BSD license.
USING: byte-arrays checksums checksums.md5 checksums.multi
checksums.sha io io.encodings.binary io.files namespaces tools.test ;
IN: checksums.multi.tests
{
{
B{ 9 143 107 205 70 33 211 115 202 222 78 131 38 39 180 246 }
B{
169 74 143 229 204 177 155 166 28 76 8 115 211 145 233
135 152 47 187 211
}
}
} [
"test" >byte-array { md5 sha1 } <multi-checksum> checksum-bytes
] unit-test
{
{
B{ 220 158 207 218 50 163 198 36 234 90 122 65 197 14 224 16 }
B{
132 132 148 224 101 202 198 114 38 53 127 18 70 170 108
53 25 255 174 207
}
}
} [
"LICENSE.txt" binary [
input-stream get { md5 sha1 } <multi-checksum> checksum-stream
] with-file-reader
] unit-test

View File

@ -0,0 +1,26 @@
! Copyright (C) 2018 Alexander Ilin.
! See http://factorcode.org/license.txt for BSD license.
USING: accessors checksums checksums.common destructors fry kernel
sequences ;
IN: checksums.multi
TUPLE: multi-checksum checksums ;
INSTANCE: multi-checksum block-checksum
C: <multi-checksum> multi-checksum
TUPLE: multi-state < disposable states results ;
M: multi-checksum initialize-checksum-state
checksums>> [ initialize-checksum-state ] V{ } map-as
multi-state new-disposable swap >>states ;
M: multi-state dispose*
states>> dispose-each ;
M: multi-state add-checksum-bytes
'[ [ _ add-checksum-bytes ] map! ] change-states ;
M: multi-state get-checksum
dup results>> [
dup states>> [ get-checksum ] { } map-as [ >>results ] keep
] unless* nip ;

View File

@ -0,0 +1 @@
Checksum with multiple algorithms in one pass