some parse-number cleanup, documentation work

README.txt

@@ -20,12 +20,27 @@ Primitive words have an implementation coded in the host language (C or
 Java). Compound words are executed by invoking the interpreter
 recursively on their definition, which is also a linked list.
 
+* A note about code examples
+
+Factor words are separated out into multiple ``vocabularies''. Each code
+example given here is preceeded with a series of declarations, such as
+the following:
+
+   USE: math
+   USE: streams
+
+When entering code at the interactive interpreter loop, most
+vocabularies are already in the search path, and the USE: declarations
+can be omitted. However, in a source file they must all be specified, by convention at the beginning of the file.
+
 * Control flow
 
 Control flow rests on two basic concepts: recursion, and branching.
 Words with compound definitions may refer to themselves, and there is
 exactly one primitive for performing conditional execution:
 
+    USE: combinators
+
     1 10 < [ "10 is less than 1." print ] [ "whoa!" print ] ifte
     ==> 10 is less than 1.
 
@@ -43,6 +58,8 @@ Here is an example of a word that uses these two concepts:
 
 An example:
 
+    USE: lists
+
     3 [ 1 2 3 4 ] contains?
     ==> [ 3 4 ]
     5 [ 1 2 3 4 ] contains?
@@ -51,7 +68,7 @@ An example:
 It recurses down the list, until it reaches the end, in which case the
 outer ifte's 'false' branch is executed.
 
-A quick overview of the words used here:
+A quick overview of the words used here, along with their stack effects:
 
 Shuffle words:
 
@@ -74,6 +91,9 @@ functions, are usually built with the help of another primitive that
 simply executes a quotation at the top of the stack, removing it from
 the stack:
 
+    USE: math
+    USE: prettyprint
+
     [ 2 2 + . ] call
     ==> 4
 
@@ -94,6 +114,10 @@ a list. Note that it uses 'call' to execute the given quotation:
 
 An example:
 
+    USE: lists
+    USE: math
+    USE: stack
+
     [ 1 2 3 4 ] [ dup * . ] each
     ==> 1
         4
@@ -113,8 +137,8 @@ tuck ( x y -- y x y )
 >r ( x -- r:x ) - move top of data stack to/from 'extra hand'.
 r> ( r:x -- x )
 
-Writing >r foo r> is analogous to [ foo ] in Joy. Occurrences of >r and
+Writing >r foo r> is analogous to '[ foo ] dip' in Joy. Occurrences of
-r> must be balanced within a single word definition.
+>r and r> must be balanced within a single word definition.
 
 Linked list deconstruction:
 
@@ -122,7 +146,80 @@ uncons ( [ x | y ] -- x y )
 
 * Variables
 
-* Continuations
+Factor supports a notion of ``variables''. Whereas the stack is used for
+transient, intermediate values, variables are used for more permanent
+data.
 
-* Reflection
+Variables are retreived and stored using the 'get' and 'set' words. For
+example:
+
+    USE: math
+    USE: namespaces
+    USE: prettyprint
+
+    "~" get .
+    ==> "/home/slava"
+
+    5 "x" set
+    "x" get 2 * .
+    ==> 10
+
+The set of available variables is determined using ``dynamic scope''.
+A ``namespace'' is a set of variable name/value pairs. Namespaces can be
+pushed onto the ``name stack'', and later popped. The 'get' word
+searches all namespaces on the namestack in turn. The 'set' word stores
+a variable value into the namespace at the top of the name stack.
+
+While it is possible to push/pop the namestack directly using the words
+>n and n>, most of the time using the 'bind' combinator is more
+desirable.
+
+Good examples of namespace use are found in the I/O system.
+
+Factor provides two sets of words for working with I/O streams: words
+whose stream operand is specified on the stack (freadln, fwrite,
+fprint...) and words that use the standard input/output stream (read,
+write, print...).
+
+An I/O stream is a namespace with a slot for each I/O operation. I/O
+operations taking an explicit stream operand are all defined as follows:
+
+: freadln ( stream -- string )
+    [ "freadln" get call ] bind ;
+
+: fwrite ( string stream -- )
+    [ "fwrite" get call ] bind ;
+
+: fclose ( stream -- )
+    [ "fclose" get call ] bind ;
+
+( ... et cetera )
+
+The second set of I/O operations, whose stream is the implicit 'standard
+input/output' stream, are defined as follows:
+
+: read ( -- string )
+    "stdio" get freadln ;
+
+: write ( string -- )
+    "stdio" get fwrite ;
+
+( ... et cetera )
+
+In the global namespace, the 'stdio' variable corresponds to a stream
+whose operations read/write from the standard file descriptors 0 and 1.
+
+However, the 'with-stream' combinator provides a way to rebind the
+standard input/output stream for the duration of the execution of a
+single quotation. The following example writes the source of a word
+definition to a file named 'definition.txt':
+
+    USE: prettyprint
+    USE: streams
+
+    "definition.txt" <filebw> [ "with-stream" see ] with-stream
+
+The 'with-stream' word is implemented by pushing a new namespace on the
+namestack, setting the 'stdio' variable therein, and execution the given
+quotation.
 

TODO.FACTOR.txt

@@ -1,10 +1,16 @@
+[error] SideKick$BufferChangeHandler: We have cplusplus.xml (/home/slava/jEdit/modes/) but got event for DefaultInputHandler.java (/home/slava/jEdit/org/gjt/sp/jedit/gui/)
+[error] SideKick$BufferChangeHandler: We have cplusplus.xml (/home/slava/jEdit/modes/) but got event for DefaultInputHandler.java (/home/slava/jEdit/org/gjt/sp/jedit/gui/)
+
+- dec> bin> oct> hex> throw errors
+- parse-number doesn't
+- eval with multilien strings and #!
+- redefining a word doesn't clear comments
 - quit responder breaks with multithreading
 - nicer way to combine two paths
 - don't show listener on certain commands
 - plugin should not exit jEdit on fatal errors
 - wordpreview: don't show for string literals and comments
 - alist -vs- assoc terminology
-- NPE in activate()/deactivate()
 - write-icon kind of messy; " " should be output by the listener
 - f usages. --> don't print all words
 - file responder: don't show full path in title
@@ -13,7 +19,6 @@
 - jedit ==> jedit-word, jedit takes a file name
 - introduce ifte* and ?str-head/?str-tail where appropriate
 - namespace clone drops static var bindings
-- when running (inf, .factor-rc not loaded
 
 + bignums:
 
@@ -26,7 +31,9 @@
 
 + docs:
 
-- examples of assoc usage
+- explain how log uses >polar and rect>
+- when* unless*
+- simple i/o section
 - unparse examples, and difference from prettyprint
 - review doc formatting with latex2html
 - recursion -vs- iteration in vectors chapter, and combinator

build.xml

@@ -56,6 +56,7 @@
 				<include name="org/**/*.class"/>
 				<include name="*.factor"/>
 				<include name="doc/**/*.html"/>
+				<include name="doc/**/*.png"/>
 				<include name="doc/*.html"/>
 				<include name="Factor.manifest"/>
 			</fileset>

factor/jedit/FactorPlugin.props

@@ -3,7 +3,7 @@
 plugin.factor.jedit.FactorPlugin.activate=startup
 
 plugin.factor.jedit.FactorPlugin.name=Factor
-plugin.factor.jedit.FactorPlugin.version=0.65
+plugin.factor.jedit.FactorPlugin.version=0.66
 plugin.factor.jedit.FactorPlugin.author=Slava Pestov
 plugin.factor.jedit.FactorPlugin.docs=/doc/jedit/index.html
 

library/httpd/wiki-responder.factor

@@ -32,95 +32,41 @@ USE: html
 USE: lists
 USE: logic
 USE: kernel
+USE: math
 USE: namespaces
+USE: parser
 USE: regexp
 USE: stdio
 USE: stack
 USE: strings
+USE: words
 
 USE: httpd
 USE: httpd-responder
 
-: wiki-word-regexp ( -- regexp )
+! : wiki-word? ( word -- ? )
-    "((?:[A-Z][a-z0-9]*){2,})" ;
+!     #! A WikiWord starts with a capital and contains more than
-
+!     #! one capital letter.
-: wiki-word? ( word -- ? )
+!     dup str-length 0 > [
-    wiki-word-regexp re-matches ;
+!         0 over str-nth LETTER? [
-
+!             0 swap [ LETTER? [ succ ] when ] str-each 1 = not
-: wiki-word-links ( str -- str )
+!         ] [
-    wiki-word-regexp "$1" "$1" re-replace ;
+!             drop f
-
+!         ] ifte
-: get-wiki-page ( name -- text )
+!     ] [
-    "wiki" get [ get ] bind ;
+!         drop f
-
+!     ] ifte ;
-: write-wiki-page ( text -- )
+! 
-    [ chars>entities wiki-word-links write ] call ;
+! : wiki-formatting ( str -- )
-
+!     #! If a word with this name exists in the wiki-formatting
-: wiki-nodes ( -- alist )
+!     #! vocabulary, its a special text style sequence.
-    "wiki" get [ vars-values ] bind ;
+!     [ "wiki-formatting" ] search ;
-
+! 
-: search-wiki ( string -- alist )
+! : (wiki-parser) ( text -- )
-    wiki-nodes [ dupd cdr str-contains? ] subset nip ;
+!     [
-
+!         scan dup wiki-word? [
-: get-category-text ( category -- text )
+!             <a href= dup a> write </a>
-    <% search-wiki [ car % "\n" % ] each %> ;
+!         ] [
-
+!             write
-: serve-category-page ( name text -- )
+!         ] ifte " " write
-    swap [ write-wiki-page ] html-document ;
+!     ] with-parser ;
-
-: wiki-footer ( name -- )
-    "<hr>" print
-    "Edit" swap "edit?" swap cat2 write ;
-
-: serve-existing-page ( name text -- )
-    over [ write-wiki-page wiki-footer ] html-document ;
-
-: wiki-editor ( name text -- )
-    "<form action='" write
-    swap write
-    "' method='post'>" print
-    "<textarea name='text' cols='64' rows='16'>" write
-    [ chars>entities write ] when*
-    "</textarea><p>" print
-    "<input type='Submit' value='Submit'></form>" write ;
-
-: serve-edit-page ( name text -- )
-    over [
-        over wiki-word? [
-            wiki-editor
-        ] [
-            drop "Not a wiki word: " write write
-        ] ifte
-    ] html-document ;
-
-: wiki-get-responder ( argument -- )
-    serving-html
-
-    dup "edit?" str-head? dup [
-        nip dup get-wiki-page serve-edit-page
-    ] [
-        drop dup "Category" str-head? [
-            dup get-category-text serve-category-page
-        ] [
-            dup get-wiki-page dup [
-                serve-existing-page
-            ] [
-                serve-edit-page
-            ] ifte
-        ] ifte
-    ] ifte ;
-
-: set-wiki-page ( name text -- )
-    "wiki" get [ put ] bind ;
-
-: wiki-post-responder ( argument -- )
-    #! Handle a page edit.
-    "response" get dup [
-        "text=" str-head? dup [
-            2dup set-wiki-page serve-existing-page
-        ] [
-            2drop bad-request
-        ] ifte
-    ] [
-        2drop bad-request
-    ] ifte ;

library/platform/jvm/processes.factor

@@ -39,15 +39,13 @@ USE: stack
     jinvoke-static ;
 
 : (pipe) ( args -- process )
-    f cwd <file> jvm-runtime
+    jvm-runtime
-    [
+    [ [ "java.lang.String" ] ]
-        [ "java.lang.String" ]
+    "java.lang.Runtime" "exec" jinvoke ;
-        [ "java.lang.String" ]
-        "java.io.File"
-    ] "java.lang.Runtime" "exec" jinvoke ;
 
 : close-stderr ( process -- )
-    [ ] "java.lang.Process" "getErrorStream" jinvoke close ;
+    [ ] "java.lang.Process" "getErrorStream" jinvoke
+    close-java-stream ;
 
 : pipe ( args -- stream )
     #! Start a process, and return a stream for communicating

library/platform/jvm/stream.factor

@@ -35,7 +35,7 @@ USE: namespaces
 USE: stack
 USE: strings
 
-: close ( stream -- )
+: close-java-stream ( stream -- )
     [
         [ "java.io.InputStream" is ] [
             [ ] "java.io.InputStream" "close" jinvoke
@@ -97,8 +97,8 @@ USE: strings
     "out" get [ ] "java.io.OutputStream" "flush" jinvoke ;
 
 : <byte-stream>/fclose ( -- )
-    "in" get  [ close ] when* 
+    "in" get  [ close-java-stream ] when* 
-    "out" get [ close ] when* ;
+    "out" get [ close-java-stream ] when* ;
 
 : <bin> ( in -- in )
     [ "java.io.InputStream" ] "java.io.BufferedInputStream" jnew ;
@@ -150,8 +150,8 @@ USE: strings
     "out" get [ ] "java.io.Writer" "flush" jinvoke ;
 
 : <char-stream>/fclose ( -- )
-    "in" get  [ close ] when* 
+    "in" get  [ close-java-stream ] when* 
-    "out" get [ close ] when* ;
+    "out" get [ close-java-stream ] when* ;
 
 : <char-stream> ( in out -- stream )
     #! Creates a new stream for reading from the

library/platform/native/parse-numbers.factor

@@ -50,52 +50,37 @@ USE: unparser
         [ drop t ] [ not-a-number ]
     ] cond ;
 
-: >digit ( n -- ch )
+: digit ( num digit base -- num )
-    dup 10 < [ CHAR: 0 + ] [ 10 - CHAR: a + ] ifte ;
+    2dup <= [ rot * + ] [ not-a-number ] ifte ;
 
-: digit ( num digit -- num )
+: (str>integer) ( str base -- num )
-    "base" get swap 2dup > [
+    over str-length 0 = [
-        >r * r> +
-    ] [
-        not-a-number
-    ] ifte ;
-
-: (str>integer) ( str -- num )
-    dup str-length 0 = [
         not-a-number
     ] [
-        0 swap [ digit> digit ] str-each
+        0 rot [ digit> pick digit ] str-each nip
     ] ifte ;
 
-: str>integer ( str -- num )
+: str>integer ( str base -- num )
-    #! Parse a string representation of an integer.
+    swap "-" ?str-head [
-    dup str-length 0 = [
+        swap (str>integer) neg
-        drop not-a-number
     ] [
-        dup "-" str-head? dup [
+        swap (str>integer)
-            nip (str>integer) neg
-        ] [
-            drop (str>integer)
-        ] ifte
     ] ifte ;
 
 : str>ratio ( str -- num )
     dup CHAR: / index-of str//
-    swap str>integer swap str>integer / ;
+    swap 10 str>integer swap 10 str>integer / ;
 
 : str>number ( str -- num )
     #! Affected by "base" variable.
     [
-        [ "/" swap str-contains? ] [ str>ratio   ]
+        [ "/" swap str-contains? ] [ str>ratio      ]
-        [ "." swap str-contains? ] [ str>float   ]
+        [ "." swap str-contains? ] [ str>float      ]
-        [ drop t                 ] [ str>integer ]
+        [ drop t                 ] [ 10 str>integer ]
     ] cond ;
 
 : base> ( str base -- num/f )
-    [
+    [ str>integer ] [ [ 2drop f ] when ] catch ;
-        "base" set
-        [ str>number ] [ [ drop f ] when ] catch
-    ] with-scope ;
 
 : bin> ( str -- num )
     #! Convert a binary string to a number.
@@ -114,4 +99,5 @@ USE: unparser
     16 base> ;
 
 ! Something really sucks about these words here
-: parse-number ( str -- num ) dec> ;
+: parse-number ( str -- num )
+    [ str>number ] [ [ drop f ] when ] catch ;

library/platform/native/parse-syntax.factor

@@ -63,7 +63,7 @@ USE: unparser
 : } nreverse list>vector parsed ; parsing
 
 ! Do not execute parsing word
-: POSTPONE: ( -- ) scan parse-word parsed ; parsing
+: POSTPONE: ( -- ) scan-word parsed ; parsing
 
 ! Colon defs
 : CREATE
@@ -186,7 +186,7 @@ USE: unparser
 
 : BASE: ( base -- )
     #! Read a number in a specific base.
-    "base" get >r "base" set scan number, r> "base" set ;
+    scan swap str>integer parsed ;
 
 : HEX: 16 BASE: ; parsing
 : DEC: 10 BASE: ; parsing

library/platform/native/parser.factor

@@ -61,9 +61,19 @@ USE: unparser
     #! read ahead in the input stream.
     t "parsing" word set-word-property ;
 
-: <parsing "line" set 0 "col" set ;
+: end? ( -- ? )
-: parsing> "line" off "col" off ;
+    "col" get "line" get str-length >= ;
-: end? ( -- ? ) "col" get "line" get str-length >= ;
+
+: (with-parser) ( quot -- )
+    end? [ drop ] [ [ call ] keep (with-parser) ] ifte ;
+
+: with-parser ( text quot -- )
+    #! Keep calling the quotation until we reach the end of the
+    #! input.
+    swap "line" set 0 "col" set
+    (with-parser)
+    "line" off "col" off ;
+
 : ch ( -- ch ) "col" get "line" get str-nth ;
 : advance ( -- ) "col" succ@ ;
 
@@ -116,12 +126,14 @@ USE: unparser
         r> substring
     ] ifte ;
 
-: parse-word ( str -- obj )
+: scan-word ( -- obj )
-    dup "use" get search dup [
+    scan dup [
-        nip
+        dup "use" get search dup [
-    ] [
+            nip
-        drop str>number
+        ] [
-    ] ifte ;
+            drop str>number
+        ] ifte
+    ] when ;
 
 : parsed| ( obj -- )
     #! Some ugly ugly code to handle [ a | b ] expressions.
@@ -137,15 +149,12 @@ USE: unparser
 : parsed ( obj -- )
     over "|" = [ nip parsed| "]" expect ] [ swons ] ifte ;
 
-: number, ( num -- )
+: (parse) ( str -- )
-    str>number parsed ;
-
-: word, ( str -- )
     [
-        parse-word dup parsing? [ execute ] [ parsed ] ifte
+        scan-word [
-    ] when* ;
+            dup parsing? [ execute ] [ parsed ] ifte
-
+        ] when*
-: (parse) <parsing [ end? not ] [ scan word, ] while parsing> ;
+    ] with-parser ;
 
 : parse ( str -- code )
     #! Parse the string into a parse tree that can be executed.

library/platform/native/unparser.factor

@@ -39,6 +39,9 @@ USE: stdio
 USE: strings
 USE: words
 
+: >digit ( n -- ch )
+    dup 10 < [ CHAR: 0 + ] [ 10 - CHAR: a + ] ifte ;
+
 : integer% ( num radix -- )
     tuck /mod >digit % dup 0 > [
         swap integer%
@@ -118,6 +121,13 @@ DEFER: unparse
     #! output.
     "." over str-contains? [ ".0" cat2 ] unless ;
 
+: unparse-unknown ( obj -- str )
+    <% "#<" %
+    dup type-of type-name %
+    " @ " % 
+    address-of unparse %
+    ">" % %> ;
+
 : unparse ( obj -- str )
     [
         [ t eq?    ] [ drop "t" ]
@@ -128,5 +138,5 @@ DEFER: unparse
         [ float?   ] [ unparse-float fix-float ]
         [ complex? ] [ unparse-complex ]
         [ string?  ] [ unparse-str ]
-        [ drop t   ] [ <% "#<" % type-of type-name % ">" % %> ]
+        [ drop t   ] [ unparse-unknown ]
     ] cond ;

library/prettyprint.factor

@@ -218,3 +218,8 @@ DEFER: prettyprint*
 : .s datastack  . ;
 : .r callstack  . ;
 : .c catchstack . ;
+
+! For integers only
+: .b >bin print ;
+: .o >oct print ;
+: .h >hex print ;

library/stdio.factor

@@ -81,10 +81,11 @@ USE: streams
     #! Print a newline to standard output.
     "\n" write ;
 
+: close ( -- )
+    "stdio" get fclose ;
+
 : with-stream ( stream quot -- )
-    [
+    [ swap "stdio" set  [ close rethrow ] catch ] with-scope ;
-        swap "stdio" set [ "stdio" get fclose rethrow ] catch
-    ] with-scope ;
 
 : with-string ( quot -- str )
     #! Execute a quotation, and push a string containing all

library/test/parse-number.factor

@@ -7,130 +7,130 @@ USE: unparser
 
 [ f ]
 [ f ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ "12345abcdef" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ t ]
 [ "-12" ]
-[ dec> 0 < ]
+[ parse-number 0 < ]
 test-word
 
 [ f ]
 [ "--12" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ "-" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ "e" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ "100.0" ]
 [ "1.0e2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ "-100.0" ]
 [ "-1.0e2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ "0.01" ]
 [ "1.0e-2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ "-0.01" ]
 [ "-1.0e-2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ f ]
 [ "-1e-2e4" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ "3.14" ]
 [ "3.14" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ f ]
 [ "." ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ ".e" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ "101.0" ]
 [ "1.01e2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ "-101.0" ]
 [ "-1.01e2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ "1.01" ]
 [ "101.0e-2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ "-1.01" ]
 [ "-101.0e-2" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word
 
 [ 5 ]
 [ "10/2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ -5 ]
 [ "-10/2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ -5 ]
 [ "10/-2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ 5 ]
 [ "-10/-2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ "10.0/2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ "1e1/2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ f ]
 [ "e/2" ]
-[ dec> ]
+[ parse-number ]
 test-word
 
 [ "33/100" ]
 [ "66/200" ]
-[ dec> unparse ]
+[ parse-number unparse ]
 test-word