Documentation and module system updates

contrib/splay-trees.factor

@@ -112,6 +112,6 @@ USING: namespaces words ;
 
 <splay-tree> "foo" set
 all-words [ dup word-name "foo" get set-splay ] each
-all-words [ dup word-name "foo" get get-splay drop ] each
+all-words [ word-name "foo" get get-splay drop ] each
 
 PROVIDE: splay-trees ;

doc/handbook/cookbook.facts

@@ -179,8 +179,10 @@ ARTICLE: "cookbook-sources" "Source file cookbook"
 "You can load source files from the Factor source code directory using " { $link run-resource } ":"
 { $code "\"/examples/lcd.factor\" run-resource" }
 { $references
-    "You can learn more about parsing external content:"
+    { }
-    "parse-stream"
+    "sources"
+    "modules"
+    "parser"
 } ;
 
 ARTICLE: "cookbook-io" "I/O cookbook"

doc/handbook/parser.facts

@@ -8,26 +8,17 @@ $terpri
 "As documented in " { $link "vocabulary-search" } ", the parser looks up words in the vocabulary search path. New word definitions are added to the current vocabulary. These two parameters are stored in a pair of variables:"
 { $subsection use }
 { $subsection in }
-"There are two simple ways to call the parser:"
+"There are two simple ways to call the parser which use the vocabulary search path currently in scope, which is usually the default listener search path:"
 { $subsection parse }
 { $subsection eval }
-"More sophisticated facilities exist, too."
+"The parser can also take input from a stream (" { $link "streams" } "):"
-{ $subsection "parse-stream" }
+{ $subsection parse-stream }
-"The parser can be extended with new parsing word definitions."
-{ $subsection "parsing-words" } ;
-
-ARTICLE: "parse-stream" "Parsing from streams"
 "By convention, words for parsing input from streams use a certain default vocabulary search path:"
 { $subsection file-vocabs }
-"The central word for parsing input from a stream:"
+"More details on working with source files can be found in " { $link "sources" } "."
-{ $subsection parse-stream }
+$terpri
-"Utilities for working with files:"
+"The parser can be extended with new parsing word definitions."
-{ $subsection parse-file }
+{ $subsection "parsing-words" } ;
-{ $subsection run-file }
-"Utilities for working with Factor libarary files:"
-{ $subsection resource-path }
-{ $subsection parse-resource }
-{ $subsection run-resource } ;
 
 ARTICLE: "parsing-words" "Parsing words"
 "Parsing words execute at parse time, and therefore can access and modify the state of the parser, as well as add objects to the parse tree. Parsing words are a difficult concept to grasp, so this section has several examples and explains the workings of some of the parsing words provided in the library."

doc/handbook/tools.facts

@@ -1,17 +1,24 @@
-USING: errors help image inspector kernel listener memory parser
+USING: errors help image inspector io kernel listener memory
-prettyprint sequences test walker words ;
+modules parser prettyprint sequences test walker words ;
 
 ARTICLE: "tools" "Development tools"
 "This section covers words which are used during development, and not usually invoked directly by user code."
+$terpri
+"Interactive development:"
 { $subsection "listener" }
 { $subsection "debugger" }
-{ $subsection "inspector" }
-{ $subsection "memory" }
 { $subsection "word-introspection" }
+{ $subsection "inspector" }
+{ $subsection "walker" }
 { $subsection "annotations" }
-{ $subsection "timing" }
+"Working on a project:"
+{ $subsection "sources" }
+{ $subsection "modules" }
+{ $subsection "images" }
 { $subsection "unit-test" }
-{ $subsection "images" } ;
+"Advanced features:"
+{ $subsection "memory" }
+{ $subsection "timing" } ;
 
 ARTICLE: "listener" "The listener"
 "The listener evaluates Factor expressions read from a stream. The listener is the primary interface to the Factor runtime. Typically, you write Factor code in a text editor, then load it using the listener and test it."
@@ -26,9 +33,6 @@ $terpri
 { $subsection .s }
 "Note that calls to " { $link .s } " can also be included inside words as a debugging aid, however a more convenient way to achieve this is to use the annotation facility. See " { $link "annotations" } "."
 $terpri
-"Source files can be loaded in:"
-{ $subsection run-file }
-{ $subsection reload }
 "You can start a nested listener or exit a listener using the following words:"
 { $subsection listener }
 { $subsection bye }
@@ -39,6 +43,42 @@ $terpri
 "Finally, the multi-line expression reading word can be used independently of the rest of the listener:"
 { $subsection read-multiline } ;
 
+ARTICLE: "sources" "Source files"
+"The simplest way to distribute a piece of Factor code is in the form of a source file. Source files can be loaded in the listener:"
+{ $subsection run-file }
+"Another way to load a source file is to provide a path relative to the Factor installation directory:"
+{ $subsection run-resource }
+"Words remember which source file defines them; this can be used to update word definitions during development:"
+{ $subsection reload }
+"Details on the Factor source parser itself can be found in " { $link "parser" } "."
+$terpri
+"User-contributed libraries in the " { $snippet "contrib/" } " directory of the Factor distribution should be loaded via the high-level module system instead of the above words (" { $link "modules" } ")." ;
+
+ARTICLE: "modules" "Modules and contributed libraries"
+"The Factor distribution includes a selection of contributed libraries in the " { $snippet "contrib/" } " directory, which are managed by a simple module system."
+$terpri
+"To load " { $snippet "contrib/aim/" } " for instance, you simply issue the following command in the listener:"
+{ $code "\"aim\" require" }
+"The " { $link require } " word will load all dependencies and source files of the " { $snippet "aim" } " module."
+{ $subsection require }
+{ $subsection (require) }
+"To define a new module named " { $snippet "frob" } ", create one of the following two files:"
+{ $code "contrib/frob.factor" "contrib/frob/load.factor" }
+"The module definition file should first list all required modules:"
+{ $subsection POSTPONE: REQUIRES: }
+"These modules will be loaded first. Next, the source files and unit tests, if any, have to be registered:"
+{ $subsection POSTPONE: PROVIDE: }
+"It is important that the module path name matches the module name passed to " { $link POSTPONE: PROVIDE: } ", which should be " { $snippet "frob" } " in the above example."
+$terpri
+"Here is a simple module definition taken from " { $snippet "contrib/concurrency/load.factor" } ":"
+{ $code "REQUIRES: dlists ;"
+"PROVIDE: concurrency"
+"{ \"concurrency.factor\" }"
+"{ \"concurrency-examples.factor\" \"concurrency-tests.factor\" } ;" }
+"The following two words are useful during development of modules:"
+{ $subsection load-module }
+{ $subsection test-module } ;
+
 ARTICLE: "debugger" "The debugger"
 "If an expression entered in the listener throws an error, the error is printed to the output stream. A number of words facilitate interactive debugging of the error:"
 { $subsection :s }

library/help/markup.factor

@@ -150,6 +150,8 @@ M: link summary "Link: " swap link-name append ;
 : $definition ( content -- )
     "Definition" $heading $see ;
 
+: $call ( content -- ) first call ;
+
 : $curious ( content -- )
     "For the curious..." $heading print-element ;
 

library/help/syntax.facts

@@ -0,0 +1,24 @@
+USING: help ;
+
+HELP: HELP: "word stack-effect content..."
+{ $values { "word" "a word" } { "stack-effect" "a stack effect or syntax string" } { "content" "markup elements" } }
+{ $description "Defines documentation for a word." }
+{ $examples
+    { $code
+        ": foo 2 + ;"
+        "HELP: foo \"( m -- n )\""
+        "{ $values { \"m\" \"an integer\" } { \"n\" \"an integer\" } }"
+        "{ $description \"Increments a value by 2.\" } ;"
+        "\ foo help"
+    }
+} ;
+
+HELP: ARTICLE: "topic title content..."
+{ $values { "topic" "a string" } { "title" "a string" } { "content" "markup elements" } }
+{ $description "Defines a help article." }
+{ $examples
+    { $code
+        "ARTICLE: \"example\" \"An example article\""
+        "\"Hello world.\" ;"
+    }
+} ;

library/io/c-streams.facts

@@ -45,7 +45,7 @@ HELP: <c-stream> "( in out -- stream )"
 { $notes "Usually C streams are only used during bootstrap, and non-blocking OS-specific I/O routines are used during normal operation." } ;
 
 HELP: fopen "( path mode -- alien )"
-{ $values { "path" "a pathname string" } { "mode" "an access mode specifier" } { "alien" "a C FILE* handle" } }
+{ $values { "path" "a path name string" } { "mode" "an access mode specifier" } { "alien" "a C FILE* handle" } }
 { $description "Opens a file named by " { $snippet "path" } ". The " { $snippet "mode" } " parameter should be something like " { $snippet "\"r\"" } " or " { $snippet "\"rw\"" } "; consult the " { $snippet "fopen(3)" } " manual page for details." }
 { $errors "Throws an error if the file could not be opened." }
 { $notes "User code should call " { $link <file-reader> } " or " { $link <file-writer> } " to get a high level stream." } ;

library/io/files.facts

@@ -1,17 +1,17 @@
 USING: help io ;
 
 HELP: cwd "( -- path )"
-{ $values { "path" "a pathname string" } }
+{ $values { "path" "a path name string" } }
 { $description "Outputs the current working directory of the Factor process." }
 { $see-also cd } ;
 
 HELP: cd "( path -- )"
-{ $values { "path" "a pathname string" } }
+{ $values { "path" "a path name string" } }
 { $description "Changes the current working directory of the Factor process." }
 { $see-also cwd } ;
 
 HELP: stat "( path -- array/f )"
-{ $values { "path" "a pathname string" } { "array/f" "a four-element array or " { $link f } } }
+{ $values { "path" "a path name string" } { "array/f" "a four-element array or " { $link f } } }
 { $description
     "If the file does not exist, outputs " { $link f } ". Otherwise, outputs a four-element array:"
     { $list

library/modules.factor

@@ -2,7 +2,7 @@
 ! See http://factorcode.org/license.txt for BSD license.
 IN: modules
 USING: compiler hashtables io kernel namespaces parser sequences
-words ;
+test words ;
 
 TUPLE: module name files tests ;
 
@@ -27,26 +27,23 @@ H{ } clone modules set-global
 
 : module modules get hash ;
 
-: require ( name -- )
+: load-module ( name -- )
-    dup module [
+    "Loading module " write dup write "..." print
-        drop
+    [ dup module-def run-resource ] assert-depth drop ;
-    ] [
+
-        "Loading module " write dup write "..." print
+: (require) ( name -- )
-        module-def run-resource
+    dup module [ drop ] [ load-module ] if ;
-    ] if ;
+
+: require ( name -- ) (require) compile-all ;
 
 : run-resources ( seq -- )
     bootstrapping? get
     [ parse-resource % ] [ run-resource ] ? each ;
 
-: load-module ( module -- ) module-files run-resources ;
-
 : provide ( name files tests -- )
-    <module> dup load-module
+    <module> dup module-files run-resources
     dup module-name modules get set-hash ;
 
-: reload-module ( name -- ) module load-module ;
-
 : test-module ( name -- ) module module-tests run-resources ;
 
 : test-modules ( -- ) modules hash-keys [ test-module ] each ;

library/modules.facts

@@ -0,0 +1,57 @@
+IN: modules
+USING: help io ;
+
+HELP: module-path "( name -- path )"
+{ $values { "name" "a module name string" } { "path" "a path name string" } }
+{ $description "Outputs the location of the module named " { $snippet "name" } "." } ;
+
+HELP: module-paths "( name seq -- newseq )"
+{ $values { "name" "a module name string" } { "seq" "a sequence of strings" } { "newseq" "a new sequence of path name strings" } }
+{ $description "Prepend the location of the module named " { $snippet "name" } " to every file name in " { $snippet "seq" } "." } ;
+
+HELP: module-def "( name -- path )"
+{ $values { "name" "a module name string" } { "path" "a path name string" } }
+{ $description "Outputs the location of the module definition file. This word looks for the module definition in two locations:"
+    { $list
+        { $snippet "contrib/" { $emphasis "name" } ".factor" }
+        { $snippet "contrib/" { $emphasis "name" } "/load.factor" }
+    }
+}
+{ $notes "This file is loaded by " { $link require } ", and should contain the necessary " { $link POSTPONE: REQUIRES: } " and " { $link POSTPONE: PROVIDE: } " declarations for the module." } ;
+
+HELP: modules f
+{ $description "Variable. Hashtable mapping loaded module names to " { $link module } " instances." }
+{ $see-also require reload-module } ;
+
+HELP: load-module "( name -- )"
+{ $values { "name" "a module name string" } }
+{ $description "Runs the module definition file given by " { $link module-def } ", which loads the module's dependencies and source files." }
+{ $notes "Unless you want to explicitly reload the sources of a module (for example, after making changes), you should use " { $link (require) } " or " { $link require } " instead." } ;
+
+HELP: (require) "( name -- )"
+{ $values { "name" "a module name string" } }
+{ $description "Ensures that a module has been loaded, along with all its dependencies."
+$terpri
+"If this module is already listed in the " { $link modules } " hashtable, this word does nothing. Otherwise, it calls " { $link load-module } "." }
+{ $notes "Module definitions should use the " { $link POSTPONE: REQUIRES: } " parsing word instead. In the listener, the " { $link require } " word might be more useful since it recompiles new words after loading the module." } ;
+
+HELP: require "( name -- )"
+{ $values { "name" "a module name string" } }
+{ $description "Ensures that a module has been loaded, along with all its dependencies, and recompiles all new words. This word is only for interactive use in the listener; to avoid recompiling words multiple times, any user code which needs to load modules should call " { $link (require) } " instead." } ;
+
+HELP: run-resources "( seq -- )"
+{ $values { "seq" "a sequence of strings" } }
+{ $description "Load a collection of source files identified by resource paths (see " { $link resource-path } ")."
+$terpri
+"If bootstrapping, this word appends the top-level forms to the currently constructing quotation instead." } ;
+
+HELP: provide "( name files tests -- )"
+{ $values { "name" "a string" } { "files" "a sequence of strings" } { "tests" "a sequence of strings" } }
+{ $description "Registers a module definition and loads the source files. Usually instead of calling this word, module definitions use the parsing word " { $link POSTPONE: PROVIDE: } " instead." } ;
+
+HELP: test-module "( name -- )"
+{ $values { "name" "a module name string" } }
+{ $description "Runs the unit test files associated to the module by a previous call to " { $link provide } " or " { $link POSTPONE: PROVIDE: } "." } ;
+
+HELP: test-modules "( -- )"
+{ $description "Runs unit test files for all loaded modules." } ;

library/syntax/parse-syntax.factor

@@ -77,8 +77,6 @@ DEFER: !PRIMITIVE: parsing
 : !PROVIDE:
     scan [ { { } { } } append first2 provide ] f ; parsing
 
-: !REQUIRE: scan require compile-all ; parsing
-
 : !REQUIRES:
     string-mode on
-    [ string-mode off [ require ] each ] f ; parsing
+    [ string-mode off [ (require) ] each ] f ; parsing

library/syntax/parser.factor

@@ -93,7 +93,7 @@ global [
         "scratchpad" "syntax" "arrays" "compiler"
         "errors" "generic" "hashtables" "help" "inference"
         "inspector" "io" "jedit" "kernel" "listener" "math"
-        "memory" "namespaces" "parser" "prettyprint"
+        "memory" "modules" "namespaces" "parser" "prettyprint"
         "sequences" "shells" "strings" "styles" "test"
         "threads" "vectors" "walker" "words"
     } set-use