Note that this handbook is not a tutorial or introductory guide, nor does it cover some background material that you are expected to understand, such as object-oriented programming, higher-order functions, continuations, or general algorithm and program design.
\item[\texttt{foo/bar}] either \texttt{foo} or \texttt{bar}. For example, \texttt{str/f} means either a string, or \texttt{f}
\end{description}
If the stack effect identifies quotations, the stack effect of each quotation may be given after suffixing \texttt{|} to the whole string. For example, the following denotes a word that takes a list and a quotation and produces a new list, calling the quotation with elements of the list.
The following naming conventions are used in the Factor library.
\begin{description}
\item[\texttt{FOO:}] a parsing word that reads ahead from the input string
\item[\texttt{FOO}] a parsing word that does not read ahead, but rather takes a fixed action at parse time
\item[\texttt{FOO"}] a parsing word that reads characters from the input string until the next occurrence of \texttt{"}
\item[\texttt{foo?}] a predicate returning a boolean or generalized boolean value
\item[\texttt{foo.}] a word whose primary action is to print something, rather than to return a value. The basic case is the \texttt{.}~word, which prints the object at the top of the stack
\item[\texttt{foo*}] a variation of the \texttt{foo} word that takes more parameters
\item[\texttt{(foo)}] a word that is only useful for the implementation of \texttt{foo}
\item[\texttt{>to}] converts the object at the top of the stack to the \texttt{to} class
\item[\texttt{from>}] converts an instance of the \texttt{from} class into some canonical form
\item[\texttt{from>to}] convert an instance of the \texttt{from} class to the \texttt{to} class
\item[\texttt{>s}] move top of data stack to the \texttt{s} stack, where \texttt{s} is either \texttt{r} (call stack), \texttt{n} (name stack), or \texttt{c} (catch stack). Sometimes, libraries will define their own words following this naming convention, to implement user-defined stacks, typically stored in variables
\item[\texttt{foo-with}] a form of the \texttt{foo} combinator that takes an extra object, and passes this object on each iteration of the quotation; for example, \texttt{each-with} and \texttt{map-with}
\item[\texttt{with-foo}] executes a quotation in a namespace where \texttt{foo} is configured in a special manner; for example, \texttt{with-stream}
\item[\texttt{make-foo}] executes a quotation in a namespace where a sequence of type \texttt{foo} is being constructed; for example, \texttt{make-string}
description={a set of words in the \texttt{parser} vocabulary, primarily \texttt{parse}, \texttt{eval}, \texttt{parse-file} and \texttt{run-file}, that creates objects from their printed representations, and adds word definitions to the dictionary}}}
In Factor, an \emph{object} is a piece of data that can be identified. Code is data, so Factor syntax is actually a syntax for describing objects, of which code is a special case. Factor syntax is read by the parser. The parser performs two kinds of tasks -- it creates objects from their \emph{printed representations}, and it adds \emph{word definitions} to the dictionary (\ref{words}). The parser can be extended (\ref{parser}).
Factor syntax consists of whitespace-separated tokens. The parser tokenizes the input on whitespace boundaries, where whitespace is defined as a sequence or one or more space, tab, newline or carriage-return characters. The parser is case-sensitive, so
the following three expressions tokenize differently:
\begin{verbatim}
2X+
2 X +
2 x +
\end{verbatim}
As the parser reads tokens it makes a distinction between numbers, ordinary words, and
parsing words. Tokens are appended to the parse tree, the top level of which is a list
returned by the original parser invocation. Nested levels of the parse tree are created
by parsing words.
Here is the parser algorithm in more detail -- some of the concepts therein will be defined shortly:
\begin{itemize}
\item If the current character is a double-quote (\texttt{"}), the \texttt{"} parsing word is executed, causing a string to be read.
\item Otherwise, the next token is taken from the input. The parser searches for a word named by the token in the currently used set of vocabularies. If the word is found, one of the following two actions is taken:
\begin{itemize}
\item If the word is an ordinary word, it is appended to the parse tree.
\item If the word is a parsing word, it is executed.
\end{itemize}
Otherwise if the token does not represent a known word, the parser attempts to parse it as a number. If the token is a number, the number object is added to the parse tree. Otherwise, an error is raised and parsing halts.
description={a parser mode where token are added to the parse tree as strings, without being looked up in the dictionary or converted into numbers first. Activated by switching on the \texttt{string-mode} variable}}}
There is one exception to the above process; the parser might be placed in \emph{string mode}, in which case it simply reads tokens and appends them to the parse tree as strings. String mode is activated and deactivated by certain parsing words wishing to read input in an unstructured but tokenized manner -- see \ref{string-mode}.
description={a word that is run at parse time. Parsing words can be defined by suffixing the compound definition with \texttt{parsing}. Parsing words have the \texttt{\dq{}parsing\dq{}} word property set to true, and respond with true to the \texttt{parsing?}~word}}}
description={an object holding a code definition and set of properties. Words are organized into vocabularies, and are uniquely identified by name within a vocabulary.}}}
\wordglos
\newcommand{\vocabglos}{\glossary{
name=vocabulary,
description={a collection of words, uniquely identified by name. The hashtable of vocabularies is stored in the \texttt{vocabularies} global variable, and the \texttt{USE:}~and \texttt{USING:}~parsing words add vocabularies to the parser's search path}}}
At the interactive listener, the default search path contains many more vocabularies. The default search path depends on how the parser was invoked (\ref{parsing-quotations}).
description={the list of vocabularies that the parser looks up tokens in. You can add to this list with the \texttt{USE:} and \texttt{USING:} parsing words}}}
\useglos
The \texttt{USE:} parsing word adds a new vocabulary at the front of the search path. Subsequent word lookups by the parser will search this vocabulary first.
Consecutive \texttt{USE:} declarations can be merged into a single \texttt{USING:} declaration.
\begin{alltt}
USING: lists strings vectors ;
\end{alltt}
Due to the way the parser works, words cannot be referenced before they are defined; that is, source files must order definitions in a strictly bottom-up fashion. For a way around this, see \ref{deferred}.
description={an instance of the \texttt{integer} class, which is a disjoint union of the \texttt{fixnum} and \texttt{bignum} classes}}}
\numberglos
\newcommand{\fixnumglos}{\glossary{
name=fixnum,
description={an instance of the \texttt{fixnum} class, representing a fixed precision integer. On 32-bit systems, an element of the interval $(-2^{-29},2^{29}]$, and on 64-bit systems, the interval $(-2^{-61},2^{61}]$}}}
\fixnumglos
\newcommand{\bignumglos}{\glossary{
name=bignum,
description={an instance of the \texttt{bignum} class, representing an arbitrary-precision integer whose value is bounded by available object memory}}}
\bignumglos
The printed representation of an integer consists of a sequence of digits, optionally prefixed by a sign.
\begin{alltt}
123456
-10
2432902008176640000
\end{alltt}
Integers are entered in base 10 unless prefixed with a base change parsing word.
description={an instance of the \texttt{ratio} class, representing an exact ratio of two integers}}}
\ratioglos
The printed representation of a ratio is a pair of integers separated by a slash (\texttt{/}).
No intermediate whitespace is permitted. Either integer may be signed, however the ratio will be normalized into a form where the denominator is positive and the greatest common divisor
description={an instance of the \texttt{complex} class, representing a complex number with real and imaginary components, where both components are real numbers}}}
Many different types of objects can be constructed at parse time via literal syntax. Numbers are a special case since support for reading them is built-in to the parser. All other literals are constructed via parsing words.
If a quotation contains a literal object, the same literal object instance is used each time the quotation executes; that is, literals are ``live''.
description={an instance of the \texttt{boolean} class, either \texttt{f} or \texttt{t}. See generalized boolean}}
\glossary{
name=generalized boolean,
description={an object used as a truth value. The \texttt{f} object is false and anything else is true. See boolean}}
\glossary{
name=t,
description={the canonical truth value. The \texttt{t} class, whose sole instance is the \texttt{t} object. Note that the \texttt{t} class is not equal to the \texttt{t} object}}
\glossary{
name=f,
description={the canonical false value; anything else is true. The \texttt{f} class, whose sole instance is the \texttt{f} object. Note that the \texttt{f} class is not equal to the \texttt{f} object}}
}
\boolglos
Any Factor object may be used as a truth value in a conditional expression. The \texttt{f} object is false and anything else is true. The \texttt{f} object is also used to represent the empty list, as well as the concept of a missing value. The canonical truth value is the \texttt{t} object.
Adds the \texttt{f} and \texttt{t} objects to the parse tree.
Note that the \texttt{f} parsing word and class is not the same as the \texttt{f} object. The former can be obtained by writing \texttt{\bs~f} inside a quotation, or \texttt{POSTPONE: f} inside a list that will not be evaluated.
description={an integer whose value denotes a Unicode code point. Character values are limited to the range from $0$ to $2^{16}-1$ inclusive, however in a later release this can be upgraded to the full 21-bit Unicode space without requiring any changes to user code}}}
Adds the Unicode code point of the character represented by \emph{token} to the parse tree.
\newcommand{\escapeglos}{\glossary{
name=escape,
description={a sequence allowing a non-literal character to be inserted in a string. For a list of escapes, see \ref{escape}}}}
\escapeglos
If the token is a single-character string other than whitespace or backslash, the character is taken to be this token. If the token begins with a backslash, it denotes one of the following escape codes.
A Unicode character can be specified by its code number by writing \texttt{\bs{}u} followed by a four-digit hexadecimal. That is, the following two expressions are equivalent:
\begin{alltt}
CHAR: \bs{}u0078
78
\end{alltt}
While not useful for single characters, this syntax is also permitted inside strings.
description={an instance of the \texttt{list} class, storing a sequence of elements as a chain of zero or more conses, where the car of each cons is an element, and the cdr is either \texttt{f} or another list}}
Parses two components making up a cons cell. Note that the lists parsed with \texttt{[} and \texttt{]} are just a special case of \texttt{[[} and \texttt{]]}. The following two lines are equivalent.
\begin{alltt}
[ 1 2 3 ]
[[ 1 [[ 2 [[ 3 f ]] ]] ]]
\end{alltt}
The empty list is denoted by \texttt{f}, along with boolean falsity, and the concept of a missing value. The expression \texttt{[ ]} parses to the same object as \texttt{f}.
While words parse as themselves, a word occurring inside a quotation is executed when the quotation is called. Sometimes it is desirable to have a word be pushed on the data stack during the execution of a quotation, usually for reflective access to the word's slots.
Reads the next word from the input string and appends some \emph{code} to the parse tree that pushes the word on the stack when the code is called. The following two lines are equivalent:
Reads the next word from the input string and appends the word to the parse tree, even if it is a parsing word. For an word \texttt{foo}, \texttt{POSTPONE: foo} and \texttt{foo} are equivalent; however, if \texttt{foo} is a parsing word, the latter will execute it at parse time, while the former will execute it at runtime. Usually used inside parsing words that wish to delegate some action to a further parsing word.
Using mutable object literals in word definitions requires care, since if those objects
are mutated, the actual word definition will be changed, which is in most cases not what you would expect. Strings and lists are immutable; string buffers, vectors, hashtables and tuples are mutable.
description={an instance of a user-defined class whose metaclass is the \texttt{tuple} metaclass, storing a fixed set of elements in named slots, with optional delegation method dispatch semantics}}}
Parses a tuple. The tuple's class must follow \texttt{<<}. The element after that is always the tuple's delegate. Further elements until \texttt{>>} are specified according to the tuple's slot definition, and an error is raised if an incorrect number of elements is given.
description={an instance of the \texttt{matrix} class, representing a mathematical matrix of numbers}}}
\matrixglos
\wordtable{
\vocabulary{syntax}
\parsingword{M[}{M[}
\parsingword{]M}{]M}
}
Parses a matrix. A matrix is specified as a set of rows, and each row is written like a list and must have the same length. The following is an example:
\begin{verbatim}
M[ [ 3 -5 1 ]
[ -2 7 1/2 ] ]M
\end{verbatim}
It corresponds to the following mathematical matrix:
description={a comment describing the usage of a word. Delimited by the \texttt{\#"!} parsing word, they appear at the start of a word definition and are stored in the \texttt{""documentation""} word property}}}
\doccommentglos
Comments that begin with \texttt{\#!} are called \emph{documentation comments}.
A documentation comment has no effect on the generated parse tree, but if it is the first thing inside a word definition, the comment text is appended to the string stored in the word's \texttt{"documentation"} property.
description={A string of the form \texttt{( \emph{inputs} -- \emph{outputs} )}, where the inputs and outputs are a whitespace-separated list of names or types. The top of the stack is the right-most token on both sides.}}
\newcommand{\stackcommentglos}{\glossary{
name=stack effect comment,
description={a comment describing the inputs and outputs of a word. Delimited by \texttt{(} and \texttt{}), they appear at the start of a word definition and are stored in the \texttt{""stack-effect""} word property}}}
\stackcommentglos
Comments delimited by \texttt{(} and \texttt{)} are called \emph{stack effect comments}. By convention they are placed at the beginning of a word definition to document the word's inputs and outputs:
A stack effect comment has no effect on the generated parse tree, but if it is the first thing inside a word definition, the word's \texttt{"stack-effect"} property is set to the comment text.
Word properties are described in \ref{word-props}.
as the next word in the quotation would expect them. Their behavior can be understood entirely in terms of their stack effects, which are given in table \ref{shuffles}.
Try to avoid the complex shuffle words such as \texttt{rot} and \texttt{2dup} as much as possible, for they make data flow harder to understand. If you find yourself using too many shuffle words, or you're writing
a stack effect comment in the middle of a compound definition to keep track of stack contents, it is
a good sign that the word should probably be factored into two or
description={holds quotations waiting to be called. When a quotation is called with \texttt{call}, or when a compound word is executed, the previous call frame is pushed on the call stack, and the new quotation becomes the current call frame}}}
\csglos
\newcommand{\cfglos}{\glossary{
name=call frame,
description=the currently executing quotation}}
\cfglos
\glossary{
name=interpreter,
description=executes quotations by iterating them and recursing into nested definitions. see compiler}
The Factor interpreter executes quotations. Quotations are lists, and since lists can contain any Factor object, they can contain words. It is words that give quotations their operational behavior, as you can see in the following description of the interpreter algorithm.
\begin{itemize}
\item If the call frame is \texttt{f}, the call stack is popped and becomes the new call frame.
\item If the car of the call frame is a word, the word is executed:
\begin{itemize}
\item If the word is a symbol, it is pushed on the data stack. See \ref{symbols}.
\item If the word is a compound definition, the current call frame is pushed on the call stack, and the new call frame becomes the word definition. See \ref{colondefs}.
\item If the word is compiled or primitive, the interpreter jumps to a machine code definition. See \ref{primitives}.
\item If the word is undefined, an error is raised. See \ref{deferred}.
\end{itemize}
\item Otherwise, the car of the call frame is pushed on the data stack.
\item The call frame is set to the cdr, and the loop continues.
description=a word taking quotations or other words as input}
The following pair of words invokes the interpreter reflectively. They are used to implement \emph{combinators}, which are words that take code from the stack. Combinator definitions must be followed by the \texttt{inline} word to mark them as inline in order to compile; for example:
Push the current call frame on the call stack, and set the call stack to the given quotation. Conceptually: calls the quotation, as if its definition was substituted at the location of the \texttt{call}.
description=the elimination of call stack pushes when making a tail call}}
When a call is made to a quotation from the last word in the call frame, there is no
purpose in pushing the empty call frame on the call stack. Therefore the last call in a quotation does not grow the call stack, and tail recursion executes in bounded space.
Because of the way the interpreter is described in \ref{quotations}, the top of the call stack is not accessed during the execution of a quotation; it is only popped when the interpreter reaches the end of the quotation. In effect, the call stack can be used as a temporary storage area, as long as pushes and pops are balanced out within a single quotation.
It is very important to balance usages of \texttt{>r} and \texttt{r>} within a single quotation or word definition.
\begin{verbatim}
: the-good >r 2 + r> * ; ! Okay
: the-bad >r 2 + ; ! Runtime error
: the-ugly r> ; ! Runtime error
\end{verbatim}
Basically, the rule is you must leave the call stack in the same state as you found it, so that when the current quotation finishes executing, the interpreter can return to the caller.
One exception is that when \texttt{ifte} occurs as the last word in a definition, values may be pushed on the call stack before the condition value is computed, as long as both branches of the \texttt{ifte} pop the values off the call stack before returning.
There are some words that combine shuffle words with \texttt{call}. They are useful in the implementation of higher-order words taking quotations as inputs.
The \texttt{cond} is a generalized boolean. If it is \texttt{f}, the \texttt{false} quotation is called, and if \texttt{cond} is any other value, the \texttt{true} quotation is called. The condition flag is removed from the stack before either quotation executes.
Note that in general, both branches should have the same stack effect. Not only is this good style that makes the word easier to understand, but also unbalanced conditionals cannot be compiled (\ref{compiler}).
This pair are minor variations on \texttt{ifte} where only one branch is specified. The other is implicitly \texttt{[ ]}. They are implemented in the trivial way:
\begin{verbatim}
: when [ ] ifte ; inline
: unless [ ] swap ifte ; inline
\end{verbatim}
The \texttt{ifte} word removes the condition flag from the stack before calling either quotation. Sometimes this is not desirable, if the condition flag is serving a dual purpose as a value to be consumed by the \texttt{true} quotation. The \texttt{ifte*} word exists for this purpose.
If the condition is true, it is retained on the stack before the \texttt{true} quotation is called. Otherwise, the condition is removed from the stack and the \texttt{false} quotation is called. The following two lines are equivalent:
If the condition is \texttt{f}, the \texttt{false} quotation is called with the \texttt{default} value on the stack. Otherwise, the \texttt{true} quotation is called with the condition on the stack. The following two lines are equivalent:
Outputs \texttt{t} if at least one of the inputs is true.
An alternative set of logical operations operate on individual bits of integers bitwise, rather than generalized boolean truth values. They are documented in \ref{bitwise}.
description=an object representing the future of the computation}}
\contglos
At any point in the execution of a Factor program, the \emph{current continuation} represents the future of the computation. This object can be captured with the \texttt{callcc0} and \texttt{callcc1} words.
Calling one of these words calls the given quotation with the continuation on the stack. The continuation is itself a quotation, and calling it \emph{continues execution} at the point after the call to \texttt{callcc0} and \texttt{callcc1}. Essentially, a continuation is a snapshot of the four stacks that can be restored at a later time.
The difference between \texttt{callcc0} and \texttt{callcc1} lies in the continuation object. When \texttt{callcc1} is used, calling the continuation takes one value from the top of the data stack, and places it back on the \emph{restored} data stack. This allows idioms such as exception handling, co-routines and generators to be implemented via continuations.
description=an object representing an exceptional situation that has been detected}
Support for handling exceptional situations such as bad user input, implementation bugs, and input/output errors is provided by a pair of words, \texttt{throw} and \texttt{catch}.
Raises an exception. Execution does not continue at the point after the \texttt{throw} call. Rather, the innermost catch block is invoked, and execution continues at that point. Passing \texttt{f} as an exception will cause \texttt{throw} to do nothing.
An exception handler is established, and the \texttt{try} quotation is called.
If the \texttt{try} quotation throws an error and no nested \texttt{catch} is established, the following sequence of events takes place:
\begin{itemize}
\item the stacks are restored to their state prior to the \texttt{catch} call,
\item the exception is pushed on the data stack,
\item the \texttt{handler} quotation is called.
\end{itemize}
If the \texttt{try} quotation completes successfully, the stacks are \emph{not} restored. The \texttt{f} object is pushed, and the \texttt{handler} quotation is called.
A common idiom is that the \texttt{catch} block cleans up from the error in some fashion, then passes it on to the next-innermost catch block. The following word is used for this purpose.
Raises an exception, without saving the current stacks for post-mortem inspection. This is done so that inspecting the error stacks sheds light on the original cause of the exception, rather than the point where it was rethrown.
Here is a simple example of a word definition that attempts to convert a string representing a hexadecimal number into an integer, and instead of halting execution when the string is not valid, it simply outputs \texttt{f}.
\begin{verbatim}
: catch-hex> ( str -- n/f )
[ hex> ] [ [ drop f ] when ] catch ;
\end{verbatim}
Exception handling is implemented using a \emph{catch stack}. The \texttt{catch} word pushes the current continuation on the catch stack, and \texttt{throw} calls the continuation at the top of the catch stack with the raised exception.
\glossary{name=catch stack,
description={a stack of exception handler continuations, pushed and popped by \texttt{catch}}}
Factor implements co-operative multitasking, where the thread of control switches between tasks at explicit calls to \texttt{yield}, as well as when blocking I/O is performed. Multitasking is implemented via continuations.
Calls \texttt{quot} in a co-operative thread. The new thread begins executing immediately, and the current thread resumes when the quotation yields, either from blocking
I/O or an explicit call to \texttt{yield}. This is implemented by adding the current continuation to the run queue, then calling \texttt{quot}, and finally executing \texttt{stop} after \texttt{quot} returns.
The current state of the interpreter is determined by the contents of the four stacks. A set of words for getting and setting stack contents are the primitive building blocks for continuations, and in turn abstractions such as exception handling and multitasking.
Save and restore the call stack contents. The call stack does not include the currently executing quotation that made the call to \texttt{callstack}, since the current quotation is held in the call frame -- \ref{quotations} has details. Similarly, calling \texttt{set-callstack} will continue executing the current quotation until it returns, at which point control transfers to the quotation at the top of the new call stack.
Words are the fundamental unit of code in Factor, analogous to functions or procedures in other languages. Words are also objects, and this concept forms the basis for Factor's meta-programming facilities. Words hold two distinct pieces of information:
Sets the current vocabulary for new word definitions, and adds the vocabulary to the search path (\ref{vocabsearch}).
Parsing words add definitions to the current vocabulary. When a source file is being parsed, the current vocabulary is initially set to \texttt{scratchpad}.
Words whose names are known at parse time -- that is, most words making up your program -- can be referenced by stating their name. However, the parser itself, and sometimes code you write, will need to look up words dynamically.
The \texttt{vocabs} parameter is a list of vocabulary names. If a word with the given name is found, it is pushed on the stack, otherwise, \texttt{f} is pushed.
Creates a new word \texttt{name} in the current vocabulary. This word is intended to be called from parsing words (\ref{parsing-words}), and in fact is defined as follows:
\item Using defining words at run-time. This is a more dynamic feature that can be used to implement code generation and such, and in fact parse-time defining words are implemented in terms of run-time defining words.
A word \texttt{name} is created in the current vocabulary, and is associated with \texttt{definition}.
\begin{verbatim}
: ask-name ( -- name )
"What is your name? " write read-line ;
: greet ( name -- )
"Greetings, " write print ;
: friend ( -- )
ask-name greet ;
\end{verbatim}
By convention, the word name should be followed by a stack effect comment, and for more complex definitions, a documentation comment; see \ref{comments}.
A word \texttt{name} is created in the current vocabulary that pushes itself on the stack when executed. Symbols are used to identify variables (\ref{namespaces}) as well as for storing crufties in their properties (\ref{word-props}).
description=a word implemented as native code in the Factor runtime}}
\symbolglos
Executing a primitive invokes native code in the Factor runtime. Primitives cannot be defined through Factor code. Compiled definitions behave similarly to primitives in that the interpreter jumps to native code upon encountering them.
description={a word without a definition, created by the \texttt{DEFER:}~parsing word}}
Due to the way the parser works, words cannot be referenced before they are defined; that is, source files must order definitions in a strictly bottom-up fashion. Mutually-recursive pairs of words can be implemented by \emph{deferring} one of the words in the pair so that the second word in the pair can parse, then by replacing the deferred definition with a real one.
Create a word \texttt{name} in the current vocabulary that simply raises an error when executed. Usually, the word will be replaced with a real definition later.
Removes the word \texttt{name} from its vocabulary. Existing definitions that reference the word will continue to work, but newly-parsed occurrences of the word will not locate the forgotten definition. No exception is thrown if no such word exists.
A compound or generic word (\ref{generic}) can be given special behavior with one of the below parsing words.
\wordtable{
\vocabulary{syntax}
\parsingword{inline}{inline}
}
Marks the most recently defined word as an inline word. The compiler copies the definitions of inline words directly into the word being compiled. Combinators must be inlined in order to compile. For any other word, inlining is merely an optimization; see \ref{compiler}. Inlining does not affect the execution of the word in the interpreter, nor is inlining visible when you \texttt{see} the word (\ref{exploring-vocabs}).
\wordtable{
\vocabulary{syntax}
\parsingword{parsing}{parsing}
}
Marks the most recently defined word as a parsing word. Parsing words run at parse time. Se \ref{parsing-words}.
Retrieve and store word properties. Note that the stack effect is designed so that it is most convenient when \texttt{name} is a literal that is pushed on the stack right before executing these words. This is usually the case.
Retreives and stores a word's primitive parameter. This parameter is only used if the primitive number is 1 (compound definitions) or 2 (symbols). Note that to define a compound definition or symbol, you must use \texttt{define-compound} or \texttt{define-symbol}, as these words do not update the cross-referencing of word dependencies.
This is an even lower-level facility for working with the address containing native code to be invoked when the word is executed. The compiler sets the execution token to a location in memory containing generated code.
Updates a word's execution token according to its primitive number. When called with a compiled word, has the effect of decompiling the word. The execution token is automatically updated after a call to \texttt{set-word-primitive}.
Updates the cross-referencing database, which you will probably need to do if you mess around with any of the words in this section -- assuming Factor does not crash first, that is.
Everything in Factor is an object, where an object is a collection of slots. Each object has a unique identity, and references to objects are passed by value on the stack. It is possible to have two references to the same object, and if the object is mutated through one reference, the changes will be visible through the other reference. Not all objects are mutable; the documentation for each class details if its instances are mutable or not.
description={two objects are equal if they have the same class and if their slots are equal, or alternatively, if both are numbers that denote the same value}}
There are two distinct notions of ``sameness'' when it comes to objects. You can test if two references point to the same object, or you can test if two objects are equal in some sense, usually by having the same type and equal slot values.
Output \texttt{t} if two objects are equal, and \texttt{f} otherwise. The precise meaning of equality depends on the object's class, however usually two objects are equal if their slot values are equal. If two objects are equal, they have the same printed representation, although the converse is not always true. In particular:
\begin{itemize}
\item If no more specific method is defined, \texttt{=} calls \texttt{eq?}.
\item Two numbers are equal if they have the same numerical value.
\item Two sequences are equal if they are both instances of the same class, and if they have the same length, and elements.
\item Two hashtables are equal if they hold the same set of key/value pairs.
\item Two tuples are equal if they are of the same class and their slots are equal.
\item Two words are equal if they are the same object.
Make a fresh object that is equal to the given object. This is not guaranteed to actually copy the object; it does nothing with immutable objects, and does not copy words either. However, sequences and tuples can be cloned to obtain a new shallow copy of the original.
description={a word defined using the \texttt{GENERIC:}~parsing word. The behavior of generic words depends on the class of the object at the top of the stack. A generic word is composed of methods, where each method is specialized on a class}}
\glossary{name=method,
description={gives a generic word behavior when the top of the stack is an instance of a specific class}}
Sometimes you want a word's behavior to depend on the class of the object at the top of the stack, however implementing the word as a set of nested conditional tests is undesirable since it leads to unnecessary coupling -- adding support for a new class requires modifying the original definition of the word.
A generic word is a word whose behavior depends on the class of the
object at the top of the stack, however this behavior is defined in a
If two classes have a non-empty intersection, there is no guarantee that one is a subclass of the other. This means there is no canonical linear ordering of classes. The methods of a generic word are linearly ordered, though, and you can inspect this order using the \texttt{order} word.
Suppose you have the following definitions:
\begin{verbatim}
GENERIC: foo
M: integer foo 1 + ;
M: number foo 1 - ;
M: object foo dup 2list ;
\end{verbatim}
Since the \texttt{integer} class is strictly smaller than the \texttt{number} class, which in turn is strictly smaller than the \texttt{object} class, the ordering of methods is not surprising in this case:
However, suppose we had the following set of definitions:
\begin{verbatim}
GENERIC: describe
M: general-t describe drop "a true value" print ;
M: general-list describe drop "a list" print ;
M: object describe drop "an object" print ;
\end{verbatim}
Neither \texttt{general-t} nor \texttt{general-list} contains the other, and their intersection is the non-empty \texttt{cons} class. So the generic word system will place \texttt{object} first in the method order, however either \texttt{general-t} or \texttt{general-list} may come next, and it is pretty much a random choice that depends on hashing:
description={an object invariant that describes its shape. An object's type is constant for the lifetime of the object, and there is only a fixed number of types built-in to the run-time. See class}}
\glossary{name=built-in class,
description=see type}
Every object is an instance of to exactly one type, and the type is constant for the lifetime of the object. There is only a fixed number of types built-in to the run-time, and corresponding to each type is a \emph{built-in class}:
Outputs the canonical class of a given object. While an object may be an instance of more than one class, the canonical class is either the built-in class, or if the object is a tuple, the tuple class. Examples:
description={a class whose set of instances is the union of the set of instances of a list of member classes}}
An object is an instance of a union class if it is an instance of one of its members. Union classes are used to associate the same method with several different classes, as well as to conveniently define predicates.
description={a word with stack effect \texttt{( object -- ?~)}, or more alternatively, a class whose instances are the instances of a superclass that satisfy an arbitrary predicate}}
An object is an instance of a predicate classes if it is an instance of the predicate's parent class, and if it satisfies the predicate definition.
Defines a predicate class deriving from \texttt{parent} whose instances are the instances of \texttt{superclass} that satisfy the \texttt{predicate} quotation. The predicate quotation must have stack effect \texttt{( object -- ?~)}.
Intersection and union of classes. Note that the returned class might not be the exact desired class; for example, \texttt{object} is output if no suitable class definition could be found at all.
The constructor takes slots in left-to-right order from the stack. After construction, slots are read and written using various automatically-defined words with names of the
Define a \texttt{<class>} word that creates a tuple instance of the \texttt{class}, then applies the \texttt{definition} to this new tuple. The \texttt{definition} quotation must have stack effect \texttt{( tuple -- tuple )}.
Returns an object's delegate, or \texttt{f} if no delegate is set. Note that in this case, undefined methods will be passed to \texttt{f}; rather an error is raised immediately.
Class membership test pridicates only test if an object is a direct instance of that class. Sometimes, you need to check if an object \emph{or its delegate} is an instance of a class. This can be done with the \verb|is?| combinator.
\wordtable{
\vocabulary{generic}
\ordinaryword{is?}{is?~( object quot -- ?~)}
\texttt{quot:~obj -- ?~)}\\
}
Tests if the quotation outputs a true value when applied to the object or some object that it delegates to.
A handful of ``virtual'' sequences are provided by the library. These sequences are not backed by actual storage, but instead either compute their values, or take them from an underlying sequence. Virtual sequences are documented in \ref{virtual-seq} and include:
The following set of generic words constitutes the sequence protocol. The mutating words are not supported by all sequences; in particular, lists and strings are immutable.
An object that is an instance of a class implementing these generic words can be thought of as a sequence, and given to the words in the following sections.
Outputs the $n$th element of the sequence. Elements are numbered starting from 0, so the last element has an index one less than the length of the sequence. An exception should be thrown if an out-of-bounds index is accessed. All sequences support this operation, however with lists it has non-constant running time.
Sets the $n$th element of the sequence. Storing beyond the end of a resizable sequence such as a vector or string buffer grows the sequence. Storing to a negative index is always an error.
Tests if the two sequences have the same length and elements. This is weaker than \texttt{=}, since it does not ensure that the sequences are instances of the same class.
\wordtable{
\vocabulary{sequences}
\ordinaryword{lexi}{lexi~( s1 s2 -- n )}
}
Compares two sequences of integers lexicographically (dictionary order). The output value is one of the following:
\begin{description}
\item[Positive] indicating that \texttt{s1} follows \texttt{s2}
\item[Zero] indicating that \texttt{s1} is equal to \texttt{s2}
\item[Negative] indicating that \texttt{s1} precedes \texttt{s2}
\end{description}
\wordtable{
\vocabulary{sequences}
\ordinaryword{lexi>}{lexi> ( s1 s2 -- ?~)}
}
Tests if \texttt{s1} follows \texttt{s2}. Implemented as follows:
Combines successive elements of the sequence using a binary operation. The first input value at each iteration except the first one is the result of the previous iteration. The first input value at the first iteration is \verb|ident|. For example, the \verb|sum| word adds a sequence of numbers together, and is defined as follows:
\begin{alltt}
: sum ( seq -- n ) 0 [ + ] reduce ;
\end{alltt}
The \verb|reduce| word has a very simple implementation:
\begin{verbatim}
: reduce ( seq ident quot -- result ) swapd each ; inline
\end{verbatim}
So indeed, it is an expression of an idiom rather than an algorithm. Various words that combine the \verb|reduce| combinator together with an identity and mathematical operation are defined in \ref{reductions}.
Like \verb|reduce|, but instead outputs a sequence of intermediate values. The first element of the resulting sequence is always \verb|ident|. For example,
Applies the quotation to each element of the sequence. Elements that are themselves sequences are iterated recursively. Note that this word only operates on lists, strings, string buffers and vectors, not on virtual sequences or user-defined sequences.
Applies the quotation to each element yielding a new element, storing the new elements back in the original sequence. This modifies \texttt{seq} and so throws an exception if it is immutable.
\wordtable{
\vocabulary{sequences}
\ordinaryword{2map}{2map ( s1 s2 quot -- seq )}
\texttt{quot:~e1 e2 -- element}\\
}
Applies the quotation to pairs of elements from \texttt{s1} and \texttt{s2}, yielding a new element. The new elements are collected into a sequence of the same class as \texttt{s1}. Here is an example computing the pair-wise product of the elements of two vectors:
In fact the \verb|v*| word in the \verb|matrices| vocabulary is defined to call \verb|[ * ] 2map|; see \ref{pairwise} for documentation on this and similar words.
Applies the quotation to pairs of elements from \texttt{s1} and \texttt{s2}, yielding a new element. The new element is stored back in \texttt{s1}. This modifies \texttt{s1} and so throws an exception if it is immutable.
\wordtable{
\vocabulary{sequences}
\genericword{find}{find ( seq quot -- i elt )}
\genericword{find*}{find*~( i seq quot -- i elt )}
\texttt{quot:~elt -- ?}\\
}
Applies the quotation to each element of the sequence in turn, until it outputs a true value or the end of the sequence is reached. If the quotation yields a true value for some sequence element, the element index and the element itself are output. Otherwise, outputs $-1$ and \verb|f|. The \verb|find*| combinator is a variation that takes a starting index as a parameter. Various higher-level words are built on this combinator. Usually, they are used instead:
Curried forms of the above combinators. They pass an additional object to each invocation of the quotation.
\subsection{Stack operations}
The following words allow any mutable, growable sequence to be used as a LIFO (last in, first out) stack.
\wordtable{
\vocabulary{sequences}
\genericword{empty?}{empty?~( seq -- ?~)}
}
Tests if the sequence contains any elements. The default implementation of this word tests if the length is zero; user-defined sequences can provide a custom implementation that is more efficient.
Output a new sequence consisting of the elements of \texttt{s1} followed by the elements of \texttt{s2}. The new sequence is of the same class as \texttt{s1}.
The input is a sequence of sequences. If the input is empty, the output is the empty list (\texttt{f}). Otherwise, the elements of the input sequence are concatenated together, and a new sequence of the same type as the first element is output.
Applies the quotation to the $n$th element of the sequence, and store the output back in the $n$th slot of the sequence. This modifies \texttt{seq} and so throws an exception if it is immutable.
Outputs the index of the first element in the sequence equal to \texttt{obj}. If no element is found, outputs $-1$. The \verb|index*| form allows a start index to be specified. A related word is \verb|member?|.
Outputs the start index of a subsequence, or $-1$ if the subsequence does not occur in the sequence. The \verb|start*| form allows a start index to be specified. A related word is \verb|subseq?|.
Outputs a pair of sequences that equal the original sequence when appended. The first sequence has length $n$, the second has length $l-n$ where $l$ is the length of the input.
Outputs a pair of sequences that equal the original sequence excluding the $n$th element, when appended. The first sequence has length $n$, the second has length $l-n$ where $l$ is the length of the input.
A set of words for extracting subsequences of contiguous elements, and performing operations on them.
\wordtable{
\vocabulary{sequences}
\ordinaryword{subseq?}{subseq?~( s1 s2 -- ?~)}
}
Tests if \texttt{s2} contains \texttt{s1} as a subsequence. A related word is \verb|start|.
\wordtable{
\vocabulary{sequences}
\ordinaryword{head?}{head?~( s1 s2 -- ?~)}
\ordinaryword{tail?}{tail?~( s1 s2 -- ?~)}
}
Tests if \texttt{s1} starts or ends with \texttt{s1}. If \texttt{s1} is longer than \texttt{s2}, outputs \texttt{f}.
\wordtable{
\vocabulary{sequences}
\ordinaryword{head}{head~( n seq -- seq )}
}
Outputs a new sequence consisting of the first $n$ elements of the input sequence.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail}{tail~( n seq -- seq )}
}
Outputs a new sequence consisting of all elements of the sequence, starting at the $n$th index.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail*}{tail*~( n seq -- seq )}
}
Outputs a new sequence consisting of the last $n$ elements of the input sequence.
\wordtable{
\vocabulary{sequences}
\ordinaryword{?head}{?head~( s1 s2 -- seq ?~)}
\ordinaryword{?tail}{?tail~( s1 s2 -- seq ?~)}
}
Tests if \texttt{s1} starts or ends with \texttt{s1} as a subsequence. If there is a match, outputs the subrange of \texttt{s1} excluding \texttt{s1} followed by \texttt{t}. If there is no match, outputs \texttt{s1} followed by \texttt{f}.
\ordinaryword{split1}{split1~( seq split -- before after )}
}
If \texttt{seq} does not contain \texttt{split} as a subsequence, then \texttt{before} is equal to the \texttt{seq}, and \texttt{after} is \texttt{f}. Otherwise, \texttt{before} and \texttt{after} are both sequences, and yield the input excluding \texttt{split} when appended.
\wordtable{
\vocabulary{sequences}
\ordinaryword{split}{split~( seq split -- list )}
}
Outputs a list of subsequences taken between occurrences of \texttt{split} in \texttt{seq}. If \texttt{split} does not occur in \texttt{seq}, outputs a singleton list containing \texttt{seq} only.
Splits the sequence into groups of $n$ elements and collects each group in a list. If the sequence length is not a multiple of $n$, the final subsequence in the list will be shorter than $n$.
Applies the quotation to each element, and outputs a new sequence containing the elements of the original sequence for which the quotation output a true value.
Applies the quotation to each element of the sequence. If an element is found for which the quotation outputs a true value, a true value is output. Otherwise if the end of the sequence is reached, \verb|f| is output.
\wordtable{
\vocabulary{sequences}
\ordinaryword{all?}{all?~( seq quot -- ?~)}
\texttt{quot:~element -- ?}\\
}
Outputs \texttt{t} if the quotation yields true when applied to each element, otherwise outputs \texttt{f}. Given an empty sequence, vacuously outputs \texttt{t}.
\wordtable{
\vocabulary{sequences}
\ordinaryword{fiber?}{fiber?~( seq quot -- ?~)}
\texttt{quot:~element element -- ?}\\
}
Tests if all elements of the sequence are equivalent under the relation. The quotation should be an equality relation, otherwise the result will not be useful. This is implemented by vacuously outputting \verb|t| if the sequence is empty, or otherwise, by applying the quotation to each element together with the first element in turn, and testing if it always yields a true value. Usually, this word is used to test if all elements of a sequence are equal, or the same element:
A vector is a growable, mutable sequence whose elements are stored in a contiguous range of memory. The literal syntax is covered in \ref{vector-literals}. Very few words operate specifically on vectors; most operations on vectors are done with generic sequence words.
\glossary{name=improper list,description={a sequence of cons cells where the cdr of the last cons cell is not \texttt{f}}}
\glossary{name=general list,description={a proper or improper list; that is, either \texttt{f} or a cons cell}}
Lists of values are represented with nested cons cells. The car is the first element of the list; the cdr is the rest of the list. The value \texttt{f} represents the empty list.
A \emph{general list} is either the empty list or a cons cell. A \emph{list} is either the empty list or a cons cell whose cdr is also a list. A list is sometimes also known as a \emph{proper list}, and a general list that is not a proper list is known as a \emph{improper list}.
Not all list operations will function given an improper list,
Sorts the list by comparing each pair of elements with the quotation. The quotation should output \texttt{t} if \texttt{e2} is to come before \texttt{e1} in the list. For example, to sort a list of numbers in ascending order, you can do the following:
The following set of words manages LIFO (last-in-first-out) queues. Queues are built up from cons cells, and hence are immutable; queue operations always return a new queue.
A string is an immutable sequence of characters. The literal syntax is covered in \ref{string-literals}. Characters do not have a distinct data type, so elements taken out of strings appear as integers on the stack.
Turns a sequence of integers into a string. The integer elements are interpreted as characters. Note that this is not a way to turn any object into a printable representation; for that feature, see \ref{prettyprint}.
Creates a string with \texttt{char} repeated $\max(0,l-n)$ times, where $l$ is the length of \texttt{string}, then appends this new string on the left or the right of the input string.
A string buffer is a mutable and growable sequence of characters. String buffers can be used to construct new strings by accumilating substrings and characters, however usually they are only used indirectly, since the sequence construction words in \ref{make-seq} are more convenient to use in many cases.
description={a sequence that is not backed by actual storage, but instead either computes its values, or take them from an underlying sequence}}
\glossary{name=repeated sequence,
description={an instance of the \texttt{repeated} class, which is a virtual, immutable sequence consisting of a fixed element repeated a certain number of times}}
Creates an immutable sequence consisting of all integers in the interval $[a,b)$ (if $a<b$) or $(b,a]$ (if $a>b$). If $a=b$, the resulting sequence is empty. As with repeated sequences, this is just a tuple implementing the sequence protocol.
\ordinaryword{<repeated>}{<repeated> ( n object -- seq )}
}
Creates an immutable sequence consisting of \verb|object| repeated $n$ times. No storage allocation of $n$ elements is made; rather a repeated sequence is just a tuple where the \verb|nth| word is implemented to return the same value on each invocation.
Outputs a sequence with the same length as the input sequence, that presents the elements of the input sequence in reverse order. The new sequence shares storage with the given sequence.
A related word is \verb|reverse|, see \ref{aggregation}.
\ordinaryword{<slice>}{<slice> ( a b seq -- slice )}
}
Creates a mutable sequence that is a view of a subrange of elements of an underlying sequence. Changes to the underlying sequence are reflected in the slice, and vice versa.
\wordtable{
\vocabulary{sequences}
\ordinaryword{head-slice}{head-slice ( n seq -- slice )}
}
Creates a slice viewing the first $n$ elements of the input sequence.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail-slice}{tail-slice ( n seq -- slice )}
}
Creates a slice viewing all elements of the sequence, starting at the $n$th index.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail-slice*}{tail-slice* ( n seq -- slice )}
}
Creates a slice viewing the last $n$ elements of the input sequence.
The slice words output a new virtual sequence that shares structure with the original sequence, whereas the subsequence words output a fresh copied sequence.
The library supports an idiom where sequences can be constructed without passing the partial sequence being built on the stack. This reduces stack noise, and thus simplifies code and makes it easier to understand.
\newcommand{\dynamicscopeglos}{\glossary{
name=dynamic scope,
description={a variable binding policy where bindings established in a scope are visible to all code executed while the scope is active}}}
Calls the quotation in a new \emph{dynamic scope}. The quotation and any words it calls can execute the \texttt{,} and \texttt{\%} words to add elements at the end of the sequence being constructed.
Adds the element wrapped inside a one-element list, then adds the \texttt{car} word. This is used to construct quotations with \texttt{make-list} that must push a word on the stack.
Note that the sequence construction combinators will capture any variables set inside the quotation, due to the dynamic scoping behavior. These combinators are actually implemented using variables. See \ref{namespaces}.
\texttt{hashtable}&$\surd$&&$O(1)$&Large or frequently-changing mappings
\end{tabular}
It might be tempting to just always use hashtables, however for very small mappings, association lists are just as efficient, and are easier to work with since the entire set of list words can be used with them.
Association lists are built from cons cells. They are structured like a ribbed spine, where the ``spine'' is a list and each ``rib'' is a cons cell holding a key/value pair.
These words look up a key in an association list, comparing keys in the list with the given key by equality with \texttt{=}. The list is searched starting from the beginning. The two words differ in that the latter returns the key/value pair located, whereas the former only returns the value. The \texttt{assoc*} word allows a distinction to be made between a missing value.
These words output a new association list containing the key/value pair.
They differ in that \texttt{set-assoc} removes any existing key/value pairs with the given key first. In both cases, searching for the key in the returned association list gives the new value, however with the slightly faster \texttt{acons}, the old value remains shadowed in the list.
Sometimes it is convenient to decompose an association list into two lists of equal length, containing the keys and values, respectively, in the same order as the association list. This dual representation can be manipulated with a handful of helper words.
description={a container for key/value pairs inside a hashtable. A hash function assigns each key to a bucket, with the goal of spreading the keys as evenly as possible}}
\glossary{name=hashcode,
description={an integer chosen so that equal objects have equal hashcodes, and unequal objects' hashcodes are distributed as evently as possible}}
A hashtable sorts key/value pairs into buckets using a hashing function. The number of buckets is chosen to be approximately equal to the number of key/value pairs in the hashtable, so assuming a good hash function that distributes keys evenly, lookups can be performed in constant time, with a quick hash calculation to determine a bucket, followed by testing of only one or two key/value pairs.
\item The hashcode must be a fixnum (\ref{integers})\footnote{Strictly speaking, returning a bignum will not fail, however it will result in lower overall performance since the compiler will no longer make type assumptions when compiling callers of \texttt{hashcode}.}
\item If two objects are equal under \texttt{=}, they must have the same hashcode.
\end{itemize}
If mutable objects are used as hashtable keys, they must not be mutated. Doing so will violate bucket sorting invariants and result in undefined behavior.
Creates a new empty hashtable with \texttt{n} buckets. As more elements are added to the hashtable, the number of buckets is automatically increased and the keys are re-sorted.
Looks up the value associated with a key. The two words differ in that the latter returns the key/value pair located, whereas the former only returns the value. The \texttt{hash*} word allows a distinction to be made between a missing value and a value equal to \texttt{f}.
Applies the quotation to each key/value pair, collecting the key/value pairs for which the quotation output a true value into a new hashtable.
\wordtable{
\vocabulary{hashtables}
\ordinaryword{cache}{cache ( key hash quot -- value )}
\texttt{quot:~key -- value}\\
}
If the key is present in the hashtable, return the associated value, otherwise apply the quotation to the key, yielding a new value that is then stored in the hashtable.
Creates a hashtable with the same key/value pairs as the association list. If the association list contains duplicate keys, latter keys take precedence; this behavior is the opposite of the \texttt{assoc} word, where prior keys take precedence.
Outputs a list of association lists, where each association list contains the key/value pairs in a certain bucket. Useful for debugging hashcode distribution.
Calls the quotation in a new dynamic scope. The quotation and any words it calls can execute the \texttt{hash,} word to add key/value pairs to the hashtable being constructed.
Adds a key/value pair to the hashtable currently being constructed.
As with sequence construction, care must be taken to mind the effects of dynamic scoping on variable assignment performed by the quotation. Details are in \ref{namespaces}.
A variable is an entry in a hashtable of bindings, with the hashtable being implicit rather than passed on the stack. These hashtables are termed \emph{namespaces}. Nesting of scopes is implemented with a search order on namespaces, defined by a \emph{name stack}. Since namespaces are just hashtables, any object can be used as a variable, however by convention, variables are keyed by symbols (\ref{symbols}).
The \texttt{get} and \texttt{set} words read and write variable values. The \texttt{get} word searches up the chain of nested namespaces, while \texttt{set} always sets variable values in the current namespace only. Namespaces are dynamically scoped; when a quotation is called from a nested scope, any words called by the quotation also execute in that scope.
\glossary{name=name stack,
description={a stack holding namespaces. Entering a dynamic scope pushes the name stack, leaving a scope pops it}}
\glossary{name=namespace,
description={a hashtable pushed on the name stack and used as a set of variable bindings}}
\glossary{name=current namespace,
description={the namespace at the top of the name stack}}
Calls the quotation in the dynamic scope of \texttt{ns}. When variables are looked up by the quotation, \texttt{ns} is checked first, and setting variables in the quotation stores them in \texttt{ns}. The \texttt{extend} word places the namespace back on the data stack when the quotation returns.
Factor attempts to preserve natural mathematical semantics for numbers. Multiplying two large integers never results in overflow, and dividing two integers yields an exact fraction rather than a floating point approximation. Floating point numbers are also supported, along with complex numbers.
The \texttt{/} word gives an exact answer where possible. These two words output the answer in other forms. The \texttt{/i} word truncates the result towards zero, and \texttt{/f} converts it to a floating point approximation.
Raises \texttt{x} to the power of \texttt{y}. If \texttt{y} is an integer the answer is computed exactly, otherwise a floating point approximation is used.
The following ordering operations are supported on real numbers only.
The simplest type of number is the integer. Integers come in two varieties -- \emph{fixnums} and \emph{bignums}. As their names suggest, a fixnum is a fixed-width quantity\footnote{On 32-bit systems, an element of the interval $(-2^{-29},2^{29}]$, and on 64-bit systems, the interval $(-2^{-61},2^{61}]$. Because fixnums automatically grow to bignums, usually you do not have to worry about details like this.}, and is a bit quicker to manipulate than an arbitrary-precision bignum.
The predicate word \texttt{integer?}~tests if the top of the stack is an integer. If this returns true, then exactly one of \texttt{fixnum?}~or \texttt{bignum?}~would return true for that object. Usually, your code does not have to worry if it is dealing with fixnums or bignums.
Integer operations automatically return bignums if the result would be too big to fit in a fixnum. Here is an example where multiplying two fixnums returns a bignum:
Integers can be entered using a different base; see \ref{integer-literals}.
The word \texttt{.} prints numbers in decimal, regardless of how they were input. A set of words in the \texttt{prettyprint} vocabulary is provided to print integers using another base.
A pair of combinators calls a quotation a fixed number of times.
\wordtable{
\vocabulary{math}
\ordinaryword{times}{times ( n quot -- )}
\texttt{quot:~-- }\\
}
Calls the quotation $n$ times. If $n<0$, the quotation is not called at all.
\wordtable{
\vocabulary{math}
\ordinaryword{repeat}{repeat ( n quot -- )}
\texttt{quot:~i -- i }\\
}
Calls \texttt{quot}$n$ times, with the parameter \texttt{i} ranging from 0 to $n-1$. The quotation must output $i$ unmodified; or indeed, if it modifies it, the loop continues from that index. That is, the value $i$ on the stack is the actual loop counter, not a copy.
If you wish to perform an iteration over a range of integers that does not begin from zero, or an iteration that starts at a specific index and decreases towards zero, use the \verb|each| combinator (\ref{iteration}) in conjunction with a \verb|<range>| sequence (\ref{virtual-seq}), for example like so:
There are two ways of looking at an integer -- as a mathematical entity, or as a string of bits. The latter representation motivates \emph{bitwise operations}.
Computes the bitwise complement of the input; that is, each bit in the input number is flipped. Because integers are represented in two's complement form, this is actually equivalent to negating the integer, and subtracting 1.
Computes a new integer consisting of the bits of the first integer, shifted to the left by $n$ positions. If $n$ is negative, the bits are shifted to the right instead, and bits that ``fall off'' are discarded.
Computes the largest integer less than or equal to $log_2 n$. The input must be positive and the result is always an integer. In most cases, the \verb|log| word (\ref{algebraic}) should be used instead, since it allows any complex number as input, and the result is not truncated to an integer.
\wordtable{
\vocabulary{math}
\ordinaryword{each-bit}{each-bit ( n quot -{}- | quot: 0/1 -{}- )}
}
Applies the quotation to each bit of the input. The input must be a positive integer.
If we add, subtract or multiply any two integers, the result is always an integer. However, this is not the case with division. When dividing a numerator by a denominator where the numerator is not a integer multiple of the denominator, a ratio is returned instead.
Ratios are printed and can be input literally in the form above. Ratios are always reduced to lowest terms by factoring out the greatest common divisor of the numerator and denominator. A ratio with a denominator of 1 becomes an integer. Trying to create a ratio with a denominator of 0 raises an error.
description={an instance of the \texttt{real} class, which is a disjoint union of the
\texttt{rational} and \texttt{float} classes}}}
\realglos
\floatglos
Rational numbers represent \emph{exact} quantities. On the other hand, a floating point number is an \emph{approximation}. While rationals can grow to any required precision, floating point numbers are fixed-width, and manipulating them is usually faster than manipulating ratios or bignums (but slower than manipulating fixnums). Floating point literals are often used to represent irrational numbers, which have no exact representation as a ratio of two integers. Floating point literals are input with a decimal point.
\subsection{Binary representation of floats}\label{float-bits}
Floating point numbers are represented internally in IEEE 754 double-precision format. This internal representation can be accessed for advanced operations and input/output purposes.
\wordtable{
\vocabulary{math}
\ordinaryword{float>bits}{float>bits ( x -- n )}
\ordinaryword{double>bits}{double>bits ( x -- n )}
}
Converts the floating point number $x$ into IEEE 754 single or double precision form, and outputs an integer holding the binary representation of the result.
\wordtable{
\vocabulary{math}
\ordinaryword{bits>float}{float>bits ( n -- x )}
\ordinaryword{bits>double}{double>bits ( n -- x )}
}
Converts an integer representation $n$ of an IEEE 754 single or double precision float into a \verb|float| object.
Complex numbers arise as solutions to quadratic equations whose graph does not intersect the $x$ axis. Their literal syntax is covered in \ref{complex-literals}.
Tests if the top of the stack is a complex number. Note that unlike math, where all real numbers are also complex numbers, Factor only considers a number to be a complex number if its imaginary part is non-zero.
Converts between complex numbers and pairs of real numbers representing them in polar form. The polar form of a complex number consists of an absolute value and argument.
Raises the number $e$ to a specified power. The number $e$ can be pushed on the stack with the \texttt{e} word, so \texttt{exp} could have been defined as follows\footnote{Of course, $e\approx2.718281828459045$}:
However, it is actually defined otherwise, for efficiency.\footnote{In fact, things are done the other way around; the word \texttt{\^{}} is actually defined in terms of \texttt{exp}, to correctly handle complex number arguments.}
The \texttt{math} vocabulary provides the full set of trigonometric and hyperbolic functions, along with inverses and reciprocals. Complex number arguments are supported.
Any Factor sequence can be used to represent a mathematical vector, not just instances of the \verb|vector| class. Anywhere a vector is mentioned in this section, keep in mind it is a mathematical term, not a Factor data type.
The usual mathematical operations on vectors are supported.
Multiplies each element of the vector by a scalar. The two words only differ in argument order.
\wordtable{
\vocabulary{matrices}
\ordinaryword{v/n}{v/n ( vec n -- vec )}
\ordinaryword{n/v}{n/v ( n vec -- vec )}
}
Divides each element of the vector by a scalar, or alternatively, divides the scalar by each element of a vector. The two words yield different results; the elements of the two resulting vector are reciprocals of each other.
These words all expect a pair of vectors of equal length. They apply a binary operation to each pair of elements, producing a new vector. They are all implemented using the \verb|2map| combinator (\ref{iteration}).
Computes the inner product of two vectors. They must be of equal length.
Mathematically speaking, this is a map $<,>: {\mathbb{C}}^n \times{\mathbb{C}}^n \rightarrow\mathbb{C}$. It is the complex inner product; that is, $<a,b> =\overline{<b,a>}$, where $\overline{z}$ is the complex conjugate.
Computes the cross product $v_1\times v_2$. The following example illustrates the fact that a cross product of two vectors is always orthogonal to either vector.
Matrix literal syntax is documented in \ref{syntax:matrices}. In addition to the literal syntax, new matrices may be created from scratch in one of several ways.
Multiplies two matrices element-wise. They must have the same dimensions. This is \emph{not} matrix multiplication in the usual mathematical sense.
\wordtable{
\vocabulary{matrices}
\ordinaryword{m.}{m.~( matrix matrix -- matrix )}
}
Composes two matrices as linear operators. This is the usual mathematical matrix multiplication, and the first matrix must have the same number of columns as the second matrix has rows.
There is a natural isomorphism between the vector space $\mathbb{C}^m$, the $m\times1$ matrices, and the $1\times m$ matrices. Additionally, a $m\times n$ matrix acts as a linear operator from the vector space $\mathbb{C}^n$ to $\mathbb{C}^m$ in the same way as multiplying the $m\times n$ matrix by a $n \times1$ matrix. In Factor, these ideas are embodied by a set of words for converting vectors to matrices, and vice-versa.
sink of characters. Streams also support formatted output, which may be used to present styled text in a manner independent of output medium. There are two stream implementations that read and write external resources:
\item[HTML streams] implement the formatted output protocol to generate HTML from styled text attributes, then direct the HTML to an underlying stream.
description={a stream that implements the \texttt{stream-readln} and \texttt{stream-read} generic words and can be used for character input}}
\glossary{name=output stream,
description={a stream that implements the \texttt{stream-write-attr}, \texttt{stream-flush} and \texttt{stream-auto-flush} generic words and can be used for character output}}
There are various subsets of the stream protocol that a class can implement so that its instances may be used as streams. The following generic word is mandatory.
Releases any external resources associated with the stream, such as file handles and network connections. No further operations can be performed on the stream after this call.
You must close streams after you are finished working with them. A convenient way to automate this is by using the \texttt{with-stream} word in \ref{stdio}.
The following two words are optional, and should be implemented on input streams.
Reads a line of text and outputs it on the stack. If the end of the stream has been reached, outputs \texttt{f}. The precise meaning of a ``line'' depends on the stream. Streams that do not support this generic word can be wrapped in a line stream that reads lines terminated by \verb|\n|, \verb|\r| or \verb|\r\n| (\ref{special-stream}). File and network streams are automatically wrapped in line streams.
Reads \texttt{n} characters from the stream. If less than \texttt{n} characters are available before the end of the stream, a shorter string is output.
The following three words are optional, and should be implemented on output streams.
\genericword{stream-write1}{stream-write1 ( ch s -- )}
}
Outputs a character to the stream. This might not result in immediate output to the underlying resource if the stream performs buffering, like all file and network streams do. Output can be forced with \verb|stream-flush|.
\wordtable{
\vocabulary{io}
\genericword{stream-write-attr}{stream-write-attr ( str attrs s -- )}
Outputs a string to the stream. As with \verb|stream-write1|, this may not result in an immediate output operation unless \verb|stream-flush| is called.
The \texttt{attrs} parameter is an association list holding style information (\ref{styles}). Most of the time no style information needs to be output, and either the \texttt{stream-write} or \texttt{stream-print} word is used. Those words wrap \verb|stream-write-attr| and are described in the next section.
Ensures all pending output operations are been complete. With many output streams, written output is buffered and not sent to the underlying resource until either the buffer is full, or an explicit call to \texttt{stream-flush} is made.
Ensures the user sees prior output. It is not as strong as \texttt{stream-flush}. The contract is as follows: if the stream is connected to an interactive end-point such as a terminal, \texttt{stream-auto-flush} should execute \texttt{stream-flush}. If the stream is a file or network stream used for ``batch'' operations, this word should have an empty definition.
The \texttt{stream-print} word executes \texttt{stream-auto-flush} after each line of output.
With some streams, the above operations may suspend the current thread and execute other threads until input data is available (\ref{threads}).
The following three words are implemented in terms of the stream protocol, and should work with any stream supporting the required underlying operations.
Outputs a string to the stream, followed by a newline, then executes \texttt{stream-auto-flush} to force the line to be displayed on interactive streams.
Calls the quotation in a new dynamic scope, with the \texttt{stdio} variable set to \texttt{stream}. The stream is closed when the quotation returns or if an exception
HTML streams (\ref{html}) and pane streams (\ref{panes}) support styled text output using the \verb|stream-write-attr| word. The style association list given to this word can contain any of the following keys:
\begin{tabular}{l|l}
Key&Description\\
\hline
\texttt{foreground}&The foreground color, as a list with red, green, blue components\\
\texttt{background}&The background color, as a list with red, green, blue components\\
\texttt{font}&A font family name\\
\texttt{font-style}&One of \verb|plain|, \verb|bold|, \verb|italic|, or \verb|bold-italic|\\
\texttt{font-size}&An integer\\
\texttt{underline}&A boolean\\
\texttt{presented}&If set, a presentation for this object is output\\
\texttt{"file"}&If set, a hyperlink to that file is output\\
\texttt{icon}&If set, the icon named by this resource path is output\\
\end{tabular}
Note that
HTML streams only use the \verb|presented| key if it is set to a word; in that case, a link to the browser responder is output. The \verb|"file"| key causes an HTML stream to output a link to the file responder. Both responders must be enabled for such links to function.
Pane streams support presentation of any object, whereas the \verb|"file"| key is not supported.
Calls the quotation in a new dynamic scope, with the \texttt{stdio} variable set to a new string buffer. Executing \texttt{write}, \texttt{write-attr} or \texttt{print} will append text to the string buffer. When the quotation returns, the string buffer is coverted to
String buffers also support the input stream protocol, with the exception of \verb|stream-readln|. Characters are read in reverse. Usually, string buffers are not used as input streams directly, since lines cannot be read and the input string must be reversed. Instead, the following pair of words is used.
Creates a new stream for reading the characters of the string in order. This converts the string to a string buffer, reverses it, and wraps it in a line stream so that \verb|stream-readln| may be used (\ref{special-streams})..
Calls the quotation in a new dynamic scope, with the \texttt{stdio} variable set to a new string reader that reads from the given string. Executing \texttt{read1}, \texttt{read} or \texttt{read-line} will take text from the string reader.
description={a representation of an integer as a sequence of bytes, ordered from most significant to least significant. This is the native byte ordering for PowerPC, SPARC, Alpha and ARM processors}}
\glossary{name=little endian,
description={a representation of an integer as a sequence of bytes, ordered from least significant to most significant. This is the native byte ordering for x86 and x86-64 processors}}
The core stream words read and write strings. Packed binary integers can be read and written by converting to and from sequences of bytes. Floating point numbers can be read and written by converting them into a their bitwise integer representation (\ref{float-bits}).
There are two ways to order the bytes making up an integer; \emph{little endian} byte order outputs the least significant byte first, and the most significant byte last, whereas \emph{big endian} is the other way around.
Consider the hexadecimal integer \texttt{HEX: cafebabe}. Big endian byte order yields the following sequence of bytes:
Converts an integer into a string of $n$ bytes in big or little endian order. Truncation will occur if the integer is not in the range $[-2^{8n},2^{8n})$.
Files are read and written in a standard way, by attaching a reader or writer stream to the file. It is vital that file streams are closed after all input/output operations have been performed; a convenient way is to use the \verb|with-stream| word (\ref{stdio}).
\ordinaryword{<client>}{<client>~( host port -- stream~)}
}
Connects to TCP/IP port number \texttt{port} on the host named by \texttt{host}, and returns a bidirectional stream. An exception is thrown if the connection attempt fails.
\ordinaryword{<server>}{<server>~( port -- server~)}
}
Begins listening for connections to \texttt{port} on all network interfaces. An exception is thrown if the port cannot be opened. The returned object can be used as an input to the \texttt{stream-close} and \texttt{accept} words.
\ordinaryword{accept}{accept~( server -- stream~)}
}
Waits for a connection to the port number that \texttt{server} is listening on, and outputs a bidirectional stream when the connection has been established. An exception is thrown if an error occurs.
\ordinaryword{<duplex-stream>}{<duplex-stream>~( in out -- stream~)}
}
Creates a duplex stream. Writing to a duplex stream will write to \texttt{out}, and reading from a duplex stream will read from \texttt{in}. Closing a duplex stream closes both the input and output streams.
A line stream provides an implementation of the \verb|stream-readln| generic word that reads lines a character at a time from the underlying stream. Lines are terminated by either \verb|\n|, \verb|\r| or \verb|\r\n|.
Creates a stream wrapping \texttt{stream}. The given stream becomes the delegate of the new wrapper stream, so calling any stream operation on the wrapper passes it on to the delegate.
You can then define your own tuple class that delegates to a wrapper stream, then override methods on this new tuple class, and use the following combinator in your method definitions.
Executes the quotation in a dynamic scope where the \texttt{stdio} variable is set to the wrapped stream.
The following example implements a stream that emits \TeX\ markup when a certain attribute is set in the \texttt{attrs} parameter to \texttt{stream-write-attr}.
description={a set of words for printing objects in readable form}}
One of Factor's key features is the ability to print almost any object in a readable form. This greatly aids debugging and provides the building blocks for light-weight object serialization facilities.
The unparser provides a basic facility for turning certain types of objects into strings. A more general facility supporting more types is the prettyprinter (\ref{prettyprint}).
Outputs a string representation of \texttt{object}. Only the following classes of objects are supported; for anything else, an unreadable string is output:
Prints the object using literal syntax that can be parsed back again. Even though the prettyprinter supports more classes of objects than \texttt{unparse}, it is still not a general serialization mechanism. The following restrictions apply:
\item Not all objects print in a readable way. Namely, the following classes do not:
\begin{verbatim}
array
byte-array
displaced-alien
\end{verbatim}
\item Circular structure is not printed in a readable way. Circular references are printed as ``\texttt{...}''.
\item Floating point numbers might not equal themselves after being printed and read, since a decimal representation of a float is inexact.
\end{itemize}
\wordtable{
\vocabulary{prettyprint}
\ordinaryword{.}{.~( object --~)}
}
Prettyprint the object, except all output is on a single line without indentation, and deeply-nested structure is not printed fully. This word is intended for interactive use at the listener.
Controls the maximum nesting depth. Printing structures that nest further than this will simply print ``\texttt{...}''. If this is set to \texttt{f}, the nesting depth is unlimited. The default is \texttt{f}. Inside calls to \texttt{.}, set to 16, which translates to four levels of nesting with the default tab size.
\wordtable{
\vocabulary{prettyprint}
\symbolword{one-line}
}
If set to true, the prettyprinter does not emit newlines. The default is \texttt{f}. Inside calls to \texttt{.}, set to \texttt{t}.
Prettyprints the given object. Unlike \texttt{prettyprint}, this word does not emit a trailing newline, and the current indent level is given. This word is also generic, so you can add methods to have it print your own data types in a nice way.
Prints the textual representation of an object as returned by \verb|unparse|. Generally \texttt{prettyprint*} is used instead; one important distinction with \verb|unparse.| is that if the given object is a parsing word, the output is not prefixed with \texttt{POSTPONE:}.
This section concerns itself with reflective access and extension of the parser. The parser algorithm and standard syntax is described in \ref{syntax}. Before the parser proper is documented, we draw attention to a set of words for parsing numbers. They are called by the parser, and are useful in their own right.
\ordinaryword{str>number}{str>number~( string -- number )}
}
Attempts to parse the string as a number. An exception is thrown if the string does not represent a number in one of the following forms:
\begin{itemize}
\item An integer; see \ref{integer-literals}
\item A ratio; see \ref{ratio-literals}
\item A float; see \ref{float-literals}
\end{itemize}
In particular, complex numbers are parsed by the \verb|#{| and \verb|}#| parsing words, not by the number parser. To parse complex number literals, use the \texttt{parse} word (\ref{parsing-quotations}).
Like \texttt{str>number}, except instead of raising an error, outputs \texttt{f} if the string is not a valid literal number.
\wordtable{
\vocabulary{parser}
\genericword{base>}{base>~( string base -- integer )}
}
Converts a string representation of an integer in the given base into an integer. Throws an exception if the string is not a valid representation of an integer.
\wordtable{
\vocabulary{parser}
\ordinaryword{bin>}{bin>~( string -- integer )}
\ordinaryword{oct>}{oct>~( string -- integer )}
\ordinaryword{dec>}{dec>~( string -- integer )}
\ordinaryword{hex>}{hex>~( string -- integer )}
}
Convenience words defined in terms of \texttt{base>} for parsing integers in base 2, 8, 10 and 16, respectively.
As documented in \ref{vocabsearch}, the parser looks up words in the vocabulary search path. New word definitions are added to the current vocabulary. These two parameters are stored in a pair of variables (\ref{namespaces}):
There are two sets of words for parsing input from streams. The first set uses the following initial values for the \texttt{"use"} and \texttt{"in"} variables:
Parses lines of text from the stream and outputs a quotation. The \texttt{name} parameter identifies the stream in error messages. The stream is closed when the end is reached.
The next set of stream parsing words takes the vocabulary search path and current vocabulary from the current scope. These words are used to load the \texttt{.factor-rc} file on startup, so that any \texttt{USE:}~and \texttt{USING:}~declarations set in that file take effect in the listener (\ref{listener}).
Parsing words execute at parse time, and therefore can access and modify the state of the parser, as well as add objects to the parse tree. Parsing words are a difficult concept to grasp, so this section has several examples and explains the workings of some of the parsing words provided in the library.
To define a parsing word, suffix the colon definition with the \texttt{parsing} word.
\wordtable{
\vocabulary{syntax}
\parsingword{parsing}{parsing}
}
Marks the most recently defined word as a parsing word. For example:
\begin{verbatim}
: hello "Hello world" print ; parsing
\end{verbatim}
Now writing \texttt{hello} anywhere will print the message \texttt{"Hello world"} at parse time. Of course, this is a useless definition. In the sequel, we will look into writing useful parsing words that modify parser state.
The first thing to look at is how the parse tree is built. When parsing begins, the empty list is pushed on the data stack; whenever the parser algorithm appends an object to the parse tree, it conses the object onto the quotation at the top of the stack. This builds the quotation in reverse order, so when parsing is done, the quotation is reversed before it is called.
Lets look at a simple example; the parsing of \texttt{"1 2 3"}:
\begin{tabular}{l|l|l}
\hline
Token&Stack before&Stack after\\
\hline
\verb|1|&\verb|[ ]|&\verb|[ 1 ]|\\
\verb|2|&\verb|[ 1 ]|&\verb|[ 2 1 ]|\\
\verb|3|&\verb|[ 2 1 ]|&\verb|[ 3 2 1 ]|
\end{tabular}
Once the end of the string has been reached, the quotation is reversed, and the output, as you would expect, is \verb|[ 1 2 3 ]|.
Nested structure is a bit more involved. The basic idea is that parsing words can push an empty list on the stack, then all subsequent tokens are consed onto this quotation, until another parsing word adds this quotation to the quotation underneath.
The following definitions of the \verb|[| and \verb|]| parsing words illustrate the idiom:
\begin{verbatim}
: [ f ; parsing
: ] reverse swons ; parsing
\end{verbatim}
Let us look at how the following string parses:
\begin{verbatim}
"1 [ 2 3 ] 4"
\end{verbatim}
\begin{tabular}{l|l|l|l}
\hline
Token&Stack before&Stack after&Note\\
\hline
\verb|1|&\verb|[ ] [ ]|&\verb|[ ] [ 1 ]|&\\
\textbf{\texttt{[}}&\verb|[ 1 ]|&\verb|[ 1 ] [ ]|&pushes an empty list\\
description=a parsing word reads ahead of it scans following tokens from the input string}
The next idiom to look at is parsing words that read ahead. The first example is the \verb|HEX:| word, documented in \ref{integer-literals}. This word is defined so that the following two lines are equivalent:
\begin{verbatim}
HEX: deadbeef
3735928559
\end{verbatim}
It is defined in terms of a lower-level \texttt{(BASE)} word that takes the numerical base on the data stack, reads the next token from the string, then calls \texttt{base>} (\ref{parsing-numbers}):
\begin{verbatim}
: (BASE) ( base -- ) scan swap base> swons ;
: HEX: 16 (BASE) ; parsing
\end{verbatim}
The key word here is \texttt{scan}.
\wordtable{
\vocabulary{parser}
\ordinaryword{scan}{scan ( -- string )}
}
Outputs the next token as a string, or \texttt{f} if the end of the input has been reached. Advances the parser state to after this token.
The next example of a parsing word we will look at is the \verb|\| word. It reads the next token from the input, and appends code to push that word literally on the stack. That is, the following two phrases both have the effect of pushing the word \verb|+| on the stack, rather than executing it:
\begin{verbatim}
\ +
[ + ] car
\end{verbatim}
We can look at how \verb|\| is implemented:
\begin{verbatim}
: \ scan-word unit swons \ car swons ; parsing
\end{verbatim}
The key word here is \verb|scan-word|. It combines \texttt{scan} word with vocabulary search.
\wordtable{
\vocabulary{parser}
\ordinaryword{scan-word}{scan-word ( -- word )}
}
Reads the next token from the input and looks up a word with this name. If the lookup fails, attempts to parse the word as a number by calling \verb|str>number|.
Defining words add definitions to the dictionary without modifying the parse tree.
The first example to look at is the \verb|SYMBOL:| word. It reads the next token from the input stream, creates a word with that name, and makes it a symbol (\ref{symbols}). The next
example is the common \verb|:| word, which creates a colon definition. First, it reads the
name of the new word, then the definition is built up until \verb|;|. The latter
example will demonstrate building nested structure in defining words.
First, let us look at the \verb|SYMBOL:| word (\ref{symbols}).
\begin{verbatim}
: SYMBOL: CREATE define-symbol ; parsing
\end{verbatim}
The key factor the above definition is \verb|CREATE|, which reads a token from the input and creates a word with that name. This word is then passed to \verb|define-symbol|.
\wordtable{
\vocabulary{parser}
\ordinaryword{CREATE}{CREATE ( -- word )}
}
Reads the next token from the input and creates a word in the current vocabulary with that name. It uses \verb|create-in| to do this (\ref{creating-words}).
The definition of \verb|:| introduces the next idiom, and that is building a quotation and then adding a definition using \verb|;|.
\begin{verbatim}
: :
CREATE [ define-compound ] [ ]
"in-definition" on ; parsing
\end{verbatim}
The factors of the word are, in order:
\begin{description}
\item[\texttt{CREATE}] reads the following token and pushes a new word on the stack,
\item[\texttt{[ define-compound ]}] a quotation to be called by \verb|;|,
\item[\texttt{[ ]}] an empty list that the parser will build the colon definition on,
\item[\texttt{"in-definition" on}] sets a flag that subsequent parsing words can query.
\end{description}
While \verb|:| is very specific, \verb|;| is quite general because it takes a quotation pushed by a previous parsing word. You can use \verb|;| in your own parsing words.
\wordtable{
\parsingword{;}{;~( definer parsed -- )}
\texttt{definer:~parsed --}\\
}
Reverses the \verb|parsed| quotation, and passes it as input to the \verb|definer| quotation.
The definition of this word is in some sense dual to \verb|:| even thought it is more general:
\begin{verbatim}
: ; "in-definition" off reverse swap call ; parsing
\end{verbatim}
Suppose we are parsing the following string:
\begin{verbatim}
: sq dup * ;
\end{verbatim}
We can trace the parsing as before.
\begin{tabular}{l|l|l}
\hline
Token&Stack after&Note\\
\hline
\verb|:|&\verb|[ ] sq [ define-compound ] [ ]|&reads the next token\\
\item[\texttt{swap call}] calls \texttt{[ define-compound ]}. Thus, \verb|define-compound| is called to define \verb|sq| as the quotation \verb|[ dup * ]|.
String mode allows custom parsing of tokenized input. For even more esoteric situations, the input text can be accessed directly.
String mode is controlled by the \verb|string-mode| variable.
\wordtable{
\vocabulary{parser}
\symbolword{string-mode}
}
When enabled, the parser adds tokens to the parse tree as strings. This creates a paradox because further parsing words are not executed while string mode is on. However, if the token \verb|";"| is read, there is a special case that calls the \verb|;| parsing word. This parsing word reverses the quotation at the top of the stack, and calls the quotation underneath it, as usual.
An illustration of this idiom is found in the \verb|USING:| parsing word. It reads a list of vocabularies, terminated by \verb|;|. However, the vocabulary names do not name words, except by coincidence; so string mode is used to read them.
\begin{verbatim}
: USING:
string-mode on [
string-mode off [ use+ ] each
] [ ] ; parsing
\end{verbatim}
Make note of the quotation that is left in position for \verb|;| to call. It switches off string mode, so that normal parsing can resume, then adds the given vocabularies to the search path.
If the parser features described in the earlier sections are still insufficient, you can directly access a pair of variables holding parser state:
\begin{description}
\item[\texttt{"line"}] the text being parsed,
\item[\texttt{"col"}] the column number.
\end{description}
The \verb|"col"| variable is implicitly changed the \verb|scan| word (\ref{reading-ahead}), and the following word.
\wordtable{
\vocabulary{parser}
\ordinaryword{until-eol}{until-eol ( -- string )}
}
Outputs the remainder of the line being parsed. The \verb|"col"| variable is set to point to the end of the line.
This word is used to implement end-of-line comments:
Factor includes facilities for interoperating with web-based services. This includes an HTTP client, and an HTTP server with a continuation-based web application framework.
Connects to the server specified in the URL, and makes a \verb|GET| request to retreive that resource.
\wordtable{
\vocabulary{http-client}
\ordinaryword{http-post}{http-post ( type content url -- code headers stream )}
}
Attempts to connect to the server specified in the URL, and makes a \verb|POST| request with the specified content type and content. The content is automatically URL-encoded for you.
With both words, the output values are as follows:
A useful application of the HTTP server is the built-in vocabulary browser. You can use it simply by starting the HTTP server then visiting the following location:
\begin{verbatim}
http://localhost:8888/responder/browser/
\end{verbatim}
\subsection{Serving static content}
Static content may be served by setting the \verb|"doc-root"| variable to a directory holding the content. This variable may be set in the global namespace, or indeed, individually on each virtual host.
\begin{verbatim}
"/var/www/" "doc-root" set
\end{verbatim}
If a directory holds an \verb|index.html| file, the file is served when the directory is requested, otherwise a directory listing is produced. The directory listing references icons sent via the resource responder. The icons are located in the Factor source tree, and the \verb|"resource-path"| variable may be set to the root of the source tree in order for the icons to be located:
\begin{verbatim}
"/home/slava/work/Factor/" "resource-path" set
\end{verbatim}
A facility for ad-hoc server-side scripting exists. If a file with the \verb|.factsp| filename extension is requested, the file is run with \verb|run-file| and any output it sends to the default stream is sent to the client (\ref{stdio}). These ``Factor server pages'' are slower and less powerful than responders, so it is recommended that responders be used instead.
The HTTP server listens on a port number for HTTP requests and issues requests to \emph{responders}. The following form of request is understood specially by the HTTP server:
\begin{verbatim}
http://myhost/responder/foo/bar
\end{verbatim}
Such a request results in the \verb|foo| responder being invoked with the \verb|bar| argument. Requesting a path that does not begin with \verb|/responder| simply invokes the file responder with that path, as documented in the previous section.
Responders are located in a hashtable stored in the \verb|responders| variable. This variable must be in scope when the \verb|httpd| word is invoked. This is usually the case, as the global namespace includes a default value filled out with all responders that are included as part of the library.
The following words manage the set of installed responders:
\wordtable{
\vocabulary{httpd}
\ordinaryword{set-default-responder}{set-default-responder ( name -- )}
}
Sets the default responder, that is, the one handling requests not prefixed by \verb|/responder|, to the responder \verb|name|. The initial value for the default responder is \verb|"file"|, identifying the file responder.
Adds a responder to the hashtable stored in the \verb|responders| variable that is in scope.
\subsubsection{Developing a responder}
A responder is a hashtable where the following keys must be set:
\begin{description}
\item[\texttt{"responder"}] The name of the responder
\item[\texttt{"get"}] A quotation with stack effect \verb|( resource -- )|, invoked when a client requests the path \texttt{/responder/\emph{name}/\emph{resource}} using the \texttt{GET} method
\item[\texttt{"post"}] A quotation with stack effect \verb|( resource -- )|, invoked when a client requests the path using the \texttt{POST} method
\item[\texttt{"head"}] A quotation with stack effect \verb|( resource -- )|, invoked when a client requests the path using the \texttt{HEAD} method
\end{description}
The quotations are called by the HTTP server in the dynamic scope of the responder, with the following additional variables set:
\begin{description}
\item[\texttt{"method"}] The HTTP method requested by the client; one of \texttt{"get"}, \texttt{"post"} or \texttt{"head"}
\item[\texttt{"request"}] The full URL requested by the client, sans any query parameters trailing a \verb|?| character
\item[\texttt{"query"}] An association list of URL-decoded string pairs, obtained by splitting the query string, if any
\item[\texttt{"raw-query"}] The raw query string. Usually not used
\item[\texttt{"header"}] An association list of URL-decoded string pairs, obtained from the headers sent by the client
\item[\texttt{"response"}] Only set in the \verb|POST| method, an association list of string pairs, obtained from the response sent by the client
\end{description}
\subsection{Virtual hosts}\label{vhosts}
\glossary{name=virtual hosting,
description={A technique where a server is given several host name aliases via DNS; then the server serves different web sites, depending on the particular host name alias requested by the client}}
Factor's HTTP server supports virtual hosting. This is done with an additional layer of indirection, where each virtual host has its own set of responders. Virtual hosting is optional; by default, all virtual hosts share the same set of responders.
\wordtable{
\vocabulary{httpd}
\symbolword{vhosts}
}
Virtual hosts are defined in a hashtable mapping virtual host names to hashtables of responders. An initial value for this variable is defined in the global namespace; it provides an empty default virtual host. The default virtual host is the value associated with the \verb|"default"| key.
When a client makes a request, the HTTP server first searches for a responder in the requested virtual host; if no responder is found there, the default virtual host is checked. If this also fails, the global \verb|responders| variable is tested.
As an example, suppose the machine running the HTTP server has two DNS aliases, \verb|porky.ham.net| and \verb|spam.ham.net|. The following setup will serve two different static web sites from each virtual host, whereas all other responders are taken from the global table, and are shared between the two hosts:
\begin{verbatim}
vhosts get [
"porky.ham.net" {{
[[ "doc-root" "/var/www/porky/" ]]
}} set
"spam.ham.net" {{
[[ "doc-root" "/var/www/spam/" ]]
}} set
] bind
\end{verbatim}
\section{HTML streams}\label{html}
An HTML stream wraps an existing stream. Strings written to the HTML stream have their special characters converted to HTML entities via the \verb|chars>entities| word documented below. In addition, the \texttt{attrs} parameter to the \texttt{stream-write-attr} word is inspected for style attributes that direct the stream to wrap the output in various HTML tags; see \ref{styles}.
Calls the quotation in a new dynamic scope. The \texttt{stdio} variable is set to an HTML stream wrapping the previous value of \texttt{stdio}, so calls to \texttt{write}, \texttt{write-attr} and \texttt{print} go through the HTML stream.
\wordtable{
\vocabulary{html}
\ordinaryword{html-document}{html-document ( title quot -- )}
Builds on \texttt{with-html-stream} to emit the basic structure of an HTML document, consisting of \texttt{<html>}, \texttt{<head>} and \texttt{<body>} tags. The title is output in two places; a \texttt{<title>} tag and \texttt{<h1>} tag.
Converts various special characters in the input string (or any sequence of characters) into corresponding HTML entities. The following characters are converted:
A native library must be made available to Factor under a logical name before use. This is done via command line parameters, or the \verb|add-library| word.
The following two command line parameters can be specified for each library to load; the second parameter is optional.
\begin{description}
\item[\texttt{-libraries:\emph{logical}:name=\emph{name}}] associates a logical name with a system-specific native library name,
\item[\texttt{-libraries:\emph{logical}:abi=\emph{type}}] specifies the calling convention to use; \verb|type| is either \verb|cdecl| or \verb|stdcall|. If not specified, the default is \verb|cdecl|. On Unix, all libraries follow the \verb|cdecl| convention. On Windows, most libraries (but not all) follow \verb|stdcall|.
Another option is to add libraries while Factor is running.
\wordtable{
\vocabulary{alien}
\ordinaryword{add-library}{add-library ( library name abi -- )}
}
Adds a logical library named \verb|library|. The underlying shared library name is \verb|name|, and the calling convention is \verb|abi| and must be either \verb|"cdecl"| or \verb|"stdcall"|.
Defines a new word \verb|name| that calls the C function with the same name, found in the library given by the most recent \verb|LIBRARY:| declaration.
The \verb|return| value names a C type, or \verb|void| if no return value is expected.
Parameters are given by consecutive type/name pairs, where the type is again a C type, and the name is for documentation purposes. C types are documented in \ref{aliens}.
Note that the parentheses and commas are only syntax sugar; the following two definitions are equivalent, although the former is slightly more readable:
description={an instance of the \verb|alien| class, holding a pointer to native memory outside the Factor heap}}
The alien interface can work with an assortment of native data types:
\begin{itemize}
\item integer and floating point values
\item null-terminated strings
\item structures (\ref{alien-structs})
\item unions (\ref{alien-unions})
\end{itemize}
Table \ref{c-types} lists the built-in return value and parameter types. The sizes are given for a 32-bit system. Native numbers and strings are handled in a straight-forward way. Pointers are a bit more complicated, and are wrapped inside alien objects on the Factor side.
\begin{table}
\caption{\label{c-types}Supported native types}
\begin{tabular}{l|l|l}
Name&Size&Representation\\
\hline
\texttt{char}&1& Signed integer\\
\texttt{uchar}&1& Unsigned integer\\
\texttt{short}&2& Signed integer\\
\texttt{ushort}&2& Unsigned integer\\
\texttt{int}&4& Signed integer\\
\texttt{uint}&4& Unsigned integer\\
\texttt{long}&4& Signed integer\\
\texttt{ulong}&4& Unsigned integer\\
\texttt{longlong}&8& Signed integer\\
\texttt{ulonglong}&8& Unsigned integer\\
\texttt{float}&4& Single-precision float\\
\texttt{double}&8& Double-precision float\\
\texttt{char*}&4& Pointer to null-terminated byte string\\
\texttt{ushort*}&4& Pointer to null-terminated UTF16 string\\
One way to think of a C-style \verb|struct| is that it abstracts reading and writing field values stored at a range of memory given a pointer, by associating a type and offset with each field. This is the view taken by the alien interface, where defining a C structure creates a set of words for reading and writing fields of various types, offset from a base pointer given by an alien object.
Adds a field to the structure. The \verb|type| token identifies a C type, and \verb|name| gives a name to the field. A pair of words is defined, where \verb|structure| and \verb|field| are names, respectively:
\begin{alltt}
\emph{structure}-\emph{field} ( alien -- value )
set-\emph{structure}-\emph{field} ( value alien -- )
\end{alltt}
\wordtable{
\vocabulary{alien}
\parsingword{END-STRUCT}{END-STRUCT}
}
Ends a structure definition.
Defining a structure adds two new C types, where \verb|name| is the name of the structure:
\begin{description}
\item[\texttt{\emph{name}}] the type of the structure itself; structure and union definitions can define members to be of this type.
\item[\texttt{\emph{name}*}] the type of a pointer to the structure; this type can be used with return values and parameters, and in fact it is an alias for \texttt{void*}.
\end{description}
Additionally, the following two words are defined:
\begin{description}
\item[\texttt{<\emph{name}> ( -- byte-array )}] allocates a byte array large enough to hold the structure in the Factor heap. The field accessor words can then be used to work with this byte array. This feature allows calling native functions that expect pointers to caller-allocated structures\footnote{
There is an important restriction, however; the function must not retain the pointer in a global variable after it returns. Since the structure is allocated in the Factor heap, the garbage collector is free to move it between native function calls. If this behavior is undesirable, memory can be managed manually instead (\ref{malloc}).}.
\item[\texttt{\emph{name}-nth ( n alien -- alien )}] given a pointer and index into an array of structures, returns a pointer to the structure at that index.
\end{description}
Here is an example of a structure with various fields:
A C-style \verb|union| type allocates enough space for its largest member. In the alien interface, unions are used to allocate byte arrays in the Factor heap that may hold any one of the union's members.
A C-style \verb|enum| type defines a set of integer constants. The alien interface lets you define a set of words that push integers on the stack in much the same way as you would in C. While these words can be used for any purpose, using them outside of interfacing with C is discouraged.
The alien interface is built on top of a handful of primitives. Sometimes, it is
useful to call these primitives directly for debugging purposes.
\wordtable{
\vocabulary{alien}
\classword{dll}
}
Instances of this class are handles to native libraries.
\wordtable{
\vocabulary{alien}
\ordinaryword{dlopen}{dlopen ( path -- dll )}
}
Opens the specified native library and returns a DLL object. The input parameter is the
name of a native library file,
\emph{not} a logical library name.
\wordtable{
\vocabulary{alien}
\ordinaryword{dlsym}{dlsym ( symbol dll -- address )}
}
Looks up a named symbol in a native library, and outputs it address. If the \verb|dll| is \verb|f|, the lookup is performed in the runtime executable itself.
\wordtable{
\vocabulary{alien}
\ordinaryword{dlclose}{dlclose ( dll -- )}
}
Closes a native library and frees associated native resources.
Outputs an alien pointing at an offset from the base pointer of the input alien. Displaced aliens are used to access nested structures and native arrays.
\wordtable{
\vocabulary{alien}
\ordinaryword{alien-signed-cell}{alien-signed-cell ( alien offset -- n )}
\ordinaryword{set-alien-signed-cell}{set-alien-signed-cell ( n alien offset -- )}
\ordinaryword{alien-unsigned-cell}{alien-unsigned-cell ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-cell}{set-alien-unsigned-cell( n alien offset -- )}
\ordinaryword{alien-signed-8}{alien-signed-8 ( alien offset -- n )}
\ordinaryword{set-alien-signed-8}{set-alien-signed-8 ( n alien offset -- )}
\ordinaryword{alien-unsigned-8}{alien-unsigned-8 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-8}{set-alien-unsigned-8 ( n alien offset -- )}
\ordinaryword{alien-signed-4}{alien-signed-4 ( alien offset -- n )}
\ordinaryword{set-alien-signed-4}{set-alien-signed-4 ( n alien offset -- )}
\ordinaryword{alien-unsigned-4}{alien-unsigned-4 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-4}{set-alien-unsigned-4 ( n alien offset -- )}
\ordinaryword{alien-signed-2}{alien-signed-2 ( alien offset -- n )}
\ordinaryword{set-alien-signed-2}{set-alien-signed-2 ( n alien offset -- )}
\ordinaryword{alien-unsigned-2}{alien-unsigned-2 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-2}{set-alien-unsigned-2 ( n alien offset -- )}
\ordinaryword{alien-signed-1}{alien-signed-1 ( alien offset -- n )}
\ordinaryword{set-alien-signed-1}{set-alien-signed-1 ( n alien offset -- )}
\ordinaryword{alien-unsigned-1}{alien-unsigned-1 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-1}{set-alien-unsigned-1 ( n alien offset -- )}
These primitives read and write native memory. They can be given an alien, displaced alien, or byte array. No bounds checking of any kind is performed.
If for whatever reason Factor's memory management is unsuitable for a certain task, you can
directly call the standard C memory management routines. These words are very raw and deal with addresses directly, and of course it is easy to corrupt memory or crash the runtime
this way.
\wordtable{
\vocabulary{kernel-internals}
\ordinaryword{malloc}{malloc ( size -- address )}
}
Allocate a block of size \verb|size| and output a pointer to it.
If you are used to a statically typed language, you might find Factor's tendency to only fail at runtime hard to work with at first. However, the interactive development tools outlined in this part allow a much quicker turn-around time for testing changes. Also, write unit tests -- unit testing is a great way to ensure that old bugs do not re-appear once they've been fixed.
Factor is an \emph{image-based environment}. When you compiled Factor, you also generated a file named \texttt{factor.image}. I will have more to say about images later, but for now it suffices to understand that to start Factor, you must pass the image file name on the command line:
An \texttt{\textbf{ok}} prompt is printed after the initial banner, indicating the listener is ready to execute Factor phrases. The listener is a piece of Factor code, like any other; however, it helps to think of it as the primary interface to the Factor system. The listener reads Factor code and executes it. You can try the classical first program:
Multi-line phrases are supported; if there are unclosed brackets, the listener outputs \texttt{...} instead of the \texttt{ok} prompt, and the entire phrase is executed once all brackets are closed:
On startup, Factor reads the \texttt{.factor-rc} file from your home directory. You can put
any quick definitions you want available at the listener there. To avoid loading this
file, pass the \texttt{-no-user-init} command line switch. Another way to have a set of definitions available at all times is to save a custom image, as described in the next section.
While it is possible to do all development in the listener and save your work in images, it is far more convenient to work with source files, at least until an in-image structure editor is developed.
By convention, Factor source files are saved with the \texttt{.factor} filename extension. They can be loaded into the image as follows:
In Factor, loading a source file replaces any existing definitions\footnote{But see \ref{compiler} for this is not true of compiled code.}. Each word definition remembers what source file it was loaded from (if any). To reload the source file associated with a definition, use the \texttt{reload} word:
Word definitions also retain the line number where they are located in their original source file. This allows you to open a word definition in jEdit\footnote{\texttt{http://www.jedit.org}} for editing using the
The \texttt{jedit} word will open word definitions from the Factor library once the full path of the Factor source tree is entered into the \texttt{"resource-path"} variable. One way to do this is to add a phrase like the following to your \texttt{.factor-rc}:
This is what is meant by the image being an \emph{infinite session}. When you shut down and restart Factor, what happends is much closer to a Laptop's ``suspend'' mode, than a desktop computer being fully shut down.
Probably the most important debugging tool of them all is the \texttt{.} word. It prints the object at the top of the stack in a form that can be parsed by the Factor parser. A related word is \texttt{prettyprint}. It is identical to \texttt{.} except the output is more verbose; lists, vectors and hashtables are broken up into multiple lines and indented.
Most objects print in a parsable form, but not all. One exceptions to this rule is objects with external state, such as I/O ports or aliens (pointers to native structures). Also, objects with circular or very deeply nested structure will not print in a fully parsable form, since the prettyprinter has a limit on maximum nesting. Here is an example -- a vector is created, that holds a list whose first element is the vector itself:
The prettyprinted form of a vector or list with many elements is not always readable. The \texttt{[.]} and \texttt{\tto.\ttc} words output a list or a vector, respectively, with each element on its own line. In fact, the stack printing words are defined in terms of \texttt{[.]} and \texttt{\tto.\ttc}:
\begin{verbatim}
: .s datastack {.} ;
: .r callstack {.} ;
: .n namestack [.] ;
: .c catchstack [.] ;
\end{verbatim}
Before we move on, one final set of output words comes is used to output integers in
different numeric bases. The \texttt{.b} word prints an integer in binary, \texttt{.o} in octal, and \texttt{.h} in hexadecimal.
Factor organizes code in a two-tier structure of vocabularies and words. A word is the smallest unit of code; it corresponds to a function or method in other languages. Vocabularies group related words together for easy browsing and tracking of source dependencies.
Entering \texttt{vocabs .}~in the listener produces a list of all existing vocabularies:
You can look at the definition of any word, including library words, using \texttt{see}. Keep in mind you might have to \texttt{USE:} the vocabulary first.
The \texttt{see} word shows a reconstruction of the source code, not the original source code. So in particular, formatting and some comments are lost.
The \texttt{apropos.} word is handy when searching for related words. It lists all words
whose names contain a given string. The \texttt{apropos.} word is also useful when you know the exact name of a word, but are unsure what vocabulary it is in. For example, if you're looking for ways to iterate over various collections, you can do an apropos search for \texttt{map}:
From the above output, you can see that \texttt{map} is for lists, \texttt{string-map} is for strings, and \texttt{vector-map} is for vectors.
The \texttt{usage} word finds all words that refer to a given word and pushes a list on the stack. This word is helpful in two situations; the first is for learning -- a good way to learn a word is to see it used in context. The second is during refactoring -- if you change a word's stack effect, you must also update all words that call it. Usually you print the
To see the contents of the data stack, use the \texttt{.s} word. Similarly, the other stacks can be shown with \texttt{.r} (return stack), \texttt{.n} (name stack), and \texttt{.c} (catch stack). Each stack is printed with each element on its own line; the top of the stack is the first element printed.
\textbf{The generic word car does not have a suitable method for 4
:s :r :n :c show stacks at time of error.
:get ( var -- value ) inspects the error namestack.}
\end{alltt}
The words \texttt{:s}, \texttt{:r}, \texttt{:n} and \texttt{:s} behave like their counterparts that are prefixed with \texttt{.}, except they show the stacks as they were when the error was thrown.
The return stack warrants some special attention. To successfully develop Factor, you will need to learn to understand how it works. Lets look at the first few lines of the return stack at the time of the above error:
\begin{verbatim}
[ swap cdr ]
uncons
[ r> tuck 2slip ]
(each)
[ swons ]
[ each ]
each
\end{verbatim}
You can see the sequence of calls leading up to the error was \texttt{each} calling \texttt{(each)} calling \texttt{uncons}. The error tells us that the \texttt{car} word is the one that failed. Now, you can stare at the stack dump, at notice that if the call to \texttt{car} was successful and execution returned to \texttt{(each)}, the quotation \texttt{[ r> tuck 2slip ]} would resume executing. The first word there, \texttt{r>}, would take the quotation \texttt{[ swons ]} and put it back on the data stack. After \texttt{(each)} returned, it would then continue executing the quotation \texttt{[ each ]}. So what is going on here is a recursive loop, \texttt{[ swons ] each}. If you look at the definition of \texttt{reverse}, you will see that this is exactly what is being done:
\begin{verbatim}
: reverse ( list -- list ) [ ] swap [ swons ] each ;
\end{verbatim}
So a list is being reversed, but at some stage, the \texttt{car} is taken of something that is not a number. Now, you can look at the data stack with \texttt{:s}:
\begin{verbatim}
<< no-method [ ] 4 car >>
car
4
4
[ 3 2 1 ]
\end{verbatim}
So now, the mystery has been solved: as \texttt{reverse} iterates down the input value, it hits a cons cells whose \texttt{cdr} is not a list. Indeed, if you look at the value we are passing to \texttt{reverse}, you will see why:
In the future, the debugger will be linked with the walker, documented below. Right now, the walker is a separate tool. Another caveat is that in compiled code, the return stack is not reconstructed if there is an error. Until this is fixed, you should only compile code once it is debugged. For more potential compiler pitfalls, see \ref{compiler}.
The walker lets you step through the execution of a qotation. When a compound definition is reached, you can either keep walking inside the definition, or execute it in one step. The stacks can be inspected at each stage.
There are two ways to use the walker. First of all, you can call the \texttt{walk} word explicitly, giving it a quotation:
\&get ( var -- value ) inspects the stepper namestack.
step -- single step over
into -- single step into
continue -- continue execution
bye -- exit single-stepper
[ [ 10 [ dup , ] repeat ] make-list ]
walk}
\end{alltt}
As you can see, the walker prints a brief help message, then the currently executing quotation. It changes the listener prompt from \texttt{ok} to \texttt{walk}, to remind you that there is a suspended continuation.
The first element of the quotation shown is the next object to be evaluated. If it is a literal, both \texttt{step} and \texttt{into} have the effect of pushing it on the walker data stack. If it is a compound definition, then \texttt{into} will recurse the walker into the compound definition; otherwise, the word executes in one step.
The \texttt{\&r} word shows the walker return stack, which is laid out just like the primary interpreter's return stack. In fact, a good way to understand how Factor's return stack works is to play with the walker.
Note that the walker does not automatically stop when the quotation originally given finishes executing; it just keeps on walking up the return stack, and even lets you step through the listener's code. You can invoke \texttt{continue} or \texttt{exit} to terminate the walker.
While the walker can be invoked explicitly using the \texttt{walk} word, sometimes it is more convenient to \emph{annotate} a word such that the walker is invoked automatically when the word is called. This can be done using the \texttt{break} word:
Now, when some piece of code calls \texttt{layout*}, the walker will open, and you will be able to step through execution and see exactly what's going on. An important point to keep in mind is that when the walker is invoked in this manner, \texttt{exit} will not have the desired effect; execution will continue, but the data stack will be inconsistent, and an error will most likely be raised a short time later. Always use \texttt{continue} to resume execution after a break.
The walker is very handy, but sometimes you just want to see if a word is being called at all and when, and you don't care to single-step it. In that case, you can use the \texttt{watch} word:
If you accidentally start an infinite loop, you can send the Factor runtime a \texttt{QUIT} signal. On Unix, this is done by pressing \texttt{Control-\bs} in the controlling terminal. This will cause the runtime to dump the data and return stacks in a semi-readable form. Note that this will help you find the root cause of the hang, but it will not let you interrupt the infinite loop.
Unit tests are very easy to write. They are usually placed in source files. A unit test can be executed with the \texttt{unit-test} word in the \texttt{test} vocabulary. This word takes a list and a quotation; the quotation is executed, and the resulting data stack is compared against the list. If they do not equal, the unit test has failed. Here is an example of a unit test:
\begin{verbatim}
[ "Hello, crazy world" ] [
"editor" get [ 0 caret set ] bind
", crazy" 5 "editor" get [ line-insert ] bind
"editor" get [ line-text get ] bind
] unit-test
\end{verbatim}
To have a unit test assert that a piece of code does not execute successfully, but rather throws an exception, use the \texttt{unit-test-fails} word. It takes only one quotation; if the quotation does \emph{not} throw an exception, the unit test has failed.
\begin{verbatim}
[ -3 {} vector-nth ] unit-test-fails
\end{verbatim}
Unit testing is a good habit to get into. Sometimes, writing tests first, before any code, can speed the development process too; by running your unit test script, you can gauge progress.
Factor supports heap introspection. You can find all objects in the heap that match a certain predicate using the \texttt{instances} word. For example, if you suspect a resource leak, you can find all I/O ports as follows:
The stack effect inference tool checks correctness of code before it is run.
A \emph{stack effect} is a list of input classes and a list of output classes corresponding to
the effect a quotation has on the stack when called. For example, the stack effect of \verb|[ dup * ]| is \verb|[ [ integer ] [ integer ] ]|. The stack checker is used by passing a quotation to the \texttt{infer} word. It uses a sophisticated algorithm to infer stack effects of recursive words, combinators, and other tricky constructions, however, it cannot infer the stack effect of all words. In particular, anything using continuations, such as \texttt{catch} and I/O, will stump the stack checker.
\section{Usage}
The main entry point of the stack checker is a single word.
\wordtable{
\vocabulary{inference}
\ordinaryword{infer}{infer ( quot -- effect )}
}
Takes a quotation and attempts to infer its stack effect. An exception is thrown if the stack effect cannot be inferred.
You can combine unit testing with stack effect inference by writing unit tests that check stack effects of words. In fact, this can be automated with the \texttt{infer>test.} word; it takes a quotation on the stack, and prints a code snippet that tests the stack effect of the quotation:
You can then copy and paste this snippet into a test script, and run the test script after
making changes to the word to ensure its stack effect signature has not changed.
\section{The algorithm}
The stack effect inference algorithm mirrors the interpreter algorithm. A ``meta data stack'' holds two types of entries; computed values, whose type is known but literal value will only be known at runtime, and literals, whose value is known statically. When a literal value is encountered, it is simply placed on the meta data stack. When a word is encountered, one of several actions are taken, depending on the type of the word:
\begin{itemize}
\item If the word has special stack effect inference behavior, this behavior is invoked. Shuffle words and various primitives fall into this category.
\item If the word's stack effect is already known, then the inputs are removed from the meta data stack, and output values are added. If the meta data stack contains insufficient values, more values are added, and the newly added values are placed in the input list. Since inference begins with an empty stack, the input list contains all required input values when inference is complete.
\item If the word is marked to be inlined, stack effect inference recurses into the word definition and uses the same meta data stack. See \ref{declarations}.
\item Otherwise, the word's stack effect is inferred in a fresh inferencer instance, and the stack effect is cached. The fresh inferencer is used rather than the current one, so that type information and literals on the current meta data stack do not affect the subsequently-cached stack effect.
\end{itemize}
The following two examples demonstrate some simple cases:
A simple combinator such as \verb|keep| does not by itself have a stack effect, since \verb|call| takes an arbitrary quotation from the stack, which itself may have an arbitrary stack effect.
\begin{verbatim}
IN: kernel
: keep ( x quot -- x | quot: x -- )
over >r call r> ; inline
\end{verbatim}
On the other hand, the stack effect of word that passes a literal quotation to \verb|keep| can be inferred. The quotation is a literal on the meta data stack, and since \verb|keep| is marked \verb|inline|, the special inference behavior of \verb|call| receives this quotation.
Note that if \verb|call| is applied to a computed value, for example, a quotation taken from a variable, or a quotation that is constructed immediately before the \verb|call|, the stack effect inferencer will raise an error.
\textbf{! Inference error: A literal value was expected where a
computed value was found: \#<computed @ 716167923>
! Recursive state:
:s :r :n :c show stacks at time of error.
:get ( var -- value ) inspects the error namestack.}
\end{alltt}
Another word with special inference behavior is \verb|execute|. It is used much more rarely than \verb|call|, but does pretty much the same thing, except it takes a word as input rather than a string.
\subsection{Conditionals}
Simpler than a stack effect is the concept of a stack height difference. This is simply the input value count subtracted from the output value count. A conditional's stack effect can be inferred if each branch has the same stack height difference; in this case, we say that the conditional is \emph{balanced}, and the total stack effect is computed by performing a unification of types across each branch.
The following two examples exhibit balanced conditionals:
\begin{verbatim}
[ 1 ] [ dup ] ifte
dup cons? [ unit ] when cons
\end{verbatim}
The following example is not balanced and raises an error when we attempt to infer its stack effect:
:get ( var -- value ) inspects the error namestack.}
\end{alltt}
\subsection{Recursive words}
Recursive words all have the same general form; there is a conditional, and one branch of the conditional is the \emph{base case} terminating the recursion, and the other branch is the \emph{inductive case}, which reduces the problem and recurses on the reduced problem. A key observation one must make is that in a well-formed recursion, the recursive call in the inductive case eventually results in the base case being called, so we can take the stack effect of the recursive call to be the stack effect of the base case.
Consider the following implementation of a word that measures the length of a list:
\begin{verbatim}
: length ( list -- n )
[ cdr length 1 + ] [ 0 ] ifte* ;
\end{verbatim}
The stack effect can be inferred without difficulty:
The base case is taken if the top of the stack is \verb|f|, and the base case has a stack effect \verb|[ [ object ] [ fixnum ] ]|.
On the other hand if the top of the stack is something else, the inductive case is taken. The inductive case makes a recursive call to \verb|length|, and once we substitute the stack effect of the base case into this call point, we can infer that the stack effect of the recursive case is \verb|[ [ object ] [ integer ] ]|.
If both branches contain a recursive call, the stack effect inferencer gives up.
The compiler can provide a substantial speed boost for words whose stack effect can be inferred. Words without a known stack effect cannot be compiled, and must be run in the interpreter. The compiler generates native code, and so far, x86 and PowerPC backends have been developed.
During bootstrap, all words in the library with a known stack effect are compiled. You can
circumvent this, for whatever reason, by passing the \texttt{-no-compile} switch during
bootstrap:
\begin{alltt}
\textbf{bash\$} ./f boot.image.le32 -no-compile
\end{alltt}
The compiler has two limitations you must be aware of. First, if an exception is thrown in compiled code, the return stack will be incomplete, since compiled words do not push themselves there. Second, compiled code cannot be profiled. These limitations will be resolved in a future release.
The compiler consists of multiple stages -- first, a dataflow graph is inferred, then various optimizations are done on this graph, then it is transformed into a linear representation, further optimizations are done, and finally, machine code is generated from the linear representation.
The linear IR is the second of the two intermediate
representations used by Factor. It is basically a high-level
assembly language. Linear IR operations are called VOPs. The last stage of the compiler generates machine code instructions corresponding to each \emph{virtual operation} in the linear IR.
To perform everything except for the machine code generation, use the \texttt{precompile} word. This will dump the optimized linear IR instead of generating code, which can be useful sometimes.
\item[\texttt{\%prologue}] On x86, this does nothing. On PowerPC, at the start of
each word that calls a subroutine, we store the link
register in r0, then push r0 on the C stack.
\item[\texttt{\%call-label}] On PowerPC, uses near calling convention, where the
caller pushes the return address.
\item[\texttt{\%call}] On PowerPC, if calling a primitive, compiles a sequence that loads a 32-bit literal and jumps to that address. For other compiled words, compiles an immediate branch with link, so all compiled word definitions must be within 64 megabytes of each other.
\item[\texttt{\%jump-label}] Like \texttt{\%call-label} except the return address is not saved. Used for tail calls.
\item[\texttt{\%jump}] Like \texttt{\%call} except the return address is not saved. Used for tail calls.
\item[\texttt{\%dispatch}] Compile a piece of code that jumps to an offset in a
jump table indexed by an integer. The jump table consists of \texttt{\%target-label} and \texttt{\%target} must immediately follow this VOP.
\item[\texttt{\%target}] Not supported on PowerPC.
\item[\texttt{\%target-label}] A jump table entry.
\end{description}
\subsection{Slots and objects}
\begin{description}
\item[\texttt{\%slot}] The untagged object is in \texttt{vop-out-1}, the tagged slot
number is in \texttt{vop-in-1}.
\item[\texttt{\%fast-slot}] The tagged object is in \texttt{vop-out-1}, the pointer offset is
in \texttt{vop-in-1}. the offset already takes the type tag into
account, so its just one instruction to load.
\item[\texttt{\%set-slot}] The new value is \texttt{vop-in-1}, the object is \texttt{vop-in-2}, and
the slot number is \texttt{vop-in-3}.
\item[\texttt{\%fast-set-slot}] The new value is \texttt{vop-in-1}, the object is \texttt{vop-in-2}, and
the slot offset is \texttt{vop-in-3}.
the offset already takes the type tag into account, so
it's just one instruction to load.
\item[\texttt{\%write-barrier}] Mark the card containing the object pointed by \texttt{vop-in-1}.
\item[\texttt{\%untag}] Mask off the tag bits of \texttt{vop-in-1}, store result in
\texttt{vop-in-1} (which should equal \texttt{vop-out-1}!)
\item[\texttt{\%untag-fixnum}] Shift \texttt{vop-in-1} to the right by 3 bits, store result in
\texttt{vop-in-1} (which should equal \texttt{vop-out-1}!)
\item[\texttt{\%type}] Intrinstic version of type primitive. It outputs an
unboxed value in \texttt{vop-out-1}.
\end{description}
\subsection{Alien interface}
\begin{description}
\item[\texttt{\%parameters}] Ignored on x86.
\item[\texttt{\%parameter}] Ignored on x86.
\item[\texttt{\%unbox}] An unboxer function takes a value from the data stack
and converts it into a C value.
\item[\texttt{\%box}] A boxer function takes a C value as a parameter and
converts into a Factor value, and pushes it on the data
stack.
On x86, C functions return integers in EAX.
\item[\texttt{\%box-float}] On x86, C functions return floats on the FP stack.
\item[\texttt{\%box-double}] On x86, C functions return doubles on the FP stack.
\item[\texttt{\%cleanup}] Ignored on PowerPC.
On x86, in the cdecl ABI, the caller must pop input
parameters off the C stack. In stdcall, the callee does
