What follows is a detailed guide to the Factor language and development environment. It is not a tutorial or introductory guide, nor does it cover some background material that you are expected to understand, such as object-oriented programming, higher-order functions, continuations, or general issues of algorithm and program design.
Factor is a programming language combinding a postfix syntax with a functional and object-oriented
flavor, building on ideas from Forth, Joy and Lisp.
Factor is \emph{dynamic}. This means that all objects in the language are fully reflective at run time, and that new definitions can be entered without restarting the runtime. Factor code can be used interchangably as data, meaning that sophisticated language extensions can be realized as libraries of words.
Factor is \emph{safe}. This means all code executes in an object-oriented runtime that provides
garbage collection and prohibits direct pointer arithmetic. There is no way to get a dangling reference by deallocating a live object, and it is not possible to corrupt memory by overwriting the bounds of an array.
\item[\texttt{foo/bar}] either \texttt{foo} or \texttt{bar}. For example, \texttt{str/f} means either a string, or \texttt{f}
\end{description}
If the stack effect identifies quotations, the stack effect of each quotation may be given after suffixing \texttt{|} to the whole string. For example, the following denotes a word that takes a list and a quotation and produces a new list, calling the quotation with elements of the list.
\begin{verbatim}
( list quot -- list | quot: elt -- elt )
\end{verbatim}
\subsection{Naming conventions}
The following naming conventions are used in the Factor library.
\begin{description}
\item[\texttt{FOO:}] a parsing word that reads ahead from the input string
\item[\texttt{FOO}] a parsing word that does not read ahead, but rather takes a fixed action at parse time
\item[\texttt{FOO"}] a parsing word that reads characters from the input string until the next occurrence of \texttt{"}
\item[\texttt{foo?}] a predicate returning a boolean or generalized boolean value
\item[\texttt{foo.}] a word whose primary action is to print something, rather than to return a value. The basic case is the \texttt{.}~word, which prints the object at the top of the stack
\item[\texttt{foo*}] a variation of the \texttt{foo} word that takes more parameters
\item[\texttt{(foo)}] a word that is only useful for the implementation of \texttt{foo}
\item[\texttt{>to}] converts the object at the top of the stack to the \texttt{to} class
\item[\texttt{from>}] converts an instance of the \texttt{from} class into some canonical form
\item[\texttt{from>to}] convert an instance of the \texttt{from} class to the \texttt{to} class
\item[\texttt{>s}] move top of data stack to the \texttt{s} stack, where \texttt{s} is either \texttt{r} (call stack), \texttt{n} (name stack), or \texttt{c} (catch stack). Sometimes, libraries will define their own words following this naming convention, to implement user-defined stacks, typically stored in variables
\item[\texttt{nfoo}] destructive version of \texttt{foo}, that modifies one of its inputs rather than returning a new value. The ``n'' prefix denotes ``non-constructive''. This convention is used by sequence words
\item[\texttt{foo-with}] a form of the \texttt{foo} combinator that takes an extra object, and passes this object on each iteration of the quotation; for example, \texttt{each-with} and \texttt{map-with}
\item[\texttt{with-foo}] executes a quotation in a namespace where \texttt{foo} is configured in a special manner; for example, \texttt{with-stream}
\item[\texttt{make-foo}] executes a quotation in a namespace where a sequence of type \texttt{foo} is being constructed; for example, \texttt{make-string}
description={a set of words in the \texttt{parser} vocabulary, primarily \texttt{parse}, \texttt{eval}, \texttt{parse-file} and \texttt{run-file}, that creates objects from their printed representations, and adds word definitions to the dictionary}}}
In Factor, an \emph{object} is a piece of data that can be identified. Code is data, so Factor syntax is actually a syntax for describing objects, of which code is a special case. Factor syntax is read by the parser. The parser performs two kinds of tasks -- it creates objects from their \emph{printed representations}, and it adds \emph{word definitions} to the dictionary. The latter is discussed in \ref{words}. The parser can be extended (\ref{parser}).
Factor syntax consists of whitespace-separated tokens. The parser tokenizes the input on whitespace boundaries, where whitespace is defined as a sequence or one or more space, tab, newline or carriage-return characters. The parser is case-sensitive, so
the following three expressions tokenize differently:
\begin{verbatim}
2X+
2 X +
2 x +
\end{verbatim}
As the parser reads tokens it makes a distinction between numbers, ordinary words, and
parsing words. Tokens are appended to the parse tree, the top level of which is a list
returned by the original parser invocation. Nested levels of the parse tree are created
by parsing words.
Here is the parser algorithm in more detail -- some of the concepts therein will be defined shortly:
\begin{itemize}
\item If the current character is a double-quote (\texttt{"}), the \texttt{"} parsing word is executed, causing a string to be read.
\item Otherwise, the next token is taken from the input. The parser searches for a word named by the token in the currently used set of vocabularies. If the word is found, one of the following two actions is taken:
\begin{itemize}
\item If the word is an ordinary word, it is appended to the parse tree.
\item If the word is a parsing word, it is executed.
\end{itemize}
Otherwise if the token does not represent a known word, the parser attempts to parse it as a number. If the token is a number, the number object is added to the parse tree. Otherwise, an error is raised and parsing halts.
description={a parser mode where token are added to the parse tree as strings, without being looked up in the dictionary or converted into numbers first. Activated by switching on the \texttt{string-mode} variable}}}
There is one exception to the above process; the parser might be placed in \emph{string mode}, in which case it simply reads tokens and appends them to the parse tree as strings. String mode is activated and deactivated by certain parsing words wishing to read input in an unstructured but tokenized manner -- see \ref{string-mode}.
description={a word that is run at parse time. Parsing words can be defined by suffixing the compound definition with \texttt{parsing}. Parsing words have the \texttt{\dq{}parsing\dq{}} word property set to true, and respond with true to the \texttt{parsing?}~word}}}
Parsing words play a key role in parsing; while ordinary words and numbers are simply
added to the parse tree, parsing words execute in the context of the parser, and can
do their own parsing and create nested data structures in the parse tree. Parsing words
are also able to define new words.
While parsing words supporting arbitrary syntax can be defined, the default set is found
in the \texttt{syntax} vocabulary and provides the basis for all further syntactic
interaction with Factor.
\subsection{\label{vocabsearch}Vocabulary search}
\newcommand{\wordglos}{\glossary{
name=word,
description={an object holding a code definition and set of properties. Words are organized into vocabularies, and are uniquely identified by name within a vocabulary.}}}
\wordglos
\newcommand{\vocabglos}{\glossary{
name=vocabulary,
description={a collection of words, uniquely identified by name. The hashtable of vocabularies is stored in the \texttt{vocabularies} global variable, and the \texttt{USE:}~and \texttt{USING:}~parsing words add vocabularies to the parser's search path}}}
At the interactive listener, the default search path contains many more vocabularies. The default search path depends on how the parser was invoked (\ref{parsing-quotations}).
description={the list of vocabularies that the parser looks up tokens in. You can add to this list with the \texttt{USE:} and \texttt{USING:} parsing words}}}
\useglos
The \texttt{USE:} parsing word adds a new vocabulary at the front of the search path. Subsequent word lookups by the parser will search this vocabulary first.
Consecutive \texttt{USE:} declarations can be merged into a single \texttt{USING:} declaration.
\begin{alltt}
USING: lists strings vectors ;
\end{alltt}
Due to the way the parser works, words cannot be referenced before they are defined; that is, source files must order definitions in a strictly bottom-up fashion. For a way around this, see \ref{deferred}.
\subsection{Numbers}
\newcommand{\numberglos}{\glossary{
name=number,
description={an instance of the \texttt{number} class}}}
\numberglos
If a vocabulary lookup of a token fails, the parser attempts to parse it as a number.
description={an instance of the \texttt{integer} class, which is a disjoint union of the \texttt{fixnum} and \texttt{bignum} classes}}}
\numberglos
\newcommand{\fixnumglos}{\glossary{
name=fixnum,
description={an instance of the \texttt{fixnum} class, representing a fixed precision integer. On 32-bit systems, an element of the interval $(-2^{-29},2^{29}]$, and on 64-bit systems, the interval $(-2^{-61},2^{61}]$}}}
\fixnumglos
\newcommand{\bignumglos}{\glossary{
name=bignum,
description={an instance of the \texttt{bignum} class, representing an arbitrary-precision integer whose value is bounded by available object memory}}}
\bignumglos
The printed representation of an integer consists of a sequence of digits, optionally prefixed by a sign.
\begin{alltt}
123456
-10
2432902008176640000
\end{alltt}
Integers are entered in base 10 unless prefixed with a base change parsing word.
description={an instance of the \texttt{ratio} class, representing an exact ratio of two integers}}}
\ratioglos
The printed representation of a ratio is a pair of integers separated by a slash (\texttt{/}).
No intermediate whitespace is permitted. Either integer may be signed, however the ratio will be normalized into a form where the denominator is positive and the greatest common divisor
description={an instance of the \texttt{complex} class, representing a complex number with real and imaginary components, where both components are real numbers}}}
Many different types of objects can be constructed at parse time via literal syntax. Numbers are a special case since support for reading them is built-in to the parser. All other literals are constructed via parsing words.
If a quotation contains a literal object, the same literal object instance is used each time the quotation executes; that is, literals are ``live''.
\subsubsection{\label{boolean}Booleans}
\newcommand{\boolglos}{
\glossary{
name=boolean,
description={an instance of the \texttt{boolean} class, either \texttt{f} or \texttt{t}. See generalized boolean}}
\glossary{
name=generalized boolean,
description={an object used as a truth value. The \texttt{f} object is false and anything else is true. See boolean}}
\glossary{
name=t,
description={the canonical truth value. The \texttt{t} class, whose sole instance is the \texttt{t} object. Note that the \texttt{t} class is not equal to the \texttt{t} object}}
\glossary{
name=f,
description={the canonical false value; anything else is true. The \texttt{f} class, whose sole instance is the \texttt{f} object. Note that the \texttt{f} class is not equal to the \texttt{f} object}}
}
\boolglos
Any Factor object may be used as a truth value in a conditional expression. The \texttt{f} object is false and anything else is true. The \texttt{f} object is also used to represent the empty list, as well as the concept of a missing value. The canonical truth value is the \texttt{t} object.
Adds the \texttt{f} and \texttt{t} objects to the parse tree.
Note that the \texttt{f} parsing word and class is not the same as the \texttt{f} object. The former can be obtained by writing \texttt{\bs~f} inside a quotation, or \texttt{POSTPONE: f} inside a list that will not be evaluated.
\begin{alltt}
\textbf{ok} f \bs f = .
\textbf{f}
\end{alltt}
An analogous distinction holds for the \texttt{t} class and object.
description={an integer whose value denotes a Unicode code point. Character values are limited to the range from $0$ to $2^{16}-1$ inclusive, however in a later release this can be upgraded to the full 21-bit Unicode space without requiring any changes to user code}}}
Adds the Unicode code point of the character represented by \emph{token} to the parse tree.
\newcommand{\escapeglos}{\glossary{
name=escape,
description={a sequence allowing a non-literal character to be inserted in a string. For a list of escapes, see \ref{escape}}}}
\escapeglos
If the token is a single-character string other than whitespace or backslash, the character is taken to be this token. If the token begins with a backslash, it denotes one of the following escape codes.
\begin{table}[Special character escape codes]
\label{escape}
\begin{tabular}{l|l}
Escape code&Character\\
\hline
\texttt{\bs{}\bs}&Backslash (\texttt{\bs})\\
\texttt{\bs{}s}&Space\\
\texttt{\bs{}t}&Tab\\
\texttt{\bs{}n}&Newline\\
\texttt{\bs{}t}&Carriage return\\
\texttt{\bs{}0}&Null byte (ASCII 0)\\
\texttt{\bs{}e}&Escape (ASCII 27)\\
\texttt{\bs{}"}&Double quote (\texttt{"})\\
\end{tabular}
\end{table}
Examples:
\begin{alltt}
\textbf{ok} CHAR: a .
\textbf{97}
\textbf{ok} CHAR: \bs{}0 .
\textbf{0}
\textbf{ok} CHAR: \bs{}n .
\textbf{10}
\end{alltt}
A Unicode character can be specified by its code number by writing \texttt{\bs{}u} followed by a four-digit hexadecimal. That is, the following two expressions are equivalent:
\begin{alltt}
CHAR: \bs{}u0078
78
\end{alltt}
While not useful for single characters, this syntax is also permitted inside strings.
description={an instance of the \texttt{list} class, storing a sequence of elements as a chain of zero or more conses, where the car of each cons is an element, and the cdr is either \texttt{f} or another list}}
Parses two components making up a cons cell. Note that the lists parsed with \texttt{[} and \texttt{]} are just a special case of \texttt{[[} and \texttt{]]}. The following two lines are equivalent.
\begin{alltt}
[ 1 2 3 ]
[[ 1 [[ 2 [[ 3 f ]] ]] ]]
\end{alltt}
The empty list is denoted by \texttt{f}, along with boolean falsity, and the concept of a missing value. The expression \texttt{[ ]} parses to the same object as \texttt{f}.
While words parse as themselves, a word occurring inside a quotation is executed when the quotation is called. Sometimes it is desirable to have a word be pushed on the data stack during the execution of a quotation, usually for reflective access to the word's slots.
Reads the next word from the input string and appends some \emph{code} to the parse tree that pushes the word on the stack when the code is called. The following two lines are equivalent:
Reads the next word from the input string and appends the word to the parse tree, even if it is a parsing word. For an word \texttt{foo}, \texttt{POSTPONE: foo} and \texttt{foo} are equivalent; however, if \texttt{foo} is a parsing word, the latter will execute it at parse time, while the former will execute it at runtime. Usually used inside parsing words that wish to delegate some action to a further parsing word.
Using mutable object literals in word definitions requires care, since if those objects
are mutated, the actual word definition will be changed, which is in most cases not what you would expect. Strings and lists are immutable; string buffers, vectors, hashtables and tuples are mutable.
description={an instance of a user-defined class whose metaclass is the \texttt{tuple} metaclass, storing a fixed set of elements in named slots, with optional delegation method dispatch semantics}}}
Parses a tuple. The tuple's class must follow \texttt{<<}. The element after that is always the tuple's delegate. Further elements until \texttt{>>} are specified according to the tuple's slot definition, and an error is raised if an incorrect number of elements is given.
description={an instance of the \texttt{matrix} class, representing a mathematical matrix of numbers}}}
\matrixglos
\wordtable{
\vocabulary{syntax}
\parsingword{M[}{M[}
\parsingword{]M}{]M}
}
Parses a matrix. A matrix is specified as a set of rows, and each row is written like a list and must have the same length. The following is an example:
\begin{verbatim}
M[ [ 3 -5 1 ]
[ -2 7 1/2 ] ]M
\end{verbatim}
It corresponds to the following mathematical matrix:
description={a comment describing the usage of a word. Delimited by the \texttt{\#"!} parsing word, they appear at the start of a word definition and are stored in the \texttt{""documentation""} word property}}}
\doccommentglos
Comments that begin with \texttt{\#!} are called \emph{documentation comments}.
A documentation comment has no effect on the generated parse tree, but if it is the first thing inside a word definition, the comment text is appended to the string stored in the word's \texttt{"documentation"} property.
description={A string of the form \texttt{( \emph{inputs} -- \emph{outputs} )}, where the inputs and outputs are a whitespace-separated list of names or types. The top of the stack is the right-most token on both sides.}}
\newcommand{\stackcommentglos}{\glossary{
name=stack effect comment,
description={a comment describing the inputs and outputs of a word. Delimited by \texttt{(} and \texttt{}), they appear at the start of a word definition and are stored in the \texttt{""stack-effect""} word property}}}
\stackcommentglos
Comments delimited by \texttt{(} and \texttt{)} are called \emph{stack effect comments}. By convention they are placed at the beginning of a word definition to document the word's inputs and outputs:
A stack effect comment has no effect on the generated parse tree, but if it is the first thing inside a word definition, the word's \texttt{"stack-effect"} property is set to the comment text.
Word properties are described in \ref{word-props}.
as the next word in the quotation would expect them. Their behavior can be understood entirely in terms of their stack effects, which are given in table \ref{shuffles}.
Try to avoid the complex shuffle words such as \texttt{rot} and \texttt{2dup} as much as possible, for they make data flow harder to understand. If you find yourself using too many shuffle words, or you're writing
a stack effect comment in the middle of a compound definition to keep track of stack contents, it is
a good sign that the word should probably be factored into two or
more smaller words.
\subsection{\label{quotations}Quotations}
\newcommand{\csglos}{\glossary{
name=return stack,
description=see call stack}
\glossary{
name=call stack,
description={holds quotations waiting to be called. When a quotation is called with \texttt{call}, or when a compound word is executed, the previous call frame is pushed on the call stack, and the new quotation becomes the current call frame}}}
\csglos
\newcommand{\cfglos}{\glossary{
name=call frame,
description=the currently executing quotation}}
\cfglos
\glossary{
name=interpreter,
description=executes quotations by iterating them and recursing into nested definitions. see compiler}
The Factor interpreter executes quotations. Quotations are lists, and since lists can contain any Factor object, they can contain words. It is words that give quotations their operational behavior, as you can see in the following description of the interpreter algorithm.
\begin{itemize}
\item If the call frame is \texttt{f}, the call stack is popped and becomes the new call frame.
\item If the car of the call frame is a word, the word is executed:
\begin{itemize}
\item If the word is a symbol, it is pushed on the data stack. See \ref{symbols}.
\item If the word is a compound definition, the current call frame is pushed on the call stack, and the new call frame becomes the word definition. See \ref{colondefs}.
\item If the word is compiled or primitive, the interpreter jumps to a machine code definition. See \ref{primitives}.
\item If the word is undefined, an error is raised. See \ref{deferred}.
\end{itemize}
\item Otherwise, the car of the call frame is pushed on the data stack.
\item The call frame is set to the cdr, and the loop continues.
description=a word taking quotations or other words as input}
The following pair of words invokes the interpreter reflectively. They are used to implement \emph{combinators}, which are words that take code from the stack. Combinator definitions must be followed by the \texttt{inline} word to mark them as inline in order to compile; for example:
Push the current call frame on the call stack, and set the call stack to the given quotation. Conceptually: calls the quotation, as if its definition was substituted at the location of the \texttt{call}.
Execute a word definition, taking action based on the word definition, as above.
\begin{alltt}
\textbf{ok} : hello "Hello world" print ;
\textbf{ok} : twice dup execute execute ;
\textbf{ok}\bs hello twice
\textbf{Hello world}
\textbf{Hello world}
\end{alltt}
\subsubsection{Tail call optimization}
\newcommand{\tailglos}{\glossary{
name=tail call,
description=the last call in a quotation}
\glossary{
name=tail call optimization,
description=the elimination of call stack pushes when making a tail call}}
When a call is made to a quotation from the last word in the call frame, there is no
purpose in pushing the empty call frame on the call stack. Therefore the last call in a quotation does not grow the call stack, and tail recursion executes in bounded space.
\subsubsection{Call stack manipulation}
Because of the way the interpreter is described in \ref{quotations}, the top of the call stack is not accessed during the execution of a quotation; it is only popped when the interpreter reaches the end of the quotation. In effect, the call stack can be used as a temporary storage area, as long as pushes and pops are balanced out within a single quotation.
Moves the top of the call stack to the data stack.
The top of the data stack is ``hidden'' between \texttt{>r} and \texttt{r>}.
\begin{alltt}
\textbf{ok} 1 2 3 >r .s r>
\textbf{2
1}
\end{alltt}
It is very important to balance usages of \texttt{>r} and \texttt{r>} within a single quotation or word definition.
\begin{verbatim}
: the-good >r 2 + r> * ; ! Okay
: the-bad >r 2 + ; ! Runtime error
: the-ugly r> ; ! Runtime error
\end{verbatim}
Basically, the rule is you must leave the call stack in the same state as you found it, so that when the current quotation finishes executing, the interpreter can return to the caller.
One exception is that when \texttt{ifte} occurs as the last word in a definition, values may be pushed on the call stack before the condition value is computed, as long as both branches of the \texttt{ifte} pop the values off the call stack before returning.
\begin{verbatim}
: foo ( m ? n -- m+n/n )
>r [ r> + ] [ drop r> ] ifte ; ! Okay
\end{verbatim}
\subsubsection{Quotation variants}
There are three words that combine shuffle words with \texttt{call}. They are useful in the implementation of higher-order words taking quotations as inputs.
The \texttt{cond} is a generalized boolean. If it is \texttt{f}, the \texttt{false} quotation is called, and if \texttt{cond} is any other value, the \texttt{true} quotation is called. The condition flag is removed from the stack before either quotation executes.
Note that in general, both branches should have the same stack effect. Not only is this good style that makes the word easier to understand, but also unbalanced conditionals cannot be compiled (\ref{compiler}).
This pair are minor variations on \texttt{ifte} where only one branch is specified. The other is implicitly \texttt{[ ]}. They are implemented in the trivial way:
\begin{verbatim}
: when [ ] ifte ; inline
: unless [ ] swap ifte ; inline
\end{verbatim}
The \texttt{ifte} word removes the condition flag from the stack before calling either quotation. Sometimes this is not desirable, if the condition flag is serving a dual purpose as a value to be consumed by the \texttt{true} quotation. The \texttt{ifte*} word exists for this purpose.
If the condition is true, it is retained on the stack before the \texttt{true} quotation is called. Otherwise, the condition is removed from the stack and the \texttt{false} quotation is called. The following two lines are equivalent:
If the condition is \texttt{f}, the \texttt{false} quotation is called with the \texttt{default} value on the stack. Otherwise, the \texttt{true} quotation is called with the condition on the stack. The following two lines are equivalent:
\begin{verbatim}
X [ Y ] [ Z ] ?ifte
X dup [ nip Y ] [ drop Z ] ifte
\end{verbatim}
\subsubsection{Boolean logic}
The \texttt{?}~word chooses between two values, rather than two quotations.
Outputs \texttt{t} if exactly one of the inputs is true.
An alternative set of logical operations operate on individual bits of integers bitwise, rather than generalized boolean truth values. They are documented in \ref{bitwise}.
\subsection{Continuations}
\newcommand{\contglos}{
\glossary{name=continuation,
description=an object representing the future of the computation}}
\contglos
At any point in the execution of a Factor program, the \emph{current continuation} represents the future of the computation. This object can be captured with the \texttt{callcc0} and \texttt{callcc1} words.
Calling one of these words calls the given quotation with the continuation on the stack. The continuation is itself a quotation, and calling it \emph{continues execution} at the point after the call to \texttt{callcc0} and \texttt{callcc1}. Essentially, a continuation is a snapshot of the four stacks that can be restored at a later time.
The difference between \texttt{callcc0} and \texttt{callcc1} lies in the continuation object. When \texttt{callcc1} is used, calling the continuation takes one value from the top of the data stack, and places it back on the \emph{restored} data stack. This allows idioms such as exception handling, co-routines and generators to be implemented via continuations.
description=an object representing an exceptional situation that has been detected}
Support for handling exceptional situations such as bad user input, implementation bugs, and input/output errors is provided by a pair of words, \texttt{throw} and \texttt{catch}.
Raises an exception. Execution does not continue at the point after the \texttt{throw} call. Rather, the innermost catch block is invoked, and execution continues at that point. Passing \texttt{f} as an exception will cause \texttt{throw} to do nothing.
An exception handler is established, and the \texttt{try} quotation is called.
If the \texttt{try} quotation throws an error and no nested \texttt{catch} is established, the following sequence of events takes place:
\begin{itemize}
\item the stacks are restored to their state prior to the \texttt{catch} call,
\item the exception is pushed on the data stack,
\item the \texttt{handler} quotation is called.
\end{itemize}
If the \texttt{try} quotation completes successfully, the stacks are \emph{not} restored. The \texttt{f} object is pushed, and the \texttt{handler} quotation is called.
A common idiom is that the \texttt{catch} block cleans up from the error in some fashion, then passes it on to the next-innermost catch block. The following word is used for this purpose.
Raises an exception, without saving the current stacks for post-mortem inspection. This is done so that inspecting the error stacks sheds light on the original cause of the exception, rather than the point where it was rethrown.
Here is a simple example of a word definition that attempts to convert a string representing a hexadecimal number into an integer, and instead of halting execution when the string is not valid, it simply outputs \texttt{f}.
\begin{verbatim}
: catch-hex> ( str -- n/f )
[ hex> ] [ [ drop f ] when ] catch ;
\end{verbatim}
Exception handling is implemented using a \emph{catch stack}. The \texttt{catch} word pushes the current continuation on the catch stack, and \texttt{throw} calls the continuation at the top of the catch stack with the raised exception.
\glossary{name=catch stack,
description={a stack of exception handler continuations, pushed and popped by \texttt{catch}}}
Factor implements co-operative multitasking, where the thread of control switches between tasks at explicit calls to \texttt{yield}, as well as when blocking I/O is performed. Multitasking is implemented via continuations.
Calls \texttt{quot} in a co-operative thread. The new thread begins executing immediately, and the current thread resumes when the quotation yields, either from blocking
I/O or an explicit call to \texttt{yield}. This is implemented by adding the current continuation to the run queue, then calling \texttt{quot}, and finally executing \texttt{stop} after \texttt{quot} returns.
Call the continuation at the front of run queue, without saving the current continuation. In effect, this stops the current thread.
\subsubsection{Interpreter state}
The current state of the interpreter is determined by the contents of the four stacks. A set of words for getting and setting stack contents are the primitive building blocks for continuations, and in turn abstractions such as exception handling and multitasking.
Save and restore the call stack contents. The call stack does not include the currently executing quotation that made the call to \texttt{callstack}, since the current quotation is held in the call frame -- \ref{quotations} has details. Similarly, calling \texttt{set-callstack} will continue executing the current quotation until it returns, at which point control transfers to the quotation at the top of the new call stack.
Words are the fundamental unit of code in Factor, analogous to functions or procedures in other languages. Words are also objects, and this concept forms the basis for Factor's meta-programming facilities. Words hold two distinct pieces of information:
Sets the current vocabulary for new word definitions, and adds the vocabulary to the search path (\ref{vocabsearch}).
Parsing words add definitions to the current vocabulary. When a source file is being parsed, the current vocabulary is initially set to \texttt{scratchpad}.
\subsubsection{Searching for words}
Words whose names are known at parse time -- that is, most words making up your program -- can be referenced by stating their name. However, the parser itself, and sometimes code you write, will need to look up words dynamically.
The \texttt{vocabs} parameter is a list of vocabulary names. If a word with the given name is found, it is pushed on the stack, otherwise, \texttt{f} is pushed.
Creates a new word \texttt{name} in the current vocabulary. This word is intended to be called from parsing words (\ref{parsing-words}), and in fact is defined as follows:
\item Using defining words at run-time. This is a more dynamic feature that can be used to implement code generation and such, and in fact parse-time defining words are implemented in terms of run-time defining words.
A word \texttt{name} is created in the current vocabulary, and is associated with \texttt{definition}.
\begin{verbatim}
: ask-name ( -- name )
"What is your name? " write read-line ;
: greet ( name -- )
"Greetings, " write print ;
: friend ( -- )
ask-name greet ;
\end{verbatim}
By convention, the word name should be followed by a stack effect comment, and for more complex definitions, a documentation comment; see \ref{comments}.
A word \texttt{name} is created in the current vocabulary that pushes itself on the stack when executed. Symbols are used to identify variables (\ref{namespaces}) as well as for storing crufties in their properties (\ref{word-props}).
description=a word implemented as native code in the Factor runtime}}
\symbolglos
Executing a primitive invokes native code in the Factor runtime. Primitives cannot be defined through Factor code. Compiled definitions behave similarly to primitives in that the interpreter jumps to native code upon encountering them.
\subsubsection{\label{deferred}Deferred words and mutual recursion}
\glossary{
name=deferred word,
description={a word without a definition, created by the \texttt{DEFER:}~parsing word}}
Due to the way the parser works, words cannot be referenced before they are defined; that is, source files must order definitions in a strictly bottom-up fashion. Mutually-recursive pairs of words can be implemented by \emph{deferring} one of the words in the pair so that the second word in the pair can parse, then by replacing the deferred definition with a real one.
Create a word \texttt{name} in the current vocabulary that simply raises an error when executed. Usually, the word will be replaced with a real definition later.
Removes the word \texttt{name} from its vocabulary. Existing definitions that reference the word will continue to work, but newly-parsed occurrences of the word will not locate the forgotten definition. No exception is thrown if no such word exists.
A compound or generic word (\ref{generic}) can be given special behavior with one of the below parsing words.
\wordtable{
\vocabulary{syntax}
\parsingword{inline}{inline}
}
Marks the most recently defined word as an inline word. The compiler copies the definitions of inline words directly into the word being compiled. Combinators must be inlined in order to compile. For any other word, inlining is merely an optimization; see \ref{compiler}. Inlining does not affect the execution of the word in the interpreter, nor is inlining visible when you \texttt{see} the word (\ref{exploring-vocabs}).
\wordtable{
\vocabulary{syntax}
\parsingword{parsing}{parsing}
}
Marks the most recently defined word as a parsing word. Parsing words run at parse time. Se \ref{parsing-words}.
Retrieve and store word properties. Note that the stack effect is designed so that it is most convenient when \texttt{name} is a literal that is pushed on the stack right before executing these words. This is usually the case.
Retreives and stores a word's primitive parameter. This parameter is only used if the primitive number is 1 (compound definitions) or 2 (symbols). Note that to define a compound definition or symbol, you must use \texttt{define-compound} or \texttt{define-symbol}, as these words do not update the cross-referencing of word dependencies.
This is an even lower-level facility for working with the address containing native code to be invoked when the word is executed. The compiler sets the execution token to a location in memory containing generated code.
Updates a word's execution token according to its primitive number. When called with a compiled word, has the effect of decompiling the word. The execution token is automatically updated after a call to \texttt{set-word-primitive}.
Updates the cross-referencing database, which you will probably need to do if you mess around with any of the words in this section -- assuming Factor does not crash first, that is.
Everything in Factor is an object, where an object is a collection of slots. Each object has a unique identity, and references to objects are passed by value on the stack. It is possible to have two references to the same object, and if the object is mutated through one reference, the changes will be visible through the other reference. Not all objects are mutable; the documentation for each class details if its instances are mutable or not.
description={two objects are equal if they have the same class and if their slots are equal, or alternatively, if both are numbers that denote the same value}}
There are two distinct notions of ``sameness'' when it comes to objects. You can test if two references point to the same object, or you can test if two objects are equal in some sense, usually by having the same type and equal slot values.
Output \texttt{t} if two objects are equal, and \texttt{f} otherwise. The precise meaning of equality depends on the object's class, however usually two objects are equal if their slot values are equal. If two objects are equal, they have the same printed representation, although the converse is not always true. In particular:
\begin{itemize}
\item If no more specific method is defined, \texttt{=} calls \texttt{eq?}.
\item Two numbers are equal if they have the same numerical value.
\item Two sequences are equal if they are both instances of the same class, and if they have the same length, and elements.
\item Two hashtables are equal if they hold the same set of key/value pairs.
\item Two tuples are equal if they are of the same class and their slots are equal.
\item Two words are equal if they are the same object.
Make a fresh object that is equal to the given object. This is not guaranteed to actually copy the object; it does nothing with immutable objects, and does not copy words either. However, sequences and tuples can be cloned to obtain a new shallow copy of the original.
description={a word defined using the \texttt{GENERIC:}~parsing word. The behavior of generic words depends on the class of the object at the top of the stack. A generic word is composed of methods, where each method is specialized on a class}}
\glossary{name=method,
description={gives a generic word behavior when the top of the stack is an instance of a specific class}}
Sometimes you want a word's behavior to depend on the class of the object at the top of the stack, however implementing the word as a set of nested conditional tests is undesirable since it leads to unnecessary coupling -- adding support for a new class requires modifying the original definition of the word.
A generic word is a word whose behavior depends on the class of the
object at the top of the stack, however this behavior is defined in a
If two classes have a non-empty intersection, there is no guarantee that one is a subclass of the other. This means there is no canonical linear ordering of classes. The methods of a generic word are linearly ordered, though, and you can inspect this order using the \texttt{order} word.
Suppose you have the following definitions:
\begin{verbatim}
GENERIC: foo
M: integer foo 1 + ;
M: number foo 1 - ;
M: object foo dup 2list ;
\end{verbatim}
Since the \texttt{integer} class is strictly smaller than the \texttt{number} class, which in turn is strictly smaller than the \texttt{object} class, the ordering of methods is not surprising in this case:
\begin{alltt}
\textbf{ok}\bs foo order .
\textbf{[ object number integer ]}
\end{alltt}
However, suppose we had the following set of definitions:
\begin{verbatim}
GENERIC: describe
M: general-t describe drop "a true value" print ;
M: general-list describe drop "a list" print ;
M: object describe drop "an object" print ;
\end{verbatim}
Neither \texttt{general-t} nor \texttt{general-list} contains the other, and their intersection is the non-empty \texttt{cons} class. So the generic word system will place \texttt{object} first in the method order, however either \texttt{general-t} or \texttt{general-list} may come next, and it is pretty much a random choice that depends on hashing:
\item[object] there is no need for a predicate word, since
every object is an instance of this class.
\item[f] the only instance of this class is the singleton
\texttt{f} signifying falsity, missing value, and empty list, and the predicate testing for this is the built-in library word \texttt{not}.
\end{description}
\subsubsection{Built-in classes}
\glossary{name=type,
description={an object invariant that describes its shape. An object's type is constant for the lifetime of the object, and there is only a fixed number of types built-in to the run-time. See class}}
\glossary{name=built-in class,
description=see type}
Every object is an instance of to exactly one type, and the type is constant for the lifetime of the object. There is only a fixed number of types built-in to the run-time, and corresponding to each type is a \emph{built-in class}:
Outputs the canonical class of a given object. While an object may be an instance of more than one class, the canonical class is either the built-in class, or if the object is a tuple, the tuple class. Examples:
\begin{alltt}
\textbf{ok} 1.0 class .
\textbf{float}
\textbf{ok} TUPLE: point x y z ;
\textbf{ok} << point f 1 2 3 >> class .
\textbf{point}
\end{alltt}
\subsubsection{Unions}
\glossary{name=union,
description={a class whose set of instances is the union of the set of instances of a list of member classes}}
An object is an instance of a union class if it is an instance of one of its members. Union classes are used to associate the same method with several different classes, as well as to conveniently define predicates.
Defines a complement class. For example, the class of all values denoting ``true'' is defined as follows:
\begin{verbatim}
COMPLEMENT: general-t f
\end{verbatim}
\subsubsection{Predicates}
\glossary{name=predicate,
description={a word with stack effect \texttt{( object -- ?~)}, or more alternatively, a class whose instances are the instances of a superclass that satisfy an arbitrary predicate}}
An object is an instance of a predicate classes if it is an instance of the predicate's parent class, and if it satisfies the predicate definition.
Defines a predicate class deriving from \texttt{parent} whose instances are the instances of \texttt{superclass} that satisfy the \texttt{predicate} quotation. The predicate quotation must have stack effect \texttt{( object -- ?~)}.
Intersection and union of classes. Note that the returned class might not be the exact desired class; for example, \texttt{object} is output if no suitable class definition could be found at all.
The constructor takes slots in left-to-right order from the stack. After construction, slots are read and written using various automatically-defined words with names of the
Define a \texttt{<class>} word that creates a tuple instance of the \texttt{class}, then applies the \texttt{definition} to this new tuple. The \texttt{definition} quotation must have stack effect \texttt{( tuple -- tuple )}.
\subsubsection{Delegation}
\glossary{name=delegate,
description={a fa\,cade object's delegate receives unhandled methods that are called on the fa\,cade}}
\glossary{name={fa\,cade},
description=an object with a delegate}
Each tuple can have an optional delegate tuple. Generic words called on
the tuple that do not have a method for the tuple's class will be passed on
to the delegate. Note that delegation to objects that are not tuples is not fully supported at this stage and might not work as you might expect.
Returns an object's delegate, or \texttt{f} if no delegate is set. Note that in this case, undefined methods will be passed to \texttt{f}; rather an error is raised immediately.
A handful of ``virtual'' sequences are provided by the library. These sequences are not backed by actual storage, but instead either compute their values, or take them from an underlying sequence. Virtual sequences are documented in \ref{virtual-seq} and include:
\begin{verbatim}
repeated
range
slice
\end{verbatim}
User-defined classes can also implement the sequence protocol and gain the ability to reuse many of the words in this section.
The following set of generic words is the core of the sequence protocol. The mutating words are not supported by all sequences; in particular, lists and strings are immutable.
\glossary{name=resizable sequence,
description={a sequence implementing the \texttt{set-length} generic word. For example, vectors and string buffers}}
\glossary{name=mutable sequence,
description={a sequence implementing the \texttt{set-nth} generic word. For example, vectors and string buffers}}
The sequence protocol consists of a set of generic words. Any object that is an instance of a class implementing these generic words can be thought of as a sequence, and given to the words in the following sections.
Outputs the $n$th element of the sequence. Elements are numbered starting from 0, so the last element has an index one less than the length of the sequence. An exception should be thrown if an out-of-bounds index is accessed. All sequences support this operation, however with lists it has non-constant running time.
Sets the $n$th element of the sequence. Storing beyond the end of a resizable sequence such as a vector or string buffer grows the sequence. Storing to a negative index is always an error.
Tests if the sequence contains any elements. The default implementation of this word tests if the length is zero; user-defined sequences can provide a custom implementation that is more efficient.
Tests if the two sequences have the same length and elements. This is weaker than \texttt{=}, since it does not ensure that the sequences are instances of the same class.
Output a new sequence consisting of the elements of \texttt{s1} followed by the elements of \texttt{s2}. The new sequence is of the same class as \texttt{s1}.
The input is a sequence of sequences. If the input is empty, the output is the empty list (\texttt{f}). Otherwise, the elements of the input sequence are concatenated together, and a new sequence of the same type as the first element is output.
The following set of words do not modify their inputs.
\wordtable{
\vocabulary{sequences}
\ordinaryword{head}{head~( n seq -- seq )}
}
Outputs a new sequence consisting of the first $n$ elements of the input sequence.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail}{tail~( n seq -- seq )}
}
Outputs a new sequence consisting of all elements of the sequence, starting at the $n$th index.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail*}{tail*~( n seq -- seq )}
}
Outputs a new sequence consisting of the last $n$ elements of the input sequence.
\wordtable{
\vocabulary{sequences}
\ordinaryword{?head}{?head~( s1 s2 -- seq ?~)}
\ordinaryword{?tail}{?tail~( s1 s2 -- seq ?~)}
}
Tests if \texttt{s1} starts or ends with \texttt{s1} as a subsequence. If there is a match, outputs the subrange of \texttt{s1} excluding \texttt{s1} followed by \texttt{t}. If there is no match, outputs \texttt{s1} followed by \texttt{f}.
\wordtable{
\vocabulary{sequences}
\ordinaryword{cut}{cut ( seq n -- s1 s2 )}
}
Outputs a pair of sequences that equal the original sequence when appended. The first sequence has length $n$, the second has length $l-n$ where $l$ is the length of the input.
\begin{alltt}
\textbf{ok} "Hello world" 5 cut .s
\textbf{" world"
"Hello"}
\end{alltt}
\wordtable{
\vocabulary{sequences}
\ordinaryword{cut*}{cut* ( seq n -- s1 s2 )}
}
Outputs a pair of sequences that equal the original sequence excluding the $n$th element, when appended. The first sequence has length $n$, the second has length $l-n$ where $l$ is the length of the input.
\begin{alltt}
\textbf{ok} "Hello world" 5 cut* .s
\textbf{"world"
"Hello"}
\end{alltt}
\wordtable{
\vocabulary{sequences}
\ordinaryword{split1}{split1~( seq split -- before after )}
}
If \texttt{seq} does not contain \texttt{split} as a subsequence, then \texttt{before} is equal to the \texttt{seq}, and \texttt{after} is \texttt{f}. Otherwise, \texttt{before} and \texttt{after} are both sequences, and yield the input excluding \texttt{split} when appended.
\wordtable{
\vocabulary{sequences}
\ordinaryword{split}{split~( seq split -- list )}
}
Outputs a list of subsequences taken between occurrences of \texttt{split} in \texttt{seq}. If \texttt{split} does not occur in \texttt{seq}, outputs a singleton list containing \texttt{seq} only.
\begin{alltt}
\textbf{ok} "/usr/local/bin" "/" split .
\textbf{[ "" "usr" "local" "bin" ]}
\end{alltt}
\wordtable{
\vocabulary{sequences}
\ordinaryword{split-n}{split-n~( str n -- list )}
}
Splits the sequence into groups of $n$ elements and collects each group in a list. If the sequence length is not a multiple of $n$, the final subsequence in the list will be shorter than $n$.
The following set of sequence operations modify their inputs. The ``n'' prefix denotes ``non-constructive''; these words do not construct new output objects. None of these operations are permitted on immutable sequences like lists and strings.
Applies the quotation to the $n$th element of the sequence, and store the output back in the $n$th slot of the sequence. This modifies \texttt{seq} and so throws an exception if it is immutable.
Applies the quotation to each element yielding a new element, storing the new elements back in the original sequence. This modifies \texttt{seq} and so throws an exception if it is immutable.
Applies the quotation to pairs of elements from \texttt{s1} and \texttt{s2}, yielding a new element. The new elements are collected into a sequence of the same class as \texttt{s1}. Here is an example computing the pair-wise product of the elements of two vectors:
Applies the quotation to pairs of elements from \texttt{s1} and \texttt{s2}, yielding a new element. The new element is stored back in \texttt{s1}. This modifies \texttt{s1} and so throws an exception if it is immutable.
A vector is a growable, mutable sequence whose elements are stored in a contiguous range of memory. The literal syntax is covered in \ref{vector-literals}. Very few words operate specifically on vectors; most operations on vectors are done with generic sequence words.
\glossary{name=improper list,description={a sequence of cons cells where the cdr of the last cons cell is not \texttt{f}}}
\glossary{name=general list,description={a proper or improper list; that is, either \texttt{f} or a cons cell}}
Lists of values are represented with nested cons cells. The car is the first element of the list; the cdr is the rest of the list. The value \texttt{f} represents the empty list.
A \emph{general list} is either the empty list or a cons cell. A \emph{list} is either the empty list or a cons cell whose cdr is also a list. A list is sometimes also known as a \emph{proper list}, and a general list that is not a proper list is known as a \emph{improper list}.
Not all list operations will function given an improper list,
The two most frequently-used combinators are \verb|each| and \verb|map|, they can be used with any sequence and are documented in \ref{sequence-combinators}.
Applies the quotation to each element, and outputs the rest of the list upon encountering an element for which the quotation outputs true. If the quotation did not output true for any element, \texttt{some?}~outputs \texttt{f}. Note that the output is a generalized boolean; if the quotation matched any element, the result is true.
Outputs \texttt{t} if the quotation yields true when applied to each element, otherwise outputs \texttt{f}. Given the empty list, vacuously outputs \texttt{t}.
Sorts the list by comparing each pair of elements with the quotation. The quotation should output \texttt{t} if \texttt{e2} is to come before \texttt{e1} in the list. For example, to sort a list of numbers in ascending order, you can do the following:
The following set of words manages LIFO (last-in-first-out) queues. Queues are built up from cons cells, and hence are immutable; queue operations always return a new queue.
A string is an immutable sequence of characters. The literal syntax is covered in \ref{string-literals}. Characters do not have a distinct data type, so elements taken out of strings appear as integers on the stack.
Turns a sequence of integers into a string. The integer elements are interpreted as characters. Note that this is not a way to turn any object into a printable representation; for that feature, see \ref{prettyprint}.
A string buffer is a mutable and growable sequence of characters. String buffers can be used to construct new strings by accumilating substrings and characters, however usually they are only used indirectly, since the sequence construction words in \ref{make-seq} are more convenient to use in many cases.
Creates an immutable sequence consisting of \verb|object| repeated $n$ times. No storage allocation of $n$ elements is made; rather a repeated sequence is just a tuple where the \verb|nth| word is implemented to return the same value on each invocation.
Creates an immutable sequence consisting of all integers in the interval $[a,b)$ (if $a<b$) or $(b,a]$ (if $a>b$). If $a=b$, the resulting sequence is empty. As with repeated sequences, this is just a tuple implementing the sequence protocol.
\begin{alltt}
\textbf{ok} CHAR: a CHAR: z 1 + <range> .
<< range [ ] 97 123 1 >>
\textbf{ok} CHAR: a CHAR: z 1 + <range> >string .
"abcdefghijklmnopqrstuvwxyz"
\textbf{ok} CHAR: z CHAR: a 1 - <range> >string .
"zyxwvutsrqponmlkjihgfedcba"
\end{alltt}
\wordtable{
\vocabulary{sequences}
\ordinaryword{<slice>}{<slice> ( a b seq -- slice )}
}
Creates a mutable sequence that is a view of a subrange of elements of an underlying sequence. Changes to the underlying sequence are reflected in the slice, and vice versa.
\wordtable{
\vocabulary{sequences}
\ordinaryword{head-slice}{head-slice ( n seq -- slice )}
}
Creates a slice viewing the first $n$ elements of the input sequence.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail-slice}{tail-slice ( n seq -- slice )}
}
Creates a slice viewing all elements of the sequence, starting at the $n$th index.
\wordtable{
\vocabulary{sequences}
\ordinaryword{tail-slice*}{tail-slice* ( n seq -- slice )}
}
Creates a slice viewing the last $n$ elements of the input sequence.
There is a natural duality between the four slicing words above, and the four subsequence words from \ref{subseq}:
The library supports an idiom where sequences can be constructed without passing the partial sequence being built on the stack. This reduces stack noise, and thus simplifies code and makes it easier to understand.
\newcommand{\dynamicscopeglos}{\glossary{
name=dynamic scope,
description={a variable binding policy where bindings established in a scope are visible to all code executed while the scope is active}}}
Calls the quotation in a new \emph{dynamic scope}. The quotation and any words it calls can execute the \texttt{,} and \texttt{\%} words to add elements at the end of the sequence being constructed.
Adds the element wrapped inside a one-element list, then adds the \texttt{car} word. This is used to construct quotations with \texttt{make-list} that must push a word on the stack.
Note that the sequence construction combinators will capture any variables set inside the quotation, due to the dynamic scoping behavior. These combinators are actually implemented using variables. See \ref{namespaces}.
\texttt{hashtable}&$\surd$&&$O(1)$&Large or frequently-changing mappings
\end{tabular}
It might be tempting to just always use hashtables, however for very small mappings, association lists are just as efficient, and are easier to work with since the entire set of list words can be used with them.
Association lists are built from cons cells. They are structured like a ribbed spine, where the ``spine'' is a list and each ``rib'' is a cons cell holding a key/value pair.
These words look up a key in an association list, comparing keys in the list with the given key by equality with \texttt{=}. The list is searched starting from the beginning. The two words differ in that the latter returns the key/value pair located, whereas the former only returns the value. The \texttt{assoc*} word allows a distinction to be made between a missing value.
These words output a new association list containing the key/value pair.
They differ in that \texttt{set-assoc} removes any existing key/value pairs with the given key first. In both cases, searching for the key in the returned association list gives the new value, however with the slightly faster \texttt{acons}, the old value remains shadowed in the list.
Sometimes it is convenient to decompose an association list into two lists of equal length, containing the keys and values, respectively, in the same order as the association list. This dual representation can be manipulated with a handful of helper words.
description={a container for key/value pairs inside a hashtable. A hash function assigns each key to a bucket, with the goal of spreading the keys as evenly as possible}}
\glossary{name=hashcode,
description={an integer chosen so that equal objects have equal hashcodes, and unequal objects' hashcodes are distributed as evently as possible}}
A hashtable sorts key/value pairs into buckets using a hashing function. The number of buckets is chosen to be approximately equal to the number of key/value pairs in the hashtable, so assuming a good hash function that distributes keys evenly, lookups can be performed in constant time, with a quick hash calculation to determine a bucket, followed by testing of only one or two key/value pairs.
Outputs the hashcode of the object. The contract of this generic word is as follows:
\begin{itemize}
\item The hashcode must be a fixnum\footnote{Strictly speaking, returning a bignum will not fail, however it will result in lower overall performance since the compiler will no longer make type assumptions when compiling callers of \texttt{hashcode}.}
\item If two objects are equal under \texttt{=}, they must have the same hashcode.
\end{itemize}
If mutable objects are used as hashtable keys, they must not be mutated. Doing so will violate bucket sorting invariants and result in undefined behavior.
Creates a new empty hashtable with \texttt{n} buckets. As more elements are added to the hashtable, the number of buckets is automatically increased and the keys are re-sorted.
Looks up the value associated with a key. The two words differ in that the latter returns the key/value pair located, whereas the former only returns the value. The \texttt{hash*} word allows a distinction to be made between a missing value and a value equal to \texttt{f}.
Creates a hashtable with the same key/value pairs as the association list. If the association list contains duplicate keys, latter keys take precedence; this behavior is the opposite of the \texttt{assoc} word, where prior keys take precedence.
Outputs a list of association lists, where each association list contains the key/value pairs in a certain bucket. Useful for debugging hashcode distribution.
\subsubsection{Hashtable construction}
A facility analogous to sequence construction (\ref{make-seq}) exists for hashtables.
Calls the quotation in a new dynamic scope. The quotation and any words it calls can execute the \texttt{hash,} word to add key/value pairs to the hashtable being constructed.
Adds a key/value pair to the hashtable currently being constructed.
As with sequence construction, care must be taken to mind the effects of dynamic scoping on variable assignment performed by the quotation. Details are in \ref{namespaces}.
\subsection{\label{namespaces}Variables and namespaces}
A variable is an entry in a hashtable of bindings, with the hashtable being implicit rather than passed on the stack. These hashtables are termed \emph{namespaces}. Nesting of scopes is implemented with a search order on namespaces, defined by a \emph{name stack}. Since namespaces are just hashtables, any object can be used as a variable, however by convention, variables are keyed by symbols (\ref{symbols}).
The \texttt{get} and \texttt{set} words read and write variable values. The \texttt{get} word searches up the chain of nested namespaces, while \texttt{set} always sets variable values in the current namespace only. Namespaces are dynamically scoped; when a quotation is called from a nested scope, any words called by the quotation also execute in that scope.
\glossary{name=name stack,
description={a stack holding namespaces. Entering a dynamic scope pushes the name stack, leaving a scope pops it}}
\glossary{name=namespace,
description={a hashtable pushed on the name stack and used as a set of variable bindings}}
\glossary{name=current namespace,
description={the namespace at the top of the name stack}}
Calls the quotation in the dynamic scope of \texttt{ns}. When variables are looked up by the quotation, \texttt{ns} is checked first, and setting variables in the quotation stores them in \texttt{ns}. The \texttt{extend} word places the namespace back on the data stack when the quotation returns.
Factor attempts to preserve natural mathematical semantics for numbers. Multiplying two large integers never results in overflow, and dividing two integers yields an exact fraction rather than a floating point approximation. Floating point numbers are also supported, along with complex numbers.
\subsection{Number protocol}
The following usual operations are supported by all numbers.
The \texttt{/} word gives an exact answer where possible. These two words output the answer in other forms. The \texttt{/i} word truncates the result towards zero, and \texttt{/f} converts it to a floating point approximation.
Raises \texttt{x} to the power of \texttt{y}. If \texttt{y} is an integer the answer is computed exactly, otherwise a floating point approximation is used.
The following ordering operations are supported on real numbers only.
The simplest type of number is the integer. Integers come in two varieties -- \emph{fixnums} and \emph{bignums}. As their names suggest, a fixnum is a fixed-width quantity\footnote{On 32-bit systems, an element of the interval $(-2^{-29},2^{29}]$, and on 64-bit systems, the interval $(-2^{-61},2^{61}]$. Because fixnums automatically grow to bignums, usually you do not have to worry about details like this.}, and is a bit quicker to manipulate than an arbitrary-precision bignum.
The predicate word \texttt{integer?}~tests if the top of the stack is an integer. If this returns true, then exactly one of \texttt{fixnum?}~or \texttt{bignum?}~would return true for that object. Usually, your code does not have to worry if it is dealing with fixnums or bignums.
Integer operations automatically return bignums if the result would be too big to fit in a fixnum. Here is an example where multiplying two fixnums returns a bignum:
Integers can be entered using a different base; see \ref{integer-literals}.
The word \texttt{.} prints numbers in decimal, regardless of how they were input. A set of words in the \texttt{prettyprint} vocabulary is provided to print integers using another base.
A pair of combinators calls a quotation a fixed number of times.
\wordtable{
\vocabulary{math}
\ordinaryword{times}{times ( n quot -- )}
\texttt{quot:~-- }\\
}
Calls the quotation $n$ times. If $n<0$, the quotation is not called at all.
\wordtable{
\vocabulary{math}
\ordinaryword{repeat}{repeat ( n quot -- )}
\texttt{quot:~i -- i }\\
}
Calls \texttt{quot}$n$ times, with the parameter \texttt{i} ranging from 0 to $n-1$. The quotation must output $i$ unmodified; or indeed, if it modifies it, the loop continues from that index. That is, the value $i$ on the stack is the actual loop counter, not a copy.
There are two ways of looking at an integer -- as a mathematical entity, or as a string of bits. The latter representation motivates \emph{bitwise operations}.
Computes the bitwise complement of the input; that is, each bit in the input number is flipped. Because integers are represented in two's complement form, this is actually equivalent to negating the integer, and subtracting 1.
Computes a new integer consisting of the bits of the first integer, shifted to the left by $n$ positions. If $n$ is negative, the bits are shifted to the right instead, and bits that ``fall off'' are discarded.
Computes the largest integer less than or equal to $log_2 n$. The input must be positive and the result is always an integer. In most cases, the \verb|log| word (\ref{algebraic}) should be used instead, since it allows any complex number as input, and the result is not truncated to an integer.
\wordtable{
\vocabulary{math}
\ordinaryword{each-bit}{each-bit ( n quot -{}- | quot: 0/1 -{}- )}
}
Applies the quotation to each bit of the input. The input must be a positive integer.
If we add, subtract or multiply any two integers, the result is always an integer. However, this is not the case with division. When dividing a numerator by a denominator where the numerator is not a integer multiple of the denominator, a ratio is returned instead.
Ratios are printed and can be input literally in the form above. Ratios are always reduced to lowest terms by factoring out the greatest common divisor of the numerator and denominator. A ratio with a denominator of 1 becomes an integer. Trying to create a ratio with a denominator of 0 raises an error.
description={an instance of the \texttt{real} class, which is a disjoint union of the
\texttt{rational} and \texttt{float} classes}}}
\realglos
\floatglos
Rational numbers represent \emph{exact} quantities. On the other hand, a floating point number is an \emph{approximation}. While rationals can grow to any required precision, floating point numbers are fixed-width, and manipulating them is usually faster than manipulating ratios or bignums (but slower than manipulating fixnums). Floating point literals are often used to represent irrational numbers, which have no exact representation as a ratio of two integers. Floating point literals are input with a decimal point.
Complex numbers arise as solutions to quadratic equations whose graph does not intersect the $x$ axis. Their literal syntax is covered in \ref{complex-literals}.
Tests if the top of the stack is a complex number. Note that unlike math, where all real numbers are also complex numbers, Factor only considers a number to be a complex number if its imaginary part is non-zero.
Converts between complex numbers and pairs of real numbers representing them in polar form. The polar form of a complex number consists of an absolute value and argument.
Raises the number $e$\footnote{Approximately equal to $2.718281828459045$} to a specified power. The number $e$ can be pushed on the stack with the \texttt{e} word, so \texttt{exp} could have been defined as follows:
However, it is actually defined otherwise, for efficiency.\footnote{In fact, things are done the other way around; the word \texttt{\^{}} is actually defined in terms of \texttt{exp}, to correctly handle complex number arguments.}
The \texttt{math} vocabulary provides the full set of trigonometric and hyperbolic functions, along with inverses and reciprocals. Complex number arguments are supported.
Any Factor sequence can be used to represent a mathematical vector, not just instances of the \verb|vector| class. Anywhere a vector is mentioned in this section, keep in mind it is a mathematical term, not a Factor data type.
The usual mathematical operations on vectors are supported.
This is \emph{not} the mathematical dot product or cross product operation.
\wordtable{
\vocabulary{matrices}
\ordinaryword{v.}{v.~( vec vec -- n )}
}
Computes the inner product of two vectors. They must be of equal length.
Mathematically speaking, this is a map $<,>: {\mathbb{C}}^n \times{\mathbb{C}}^n \rightarrow\mathbb{C}$. It is the complex inner product; that is, $<a,b> =\overline{<b,a>}$, where $\overline{z}$ is the complex conjugate.
Computes the cross product $v_1\times v_2$. The following example illustrates the mathematical fact that a cross product of two vectors is always orthogonal to either vector.
Matrix literal syntax is documented in \ref{syntax:matrices}. In addition to the literal syntax, new matrices may be created from scratch in one of several ways.
Creates a new matrix with the given dimensions and all elements set to zero.
\wordtable{
\vocabulary{matrices}
\ordinaryword{<identity-matrix>}{<identity-matrix> ( n -- matrix )}
}
Creates a new $n\times n$ matrix where all elements on the main diagonal are 1, and all other elements are zero; for example:
\begin{alltt}
\textbf{ok} 3 <identity-matrix> prettyprint
M[ [ 1 0 0 ]
[ 0 1 0 ]
[ 0 0 1 ] ]M
\end{alltt}
The following are the usual algebraic operations on matrices.
\wordtable{
\vocabulary{matrices}
\ordinaryword{n*m}{n*m ( n matrix -- matrix )}
}
Multiplies each element of a matrix by a scalar.
\begin{alltt}
\textbf{ok} 5 2 <identity-matrix> n*m prettyprint
\textbf{M[ [ 5 0 ]
[ 0 5 ] ]M}
\end{alltt}
\wordtable{
\vocabulary{matrices}
\ordinaryword{m+}{m+ ( matrix matrix -- matrix )}
}
Adds two matrices. They must have the same dimensions.
\wordtable{
\vocabulary{matrices}
\ordinaryword{m+}{m+ ( matrix matrix -- matrix )}
}
Subtracts two matrices. They must have the same dimensions.
\wordtable{
\vocabulary{matrices}
\ordinaryword{m*}{m* ( matrix matrix -- matrix )}
}
Multiplies two matrices element-wise. They must have the same dimensions. This is \emph{not} matrix multiplication in the usual mathematical sense.
\wordtable{
\vocabulary{matrices}
\ordinaryword{m.}{m.~( matrix matrix -- matrix )}
}
Composes two matrices as linear operators. This is the usual mathematical matrix multiplication, and the first matrix must have the same number of columns as the second matrix has rows.
There is a natural isomorphism between the vector space $\mathbb{C}^m$, the $m\times1$ matrices, and the $1\times m$ matrices. Additionally, a $m\times n$ matrix acts as a linear operator from the vector space $\mathbb{C}^n$ to $\mathbb{C}^m$ in the same way as multiplying the $m\times n$ matrix by a $n \times1$ matrix. In Factor, these ideas are embodied by a set of words for converting vectors to matrices, and vice-versa.
sink of characters. Streams also support formatted output, which may be used to present styled text in a manner independent of output medium. A few stream implementations are provided by the library.
\begin{description}
\item[File streams] read and write local files.
\item[Network streams] connect to servers and accept connections from clients.
\item[HTML streams] implement the formatted output protocol to generate HTML from styled text attributes, then direct the HTML to an underlying stream.
description={a stream that implements the \texttt{stream-readln} and \texttt{stream-read} generic words and can be used for character input}}
\glossary{name=output stream,
description={a stream that implements the \texttt{stream-write-attr}, \texttt{stream-flush} and \texttt{stream-auto-flush} generic words and can be used for character output}}
There are various subsets of the stream protocol that a class can implement so that its instances may be used as streams. The following generic word is mandatory.
Releases any external resources associated with the stream, such as file handles and network connections. No further operations can be performed on the stream after this call.
You must close streams after you are finished working with them. A convenient way to automate this is by using the \texttt{with-stream} word in \ref{stdio}.
The following two words are optional, and should be implemented on input streams.
Reads a line of text and outputs it on the stack. If the end of the stream has been reached, outputs \texttt{f}. The precise meaning of a ``line'' depends on the stream; with file and network streams, it is a range of characters terminated by \verb|\n|, \verb|\r| or \verb|\r\n|.
Reads \texttt{n} characters from the stream. If less than \texttt{n} characters are available before the end of the stream, a shorter string is output.
The following three words are optional, and should be implemented on output streams.
Outputs a character or string to the stream. This might not result in immediate output to the underlying resource if the stream performs buffering, like all file and network streams do.
The \texttt{attrs} parameter is an association list holding style information, and it is ignored by most streams -- one exception is HTML streams (\ref{html}). Most of the time either the \texttt{stream-write} or \texttt{stream-print} word is used. They are described in the next section.
Ensures all pending output operations are been complete. With many output streams, written output is buffered and not sent to the underlying resource until either the buffer is full, or an explicit call to \texttt{stream-flush} is made.
Ensures the user sees prior output. It is not as strong as \texttt{stream-flush}. The contract is as follows: if the stream is connected to an interactive end-point such as a terminal, \texttt{stream-auto-flush} should execute \texttt{stream-flush}. If the stream is a file or network stream used for ``batch'' operations, this word should have an empty definition.
The \texttt{stream-print} word executes \texttt{stream-auto-flush} after each line of output.
With some streams, the above operations may suspend the current thread and execute other threads until input data is available (\ref{threads}).
\subsection{Stream utilities}
The following three words are implemented in terms of the stream protocol, and should work with any stream supporting the required underlying operations.
Outputs a character or string to the stream, followed by a newline, then executes \texttt{stream-auto-flush} to force the line to be displayed on interactive streams.
Calls the quotation in a new dynamic scope, with the \texttt{stdio} variable set to \texttt{stream}. The stream is closed when the quotation returns or if an exception
Calls the quotation in a new dynamic scope, with the \texttt{stdio} variable set to a new string buffer. Executing \texttt{write}, \texttt{write-attr} or \texttt{print} will append text to the string buffer. When the quotation returns, the string buffer is coverted to
\ordinaryword{<client>}{<client>~( host port -- stream~)}
}
Connects to TCP/IP port number \texttt{port} on the host named by \texttt{host}, and returns a bidirectional stream. An exception is thrown if the connection attempt fails.
\ordinaryword{<server>}{<server>~( port -- server~)}
}
Begins listening for connections to \texttt{port} on all network interfaces. An exception is thrown if the port cannot be opened. The returned object can be used as an input to the \texttt{stream-close} and \texttt{accept} words.
\ordinaryword{accept}{accept~( server -- stream~)}
}
Waits for a connection to the port number that \texttt{server} is listening on, and outputs a bidirectional stream when the connection has been established. An exception is thrown if an error occurs.
Creates a null stream, which ignores output written to it, and returns end of file if an attempt is made to read.
\wordtable{
\vocabulary{streams}
\ordinaryword{<duplex-stream>}{<duplex-stream>~( in out -- stream~)}
}
Creates a duplex stream. Writing to a duplex stream will write to \texttt{out}, and reading from a duplex stream will read from \texttt{in}. Closing a duplex stream closes both the input and output streams.
Creates a stream wrapping \texttt{stream}. The given stream becomes the delegate of the new wrapper stream, so calling any stream operation on the wrapper passes it on to the delegate.
You can then define your own tuple class that delegates to a wrapper stream, then override methods on this new tuple class, and use the following combinator in your method definitions.
Executes the quotation in a dynamic scope where the \texttt{stdio} variable is set to the wrapped stream.
The following example implements a stream that emits \TeX\ markup when a certain attribute is set in the \texttt{attrs} parameter to \texttt{stream-write-attr}.
description={a set of words for printing objects in readable form}}
One of Factor's key features is the ability to print almost any object in a readable form. This greatly aids debugging and provides the building blocks for light-weight object serialization facilities.
The unparser provides a basic facility for turning certain types of objects into strings. A more general facility supporting more types is the prettyprinter (\ref{prettyprint}).
Outputs a string representation of \texttt{object}. Only the following classes of objects are supported; for anything else, an unreadable string is output:
Prints the object using literal syntax that can be parsed back again. Even though the prettyprinter supports more classes of objects than \texttt{unparse}, it is still not a general serialization mechanism. The following restrictions apply:
\item Not all objects print in a readable way. Namely, the following classes do not:
\begin{verbatim}
array
byte-array
displaced-alien
\end{verbatim}
\item Circular structure is not printed in a readable way. Circular references are printed as ``\texttt{...}''.
\item Floating point numbers might not equal themselves after being printed and read, since a decimal representation of a float is inexact.
\end{itemize}
\wordtable{
\vocabulary{prettyprint}
\ordinaryword{.}{.~( object --~)}
}
Prettyprint the object, except all output is on a single line without indentation, and deeply-nested structure is not printed fully. This word is intended for interactive use at the listener.
Controls the maximum nesting depth. Printing structures that nest further than this will simply print ``\texttt{...}''. If this is set to \texttt{f}, the nesting depth is unlimited. The default is \texttt{f}. Inside calls to \texttt{.}, set to 16, which translates to four levels of nesting with the default tab size.
\wordtable{
\vocabulary{prettyprint}
\symbolword{one-line}
}
If set to true, the prettyprinter does not emit newlines. The default is \texttt{f}. Inside calls to \texttt{.}, set to \texttt{t}.
Prettyprints the given object. Unlike \texttt{prettyprint}, this word does not emit a trailing newline, and the current indent level is given. This word is also generic, so you can add methods to have it print your own data types in a nice way.
This section concerns itself with reflective access and extension of the parser. The parser algorithm and standard syntax is described in \ref{syntax}. Before the parser proper is documented, we draw attention to a set of words for parsing numbers. They are called by the parser, and are useful in their own right.
\ordinaryword{str>number}{str>number~( string -- number )}
}
Attempts to parse the string as a number. An exception is thrown if the string does not represent a number in one of the following forms:
\begin{itemize}
\item An integer; see \ref{integer-literals}
\item A ratio; see \ref{ratio-literals}
\item A float; see \ref{float-literals}
\end{itemize}
In particular, complex numbers are parsed by the \verb|#{| and \verb|}#| parsing words, not by the number parser. To parse complex number literals, use the \texttt{parse} word (\ref{parsing-quotations}).
Like \texttt{str>number}, except instead of raising an error, outputs \texttt{f} if the string is not a valid literal number.
\wordtable{
\vocabulary{parser}
\genericword{base>}{base>~( string base -- integer )}
}
Converts a string representation of an integer in the given base into an integer. Throws an exception if the string is not a valid representation of an integer.
\wordtable{
\vocabulary{parser}
\ordinaryword{bin>}{bin>~( string -- integer )}
\ordinaryword{oct>}{oct>~( string -- integer )}
\ordinaryword{dec>}{dec>~( string -- integer )}
\ordinaryword{hex>}{hex>~( string -- integer )}
}
Convenience words defined in terms of \texttt{base>} for parsing integers in base 2, 8, 10 and 16, respectively.
As documented in \ref{vocabsearch}, the parser looks up words in the vocabulary search path. New word definitions are added to the current vocabulary. These two parameters are stored in a pair of variables (\ref{namespaces}):
There are two sets of words for parsing input from streams. The first set uses the following initial values for the \texttt{"use"} and \texttt{"in"} variables:
Parses lines of text from the stream and outputs a quotation. The \texttt{name} parameter identifies the stream in error messages. The stream is closed when the end is reached.
The next set of stream parsing words takes the vocabulary search path and current vocabulary from the current scope. These words are used to load the \texttt{.factor-rc} file on startup, so that any \texttt{USE:}~and \texttt{USING:}~declarations set in that file take effect in the listener (\ref{listener}).
Parsing words execute at parse time, and therefore can access and modify the state of the parser, as well as add objects to the parse tree. Parsing words are a difficult concept to grasp, so this section has several examples and explains the workings of some of the parsing words provided in the library.
To define a parsing word, suffix the colon definition with the \texttt{parsing} word.
\wordtable{
\vocabulary{syntax}
\parsingword{parsing}{parsing}
}
Marks the most recently defined word as a parsing word. For example:
\begin{verbatim}
: hello "Hello world" print ; parsing
\end{verbatim}
Now writing \texttt{hello} anywhere will print the message \texttt{"Hello world"} at parse time. Of course, this is a useless definition. In the sequel, we will look into writing useful parsing words that modify parser state.
\subsubsection{Nested structure}
The first thing to look at is how the parse tree is built. When parsing begins, the empty list is pushed on the data stack; whenever the parser algorithm appends an object to the parse tree, it conses the object onto the quotation at the top of the stack. This builds the quotation in reverse order, so when parsing is done, the quotation is reversed before it is called.
Lets look at a simple example; the parsing of \texttt{"1 2 3"}:
\begin{tabular}{l|l|l}
\hline
Token&Stack before&Stack after\\
\hline
\verb|1|&\verb|[ ]|&\verb|[ 1 ]|\\
\verb|2|&\verb|[ 1 ]|&\verb|[ 2 1 ]|\\
\verb|3|&\verb|[ 2 1 ]|&\verb|[ 3 2 1 ]|
\end{tabular}
Once the end of the string has been reached, the quotation is reversed, and the output, as you would expect, is \verb|[ 1 2 3 ]|.
Nested structure is a bit more involved. The basic idea is that parsing words can push an empty list on the stack, then all subsequent tokens are consed onto this quotation, until another parsing word adds this quotation to the quotation underneath.
The following definitions of the \verb|[| and \verb|]| parsing words illustrate the idiom:
\begin{verbatim}
: [ f ; parsing
: ] reverse swons ; parsing
\end{verbatim}
Let us look at how the following string parses:
\begin{verbatim}
"1 [ 2 3 ] 4"
\end{verbatim}
\begin{tabular}{l|l|l|l}
\hline
Token&Stack before&Stack after&Note\\
\hline
\verb|1|&\verb|[ ] [ ]|&\verb|[ ] [ 1 ]|&\\
\textbf{\texttt{[}}&\verb|[ 1 ]|&\verb|[ 1 ] [ ]|&pushes an empty list\\
description=a parsing word reads ahead of it scans following tokens from the input string}
The next idiom to look at is parsing words that read ahead. The first example is the \verb|HEX:| word, documented in \ref{integer-literals}. This word is defined so that the following two lines are equivalent:
\begin{verbatim}
HEX: deadbeef
3735928559
\end{verbatim}
It is defined in terms of a lower-level \texttt{(BASE)} word that takes the numerical base on the data stack, reads the next token from the string, then calls \texttt{base>} (\ref{parsing-numbers}):
\begin{verbatim}
: (BASE) ( base -- ) scan swap base> swons ;
: HEX: 16 (BASE) ; parsing
\end{verbatim}
The key word here is \texttt{scan}.
\wordtable{
\vocabulary{parser}
\ordinaryword{scan}{scan ( -- string )}
}
Outputs the next token as a string, or \texttt{f} if the end of the input has been reached. Advances the parser state to after this token.
The next example of a parsing word we will look at is the \verb|\| word. It reads the next token from the input, and appends code to push that word literally on the stack. That is, the following two phrases both have the effect of pushing the word \verb|+| on the stack, rather than executing it:
\begin{verbatim}
\ +
[ + ] car
\end{verbatim}
We can look at how \verb|\| is implemented:
\begin{verbatim}
: \ scan-word unit swons \ car swons ; parsing
\end{verbatim}
The key word here is \verb|scan-word|. It combines \texttt{scan} word with vocabulary search.
\wordtable{
\vocabulary{parser}
\ordinaryword{scan-word}{scan-word ( -- word )}
}
Reads the next token from the input and looks up a word with this name. If the lookup fails, attempts to parse the word as a number by calling \verb|str>number|.
\subsubsection{Defining words}
\definingwordglos
Defining words add definitions to the dictionary without modifying the parse tree.
The first example to look at is the \verb|SYMBOL:| word. It reads the next token from the input stream, creates a word with that name, and makes it a symbol (\ref{symbols}). The next
example is the common \verb|:| word, which creates a colon definition. First, it reads the
name of the new word, then the definition is built up until \verb|;|. The latter
example will demonstrate building nested structure in defining words.
First, let us look at the \verb|SYMBOL:| word (\ref{symbols}).
\begin{verbatim}
: SYMBOL: CREATE define-symbol ; parsing
\end{verbatim}
The key factor the above definition is \verb|CREATE|, which reads a token from the input and creates a word with that name. This word is then passed to \verb|define-symbol|.
\wordtable{
\vocabulary{parser}
\ordinaryword{CREATE}{CREATE ( -- word )}
}
Reads the next token from the input and creates a word in the current vocabulary with that name. It uses \verb|create-in| to do this (\ref{creating-words}).
The definition of \verb|:| introduces the next idiom, and that is building a quotation and then adding a definition using \verb|;|.
\begin{verbatim}
: :
CREATE [ define-compound ] [ ]
"in-definition" on ; parsing
\end{verbatim}
The factors of the word are, in order:
\begin{description}
\item[\texttt{CREATE}] reads the following token and pushes a new word on the stack,
\item[\texttt{[ define-compound ]}] a quotation to be called by \verb|;|,
\item[\texttt{[ ]}] an empty list that the parser will build the colon definition on,
\item[\texttt{"in-definition" on}] sets a flag that subsequent parsing words can query.
\end{description}
While \verb|:| is very specific, \verb|;| is quite general because it takes a quotation pushed by a previous parsing word. You can use \verb|;| in your own parsing words.
\wordtable{
\parsingword{;}{;~( definer parsed -- )}
\texttt{definer:~parsed --}\\
}
Reverses the \verb|parsed| quotation, and passes it as input to the \verb|definer| quotation.
The definition of this word is in some sense dual to \verb|:| even thought it is more general:
\begin{verbatim}
: ; "in-definition" off reverse swap call ; parsing
\end{verbatim}
Suppose we are parsing the following string:
\begin{verbatim}
: sq dup * ;
\end{verbatim}
We can trace the parsing as before.
\begin{tabular}{l|l|l}
\hline
Token&Stack after&Note\\
\hline
\verb|:|&\verb|[ ] sq [ define-compound ] [ ]|&reads the next token\\
\item[\texttt{swap call}] calls \texttt{[ define-compound ]}. Thus, \verb|define-compound| is called to define \verb|sq| as the quotation \verb|[ dup * ]|.
\end{description}
\subsubsection{\label{string-mode}String mode and parser variables}
\stringmodeglos
String mode allows custom parsing of tokenized input. For even more esoteric situations, the input text can be accessed directly.
String mode is controlled by the \verb|string-mode| variable.
\wordtable{
\vocabulary{parser}
\symbolword{string-mode}
}
When enabled, the parser adds tokens to the parse tree as strings. This creates a paradox because further parsing words are not executed while string mode is on. However, if the token \verb|";"| is read, there is a special case that calls the \verb|;| parsing word. This parsing word reverses the quotation at the top of the stack, and calls the quotation underneath it, as usual.
An illustration of this idiom is found in the \verb|USING:| parsing word. It reads a list of vocabularies, terminated by \verb|;|. However, the vocabulary names do not name words, except by coincidence; so string mode is used to read them.
\begin{verbatim}
: USING:
string-mode on [
string-mode off [ use+ ] each
] [ ] ; parsing
\end{verbatim}
Make note of the quotation that is left in position for \verb|;| to call. It switches off string mode, so that normal parsing can resume, then adds the given vocabularies to the search path.
If the parser features described in the earlier sections are still insufficient, you can directly access a pair of variables holding parser state:
\begin{description}
\item[\texttt{"line"}] the text being parsed,
\item[\texttt{"col"}] the column number.
\end{description}
The \verb|"col"| variable is implicitly changed the \verb|scan| word (\ref{reading-ahead}), and the following word.
\wordtable{
\vocabulary{parser}
\ordinaryword{until-eol}{until-eol ( -- string )}
}
Outputs the remainder of the line being parsed. The \verb|"col"| variable is set to point to the end of the line.
This word is used to implement end-of-line comments:
An HTML stream wraps an existing stream. Strings written to the HTML stream have their special characters converted to HTML entities before being passed on to the wrapped stream. Also, the \texttt{attrs} parameter to the \texttt{stream-write-attr} word may be filled out to wrap the text being written in various HTML tags.
Calls the quotation in a new dynamic scope. The \texttt{stdio} variable is set to an HTML stream wrapping the previous value of \texttt{stdio}, so calls to \texttt{write}, \texttt{write-attr} and \texttt{print} go through the HTML stream.
\wordtable{
\vocabulary{html}
\ordinaryword{html-document}{html-document ( title quot -- )}
Builds on \texttt{with-html-stream} to emit the basic structure of an HTML document, consisting of \texttt{<html>}, \texttt{<head>} and \texttt{<body>} tags. The title is output in two places; a \texttt{<title>} tag and \texttt{<h1>} tag.
\ordinaryword{simple-html-document}{simple-html-document ( title quot -- )}
}
Like \texttt{html-document}, except the output is wrapped inside a \texttt{<pre>} tag.
Calls to \texttt{write-attr} inside a quotation given to one of the above combinators will perform HTML output. The following keys may be set in the \texttt{attrs} association list given to \texttt{write-attr}:
\begin{tabular}{l|l}
Key&Description\\
\hline
\texttt{"fg"}&The foreground color, as a list with red, green, blue components\\
\texttt{"bg"}&The background color, as a list with red, green, blue components\\
\texttt{"bold"}&A boolean\\
\texttt{"italics"}&A boolean\\
\texttt{"underline"}&A boolean\\
\texttt{"size"}&An integer\\
\texttt{"file"}&If set, a hyperlink to that file is output\\
\texttt{"word"}&If set, a hyperlink to that word is output\\
\texttt{"vocab"}&Must be set of \texttt{"word"} is set
\end{tabular}
Hyperlinks to files and words point to the file and browser responders, respectively. These responders must be enabled for such links to function.
Factor's alien inteface provides a means of directly calling native libraries written in C and other languages. There are no
wrappers to write, other than having to specify the return type and parameter types for
the functions you wish to call.
\subsection{Loading native libraries}
A native library must be made available to Factor under a logical name before use. This is done via command line parameters, or the \verb|add-library| word.
The following two command line parameters can be specified for each library to load; the second parameter is optional.
\begin{description}
\item[\texttt{-libraries:\emph{logical}:name=\emph{name}}] associates a logical name with a system-specific native library name,
\item[\texttt{-libraries:\emph{logical}:abi=\emph{type}}] specifies the calling convention to use; \verb|type| is either \verb|cdecl| or \verb|stdcall|. If not specified, the default is \verb|cdecl|. On Unix, all libraries follow the \verb|cdecl| convention. On Windows, most libraries (but not all) follow \verb|stdcall|.
Another option is to add libraries while Factor is running.
\wordtable{
\vocabulary{alien}
\ordinaryword{add-library}{add-library ( library name abi -- )}
}
Adds a logical library named \verb|library|. The underlying shared library name is \verb|name|, and the calling convention is \verb|abi| and must be either \verb|"cdecl"| or \verb|"stdcall"|.
Invokes the function named \verb|func| in the library with logical name \verb|lib|.
The \verb|return| value is a string naming a C type, and maybe set to \verb|void|, in the case of the native function not returning a value.
The \verb|parameters| value is a list
of strings naming C types. C types are listed in table \ref{c-types}.
For example, suppose you have a \verb|foo| library exporting the following function:
\begin{verbatim}
void the_answer(char* question, int value) {
printf("The answer to %s is %d.\n",question,value);
}
\end{verbatim}
You can define a word for invoking it:
\begin{verbatim}
: the-answer ( question value -- answer )
"void" "foo" "the_answer" [ "char*" "int" ]
alien-invoke ;
\end{verbatim}
Now, after being compiled, the word can be executed with two parameters on the stack:
\begin{alltt}
\textbf{ok}\bs the-answer compile
\textbf{Compiling the-answer}
\textbf{ok} "the question" 42 the-answer
\textbf{The answer to the question is 42.}
\end{alltt}
\subsection{\label{aliens}Alien objects}
\glossary{
name=alien,
description={an instance of the \verb|alien| class, holding a pointer to native memory outside the Factor heap}}
The alien interface can work with an assortment of native data types:
\begin{itemize}
\item integer and floating point values
\item null-terminated strings
\item structures (\ref{alien-structs})
\item unions (\ref{alien-unions})
\end{itemize}
Table \ref{c-types} lists the built-in return value and parameter types. The sizes are given for a 32-bit system. Native numbers and strings are handled in a straight-forward way. Pointers are a bit more complicated, and are wrapped inside alien objects on the Factor side.
\begin{table}
\caption{\label{c-types}Supported native types}
\begin{tabular}{l|l|l}
Name&Size&Representation\\
\hline
\texttt{char}&1& Signed integer\\
\texttt{uchar}&1& Unsigned integer\\
\texttt{short}&2& Signed integer\\
\texttt{ushort}&2& Unsigned integer\\
\texttt{int}&4& Signed integer\\
\texttt{uint}&4& Unsigned integer\\
\texttt{long}&4& Signed integer\\
\texttt{ulong}&4& Unsigned integer\\
\texttt{longlong}&8& Signed integer\\
\texttt{ulonglong}&8& Unsigned integer\\
\texttt{float}&4& Single-precision float\\
\texttt{double}&8& Double-precision float\\
\texttt{char*}&4& Pointer to null-terminated byte string\\
\texttt{ushort*}&4& Pointer to null-terminated UTF16 string\\
\texttt{void*}&4& Generic pointer
\end{tabular}
\end{table}
\wordtable{
\vocabulary{alien}
\ordinaryword{c-size}{c-size ( type -- n )}
}
Outputs the size of the given C type. This is just like the \verb|sizeof| operator in C.
Many native functions expect you to specify sizes for input and output parameters, and
this word can be used for that purpose.
\wordtable{
\vocabulary{alien}
\classword{alien}
}
Pointers to native memory, including \verb|void*| and other types, are represented as objects of the \verb|alien| class.
\wordtable{
\vocabulary{alien}
\predword{alien?}
}
Tests if the object at the top of the stack is an alien pointer.
\subsubsection{\label{alien-structs}Structures}
One way to think of a C-style \verb|struct| is that it abstracts reading and writing field values stored at a range of memory given a pointer, by associating a type and offset with each field. This is the view taken by the alien interface, where defining a C structure creates a set of words for reading and writing fields of various types, offset from a base pointer given by an alien object.
Adds a field to the structure. The \verb|type| token identifies a C type, and \verb|name| gives a name to the field. A pair of words is defined, where \verb|structure| and \verb|field| are names, respectively:
\begin{alltt}
\emph{structure}-\emph{field} ( alien -- value )
set-\emph{structure}-\emph{field} ( value alien -- )
\end{alltt}
\wordtable{
\vocabulary{alien}
\parsingword{END-STRUCT}{END-STRUCT}
}
Ends a structure definition.
Defining a structure adds two new C types, where \verb|name| is the name of the structure:
\begin{description}
\item[\texttt{\emph{name}}] the type of the structure itself; structure and union definitions can define members to be of this type.
\item[\texttt{\emph{name}*}] the type of a pointer to the structure; this type can be used with return values and parameters, and in fact it is an alias for \texttt{void*}.
\end{description}
Additionally, the following two words are defined:
\begin{description}
\item[\texttt{<\emph{name}> ( -- byte-array )}] allocates a byte array large enough to hold the structure in the Factor heap. The field accessor words can then be used to work with this byte array. This feature allows calling native functions that expect pointers to caller-allocated structures\footnote{
There is an important restriction, however; the function must not retain the pointer in a global variable after it returns. Since the structure is allocated in the Factor heap, the garbage collector is free to move it between native function calls. If this behavior is undesirable, memory can be managed manually instead (\ref{malloc}).}.
\item[\texttt{\emph{name}-nth ( n alien -- alien )}] given a pointer and index into an array of structures, returns a pointer to the structure at that index.
\end{description}
Here is an example of a structure with various fields:
\begin{verbatim}
BEGIN-STRUCT: surface
FIELD: uint flags
FIELD: format* format
FIELD: int w
FIELD: int h
FIELD: ushort pitch
FIELD: void* pixels
FIELD: int offset
FIELD: void* hwdata
FIELD: short clip-x
FIELD: short clip-y
FIELD: ushort clip-w
FIELD: ushort clip-h
FIELD: uint unused1
FIELD: uint locked
FIELD: int map
FIELD: uint format_version
FIELD: int refcount
END-STRUCT
\end{verbatim}
\subsubsection{\label{alien-unions}Unions}
A C-style \verb|union| type allocates enough space for its largest member. In the alien interface, unions are used to allocate byte arrays in the Factor heap that may hold any one of the union's members.
The alien interface is built on top of a handful of primitives. Sometimes, it is
useful to call these primitives directly for debugging purposes.
\wordtable{
\vocabulary{alien}
\classword{dll}
}
Instances of this class are handles to native libraries.
\wordtable{
\vocabulary{alien}
\ordinaryword{dlopen}{dlopen ( path -- dll )}
}
Opens the specified native library and returns a DLL object. The input parameter is the
name of a native library file,
\emph{not} a logical library name.
\wordtable{
\vocabulary{alien}
\ordinaryword{dlsym}{dlsym ( symbol dll -- address )}
}
Looks up a named symbol in a native library, and outputs it address. If the \verb|dll| is \verb|f|, the lookup is performed in the runtime executable itself.
\wordtable{
\vocabulary{alien}
\ordinaryword{dlclose}{dlclose ( dll -- )}
}
Closes a native library and frees associated native resources.
Outputs an alien pointing at an offset from the base pointer of the input alien. Displaced aliens are used to access nested structures and native arrays.
\wordtable{
\vocabulary{alien}
\ordinaryword{alien-signed-cell}{alien-signed-cell ( alien offset -- n )}
\ordinaryword{set-alien-signed-cell}{set-alien-signed-cell ( n alien offset -- )}
\ordinaryword{alien-unsigned-cell}{alien-unsigned-cell ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-cell}{set-alien-unsigned-cell( n alien offset -- )}
\ordinaryword{alien-signed-8}{alien-signed-8 ( alien offset -- n )}
\ordinaryword{set-alien-signed-8}{set-alien-signed-8 ( n alien offset -- )}
\ordinaryword{alien-unsigned-8}{alien-unsigned-8 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-8}{set-alien-unsigned-8 ( n alien offset -- )}
\ordinaryword{alien-signed-4}{alien-signed-4 ( alien offset -- n )}
\ordinaryword{set-alien-signed-4}{set-alien-signed-4 ( n alien offset -- )}
\ordinaryword{alien-unsigned-4}{alien-unsigned-4 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-4}{set-alien-unsigned-4 ( n alien offset -- )}
\ordinaryword{alien-signed-2}{alien-signed-2 ( alien offset -- n )}
\ordinaryword{set-alien-signed-2}{set-alien-signed-2 ( n alien offset -- )}
\ordinaryword{alien-unsigned-2}{alien-unsigned-2 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-2}{set-alien-unsigned-2 ( n alien offset -- )}
\ordinaryword{alien-signed-1}{alien-signed-1 ( alien offset -- n )}
\ordinaryword{set-alien-signed-1}{set-alien-signed-1 ( n alien offset -- )}
\ordinaryword{alien-unsigned-1}{alien-unsigned-1 ( alien offset -- n )}
\ordinaryword{set-alien-unsigned-1}{set-alien-unsigned-1 ( n alien offset -- )}
These primitives read and write native memory. They can be given an alien, displaced alien, or byte array. No bounds checking of any kind is performed.
If for whatever reason Factor's memory management is unsuitable for a certain task, you can
directly call the standard C memory management routines. These words are very raw and deal with addresses directly, and of course it is easy to corrupt memory or crash the runtime
this way.
\wordtable{
\vocabulary{kernel-internals}
\ordinaryword{malloc}{malloc ( size -- address )}
}
Allocate a block of size \verb|size| and output a pointer to it.
If you are used to a statically typed language, you might find Factor's tendency to only fail at runtime hard to work with at first. However, the interactive development tools outlined in this chapter allow a much quicker turn-around time for testing changes. Also, write unit tests -- unit testing is a great way to ensure that old bugs do not re-appear once they've been fixed.
Factor is an \emph{image-based environment}. When you compiled Factor, you also generated a file named \texttt{factor.image}. I will have more to say about images later, but for now it suffices to understand that to start Factor, you must pass the image file name on the command line:
An \texttt{\textbf{ok}} prompt is printed after the initial banner, indicating the listener is ready to execute Factor phrases. The listener is a piece of Factor code, like any other; however, it helps to think of it as the primary interface to the Factor system. The listener reads Factor code and executes it. You can try the classical first program:
\begin{alltt}
\textbf{ok} "Hello, world." print
\textbf{Hello, world.}
\end{alltt}
Multi-line phrases are supported; if there are unclosed brackets, the listener outputs \texttt{...} instead of the \texttt{ok} prompt, and the entire phrase is executed once all brackets are closed:
\begin{alltt}
\textbf{ok} [ 1 2 3 ] [
\textbf{...} .
\textbf{...} ] each
\textbf{1
2
3}
\end{alltt}
The listener knows when to print a continuation prompt by looking at the height of the
stack. Parsing words such as \texttt{[} and \texttt{:} leave elements on the parser
stack; these elements are popped by \texttt{]} and \texttt{;}.
On startup, Factor reads the \texttt{.factor-rc} file from your home directory. You can put
any quick definitions you want available at the listener there. To avoid loading this
file, pass the \texttt{-no-user-init} command line switch. Another way to have a set of definitions available at all times is to save a custom image, as described in the next section.
While it is possible to do all development in the listener and save your work in images, it is far more convenient to work with source files, at least until an in-image structure editor is developed.
By convention, Factor source files are saved with the \texttt{.factor} filename extension. They can be loaded into the image as follows:
In Factor, loading a source file replaces any existing definitions\footnote{But see \ref{compiler} for this is not true of compiled code.}. Each word definition remembers what source file it was loaded from (if any). To reload the source file associated with a definition, use the \texttt{reload} word:
\begin{alltt}
\textbf{ok}\bs draw reload
\end{alltt}
Word definitions also retain the line number where they are located in their original source file. This allows you to open a word definition in jEdit\footnote{\texttt{http://www.jedit.org}} for editing using the
\texttt{jedit} word:
\begin{alltt}
\textbf{ok}\bs compile jedit
\end{alltt}
This word requires that a jEdit instance is already running.
The \texttt{jedit} word will open word definitions from the Factor library once the full path of the Factor source tree is entered into the \texttt{"resource-path"} variable. One way to do this is to add a phrase like the following to your \texttt{.factor-rc}:
The \texttt{factor.image} file is basically a dump of all objects in the heap. A new image can be saved as follows:
\begin{alltt}
\textbf{ok} "work.image" save-image
\textbf{Saving work.image...}
\end{alltt}
When you save an image before exiting Factor, then start Factor again, everything will be almost as you left it. Try the following:
\begin{alltt}
./f factor.image
\textbf{ok} "Learn Factor" "reminder" set
\textbf{ok} "factor.image" save-image bye
\textbf{Saving factor.image...}
\end{alltt}
Factor will save the image and exit. Now start it again and see that the reminder is still there:
\begin{alltt}
./f factor.image
\textbf{ok} "reminder" get .
\textbf{"Learn Factor"}
\end{alltt}
This is what is meant by the image being an \emph{infinite session}. When you shut down and restart Factor, what happends is much closer to a Laptop's ``suspend'' mode, than a desktop computer being fully shut down.
\subsection{Looking at objects}
Probably the most important debugging tool of them all is the \texttt{.} word. It prints the object at the top of the stack in a form that can be parsed by the Factor parser. A related word is \texttt{prettyprint}. It is identical to \texttt{.} except the output is more verbose; lists, vectors and hashtables are broken up into multiple lines and indented.
Most objects print in a parsable form, but not all. One exceptions to this rule is objects with external state, such as I/O ports or aliens (pointers to native structures). Also, objects with circular or very deeply nested structure will not print in a fully parsable form, since the prettyprinter has a limit on maximum nesting. Here is an example -- a vector is created, that holds a list whose first element is the vector itself:
The prettyprinted form of a vector or list with many elements is not always readable. The \texttt{[.]} and \texttt{\tto.\ttc} words output a list or a vector, respectively, with each element on its own line. In fact, the stack printing words are defined in terms of \texttt{[.]} and \texttt{\tto.\ttc}:
\begin{verbatim}
: .s datastack {.} ;
: .r callstack {.} ;
: .n namestack [.] ;
: .c catchstack [.] ;
\end{verbatim}
Before we move on, one final set of output words comes is used to output integers in
different numeric bases. The \texttt{.b} word prints an integer in binary, \texttt{.o} in octal, and \texttt{.h} in hexadecimal.
Factor organizes code in a two-tier structure of vocabularies and words. A word is the smallest unit of code; it corresponds to a function or method in other languages. Vocabularies group related words together for easy browsing and tracking of source dependencies.
Entering \texttt{vocabs .}~in the listener produces a list of all existing vocabularies:
You can look at the definition of any word, including library words, using \texttt{see}. Keep in mind you might have to \texttt{USE:} the vocabulary first.
The \texttt{see} word shows a reconstruction of the source code, not the original source code. So in particular, formatting and some comments are lost.
\subsection{Cross-referencing words}
The \texttt{apropos.} word is handy when searching for related words. It lists all words
whose names contain a given string. The \texttt{apropos.} word is also useful when you know the exact name of a word, but are unsure what vocabulary it is in. For example, if you're looking for ways to iterate over various collections, you can do an apropos search for \texttt{map}:
\begin{alltt}
\textbf{ok} "map" apropos.
\textbf{IN: inference
type-value-map
IN: lists
map
map-with
IN: sdl
set-surface-map
surface-map
IN: strings
string-map
IN: vectors
vector-map}
\end{alltt}
From the above output, you can see that \texttt{map} is for lists, \texttt{string-map} is for strings, and \texttt{vector-map} is for vectors.
The \texttt{usage} word finds all words that refer to a given word and pushes a list on the stack. This word is helpful in two situations; the first is for learning -- a good way to learn a word is to see it used in context. The second is during refactoring -- if you change a word's stack effect, you must also update all words that call it. Usually you print the
return value of \texttt{usage} using \texttt{.}:
\begin{alltt}
\textbf{ok}\bs string-map usage .
\textbf{schars>entities
filter-null
url-encode}
\end{alltt}
Another useful word is \texttt{usages}. Unlike \texttt{usage}, it finds all usages, even
indirect ones -- so if a word refers to another word that refers to the given word,
both words will be in the output list.
\subsection{Exploring classes}
Factor supports object-oriented programming via generic words. Generic words are called
like ordinary words, however they can have multiple definitions, one per class, and
these definitions do not have to appear in the same source file. Such a definition is
termed a \emph{method}, and the method is said to \emph{specialize} on a certain
class. A class in the most
general sense is just a set of objects. You can output a list of classes in the system
client-stream fd-stream null-stream server string-output
wrapper-stream LETTER blank digit letter printable sbuf
string text POSTPONE: f POSTPONE: t vector compound
primitive symbol undefined word ]}
\end{alltt}
If you \texttt{see} a generic word, all methods defined on the generic word are shown.
Alternatively, you can use \texttt{methods.} to print all methods specializing on a
given class:
\begin{alltt}
\textbf{ok}\bs list methods.
\textbf{PREDICATE: general-list list
dup [
last* cdr
] when not ;
IN: gadgets
M: list custom-sheet
[
length count
] keep zip alist>sheet "Elements:" <titled> ;
IN: prettyprint
M: list prettyprint*
[
[
POSTPONE: [
] car swap [
POSTPONE: ]
] car prettyprint-sequence
] check-recursion ;}
\end{alltt}
\subsection{Browsing via the HTTP server}
A more sophisticated way to browse the library is using the integrated HTTP server. You can start the HTTP server using the following pair of commands:
To see the contents of the data stack, use the \texttt{.s} word. Similarly, the other stacks can be shown with \texttt{.r} (return stack), \texttt{.n} (name stack), and \texttt{.c} (catch stack). Each stack is printed with each element on its own line; the top of the stack is the first element printed.
\subsection{The debugger}
If the execution of a phrase in the listener causes an error to be thrown, the error
is printed and the stacks at the time of the error are saved. If you're spent any
time with Factor at all, you are probably familiar with this type of message:
\begin{alltt}
\textbf{ok} [ 1 2 3 ] 4 append reverse
\textbf{The generic word car does not have a suitable method for 4
:s :r :n :c show stacks at time of error.
:get ( var -- value ) inspects the error namestack.}
\end{alltt}
The words \texttt{:s}, \texttt{:r}, \texttt{:n} and \texttt{:s} behave like their counterparts that are prefixed with \texttt{.}, except they show the stacks as they were when the error was thrown.
The return stack warrants some special attention. To successfully develop Factor, you will need to learn to understand how it works. Lets look at the first few lines of the return stack at the time of the above error:
\begin{verbatim}
[ swap cdr ]
uncons
[ r> tuck 2slip ]
(each)
[ swons ]
[ each ]
each
\end{verbatim}
You can see the sequence of calls leading up to the error was \texttt{each} calling \texttt{(each)} calling \texttt{uncons}. The error tells us that the \texttt{car} word is the one that failed. Now, you can stare at the stack dump, at notice that if the call to \texttt{car} was successful and execution returned to \texttt{(each)}, the quotation \texttt{[ r> tuck 2slip ]} would resume executing. The first word there, \texttt{r>}, would take the quotation \texttt{[ swons ]} and put it back on the data stack. After \texttt{(each)} returned, it would then continue executing the quotation \texttt{[ each ]}. So what is going on here is a recursive loop, \texttt{[ swons ] each}. If you look at the definition of \texttt{reverse}, you will see that this is exactly what is being done:
\begin{verbatim}
: reverse ( list -- list ) [ ] swap [ swons ] each ;
\end{verbatim}
So a list is being reversed, but at some stage, the \texttt{car} is taken of something that is not a number. Now, you can look at the data stack with \texttt{:s}:
\begin{verbatim}
<< no-method [ ] 4 car >>
car
4
4
[ 3 2 1 ]
\end{verbatim}
So now, the mystery has been solved: as \texttt{reverse} iterates down the input value, it hits a cons cells whose \texttt{cdr} is not a list. Indeed, if you look at the value we are passing to \texttt{reverse}, you will see why:
\begin{alltt}
\textbf{ok} [ 1 2 3 ] 4 append .
[[ 1 [[ 2 [[ 3 4 ]] ]] ]]
\end{alltt}
In the future, the debugger will be linked with the walker, documented below. Right now, the walker is a separate tool. Another caveat is that in compiled code, the return stack is not reconstructed if there is an error. Until this is fixed, you should only compile code once it is debugged. For more potential compiler pitfalls, see \ref{compiler}.
\subsection{The walker}
The walker lets you step through the execution of a qotation. When a compound definition is reached, you can either keep walking inside the definition, or execute it in one step. The stacks can be inspected at each stage.
There are two ways to use the walker. First of all, you can call the \texttt{walk} word explicitly, giving it a quotation:
\&get ( var -- value ) inspects the stepper namestack.
step -- single step over
into -- single step into
continue -- continue execution
bye -- exit single-stepper
[ [ 10 [ dup , ] repeat ] make-list ]
walk}
\end{alltt}
As you can see, the walker prints a brief help message, then the currently executing quotation. It changes the listener prompt from \texttt{ok} to \texttt{walk}, to remind you that there is a suspended continuation.
The first element of the quotation shown is the next object to be evaluated. If it is a literal, both \texttt{step} and \texttt{into} have the effect of pushing it on the walker data stack. If it is a compound definition, then \texttt{into} will recurse the walker into the compound definition; otherwise, the word executes in one step.
The \texttt{\&r} word shows the walker return stack, which is laid out just like the primary interpreter's return stack. In fact, a good way to understand how Factor's return stack works is to play with the walker.
Note that the walker does not automatically stop when the quotation originally given finishes executing; it just keeps on walking up the return stack, and even lets you step through the listener's code. You can invoke \texttt{continue} or \texttt{exit} to terminate the walker.
While the walker can be invoked explicitly using the \texttt{walk} word, sometimes it is more convenient to \emph{annotate} a word such that the walker is invoked automatically when the word is called. This can be done using the \texttt{break} word:
\begin{alltt}
\textbf{ok}\bs layout* break
\end{alltt}
Now, when some piece of code calls \texttt{layout*}, the walker will open, and you will be able to step through execution and see exactly what's going on. An important point to keep in mind is that when the walker is invoked in this manner, \texttt{exit} will not have the desired effect; execution will continue, but the data stack will be inconsistent, and an error will most likely be raised a short time later. Always use \texttt{continue} to resume execution after a break.
The walker is very handy, but sometimes you just want to see if a word is being called at all and when, and you don't care to single-step it. In that case, you can use the \texttt{watch} word:
\begin{alltt}
\textbf{ok}\bs draw-shape break
\end{alltt}
Now when \texttt{draw-shape} is called, a message will be printed to that effect.
You can undo the effect of \texttt{break} or \texttt{watch} by reloading the original source file containing the word definition in question:
\begin{alltt}
\textbf{ok}\bs layout* reload
\textbf{ok}\bs draw-shape reload
\end{alltt}
\subsection{Dealing with hangs}
If you accidentally start an infinite loop, you can send the Factor runtime a \texttt{QUIT} signal. On Unix, this is done by pressing \texttt{Control-\bs} in the controlling terminal. This will cause the runtime to dump the data and return stacks in a semi-readable form. Note that this will help you find the root cause of the hang, but it will not let you interrupt the infinite loop.
\section{Defensive coding}
\subsection{Unit testing}
Unit tests are very easy to write. They are usually placed in source files. A unit test can be executed with the \texttt{unit-test} word in the \texttt{test} vocabulary. This word takes a list and a quotation; the quotation is executed, and the resulting data stack is compared against the list. If they do not equal, the unit test has failed. Here is an example of a unit test:
\begin{verbatim}
[ "Hello, crazy world" ] [
"editor" get [ 0 caret set ] bind
", crazy" 5 "editor" get [ line-insert ] bind
"editor" get [ line-text get ] bind
] unit-test
\end{verbatim}
To have a unit test assert that a piece of code does not execute successfully, but rather throws an exception, use the \texttt{unit-test-fails} word. It takes only one quotation; if the quotation does \emph{not} throw an exception, the unit test has failed.
\begin{verbatim}
[ -3 {} vector-nth ] unit-test-fails
\end{verbatim}
Unit testing is a good habit to get into. Sometimes, writing tests first, before any code, can speed the development process too; by running your unit test script, you can gauge progress.
\subsection{Stack effect inference}
While most programming errors in Factor are only caught at runtime, the stack effect checker can be useful for checking correctness of code before it is run. It can also help narrow down problems with stack shuffling. The stack checker is used by passing a quotation to the \texttt{infer} word. It uses a sophisticated algorithm to infer stack effects of recursive words, combinators, and other tricky constructions, however, it cannot infer the stack effect of all words. In particular, anything using continuations, such as \texttt{catch} and I/O, will stump the stack checker. Despite this fault, it is still a useful tool.
\begin{alltt}
\textbf{ok} [ pile-fill * >fixnum over pref-size dup y
\texttt{...} [ + ] change ] infer .
\textbf{[ [ tuple number tuple ] [ tuple fixnum object number ] ]}
The stack checker will report an error if it cannot infer the stack effect of a quotation. The ``recursive state'' dump is similar to a return stack, but it is not a real return stack, since only a code walk is taking place, not full evaluation. Understanding recursive state dumps is an art, much like understanding return stacks.
:get ( var -- value ) inspects the error namestack.}
\end{alltt}
One reason stack inference might fail is if the quotation contains unbalanced branches, as above. For the inference to work, both branches of a conditional must exit with the same stack height.
Another situation when it fails is if your code calls quotations that are not statically known. This can happen if the word in question uses continuations, or if it pulls a quotation from a variable and calls it. This can also happen if you wrote your own combinator, but forgot to mark it as \texttt{inline}. For example, the following will fail:
\begin{alltt}
\textbf{ok} : dip swap >r call r> ;
\textbf{ok} [ [ + ] dip * ] infer .
! Inference error: A literal value was expected where a
computed value was found: \#<computed @ 679711507>
...
\end{alltt}
However, defining \texttt{dip} to be inlined will work:
\begin{alltt}
\textbf{ok} : dip swap >r call r> ; inline
\textbf{ok} [ [ + ] dip * ] infer .
\textbf{[ [ number number number ] [ number ] ]}
\end{alltt}
You can combine unit testing with stack effect inference by writing unit tests that check stack effects of words. In fact, this can be automated with the \texttt{infer>test.} word; it takes a quotation on the stack, and prints a code snippet that tests the stack effect of the quotation:
\begin{alltt}
\textbf{ok} [ draw-shape ] infer>test.
\textbf{[ [ [ object ] [ ] ] ]
[ [ draw-shape ] infer ]
unit-test}
\end{alltt}
You can then copy and paste this snippet into a test script, and run the test script after
making changes to the word to ensure its stack effect signature has not changed.
\section{Optimization}
While both the Factor interpreter and compiler are relatively slow at this stage, there
are still ways you can make your Factor code go faster. The key is to find bottlenecks,
and optimize them.
\subsection{Timing code}
The \texttt{time} word reports the time taken to execute a quotation, in milliseconds. The portion of time spent in garbage collection is also shown:
\begin{alltt}
\textbf{ok} [ 1000000 [ f f cons drop ] repeat ] time
\textbf{515 milliseconds run time
11 milliseconds GC time}
\end{alltt}
\subsection{Exploring memory usage}
Factor supports heap introspection. You can find all objects in the heap that match a certain predicate using the \texttt{instances} word. For example, if you suspect a resource leak, you can find all I/O ports as follows:
The \texttt{references} word finds all objects that refer to a given object:
\begin{alltt}
\textbf{ok} [ float? ] instances car references .
\textbf{[ \#<array @ 805542171> [ -1.0 0.0 / ] ]}
\end{alltt}
You can print a memory usage summary with \texttt{room.}:
\begin{alltt}
\textbf{ok} room.
\textbf{Data space: 16384 KB total 2530 KB used 13853 KB free
Code space: 16384 KB total 490 KB used 15893 KB free}
\end{alltt}
And finally, a detailed memory allocation breakdown by type with \texttt{heap-stats.}:
\begin{alltt}
\textbf{ok} heap-stats.
\textbf{bignum: 312 bytes, 17 instances
cons: 850376 bytes, 106297 instances
float: 112 bytes, 7 instances
t: 8 bytes, 1 instances
array: 202064 bytes, 3756 instances
hashtable: 54912 bytes, 3432 instances
vector: 5184 bytes, 324 instances
string: 391024 bytes, 7056 instances
sbuf: 64 bytes, 4 instances
port: 112 bytes, 2 instances
word: 96960 bytes, 3030 instances
tuple: 688 bytes, 22 instances}
\end{alltt}
\subsection{The profiler}
Factor provides a statistical sampling profiler for narrowing down memory and processor bottlenecks.
The profiler is only supported on Unix platforms. On FreeBSD 4.x, the Factor runtime must
be compiled without the \texttt{-pthread} switch, since FreeBS 4.x userspace threading makes
use of a signal that conflicts with the signal used for profiling.
The \texttt{allot-profile} word executes a quotation with the memory profiler enabled, then prints a list of all words that allocated memory, along with the bytes allocated. Note that during particularly long executions, or executions where a lot of memory is allocated, these counters may overrun.
The \texttt{call-profile} word executes a quotation with the CPU profiler enabled, then prints a list of all words that were found on the return stack, along with the number of times they were seen there. This gives a rough idea of what words are taking up the majority of execution time.
Normally, the memory and CPU profilers run every millisecond, and increment counters for all words on the return stack. The \texttt{only-top} variable can be switched on, in which case only the counter for the word at the top of the return stack is incremented. This gives a more localized picture of CPU and memory usage.
\subsection{\label{compiler}The compiler}
The compiler can provide a substantial speed boost for words whose stack effect can be inferred. Words without a known stack effect cannot be compiled, and must be run in the interpreter. The compiler generates native code, and so far, x86 and PowerPC backends have been developed.
To compile a single word, call \texttt{compile}:
\begin{alltt}
\textbf{ok}\bs pref-size compile
\textbf{Compiling pref-size}
\end{alltt}
During bootstrap, all words in the library with a known stack effect are compiled. You can
circumvent this, for whatever reason, by passing the \texttt{-no-compile} switch during
bootstrap:
\begin{alltt}
\textbf{bash\$} ./f boot.image.le32 -no-compile
\end{alltt}
The compiler has two limitations you must be aware of. First, if an exception is thrown in compiled code, the return stack will be incomplete, since compiled words do not push themselves there. Second, compiled code cannot be profiled. These limitations will be resolved in a future release.
The compiler consists of multiple stages -- first, a dataflow graph is inferred, then various optimizations are done on this graph, then it is transformed into a linear representation, further optimizations are done, and finally, machine code is generated from the linear representation. To perform everything except for the machine code generation, use the \texttt{precompile} word. This will dump the optimized linear IR instead of generating code, which can be useful sometimes.
