factor/doc/cont-responder-tutorial.txt

Overview
========

The 'cont-responder' library is a continuation based web server
for writing web applications in Factor. Each 'web application' is a
standard Factor httpd responder.

This document outlines how to write simple web applications using
'cont-responder' by showing examples. It does not attempt to go into
the technical details of continuation based web applications or how it
is implemented in Factor. Instead it uses a series of examples that
can be immediately tried at the Factor prompt to get a feel for how
things work.

Getting Started
===============
To get started you will first need to use the 'cont-responder'
vocabulary:

  USE: cont-responder

The responders that you will be writing will require an instance of
the httpd server to be running. It will be run in a background thread
to enable the interactive development of the applications. The
following is a simple function to start the server on port 8888:

  USE: httpd
  USE: threads
  : start-httpd [ 8888 httpd ] in-thread  ;
  start-httpd

Responders
==========
A 'responder' is a word that is registered with the httpd server that
gets run when the client accesses a particular URL. When run that word
has 'standard output' bound in such a way that all output goes to the
clients web browser. 

In the 'cont-responder' system there are two words used to set output
to go to the web browser and display a page. They are 'show' and
'show-final'. Think of them as 'show a page to the client'. 'show' and
'show-final' both take a single item on the stack and that is a 'page
generation' quotation.

A 'page generation' quotation is a quotation which when called will
output HTML to stdout. In the httpd system, stdout is bound to the
socket connection to the clients web browser. 

The 'page generation' quotation passed to 'show' should have stack
effect ( string -- ) while that for 'show-final' has stack effect 
( -- ). The two words have very similar uses. 

The big difference is with 'show'. It provides an URL to the page
generation quotation that when requested will proceed with execution
immediately following the 'show', and any POST request data will be on
the stack. With 'show-final' no URL is passed so it is not possible to
'resume' computation. This is explained more fully later.

Hello World 1
=============
A simple 'hello world' responder would be:

  : hello-world1 ( -- )
    [
      "<html><head><title>Hello World</title></head>" write
      "<body>Hello World!</body></html>" write
    ] show-final ;

When installed this will show a single page which is simple HTML to
display 'Hello World!'. 

The responder is installed using:

  "helloworld1" [ hello-world1 ] install-cont-responder

The 'install-cont-responder' word has stack effect 
( name quot -- ). It installs a responder with the given name. 

When the URL for that responder is accessed the 'quot' quotation is
run. In this case it is 'hello-world1' which displays the single HTML
page described previously.

Accessing the above responder from a web browser is via an URL like:

  http://localhost:8888/responder/helloworld1

This should display an HTML page showing 'Hello World!".

HTML Generation
===============
Generating HTML by writing strings containing HTML can be a bit of a
chore. Especially when the content is dynamic requiring concatenation
of many pieces of data. 

The 'cont-responder' system uses 'html', a library that allows writing
HTML looking output directly in factor. This system, developed for
'cont-responder', has recently been made part of the standard 'html'
library of Factor.

'html' basically allows you to write HTML-like output in a factor word
and it will be output as correct HTML. It can be tested at the console
very easily:

  USE: html
  <p> "This is a paragraph" write </p>
    => <p>This is a paragraph</p>

You can write open and close tags like orginary HTML and anything sent
to standard output in between the tags will be enclosed in the
specified tags. Attributes can also be used:

  <p "text-align: center" =style p> "More text" write </p>
    => <p style='text-align: center'>More text</p>

The attribute must be seperated from the value of that attribute via
whitespace. If you are using attributes the tag must be closed with a
'[tagname]>' where the [tagname] is the name of the tag used. See the
'<p p>' example above.

You can use any factor code at any point:
 
  "text-align: " "red" 
  <p 2dup cat2 =style p> 
    "Using style " write swap write write
  </p>
    => <p style='text-align: red'>Using style text-align: red</p>

Tags that are not normally closed are written using XML style closed
tag (ie. with a trailing slash):

  "One" write <br/> "Two" write <br/> <input "text" =type input/>
    => One<br>Two<br><input type='text'>

Hello World 2
=============

Using the HTML generation library makes writing responders more
readable. Here is the hello world example perviously using this
system:

  : hello-world2 ( -- )
    [
      <html>
        <head> <title> "Hello World" write </title> </head>
        <body> "Hello World!" write </body>
      </html>
    ] show-final ;

Install it using:

  "helloworld2" [ hello-world2 ] install-cont-responder

Dynamic Data
============

Adding dynamic data to the page is relatively easy. This example pulls
information from the 'room' word which returns memory details about
the running Factor system. It also uses 'room.' which outputs these
details to standard output and this is wrapped in a <pre> tag so it is
formatted correctly.

  : memory-stats1 ( -- )
    [
      <html>
        <head> <title> "Memory Statistics" write </title> </head>
        <body>
          <table "1" =border table>
            <tr> 
              <td> "Total Data Memory" write </td>
              <td> room unparse write </td>
            </tr>
            <tr> 
              <td> "Free Data Memory" write </td>
              <td> unparse write </td>
            </tr>
            <tr> 
              <td> "Total Code Memory" write </td>
              <td> unparse write </td>
            </tr>
            <tr> 
              <td> "Free Code Memory" write </td>
              <td> unparse write </td>
            </tr>
	  </table>
	</body>
        <pre> room. </pre>
      </html>
    ] show-final ;
    
  "memorystats1" [ memory-stats1 ] install-cont-responder

Accessing this page will show a table with the current memory
statistics. Hitting refresh will update the page with the latest
information.

The HTML output can be refactored into different words. For example:

  : memory-stats-table ( free total -- )
    #! Output a table containing the given statistics.
    <table "1" =border table>
      <tr> 
        <td> "Total Data Memory" write </td>
        <td> unparse write </td>
      </tr>
      <tr> 
        <td> "Free Data Memory" write </td>
        <td> unparse write </td>
      </tr>
    </table> ;

  : memory-stats2 ( -- )
    [
      <html>
        <head> <title> "Memory Statistics 2" write </title> </head>
        <body> room memory-stats-table 2drop </body>
      </html>
    ] show-final ;

  "memorystats2" [ memory-stats2 ] install-cont-responder

Some simple flow
================
The big advantage with continuation based web servers is being able to
write a web application in a standard procedural flow and have it
correctly served up in the HTTP request/response model.

This example demonstates a flow of three pages. Clicking an URL on the
first page displays the second. Clicking an URL on the second displays
the third.

When a 'show' call is executed the page generated by the quotation is
sent to the client. The computation of the responder is then
'suspended'. When the client accesses a special URL, computation is
resumed at the point of the end of the 'show' call. In this way
procedural flow is maintained.

This brings us to the 'URL' stack item that is available to the 'page
generation' quotation passed to 'show'. This URL is a string that
contains an URL that can be embedded in the page. When the user access
that URL, computation is resumed from the point of the end of the
'show' call as described above:

  : flow-example1 ( -- )
    [
      <html>
        <head> <title> "Flow Example 1" write </title> </head>
        <body>
          <p> "Page 1" write </p>
          <p> <a =href a> "Press to continue" write </a> </p>
        </body>
      </html>   
    ] show drop 
    [
      <html>
        <head> <title> "Flow Example 1" write </title> </head>
        <body>
          <p> "Page 2" write </p>
          <p> <a =href a> "Press to continue" write </a> </p>
        </body>
      </html>   
    ] show drop 
    [
      <html>
        <head> <title> "Flow Example 1" write </title> </head>
        <body>
          <p> "Page 3" write </p>
        </body>
      </html>   
    ] show-final ;

  "flowexample1" [ flow-example1 ] install-cont-responder

The 'flow-example1' word contains two 'show' calls in a row, followed
by a 'show-final'. The 'show' calls display simple pages with an anchor
link to the URL received on the stack. This URL when accessed resumes
the computation. The final page doesn't require resumption of the
computation so 'show-final' is used. We could have used 'show' and
dropped the URL passed to the quotation and the result following the
'show' but using 'show-final' is more efficient.

When you display this example in the browser you'll be able to click
the URL to navigate. You can use the back button to retry the URL's,
you can clone the browser window and navigate them independantly, etc.

The similarity of the functions above shows that some refactoring
would be useful. The pages are almost exactly the same so we seperate
this into a seperate word:

  : show-flow-page ( n bool -- )
  #! Show a page in the flow, using 'n' as the page number
  #! to display. If 'bool' is true display a link to the 
  #! next page.
  [ ( n bool url -- )
    <html>
      <head> <title> "Flow Example 1" write </title> </head>
      <body>
        <p> "Page " write rot unparse write </p>
        swap [
          <p> <a =href a> "Press to continue" write </a> </p>
        ] [
          drop 
        ] ifte
      </body>
    </html>   
  ] show 3drop ;

  : flow-example2 ( n -- )
  #! Display the given number of pages in a row.
  dup 1 - [ dup 1 + t show-flow-page ] repeat
  f show-flow-page ;
 
  "flowexample2" [ 5 flow-example2 ] install-cont-responder

In this example the 'show-flow-page' pulls the page number off the
stack. It also gets whether or not to display the link to the next
page.

Notice that after the show that a '3drop' is done whereas
previously we've only done a single 'drop'. This is due to a side
effect of 'show' using continuations. 

After the 'show' call returns there will be one item on the stack
(which we've been dropping and will explain later what it is). The
stack will also be set as it was before the show call. So in this case
the 'n' and 'bool' remain on the stack even though they were removed
during the page generation quotation. This is because we resumed the
continuation which, when captured, had those items on the stack. The
general rule of thumb is you will need to account for items on the
stack before the show call.

This example also demonstrates using the 'repeat' combinator to
sequence the page shows. Any Factor code can be called and the
continuation based system will sequentially display each page. The
back button, browser window cloning, etc will all continue to work.

You'll notice the URL's in the browser have an 'id' query parameter
with a sequence of characters as its value. This is the 'continuation
identifier' which is like a session id except that it identifies not
just the data you have stored but your location within the responder
as well.

Forms and POST data
===================
The web pages we've generated so far don't accept input from the
user. I've mentioned previously that 'show' returns a value on the
stack and we've been dropping it in our examples. 

The value returned is a namespace containing the field names and
values of any POST data in the request. If no POST data exists then it
is the boolean value 'f'. 

To process input from the user just put a form in the HTML with a
method of 'POST' and an action set to the URL passed in to the page
generation quotation. The show call will then return a namespace
containing this data. Here is a simple example:

  : accept-users-name ( -- name )
    #! Display an HTML requesting the users name. Push
    #! the name the user input on the stack..
    [
      <html>
        <head> <title> "Please enter your name" write </title> </head>
        <body>
          <form =action "post" =method form>
            <p> 
              "Please enter your name:" write
              <input "text" =type "20" =size "username" =name input/>
              <input "submit" =type "Ok" =value input/>
            </p>
          </form>
        </body>
      </html>
    ] show [
      "username" get
    ] bind ;

  : post-example1 ( -- )
    [
      <html>
        <head> <title> "Hello!" write </title> </head>
        <body>
          <p> accept-users-name write ", Good to see you!" write </p>
        </body>
      </html>
    ] show-final  ;
    
  "post-example1" [ post-example1 ] install-cont-responder
    
The 'accept-users-name' word displays an HTML form allowing input of
the name. When that form is submitted the namespace containing the
data is returned by 'show'. We bind to it and retrieve the 'username'
field. The name used here should be the same name used when creating
the field in the HTML.

'post-example1' then does something a bit tricky. Instead of first
calling 'accept-users-name' to push the name on the stack and then
displaying the resulting page we call 'accept-users-name' from within
the page itself when we actually need it. The magic of the
continuation system causes the 'accept-users-name' to be called when
needed displaying that page first. It is certainly possible to do it
the other way though:

  : post-example2 ( -- )
    accept-users-name
    [ ( name url -- )
      <html>
        <head> <title> "Hello!" write </title> </head>
        <body>
          <p> write ", Good to see you!" write </p>
        </body>
      </html>
    ] show-final ;

  "post-example2" [ post-example2 ] install-cont-responder

Associating URL's with words
============================
A web page can contain URL's that when clicked perform some
action. This may be to display other pages, or to affect some server
state. 

The 'cont-responder' system enables an URL to be associated with any
Factor quotation. This quotation will be run when the URL is
clicked. When that quotation exits control is returned to the page
that contained the call. 

The word that enables this is 'quot-href'. It takes two items on the
stack. One is the text to display for the link. The other is the
quotation to run when the link is clicked. This quotation should have
stack effect ( -- ).

This example displays a number which can be incremented or
decremented. 

0 "counter" set

: counter-example1 ( - )
  #! Display a global counter which can be incremented or decremented
  #! using anchors.
  [
    <html>
      <head> 
        <title> "Counter: " write "counter" get unparse dup write  </title>
      </head>
      <body>
        <h2> "Counter: " write write </h2> 
        <p>  "++" [ "counter" get 1 + "counter" set ] quot-href
             "--" [ "counter" get 1 - "counter" set ] quot-href
        </p>
      </body>
    </html>
  ] show-final ;

  "counter-example1" [ counter-example1 ] install-cont-responder

Accessing this example from the web browser will display a count of
zero. Clicking '++' or '--' will increment or decrement the count
respectively. This is done by calling a quotation that either
increments or decrements the count when the URL's are clicked. 

Because the count is 'global' in this example, if you clone the
browser window with the count set to a specific value and increment
it, and then refresh the original browser window you will see the most
recent incremented state. This gives you 'shopping cart' like state
whereby using the back button or cloning windows shows a view of a
single global value that can be modified by all browser
instances. ie. The state is not backtracked when the back button is
used.

You'll notice that when you visit the root URL for the responder that
the count is reset back to zero. This is because when the responder
was installed the value of zero was in the namespace stack. This stack
is copied when the responder is installed resulting in initial
accesses to the URL having the starting value. This gives you 'server
side session data' for free.

Local State
===========
You can also have a counter value with 'local' state. That is, cloning
the browser window will give you a new independant state value that
can be incremented. Going to the original browser window and
refreshing will show the original value which can be incremented or
decremented seperately from that value in the cloned window. With this
type of state, using the back button results in 'backtracking' the
state value.

A way to get 'local' state is to store values on the stack itself
rather than a namespace:

: counter-example2 ( count - )
  [ ( count URL -- )
    <html>
      <head> 
        <title> "Counter: " write dup unparse write  </title>
      </head>
      <body>
        <h2> "Counter: " write dup unparse write </h2> 
        <p>  "++" over [ 1 + counter-example2 ] cons quot-href
             "--" swap [ 1 - counter-example2 ] cons quot-href
        </p>
      </body>
    </html>
  ] show-final ;

  "counter-example2" [ 0 counter-example2 ] install-cont-responder

This example works by taking the value of the counter and consing it
to a code quotation that will increment or decrement it then call the
responder again. So if the counter value is '5' the two 'quot-href'
calls become the equivalent of:

  "++" [ 5 1 + counter-example2 ] cons quot-href
  "--" [ 5 1 - counter-example2 ] cons quot-href

Because it calls itself with the new count value the state is
remembered for that page only. This means that each page has an
independant count value. You can clone or use the back button and all
browser windows have an independant value.

Calling 'Subroutines'
=====================
Being able to call other page display functions from 'quot-href' gives
you subroutine like functionality in your web pages. A simple menu
that displays a sequence of pages and returns back to the main page is
very easy:

  : show-page ( n -- )
  #! Show a page in the flow, using 'n' as the page number
  #! to display. 
  [ ( n url -- )
    <html>
      <head> <title> "Page " write over unparse write </title> </head>
      <body>
        <p> "Page " write swap unparse write </p>
        <p> <a =href a> "Press to continue" write </a> </p>
      </body>
    </html>   
  ] show 2drop ;

  : show-some-pages ( n -- )
  #! Display the given number of pages in a row.
  [ dup 1 + show-page ] repeat ;

  : subroutine-example1 ( -- )
    [
      <html>
        <head> <title> "Subroutine Example 1" write </title> </head>
        <body>
          <p> "Please select:" write 
              <ol>
                <li> "Flow1" [ 1 show-some-pages ] quot-href </li>
                <li> "Flow2" [ 2 show-some-pages ] quot-href </li>
                <li> "Flow3" [ 3 show-some-pages ] quot-href </li>
              </ol>
          </p>
        </body>
      </html>
    ] show-final ;

  "subroutine-example1" [ subroutine-example1 ] install-cont-responder

Each item in the ordered list is an anchor. When pressed they will
call a quotation that displays a certain number of pages in a
row. When that quotation finishes via dropping off the end the main
menu page is displayed again.

Simple Testing
==============
Sometimes it is useful to test the responder words from the console
instead of accessing it via a web browser. This enables you to step
through or quickly check to see if a word is generating HTML
correctly.

Because the responders require some state associated with them to keep
track of continuation id's and other things you can't usually just run
them and expect them to work. The 'show' call for example will fail as
it expects some continuations to in the continuation table for that
responder.

The 'cont-testing.factor' file (in the contrib/cont-responder
directory) contains some simple words that maintains this state for
you in such a way that you can test the words from the console:

  "/contrib/cont-testing/load.factor" run-resource

For this example we'll call the 'subroutine-example1' responder from
above. First we need to put a 'testing state' object on the stack. All
the testing functions expect this on the stack and return it after
they have been called. We then put a quotation on the stack which
calls the code we want to test and call the 'test-cont-function' word:

  <cont-test-state> [ subroutine-example1 ] test-cont-function
   => 
  HTTP/1.1 302 Document Moved
  Location: ?id=8209741119458310
  Content-Length: 0
  Content-Type: text/plain

The first request is often a 'Document Moved' as above. This is
because by default the 'cont-responder' system does the
'Post-Refresh-Get' pattern which results in a redirect after each
request. This can be disabled but we'll work through the example with
it enabled.

We can see the continuation id where we are 'moved' to in the
'Location' header. To access this we use the 'test-cont-click'
function. Think of this as manually clicking the
URL. 'test-cont-click' has stack effect 
( state url post -- state). 'post' is a hashtable of post data to pass 
along with the request. We use 'f' here because we have no post
data. Remember that our previous 'test-cont-function' call left the
state on the stack:

  "8209741119458310" f test-cont-click
   =>
  HTTP/1.0 200 Document follows
  Content-Type: text/html
  <html><head><title>Subroutine Example 1</title></head>
        <body><p>Please select:
                 <ol><li><a href='?id=7687398605200513'>Flow1</a></li>
                     <li><a href='?id=7856272029924613'>Flow2</a></li>
                     <li><a href='?id=4909116160485714'>Flow3</a></li>
                 </ol>
              </p>
        </body>
   </html>

We can continue to drill down using 'test-cont-click' using the URL's
above to see the HTML for each 'click'.

Here's an example using post data. We'll test the 'post-example1' word
written previously:

  <cont-test-state> [ post-example1 ] test-cont-function
    =>
  HTTP/1.1 302 Document Moved
  Location: ?id=5829759941409535
  Content-Length: 0
  Content-Type: text/plain

Again we skip past the forward:

  "5829759941409535" f test-cont-click
    =>
  HTTP/1.0 200 Document follows
  Content-Type: text/html

  <html><head><title>Please enter your name</title></head>
        <body>
          <form action='?id=5456539333180428' method='post'>
            <p>Please enter your name:
               <input type='text'size='20'name='username'>
               <input type='submit'value='Ok'>
            </p>
          </form> 
        </body>
  </html>

Now we submit the post data along to the 'action' url:

  "5456539333180428" [ [[ "username" "Chris" ]] ] alist>hash test-cont-click
    =>
  HTTP/1.0 200 Document follows
  Content-Type: text/html

  <html>
    <head><title>Hello!</title></head>
    <body>
      <p>Chris, Good to see you!</p>
    </body>
  </html>

As you can see the post data was sent correctly.