More documentation.

[roundup.git] / doc / customizing.txt

===================
Customising Roundup
===================

:Version: $Revision: 1.20 $

.. This document borrows from the ZopeBook section on ZPT. The original is at:
   http://www.zope.org/Documentation/Books/ZopeBook/current/ZPT.stx

.. contents::


What You Can Do
---------------

Customisation of Roundup can take one of five forms:

1. `instance configuration`_ file changes
2. database, or `instance schema`_ changes
3. "definition" class `database content`_ changes
4. behavioural changes, through detectors_
5. `access controls`_

The third case is special because it takes two distinctly different forms
depending upon whether the instance has been initialised or not. The other two
may be done at any time, before or after instance initialisation. Yes, this
includes adding or removing properties from classes.


Instances in a Nutshell
-----------------------

Instances have the following structure:

+-------------------+--------------------------------------------------------+
|instance_config.py |Holds the basic instance_configuration                  |
+-------------------+--------------------------------------------------------+
|dbinit.py          |Holds the instance_schema                               |
+-------------------+--------------------------------------------------------+
|interfaces.py      |Defines the Web and E-Mail interfaces for the instance  |
+-------------------+--------------------------------------------------------+
|select_db.py       |Selects the database back-end for the instance          |
+-------------------+--------------------------------------------------------+
|db/                |Holds the instance's database                           |
+-------------------+--------------------------------------------------------+
|db/files/          |Holds the instance's upload files and messages          |
+-------------------+--------------------------------------------------------+
|detectors/         |Auditors and reactors for this instance                 |
+-------------------+--------------------------------------------------------+
|html/              |Web interface templates, images and style sheets        |
+-------------------+--------------------------------------------------------+

Instance Configuration
----------------------

The instance_config.py located in your instance home contains the basic
configuration for the web and e-mail components of roundup's interfaces. This
file is a Python module. The configuration variables available are:

**INSTANCE_HOME** - ``os.path.split(__file__)[0]``
 The instance home directory. The above default code will automatically
 determine the instance home for you.

**MAILHOST** - ``'localhost'``
 The SMTP mail host that roundup will use to send e-mail.

**MAIL_DOMAIN** - ``'your.tracker.email.domain.example'``
 The domain name used for email addresses.

**DATABASE** - ``os.path.join(INSTANCE_HOME, 'db')``
 This is the directory that the database is going to be stored in. By default
 it is in the instance home.

**TEMPLATES** - ``os.path.join(INSTANCE_HOME, 'html')``
 This is the directory that the HTML templates reside in. By default they are
 in the instance home.

**INSTANCE_NAME** - ``'Roundup issue tracker'``
 A descriptive name for your roundup instance. This is sent out in e-mails and
 appears in the heading of CGI pages.

**ISSUE_TRACKER_EMAIL** - ``'issue_tracker@%s'%MAIL_DOMAIN``
 The email address that e-mail sent to roundup should go to. Think of it as the
 instance's personal e-mail address.

**ISSUE_TRACKER_WEB** - ``'http://your.tracker.url.example/'``
 The web address that the instance is viewable at. This will be included in
 information sent to users of the tracker.

**ADMIN_EMAIL** - ``'roundup-admin@%s'%MAIL_DOMAIN``
 The email address that roundup will complain to if it runs into trouble.

**FILTER_POSITION** - ``'top'``, ``'bottom'`` or ``'top and bottom'``
 Where to place the web filtering HTML on the index page.

**ANONYMOUS_ACCESS** - ``'deny'`` or ``'allow'``
 Deny or allow anonymous access to the web interface.

**ANONYMOUS_REGISTER** - ``'deny'`` or ``'allow'``
 Deny or allow anonymous users to register through the web interface.

**ANONYMOUS_REGISTER_MAIL** - ``'deny'`` or ``'allow'``
 Deny or allow anonymous users to register through the mail interface.

**MESSAGES_TO_AUTHOR** - ``'yes'`` or``'no'``
 Send nosy messages to the author of the message.

**ADD_AUTHOR_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'``
 Does the author of a message get placed on the nosy list automatically?
 If ``'new'`` is used, then the author will only be added when a message
 creates a new issue. If ``'yes'``, then the author will be added on followups
 too. If ``'no'``, they're never added to the nosy.

**ADD_RECIPIENTS_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'``
 Do the recipients (To:, Cc:) of a message get placed on the nosy list?
 If ``'new'`` is used, then the recipients will only be added when a message
 creates a new issue. If ``'yes'``, then the recipients will be added on
 followups too. If ``'no'``, they're never added to the nosy.

**EMAIL_SIGNATURE_POSITION** - ``'top'``, ``'bottom'`` or ``'none'``
 Where to place the email signature in messages that Roundup generates.

**EMAIL_KEEP_QUOTED_TEXT** - ``'yes'`` or ``'no'``
 Keep email citations. Citations are the part of e-mail which the sender has
 quoted in their reply to previous e-mail.

**EMAIL_LEAVE_BODY_UNCHANGED** - ``'no'``
 Preserve the email body as is. Enabiling this will cause the entire message
 body to be stored, including all citations and signatures. It should be
 either ``'yes'`` or ``'no'``.

**MAIL_DEFAULT_CLASS** - ``'issue'`` or ``''``
 Default class to use in the mailgw if one isn't supplied in email
 subjects. To disable, comment out the variable below or leave it blank.

**HEADER_INDEX_LINKS** - ``['DEFAULT', 'UNASSIGNED', 'USER']``
 Define what index links are available in the header, and what their
 labels are. Each key is used to look up one of the index specifications
 below - so ``'DEFAULT'`` will use ``'DEFAULT_INDEX'``.

 Example ``DEFAULT_INDEX``::

  {
   'LABEL': 'All Issues',
   'CLASS': 'issue',
   'SORT': ['-activity'],
   'GROUP': ['priority'],
   'FILTER': ['status'],
   'COLUMNS': ['id','activity','title','creator','assignedto'],
   'FILTERSPEC': {
     'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
   },
  }

 This defines one of the index links that appears in the
 ``HEADER_INDEX_LINKS`` list.

 **LABEL** - ``'All Issues'``
  The text that appears as the link label.
 **CLASS** - ``'issue'``
  The class to display the index for.
 **SORT** - ``['-activity']``
  Sort by prop name, optionally preceeded with '-' to give descending or
  nothing for ascending sorting.
 **GROUP** - ``['priority']``
  Group by prop name, optionally preceeded with '-' or to sort in descending
  or nothing for ascending order.
 **FILTER** - ``['status']``
  Selects which props should be displayed in the filter section.
  Default is all. 
 **COLUMNS** - ``['id','activity','title','creator','assignedto']``
  Selects the columns that should be displayed. Default is all.
 **FILTERSPEC** - *a dictionary giving the filter specification*
  The ``FILTERSPEC`` gives the filtering arguments. This selects the values
  the node properties given by propname must have.

  Where the ``FILTERSPEC`` value is ``'CURRENT USER'``, it will be replaced
  by the id of the logged-in user. For example::

   'FILTERSPEC': {
     'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
     'assignedto': 'CURRENT USER',
   },

**HEADER_ADD_LINKS** - ``['issue']``
 List the classes that users are able to add nodes to.

**HEADER_SEARCH_LINKS** - ``['issue']``
 List the classes that users can search.

**SEARCH_FILTERS** - ``['ISSUE_FILTER', 'SUPPORT_FILTER']``
 List search filters per class. Like the INDEX entries above, each key is
 used to look up one of the filter specifications below - so ``'ISSUE'``
 will use ``'ISSUE_FILTER'``.

 Example ``ISSUE_FILTER``::

  ISSUE_FILTER = {
    'CLASS': 'issue',
    'FILTER': ['status', 'priority', 'assignedto', 'creator']
  }

  **CLASS** - ``'issue'``
   The class that the search page is for.
  **FILTER** - ``['status', 'priority', 'assignedto', 'creator']``
   Selects which props should be displayed on the filter page. Default is
   all.

The default instance_config.py is given below - as you
can see, the MAIL_DOMAIN must be edited before any interaction with the
instance is attempted.::

    # roundup home is this package's directory
    INSTANCE_HOME=os.path.split(__file__)[0]

    # The SMTP mail host that roundup will use to send mail
    MAILHOST = 'localhost'

    # The domain name used for email addresses.
    MAIL_DOMAIN = 'your.tracker.email.domain.example'

    # the next two are only used for the standalone HTTP server.
    HTTP_HOST = ''
    HTTP_PORT = 9080

    # This is the directory that the database is going to be stored in
    DATABASE = os.path.join(INSTANCE_HOME, 'db')

    # This is the directory that the HTML templates reside in
    TEMPLATES = os.path.join(INSTANCE_HOME, 'html')

    # A descriptive name for your roundup instance
    INSTANCE_NAME = 'Roundup issue tracker'

    # The email address that mail to roundup should go to
    ISSUE_TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN

    # The web address that the instance is viewable at
    ISSUE_TRACKER_WEB = 'http://your.tracker.url.example/'

    # The email address that roundup will complain to if it runs into trouble
    ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN

    # Somewhere for roundup to log stuff internally sent to stdout or stderr
    LOG = os.path.join(INSTANCE_HOME, 'roundup.log')

    # Where to place the web filtering HTML on the index page
    FILTER_POSITION = 'bottom'          # one of 'top', 'bottom', 'top and bottom'

    # Deny or allow anonymous access to the web interface
    ANONYMOUS_ACCESS = 'deny'           # either 'deny' or 'allow'

    # Deny or allow anonymous users to register through the web interface
    ANONYMOUS_REGISTER = 'deny'         # either 'deny' or 'allow'

    # Deny or allow anonymous users to register through the mail interface
    ANONYMOUS_REGISTER_MAIL = 'deny'    # either 'deny' or 'allow'

    # Send nosy messages to the author of the message
    MESSAGES_TO_AUTHOR = 'no'           # either 'yes' or 'no'

    # Does the author of a message get placed on the nosy list automatically?
    # If 'new' is used, then the author will only be added when a message
    # creates a new issue. If 'yes', then the author will be added on followups
    # too. If 'no', they're never added to the nosy.
    ADD_AUTHOR_TO_NOSY = 'new'          # one of 'yes', 'no', 'new'

    # Do the recipients (To:, Cc:) of a message get placed on the nosy list?
    # If 'new' is used, then the recipients will only be added when a message
    # creates a new issue. If 'yes', then the recipients will be added on followups
    # too. If 'no', they're never added to the nosy.
    ADD_RECIPIENTS_TO_NOSY = 'new'      # either 'yes', 'no', 'new'

    # Where to place the email signature
    EMAIL_SIGNATURE_POSITION = 'bottom' # one of 'top', 'bottom', 'none'

    # Keep email citations
    EMAIL_KEEP_QUOTED_TEXT = 'no'       # either 'yes' or 'no'

    # Preserve the email body as is
    EMAIL_LEAVE_BODY_UNCHANGED = 'no'   # either 'yes' or 'no'

    # Default class to use in the mailgw if one isn't supplied in email
    # subjects. To disable, comment out the variable below or leave it blank.
    # Examples:
    MAIL_DEFAULT_CLASS = 'issue'   # use "issue" class by default
    #MAIL_DEFAULT_CLASS = ''        # disable (or just comment the var out)

    # Define what index links are available in the header, and what their
    # labels are. Each key is used to look up one of the index specifications
    # below - so 'DEFAULT' will use 'DEFAULT_INDEX'.
    # Where the FILTERSPEC has 'assignedto' with a value of None, it will be
    # replaced by the id of the logged-in user.
    HEADER_INDEX_LINKS = ['DEFAULT', 'UNASSIGNED', 'USER']

    # list the classes that users are able to add nodes to
    HEADER_ADD_LINKS = ['issue']

    # list the classes that users can search
    HEADER_SEARCH_LINKS = ['issue']

    # list search filters per class
    SEARCH_FILTERS = ['ISSUE_FILTER', 'SUPPORT_FILTER']

    # Now the DEFAULT display specification. TODO: describe format
    DEFAULT_INDEX = {
      'LABEL': 'All Issues',
      'CLASS': 'issue',
      'SORT': ['-activity'],
      'GROUP': ['priority'],
      'FILTER': ['status'],
      'COLUMNS': ['id','activity','title','creator','assignedto'],
      'FILTERSPEC': {
        'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
      },
    }

    # The "unsassigned issues" index
    UNASSIGNED_INDEX = {
      'LABEL': 'Unassigned Issues',
      'CLASS': 'issue',
      'SORT': ['-activity'],
      'GROUP': ['priority'],
      'FILTER': ['status', 'assignedto'],
      'COLUMNS': ['id','activity','title','creator','status'],
      'FILTERSPEC': {
        'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
        'assignedto': ['-1'],
      },
    }

    # The "my issues" index -- note that the user's id will replace the
    # 'CURRENT USER' value of the "assignedto" filterspec
    USER_INDEX = {
      'LABEL': 'My Issues',
      'CLASS': 'issue',
      'SORT': ['-activity'],
      'GROUP': ['priority'],
      'FILTER': ['status', 'assignedto'],
      'COLUMNS': ['id','activity','title','creator','status'],
      'FILTERSPEC': {
        'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
        'assignedto': 'CURRENT USER',
      },
    }

    ISSUE_FILTER = {
      'CLASS': 'issue',
      'FILTER': ['status', 'priority', 'assignedto', 'creator']
    }

    SUPPORT_FILTER = {
      'CLASS': 'issue',
      'FILTER': ['status', 'priority', 'assignedto', 'creator']
    }


Instance Schema
---------------

Note: if you modify the schema, you'll most likely need to edit the
      `web interface`_ HTML template files and `detectors`_ to reflect
      your changes.

An instance schema defines what data is stored in the instance's database. The
two schemas shipped with Roundup turn it into a typical software bug tracker
(the extended schema allowing for support issues as well as bugs). Schemas are
defined using Python code. The "classic" schema looks like this::

    pri = Class(db, "priority", name=String(), order=String())
    pri.setkey("name")
    pri.create(name="critical", order="1")
    pri.create(name="urgent", order="2")
    pri.create(name="bug", order="3")
    pri.create(name="feature", order="4")
    pri.create(name="wish", order="5")

    stat = Class(db, "status", name=String(), order=String())
    stat.setkey("name")
    stat.create(name="unread", order="1")
    stat.create(name="deferred", order="2")
    stat.create(name="chatting", order="3")
    stat.create(name="need-eg", order="4")
    stat.create(name="in-progress", order="5")
    stat.create(name="testing", order="6")
    stat.create(name="done-cbb", order="7")
    stat.create(name="resolved", order="8")

    keyword = Class(db, "keyword", name=String())
    keyword.setkey("name")

    user = Class(db, "user", username=String(), password=String(),
        address=String(), realname=String(), phone=String(),
        organisation=String())
    user.setkey("username")
    user.create(username="admin", password=adminpw,
        address=instance_config.ADMIN_EMAIL)

    msg = FileClass(db, "msg", author=Link("user"), recipients=Multilink
        ("user"), date=Date(), summary=String(), files=Multilink("file"))

    file = FileClass(db, "file", name=String(), type=String())

    issue = IssueClass(db, "issue", assignedto=Link("user"),
        topic=Multilink("keyword"), priority=Link("priority"), status=Link
        ("status"))
    issue.setkey('title')

Classes and Properties - creating a new information store
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the instance above, we've defined 7 classes of information:

  priority
      Defines the possible levels of urgency for issues.

  status
      Defines the possible states of processing the issue may be in.

  keyword
      Initially empty, will hold keywords useful for searching issues.

  user
      Initially holding the "admin" user, will eventually have an entry for all
      users using roundup.

  msg
      Initially empty, will all e-mail messages sent to or generated by
      roundup.

  file
      Initially empty, will all files attached to issues.

  issue
      Initially emtyp, this is where the issue information is stored.

We define the "priority" and "status" classes to allow two things: reduction in
the amount of information stored on the issue and more powerful, accurate
searching of issues by priority and status. By only requiring a link on the
issue (which is stored as a single number) we reduce the chance that someone
mis-types a priority or status - or simply makes a new one up.

Class and Nodes
:::::::::::::::

A Class defines a particular class (or type) of data that will be stored in the
database. A class comprises one or more properties, which given the information
about the class nodes.
The actual data entered into the database, using class.create() are called
nodes. They have a special immutable property called id. We sometimes refer to
this as the nodeid.

Properties
::::::::::

A Class is comprised of one or more properties of the following types:
    * String properties are for storing arbitrary-length strings.
    * Password properties are for storing encoded arbitrary-length strings. The
      default encoding is defined on the roundup.password.Password class.
    * Date properties store date-and-time stamps. Their values are Timestamp
      objects.
    * A Link property refers to a single other node selected from a specified
      class. The class is part of the property; the value is an integer, the id
      of the chosen node.
    * A Multilink property refers to possibly many nodes in a specified class.
      The value is a list of integers.

FileClass
:::::::::

FileClasses save their "content" attribute off in a separate file from the rest
of the database. This reduces the number of large entries in the database,
which generally makes databases more efficient, and also allows us to use
command-line tools to operate on the files. They are stored in the files sub-
directory of the db directory in your instance.

IssueClass
::::::::::

IssueClasses automatically include the "messages", "files", "nosy", and
"superseder" properties.
The messages and files properties list the links to the messages and files
related to the issue. The nosy property is a list of links to users who wish to
be informed of changes to the issue - they get "CC'ed" e-mails when messages
are sent to or generated by the issue. The nosy reactor (in the detectors
directory) handles this action. The superceder link indicates an issue which
has superceded this one.
They also have the dynamically generated "creation", "activity" and "creator"
properties.
The value of the "creation" property is the date when a node was created, and
the value of the "activity" property is the date when any property on the node
was last edited (equivalently, these are the dates on the first and last
records in the node's journal). The "creator" property holds a link to the user
that created the issue.

setkey(property)
::::::::::::::::

Select a String property of the class to be the key property. The key property
muse be unique, and allows references to the nodes in the class by the content
of the key property. That is, we can refer to users by their username, e.g.
let's say that there's an issue in roundup, issue 23. There's also a user,
richard who happens to be user 2. To assign an issue to him, we could do either
of::

     roundup-admin set issue assignedto=2

or::

     roundup-admin set issue assignedto=richard

Note, the same thing can be done in the web and e-mail interfaces.

create(information)
:::::::::::::::::::

Create a node in the database. This is generally used to create nodes in the
"definitional" classes like "priority" and "status".


Examples of adding to your schema
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TODO


Detectors - adding behaviour to your tracker
--------------------------------------------
.. _detectors:

The detectors in your instance fire before (*auditors*) and after (*reactors*)
changes to the contents of your database. They are Python modules that sit in
your instance's ``detectors`` directory. You will have some installed by
default - have a look. You can write new detectors or modify the existing
ones. The existing detectors installed for you are:

**nosyreaction.py**
  This provides the automatic nosy list maintenance and email sending. The nosy
  reactor (``nosyreaction``) fires when new messages are added to issues.
  The nosy auditor (``updatenosy``) fires when issues are changed and figures
  what changes need to be made to the nosy list (like adding new authors etc)
**statusauditor.py**
  This provides the ``chatty`` auditor which changes the issue status from
  ``unread`` or ``closed`` to ``chatting`` if new messages appear. It also
  provides the ``presetunread`` auditor which pre-sets the status to
  ``unread`` on new nodes if the status isn't explicitly defined.

See the detectors section in the `design document`__ for details of the
interface for detectors.

__ design.html

Sample additional detectors that have been found useful will appear in the
``detectors`` directory of the Roundup distribution:

**newissuecopy.py**
  This detector sends an email to a team address whenever a new issue is
  created. The address is hard-coded into the detector, so edit it before you
  use it (look for the text 'team@team.host') or you'll get email errors!


Database Content
----------------

Note: if you modify the content of definitional classes, you'll most likely
       need to edit the instance `detectors`_ to reflect your changes.

Customisation of the special "definitional" classes (eg. status, priority,
resolution, ...) may be done either before or after the instance is
initialised. The actual method of doing so is completely different in each
case though, so be careful to use the right one.

**Changing content before instance initialisation**
    Edit the dbinit module in your instance to alter the nodes created in using
    the create() methods.


**Changing content after instance initialisation**
    Use the roundup-admin interface's create, set and retire methods to add,
    alter or remove nodes from the classes in question.



Web Interface
-------------

The web interface works behind the cgi-bin/roundup.cgi or roundup-server
scripts. In both cases, the scripts determine which instance is being accessed
(the first part of the URL path inside the scope of the CGI handler) and pass
control on to the instance interfaces.Client class which handles the rest of
the access through its main() method. This means that you can do pretty much
anything you want as a web interface to your instance.

Figuring out what is displayed
::::::::::::::::::::::::::::::

Most customisation of the web view can be done by modifying the templates in
the instance **html** directory. There are several types of files in there:

page
  defines the overall look of your tracker. When you
  view an issue, it appears inside this template. When you view an index, it
  also appears inside this template.
home
  the default page displayed when no other page is indicated by the user
home.classlist
  a special version of the default page that lists the classes in the tracker
*classname*.item
  displays an item of the *classname* class
*classname*.index
  displays a list of *classname* items
*classname*.search
  displays a search page for *classname* items
_generic.index
  used to display a list of items where there is no *classname*.index available
user.register
  a special page just for the user class that renders the registration page
style.css
  a static file that is served up as-is

How requests are processed
::::::::::::::::::::::::::

The basic processing of a web request proceeds as follows:

1. figure out who we are, defaulting to the "anonymous" user
2. figure out what the request is for - we call this the "context"
3. handle any requested action (item edit, search, ...)
4. render a template, resulting in HTML output

In some situations, exceptions occur:

- HTTP Redirect  (generally raised by an action)
- SendFile       (generally raised by determine_context)
  here we serve up a FileClass "content" property
- SendStaticFile (generally raised by determine_context)
  here we serve up a file from the tracker "html" directory
- Unauthorised   (generally raised by an action)
  here the action is cancelled, the request is rendered and an error
  message is displayed indicating that permission was not
  granted for the action to take place
- NotFound       (raised wherever it needs to be)
  this exception percolates up to the CGI interface that called the client

Determining web context
:::::::::::::::::::::::

To determine the "context" of a request, we look at the URL and the special
request variable ``:template``. The URL path after the instance identifier
is examined. Typical URL paths look like:

1.  ``/tracker/issue``
2.  ``/tracker/issue1``
3.  ``/tracker/_file/style.css``
4.  ``/cgi-bin/roundup.cgi/tracker/file1``
5.  ``/cgi-bin/roundup.cgi/tracker/file1/kitten.png``

where the "instance identifier" is "tracker" in the above cases. That means
we're looking at "issue", "issue1", "_file/style.css", "file1" and
"file1/kitten.png" in the cases above. The path is generally only one
entry long - longer paths are handled differently.

a. if there is no path, then we are in the "home" context.
b. if the path starts with "_file" (as in example 3,
   "/tracker/_file/style.css"), then the additional path entry,
   "style.css" specifies the filename of a static file we're to serve up
   from the instance "html" directory. Raises a SendStaticFile
   exception.
c. if there is something in the path (as in example 1, "issue"), it identifies
   the tracker class we're to display.
d. if the path is an item designator (as in examples 2 and 4, "issue1" and
   "file1"), then we're to display a specific item.
e. if the path starts with an item designator and is longer than
   one entry (as in example 5, "file1/kitten.png"), then we're assumed
   to be handling an item of a
   FileClass, and the extra path information gives the filename
   that the client is going to label the download with (ie
   "file1/kitten.png" is nicer to download than "file1"). This
   raises a SendFile exception.

Both b. and e. stop before we bother to
determine the template we're going to use. That's because they
don't actually use templates.

The template used is specified by the ``:template`` CGI variable,
which defaults to:

- only classname suplied:          "index"
- full item designator supplied:   "item"


Performing actions in web requests
::::::::::::::::::::::::::::::::::

When a user requests a web page, they may optionally also request for an
action to take place. As described in `how requests are processed`_, the
action is performed before the requested page is generated. Actions are
triggered by using a ``:action`` CGI variable, where the value is one of:

login
 Attempt to log a user in.
logout
 Log the user out - make them "anonymous".
register
 Attempt to create a new user based on the contents of the form and then log
 them in.
edit
 Perform an edit of an item in the database. There are some special form
 elements you may use:

 :link=designator:property and :multilink=designator:property
  The value specifies a node designator and the property on that
  node to add _this_ node to as a link or multilink.
 __note
  Create a message and attach it to the current node's
  "messages" property.
 __file
  Create a file and attach it to the current node's
  "files" property. Attach the file to the message created from
  the __note if it's supplied.
 :required=property,property,...
  The named properties are required to be filled in the form.

new
 Add a new item to the database. You may use the same special form elements
 as in the "edit" action.

editCSV
 Performs an edit of all of a class' items in one go. See also the
 *class*.csv templating method which generates the CSV data to be edited, and
 the "_generic.index" template which uses both of these features.

search
 Mangle some of the form variables.

 Set the form ":filter" variable based on the values of the
 filter variables - if they're set to anything other than
 "dontcare" then add them to :filter.

 Also handle the ":queryname" variable and save off the query to
 the user's query list.

Each of the actions is implemented by a corresponding *actionAction* (where
"action" is the name of the action) method on
the roundup.cgi.Client class, which also happens to be in your instance as
interfaces.Client. So if you need to define new actions, you may add them
there (see `definining new web actions`_).

Each action also has a corresponding *actionPermission* (where
"action" is the name of the action) method which determines
whether the action is permissible given the current user. The base permission
checks are:

login
 Determine whether the user has permission to log in.
 Base behaviour is to check the user has "Web Access".
logout
 No permission checks are made.
register
 Determine whether the user has permission to register
 Base behaviour is to check the user has "Web Registration".
edit
 Determine whether the user has permission to edit this item.
 Base behaviour is to check the user can edit this class. If we're
 editing the "user" class, users are allowed to edit their own
 details. Unless it's the "roles" property, which requires the
 special Permission "Web Roles".
new
 Determine whether the user has permission to create (edit) this item.
 Base behaviour is to check the user can edit this class. No
 additional property checks are made. Additionally, new user items
 may be created if the user has the "Web Registration" Permission.
editCSV
 Determine whether the user has permission to edit this class.
 Base behaviour is to check the user can edit this class.
search
 Determine whether the user has permission to search this class.
 Base behaviour is to check the user can view this class.


Repurcussions of changing the instance schema
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you choose to change the `instance schema`_ you will need to ensure the web
interface knows about it:

1. Index, item and search pages for the relevant classes may need to have
   properties added or removed,
2. The "page" template may require links to be changed, as might the "home"
   page's content arguments.

Overall Look - "page" template
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The "page" template in your instances
roundup.cgi_client.Class. This class is mixed-in to your instance through the
instance's interfaces module. This means you can override the header and
footer with your own code. This allows you to use a sidebar navigation scheme,
for example.


How the templates work
~~~~~~~~~~~~~~~~~~~~~~

Roundup's templates consist of two core technologies:

TAL - Template Attribute Language
  This is the syntax which is woven into the HTML using the ``tal:`` tag
  attributes. A TAL parser pulls out the TAL commands from the attributes
  runs them using some expression engine. TAL gives us the following commands:

  tal:define="variable expression; variable expression; ..."
   Define a new variable that is local to this tag and its contents. For
   example::

      <html tal:define="title request/description">
       <head><title tal:content="title"></title></head>
      </html>

   In the example, the variable "title" is defined as being the result of the
   expression "request/description". The tal:content command inside the <html>
   tag may then use the "title" variable.

  tal:condition="expression"
   Only keep this tag and its contents if the expression is true. For example::

     <p tal:condition="python:request.user.hasPermission('View', 'issue')">
      Display some issue information.
     </p>

   In the example, the <p> tag and its contents are only displayed if the
   user has the View permission for issues. We consider the number zero, a
   blank string, an empty list, and the built-in variable nothing to be false
   values. Nearly every other value is true, including non-zero numbers, and
   strings with anything in them (even spaces!).

  tal:repeat="variable expression"
   Repeat this tag and its contents for each element of the sequence that the
   expression returns, defining a new local variable and a special "repeat"
   variable for each element. For example::

     <tr tal:repeat="u user/list">
      <td tal:content="u/id"></td>
      <td tal:content="u/username"></td>
      <td tal:content="u/realname"></td>
     </tr>

   The example would iterate over the sequence of users returned by
   "user/list" and define the local variable "u" for each entry.

  tal:replace="expression"
   Replace this tag with the result of the expression. For example::

    <span tal:replace="request/user/realname"></span>

   The example would replace the <span> tag and its contents with the user's
   realname. If the user's realname was "Bruce" then the resultant output
   would be "Bruce".

  tal:content="expression"
   Replace the contents of this tag with the result of the expression. For
   example::

    <span tal:content="request/user/realname">user's name appears here</span>

   The example would replace the contents of the <span> tag with the user's
   realname. If the user's realname was "Bruce" then the resultant output
   would be "<span>Bruce</span>".

  tal:attributes="attribute expression; attribute expression; ..."
   Set attributes on this tag to the results of expressions. For example::

     <a tal:attributes="href string:user${request/user/id}">My Details</a>

   In the example, the "href" attribute of the <a> tag is set to the value of
   the "string:user${request/user/id}" expression, which will be something
   like "user123".

  tal:omit-tag="expression"
   Remove this tag (but not its contents) if the expression is true. For
   example::

      <span tal:omit-tag="python:1">Hello, world!</span>

   would result in output of::

      Hello, world!

  Note that the commands on a given tag are evaulated in the order above, so
  *define* comes before *condition*, and so on.

  Additionally, a tag is defined, tal:block, which is removed from output. Its
  content is not, but the tag itself is (so don't go using any tal:attributes
  commands on it). This is useful for making arbitrary blocks of HTML
  conditional or repeatable (very handy for repeating multiple table rows,
  which would othewise require an illegal tag placement to effect the repeat).

TALES - TAL Expression Syntax
  The expression engine used in this case is TALES, which runs the expressions
  that form the tag attribute values. TALES expressions come in three
  flavours:

  Path Expressions - eg. ``item/status/checklist``
   These are object attribute / item accesses. Roughly speaking, the path
   ``item/status/checklist`` is broken into parts ``item``, ``status``
   and ``checklist``. The ``item`` part is the root of the expression.
   We then look for a ``status`` attribute on ``item``, or failing that, a
   ``status`` item (as in ``item['status']``). If that
   fails, the path expression fails. When we get to the end, the object we're
   left with is evaluated to get a string - methods are called, objects are
   stringified. Path expressions may have an optional ``path:`` prefix, though
   they are the default expression type, so it's not necessary.

   XXX | components of expressions
   XXX "nothing" and "default"

  String Expressions - eg. ``string:hello ${user/name}``
   These expressions are simple string interpolations (though they can be just
   plain strings with no interpolation if you want. The expression in the
   ``${ ... }`` is just a path expression as above.

  Python Expressions - eg. ``python: 1+1``
   These expressions give the full power of Python. All the "root level"
   variables are available, so ``python:item.status.checklist()`` would be
   equivalent to ``item/status/checklist``, assuming that ``checklist`` is
   a method.

Information available to templates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following variables are available to templates.

.. taken from roundup.cgi.templating.RoundupPageTemplate docstring

*context*
  The current context. This is either None, a wrapper around a
  hyperdb class (an HTMLClass) or a wrapper around a hyperdb item (an
  HTMLItem).
*request*
  Includes information about the current request, including:
   - the url
   - the current index information (``filterspec``, ``filter`` args,
     ``properties``, etc) parsed out of the form. 
   - methods for easy filterspec link generation
   - *user*, the current user node as an HTMLItem instance
   - *form*
     The current CGI form information as a mapping of form argument
     name to value
*instance*
  The current instance
*db*
  The current database, through which db.config may be reached.

The context variable
::::::::::::::::::

The *context* variable is one of three things based on the current context
(see `determining web context`_ for how we figure this out):

1. if we're looking at a "home" page, then it's None
2. if we're looking at a specific hyperdb class, it's an HTMLClass instance
3. if we're looking at a specific hyperdb item, it's an HTMLItem instance

If the context is not None, we can access the properties of the class or item.
The only real difference between cases 2 and 3 above are:

1. the properties may have a real value behind them, and this will appear if
   the property is displayed through ``context/property`` or
   ``context/property/field``.
2. the context's "id" property will be a false value in the second case, but
   a real, or true value in the third. Thus we can determine whether we're
   looking at a real item from the hyperdb by testing "context/id".


The request variable
::::::::::::::::::::

The request variable is packed with information about the current request.

.. taken from roundup.cgi.templating.HTMLRequest docstring

=========== ================================================================
Variable    Holds
=========== ================================================================
form        the CGI form as a cgi.FieldStorage
env         the CGI environment variables
url         the current URL path for this request
base        the base URL for this instance
user        a HTMLUser instance for this user
classname   the current classname (possibly None)
template    the current template (suffix, also possibly None)
form        the current CGI form variables in a FieldStorage
**Index  page specific variables (indexing arguments)**
columns     dictionary of the columns to display in an index page
show        a convenience access to columns - request/show/colname will
              be true if the columns should be displayed, false otherwise
sort        index sort column (direction, column name)
group       index grouping property (direction, column name)
filter      properties to filter the index on
filterspec  values to filter the index on
search_text text to perform a full-text search on for an index
----------- ----------------------------------------------------------------


Displaying Properties
~~~~~~~~~~~~~~~~~~~~~

Properties appear in the user interface in three contexts: in indices, in
editors, and as search arguments.
For each type of property, there are several display possibilities.
For example, in an index view, a string property may just be
printed as a plain string, but in an editor view, that property may be
displayed in an editable field.


Index Views
~~~~~~~~~~~

Index View Specifiers
:::::::::::::::::::::

An index view specifier (URL fragment) looks like this (whitespace has been
added for clarity)::

     /issue?status=unread,in-progress,resolved&
            topic=security,ui&
            :group=+priority&
            :sort=-activity&
            :filters=status,topic&
            :columns=title,status,fixer

The index view is determined by two parts of the specifier: the layout part and
the filter part. The layout part consists of the query parameters that begin
with colons, and it determines the way that the properties of selected nodes
are displayed. The filter part consists of all the other query parameters, and
it determines the criteria by which nodes are selected for display.
The filter part is interactively manipulated with the form widgets displayed in
the filter section. The layout part is interactively manipulated by clicking on
the column headings in the table.

The filter part selects the union of the sets of items with values matching any
specified Link properties and the intersection of the sets of items with values
matching any specified Multilink properties.

The example specifies an index of "issue" nodes. Only items with a "status" of
either "unread" or "in-progres" or "resolved" are displayed, and only items
with "topic" values including both "security" and "ui" are displayed. The items
are grouped by priority, arranged in ascending order; and within groups, sorted
by activity, arranged in descending order. The filter section shows filters for
the "status" and "topic" properties, and the table includes columns for the
"title", "status", and "fixer" properties.

Filtering of indexes
::::::::::::::::::::

TODO

Searching Views
~~~~~~~~~~~~~~~

TODO

Item Views
~~~~~~~~~~

An item view contains an editor section and a spool section. At the top of an
item view, links to superseding and superseded items are always displayed.

Editor Section
::::::::::::::

The editor section is generated from a template containing <display> tags to
insert the appropriate widgets for editing properties.

Here's an example of a basic editor template.::

     <table>
     <tr>
         <td colspan=2>
             <display call="field('title', size=60)">
         </td>
     </tr>
     <tr>
         <td>
             <display call="field('fixer', size=30)">
         </td>
         <td>
             <display call="menu('status')>
         </td>
     </tr>
     <tr>
         <td>
             <display call="field('nosy', size=30)">
         </td>
         <td>
             <display call="menu('priority')>
         </td>
     </tr>
     <tr>
         <td colspan=2>
             <display call="note()">
         </td>
     </tr>
     </table>

As shown in the example, the editor template can also request the display of a
"note" field, which is a text area for entering a note to go along with a
change.

The <property> tag used in the index may also be used here - it checks to see
if the nominated Multilink property has any entries. This can be used to
eliminate sections of the editor section if the property has no entries::

  <td class="form-text">
    <display call="field('superseder', size=40, showid=1)">
    <display call="classhelp('issue', 'id,title', label='list', width=500)">
    <property name="superseder">
      <br>View: <display call="link('superseder', showid=1)">
    </property>
  </td>

The "View: " part with the links will only display if the superseder property
has values.

When a change is submitted, the system automatically generates a message
describing the changed properties.

If a note is given in the "note" field, the note is appended to the
description. The message is then added to the item's message spool (thus
triggering the standard detector to react by sending out this message to the
nosy list).

The message also displays all of the property values on the item and indicates
which ones have changed. An example of such a message might be this::

     Polly's taken a turn for the worse - this is now really important!
     -----
     title: Polly Parrot is dead
     priority: critical
     status: unread -> in-progress
     fixer: terry
     keywords: parrot,plumage,perch,nailed,dead

Spool Section
:::::::::::::

The spool section lists messages in the item's "messages" property. The index
of messages displays the "date", "author", and "summary" properties on the
message nodes, and selecting a message takes you to its content.

The <property> tag used in the index may also be used here - it checks to see
if the nominated Multilink property has any entries. This can be used to
eliminate sections of the spool section if the property has no entries::

     <property name="files">
      <tr class="strong-header">
       <td><b>Files</b></td>
      </tr>

      <tr>
       <td><display call="list('files')"></td>
      </tr>
     </property>


Access Controls
---------------

A set of Permissions are built in to the security module by default:

- Edit (everything)
- View (everything)

The default interfaces define:

- Web Registration
- Web Access
- Web Roles
- Email Registration
- Email Access

These are hooked into the default Roles:

- Admin (Edit everything, View everything, Web Roles)
- User (Web Access, Email Access)
- Anonymous (Web Registration, Email Registration)

And finally, the "admin" user gets the "Admin" Role, and the "anonymous" user
gets the "Anonymous" assigned when the database is initialised on installation.
The two default schemas then define:

- Edit issue, View issue (both)
- Edit file, View file (both)
- Edit msg, View msg (both)
- Edit support, View support (extended only)

and assign those Permissions to the "User" Role. New users are assigned the
Roles defined in the config file as:

- NEW_WEB_USER_ROLES
- NEW_EMAIL_USER_ROLES

You may alter the configuration variables to change the Role that new web or
email users get, for example to not give them access to the web interface if
they register through email.

You may use the ``roundup-admin`` "``security``" command to display the
current Role and Permission configuration in your instance.

Adding a new Permission
~~~~~~~~~~~~~~~~~~~~~~~

When adding a new Permission, you will need to:

1. add it to your instance's dbinit so it is created
2. enable it for the Roles that should have it (verify with
   "``roundup-admin security``")
3. add it to the relevant HTML interface templates
4. add it to the appropriate xxxPermission methods on in your instance
   interfaces module



-----------------

Back to `Table of Contents`_

.. _`Table of Contents`: index.html