![[tokkee]](http://tokkee.org/images/avatar.png){kind=avatar}
diff --git a/doc/customizing.txt b/doc/customizing.txt
index 7957eec2b44b80b407c8c6d889b1fc5236d7680d..0136601910ec8a56f34d249378bd9c36dd0a9db6 100644 (file)
--- a/doc/customizing.txt
+++ b/doc/customizing.txt
Customising Roundup
===================
-:Version: $Revision: 1.16 $
+:Version: $Revision: 1.105 $
-.. contents::
+.. This document borrows from the ZopeBook section on ZPT. The original is at:
+ http://www.zope.org/Documentation/Books/ZopeBook/current/ZPT.stx
+.. contents::
+ :depth: 1
What You Can Do
----------------
+===============
+
+Before you get too far, it's probably worth having a quick read of the Roundup
+`design documentation`_.
-Customisation of Roundup can take one of five forms:
+Customisation of Roundup can take one of six forms:
-1. `instance configuration`_ file changes
-2. database, or `instance schema`_ changes
+1. `tracker configuration`_ file changes
+2. database, or `tracker schema`_ changes
3. "definition" class `database content`_ changes
4. behavioural changes, through detectors_
5. `access controls`_
+6. change the `web interface`_
The third case is special because it takes two distinctly different forms
-depending upon whether the instance has been initialised or not. The other two
-may be done at any time, before or after instance initialisation. Yes, this
+depending upon whether the tracker has been initialised or not. The other two
+may be done at any time, before or after tracker initialisation. Yes, this
includes adding or removing properties from classes.
-Instances in a Nutshell
------------------------
+Trackers in a Nutshell
+======================
-Instances have the following structure:
-
-+-------------------+--------------------------------------------------------+
-|instance_config.py |Holds the basic instance_configuration |
-+-------------------+--------------------------------------------------------+
-|dbinit.py |Holds the instance_schema |
-+-------------------+--------------------------------------------------------+
-|interfaces.py |Defines the Web and E-Mail interfaces for the instance |
-+-------------------+--------------------------------------------------------+
-|select_db.py |Selects the database back-end for the instance |
-+-------------------+--------------------------------------------------------+
-|db/ |Holds the instance's database |
-+-------------------+--------------------------------------------------------+
-|db/files/ |Holds the instance's upload files and messages |
-+-------------------+--------------------------------------------------------+
-|detectors/ |Auditors and reactors for this instance |
-+-------------------+--------------------------------------------------------+
-|html/ |Web interface templates, images and style sheets |
-+-------------------+--------------------------------------------------------+
-
-Instance Configuration
-----------------------
+Trackers have the following structure:
+
+=================== ========================================================
+Tracker File Description
+=================== ========================================================
+config.py Holds the basic `tracker configuration`_
+dbinit.py Holds the `tracker schema`_
+interfaces.py Defines the Web and E-Mail interfaces for the tracker
+select_db.py Selects the database back-end for the tracker
+db/ Holds the tracker's database
+db/files/ Holds the tracker's upload files and messages
+detectors/ Auditors and reactors for this tracker
+html/ Web interface templates, images and style sheets
+=================== ========================================================
+
+Tracker Configuration
+=====================
+
+The ``config.py`` located in your tracker home contains the basic
+configuration for the web and e-mail components of roundup's interfaces.
+As the name suggests, this file is a Python module. This means that any
+valid python expression may be used in the file. Mostly though, you'll
+be setting the configuration variables to string values. Python string
+values must be quoted with either single or double quotes::
+
+ 'this is a string'
+ "this is also a string - use it when the value has 'single quotes'"
+ this is not a string - it's not quoted
+
+Python strings may use formatting that's almost identical to C string
+formatting. The ``%`` operator is used to perform the formatting, like
+so::
+
+ 'roundup-admin@%s'%MAIL_DOMAIN
+
+this will create a string ``'roundup-admin@tracker.domain.example'`` if
+MAIL_DOMAIN is set to ``'tracker.domain.example'``.
+
+You'll also note some values are set to::
-The instance_config.py located in your instance home contains the basic
-configuration for the web and e-mail components of roundup's interfaces. This
-file is a Python module. The configuration variables available are:
+ os.path.join(TRACKER_HOME, 'db')
-**INSTANCE_HOME** - ``os.path.split(__file__)[0]``
- The instance home directory. The above default code will automatically
- determine the instance home for you.
+or similar. This creates a new string which holds the path to the
+``'db'`` directory in the TRACKER_HOME directory. This is just a
+convenience so if the TRACKER_HOME changes you don't have to edit
+multiple valoues.
+
+The configuration variables available are:
+
+**TRACKER_HOME** - ``os.path.split(__file__)[0]``
+ The tracker home directory. The above default code will automatically
+ determine the tracker home for you, so you can just leave it alone.
**MAILHOST** - ``'localhost'``
The SMTP mail host that roundup will use to send e-mail.
-**MAIL_DOMAIN** - ``'your.tracker.email.domain.example'``
+**MAILUSER** - ``()``
+ If your SMTP mail host requires a username and password for access, then
+ specify them here. eg. ``MAILUSER = ('username', 'password')``
+
+**MAILHOST_TLS** - ``'no'``
+ If your SMTP mail host provides or requires TLS (Transport Layer
+ Security) then set ``MAILHOST_TLS = 'yes'``
+
+**MAILHOST_TLS_KEYFILE** - ``''``
+ If you're using TLS, you may also set MAILHOST_TLS_KEYFILE to the name of
+ a PEM formatted file that contains your private key.
+
+**MAILHOST_TLS_CERTFILE** - ``''``
+ If you're using TLS and have specified a MAILHOST_TLS_KEYFILE, you may
+ also set MAILHOST_TLS_CERTFILE to the name of a PEM formatted certificate
+ chain file.
+
+**MAIL_DOMAIN** - ``'tracker.domain.example'``
The domain name used for email addresses.
-**DATABASE** - ``os.path.join(INSTANCE_HOME, 'db')``
+**DATABASE** - ``os.path.join(TRACKER_HOME, 'db')``
This is the directory that the database is going to be stored in. By default
- it is in the instance home.
+ it is in the tracker home.
-**TEMPLATES** - ``os.path.join(INSTANCE_HOME, 'html')``
+**TEMPLATES** - ``os.path.join(TRACKER_HOME, 'html')``
This is the directory that the HTML templates reside in. By default they are
- in the instance home.
+ in the tracker home.
-**INSTANCE_NAME** - ``'Roundup issue tracker'``
- A descriptive name for your roundup instance. This is sent out in e-mails and
+**TRACKER_NAME** - ``'Roundup issue tracker'``
+ A descriptive name for your roundup tracker. This is sent out in e-mails and
appears in the heading of CGI pages.
-**ISSUE_TRACKER_EMAIL** - ``'issue_tracker@%s'%MAIL_DOMAIN``
+**TRACKER_EMAIL** - ``'issue_tracker@%s'%MAIL_DOMAIN``
The email address that e-mail sent to roundup should go to. Think of it as the
- instance's personal e-mail address.
+ tracker's personal e-mail address.
-**ISSUE_TRACKER_WEB** - ``'http://your.tracker.url.example/'``
- The web address that the instance is viewable at. This will be included in
- information sent to users of the tracker.
+**TRACKER_WEB** - ``'http://tracker.example/cgi-bin/roundup.cgi/bugs/'``
+ The web address that the tracker is viewable at. This will be included in
+ information sent to users of the tracker. The URL **must** include the
+ cgi-bin part or anything else that is required to get to the home page of
+ the tracker. You **must** include a trailing '/' in the URL.
**ADMIN_EMAIL** - ``'roundup-admin@%s'%MAIL_DOMAIN``
The email address that roundup will complain to if it runs into trouble.
-**FILTER_POSITION** - ``'top'``, ``'bottom'`` or ``'top and bottom'``
- Where to place the web filtering HTML on the index page.
+**EMAIL_FROM_TAG** - ``''``
+ Additional text to include in the "name" part of the ``From:`` address used
+ in nosy messages. If the sending user is "Foo Bar", the ``From:`` line is
+ usually::
-**ANONYMOUS_ACCESS** - ``'deny'`` or ``'allow'``
- Deny or allow anonymous access to the web interface.
+ "Foo Bar" <issue_tracker@tracker.example>
-**ANONYMOUS_REGISTER** - ``'deny'`` or ``'allow'``
- Deny or allow anonymous users to register through the web interface.
+ The EMAIL_FROM_TAG goes inside the "Foo Bar" quotes like so::
-**ANONYMOUS_REGISTER_MAIL** - ``'deny'`` or ``'allow'``
- Deny or allow anonymous users to register through the mail interface.
+ "Foo Bar EMAIL_FROM_TAG" <issue_tracker@tracker.example>
-**MESSAGES_TO_AUTHOR** - ``'yes'`` or``'no'``
- Send nosy messages to the author of the message.
+**MESSAGES_TO_AUTHOR** - ``'new'``, ``'yes'`` or``'no'``
+ Send nosy messages to the author of the message?
+ If 'new' is used, then the author will only be sent the message when the
+ message creates a new issue. If 'yes' then the author will always be sent
+ a copy of the message they wrote.
**ADD_AUTHOR_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'``
Does the author of a message get placed on the nosy list automatically?
Default class to use in the mailgw if one isn't supplied in email
subjects. To disable, comment out the variable below or leave it blank.
-**HEADER_INDEX_LINKS** - ``['DEFAULT', 'UNASSIGNED', 'USER']``
- Define what index links are available in the header, and what their
- labels are. Each key is used to look up one of the index specifications
- below - so ``'DEFAULT'`` will use ``'DEFAULT_INDEX'``.
-
- Example ``DEFAULT_INDEX``::
-
- {
- 'LABEL': 'All Issues',
- 'CLASS': 'issue',
- 'SORT': ['-activity'],
- 'GROUP': ['priority'],
- 'FILTER': ['status'],
- 'COLUMNS': ['id','activity','title','creator','assignedto'],
- 'FILTERSPEC': {
- 'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
- },
- }
-
- This defines one of the index links that appears in the
- ``HEADER_INDEX_LINKS`` list.
-
- **LABEL** - ``'All Issues'``
- The text that appears as the link label.
- **CLASS** - ``'issue'``
- The class to display the index for.
- **SORT** - ``['-activity']``
- Sort by prop name, optionally preceeded with '-' to give descending or
- nothing for ascending sorting.
- **GROUP** - ``['priority']``
- Group by prop name, optionally preceeded with '-' or to sort in descending
- or nothing for ascending order.
- **FILTER** - ``['status']``
- Selects which props should be displayed in the filter section.
- Default is all.
- **COLUMNS** - ``['id','activity','title','creator','assignedto']``
- Selects the columns that should be displayed. Default is all.
- **FILTERSPEC** - *a dictionary giving the filter specification*
- The ``FILTERSPEC`` gives the filtering arguments. This selects the values
- the node properties given by propname must have.
-
- Where the ``FILTERSPEC`` value is ``'CURRENT USER'``, it will be replaced
- by the id of the logged-in user. For example::
-
- 'FILTERSPEC': {
- 'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
- 'assignedto': 'CURRENT USER',
- },
-
-**HEADER_ADD_LINKS** - ``['issue']``
- List the classes that users are able to add nodes to.
-
-**HEADER_SEARCH_LINKS** - ``['issue']``
- List the classes that users can search.
-
-**SEARCH_FILTERS** - ``['ISSUE_FILTER', 'SUPPORT_FILTER']``
- List search filters per class. Like the INDEX entries above, each key is
- used to look up one of the filter specifications below - so ``'ISSUE'``
- will use ``'ISSUE_FILTER'``.
-
- Example ``ISSUE_FILTER``::
-
- ISSUE_FILTER = {
- 'CLASS': 'issue',
- 'FILTER': ['status', 'priority', 'assignedto', 'creator']
- }
-
- **CLASS** - ``'issue'``
- The class that the search page is for.
- **FILTER** - ``['status', 'priority', 'assignedto', 'creator']``
- Selects which props should be displayed on the filter page. Default is
- all.
-
-The default instance_config.py is given below - as you
+The default config.py is given below - as you
can see, the MAIL_DOMAIN must be edited before any interaction with the
-instance is attempted.::
+tracker is attempted.::
# roundup home is this package's directory
- INSTANCE_HOME=os.path.split(__file__)[0]
+ TRACKER_HOME=os.path.split(__file__)[0]
# The SMTP mail host that roundup will use to send mail
MAILHOST = 'localhost'
# The domain name used for email addresses.
MAIL_DOMAIN = 'your.tracker.email.domain.example'
- # the next two are only used for the standalone HTTP server.
- HTTP_HOST = ''
- HTTP_PORT = 9080
-
# This is the directory that the database is going to be stored in
- DATABASE = os.path.join(INSTANCE_HOME, 'db')
+ DATABASE = os.path.join(TRACKER_HOME, 'db')
# This is the directory that the HTML templates reside in
- TEMPLATES = os.path.join(INSTANCE_HOME, 'html')
+ TEMPLATES = os.path.join(TRACKER_HOME, 'html')
- # A descriptive name for your roundup instance
- INSTANCE_NAME = 'Roundup issue tracker'
+ # A descriptive name for your roundup tracker
+ TRACKER_NAME = 'Roundup issue tracker'
# The email address that mail to roundup should go to
- ISSUE_TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN
+ TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN
- # The web address that the instance is viewable at
- ISSUE_TRACKER_WEB = 'http://your.tracker.url.example/'
+ # The web address that the tracker is viewable at. This will be
+ # included in information sent to users of the tracker. The URL MUST
+ # include the cgi-bin part or anything else that is required to get
+ # to the home page of the tracker. You MUST include a trailing '/'
+ # in the URL.
+ TRACKER_WEB = 'http://tracker.example/cgi-bin/roundup.cgi/bugs/'
- # The email address that roundup will complain to if it runs into trouble
+ # The email address that roundup will complain to if it runs into
+ # trouble
ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN
- # Somewhere for roundup to log stuff internally sent to stdout or stderr
- LOG = os.path.join(INSTANCE_HOME, 'roundup.log')
-
- # Where to place the web filtering HTML on the index page
- FILTER_POSITION = 'bottom' # one of 'top', 'bottom', 'top and bottom'
-
- # Deny or allow anonymous access to the web interface
- ANONYMOUS_ACCESS = 'deny' # either 'deny' or 'allow'
-
- # Deny or allow anonymous users to register through the web interface
- ANONYMOUS_REGISTER = 'deny' # either 'deny' or 'allow'
-
- # Deny or allow anonymous users to register through the mail interface
- ANONYMOUS_REGISTER_MAIL = 'deny' # either 'deny' or 'allow'
+ # Additional text to include in the "name" part of the From: address
+ # used in nosy messages. If the sending user is "Foo Bar", the From:
+ # line is usually: "Foo Bar" <issue_tracker@tracker.example>
+ # the EMAIL_FROM_TAG goes inside the "Foo Bar" quotes like so:
+ # "Foo Bar EMAIL_FROM_TAG" <issue_tracker@tracker.example>
+ EMAIL_FROM_TAG = ""
# Send nosy messages to the author of the message
MESSAGES_TO_AUTHOR = 'no' # either 'yes' or 'no'
- # Does the author of a message get placed on the nosy list automatically?
- # If 'new' is used, then the author will only be added when a message
- # creates a new issue. If 'yes', then the author will be added on followups
- # too. If 'no', they're never added to the nosy.
+ # Does the author of a message get placed on the nosy list
+ # automatically? If 'new' is used, then the author will only be
+ # added when a message creates a new issue. If 'yes', then the
+ # author will be added on followups too. If 'no', they're never
+ # added to the nosy.
ADD_AUTHOR_TO_NOSY = 'new' # one of 'yes', 'no', 'new'
- # Do the recipients (To:, Cc:) of a message get placed on the nosy list?
- # If 'new' is used, then the recipients will only be added when a message
- # creates a new issue. If 'yes', then the recipients will be added on followups
- # too. If 'no', they're never added to the nosy.
+ # Do the recipients (To:, Cc:) of a message get placed on the nosy
+ # list? If 'new' is used, then the recipients will only be added
+ # when a message creates a new issue. If 'yes', then the recipients
+ # will be added on followups too. If 'no', they're never added to
+ # the nosy.
ADD_RECIPIENTS_TO_NOSY = 'new' # either 'yes', 'no', 'new'
# Where to place the email signature
EMAIL_LEAVE_BODY_UNCHANGED = 'no' # either 'yes' or 'no'
# Default class to use in the mailgw if one isn't supplied in email
- # subjects. To disable, comment out the variable below or leave it blank.
- # Examples:
+ # subjects. To disable, comment out the variable below or leave it
+ # blank. Examples:
MAIL_DEFAULT_CLASS = 'issue' # use "issue" class by default
#MAIL_DEFAULT_CLASS = '' # disable (or just comment the var out)
- # Define what index links are available in the header, and what their
- # labels are. Each key is used to look up one of the index specifications
- # below - so 'DEFAULT' will use 'DEFAULT_INDEX'.
- # Where the FILTERSPEC has 'assignedto' with a value of None, it will be
- # replaced by the id of the logged-in user.
- HEADER_INDEX_LINKS = ['DEFAULT', 'UNASSIGNED', 'USER']
-
- # list the classes that users are able to add nodes to
- HEADER_ADD_LINKS = ['issue']
-
- # list the classes that users can search
- HEADER_SEARCH_LINKS = ['issue']
-
- # list search filters per class
- SEARCH_FILTERS = ['ISSUE_FILTER', 'SUPPORT_FILTER']
-
- # Now the DEFAULT display specification. TODO: describe format
- DEFAULT_INDEX = {
- 'LABEL': 'All Issues',
- 'CLASS': 'issue',
- 'SORT': ['-activity'],
- 'GROUP': ['priority'],
- 'FILTER': ['status'],
- 'COLUMNS': ['id','activity','title','creator','assignedto'],
- 'FILTERSPEC': {
- 'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
- },
- }
-
- # The "unsassigned issues" index
- UNASSIGNED_INDEX = {
- 'LABEL': 'Unassigned Issues',
- 'CLASS': 'issue',
- 'SORT': ['-activity'],
- 'GROUP': ['priority'],
- 'FILTER': ['status', 'assignedto'],
- 'COLUMNS': ['id','activity','title','creator','status'],
- 'FILTERSPEC': {
- 'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
- 'assignedto': ['-1'],
- },
- }
-
- # The "my issues" index -- note that the user's id will replace the
- # 'CURRENT USER' value of the "assignedto" filterspec
- USER_INDEX = {
- 'LABEL': 'My Issues',
- 'CLASS': 'issue',
- 'SORT': ['-activity'],
- 'GROUP': ['priority'],
- 'FILTER': ['status', 'assignedto'],
- 'COLUMNS': ['id','activity','title','creator','status'],
- 'FILTERSPEC': {
- 'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
- 'assignedto': 'CURRENT USER',
- },
- }
-
- ISSUE_FILTER = {
- 'CLASS': 'issue',
- 'FILTER': ['status', 'priority', 'assignedto', 'creator']
- }
-
- SUPPORT_FILTER = {
- 'CLASS': 'issue',
- 'FILTER': ['status', 'priority', 'assignedto', 'creator']
- }
+ #
+ # SECURITY DEFINITIONS
+ #
+ # define the Roles that a user gets when they register with the
+ # tracker these are a comma-separated string of role names (e.g.
+ # 'Admin,User')
+ NEW_WEB_USER_ROLES = 'User'
+ NEW_EMAIL_USER_ROLES = 'User'
-
-Instance Schema
----------------
+Tracker Schema
+==============
Note: if you modify the schema, you'll most likely need to edit the
`web interface`_ HTML template files and `detectors`_ to reflect
your changes.
-An instance schema defines what data is stored in the instance's database. The
-two schemas shipped with Roundup turn it into a typical software bug tracker
-(the extended schema allowing for support issues as well as bugs). Schemas are
-defined using Python code. The "classic" schema looks like this::
+A tracker schema defines what data is stored in the tracker's database.
+Schemas are defined using Python code in the ``dbinit.py`` module of your
+tracker. The "classic" schema looks like this (see below for the meaning
+of ``'setkey'``)::
pri = Class(db, "priority", name=String(), order=String())
pri.setkey("name")
- pri.create(name="critical", order="1")
- pri.create(name="urgent", order="2")
- pri.create(name="bug", order="3")
- pri.create(name="feature", order="4")
- pri.create(name="wish", order="5")
stat = Class(db, "status", name=String(), order=String())
stat.setkey("name")
- stat.create(name="unread", order="1")
- stat.create(name="deferred", order="2")
- stat.create(name="chatting", order="3")
- stat.create(name="need-eg", order="4")
- stat.create(name="in-progress", order="5")
- stat.create(name="testing", order="6")
- stat.create(name="done-cbb", order="7")
- stat.create(name="resolved", order="8")
keyword = Class(db, "keyword", name=String())
keyword.setkey("name")
- user = Class(db, "user", username=String(), password=String(),
- address=String(), realname=String(), phone=String(),
- organisation=String())
+ user = Class(db, "user", username=String(), organisation=String(),
+ password=String(), address=String(), realname=String(),
+ phone=String())
user.setkey("username")
- user.create(username="admin", password=adminpw,
- address=instance_config.ADMIN_EMAIL)
- msg = FileClass(db, "msg", author=Link("user"), recipients=Multilink
- ("user"), date=Date(), summary=String(), files=Multilink("file"))
+ msg = FileClass(db, "msg", author=Link("user"), summary=String(),
+ date=Date(), recipients=Multilink("user"),
+ files=Multilink("file"))
file = FileClass(db, "file", name=String(), type=String())
- issue = IssueClass(db, "issue", assignedto=Link("user"),
- topic=Multilink("keyword"), priority=Link("priority"), status=Link
- ("status"))
+ issue = IssueClass(db, "issue", topic=Multilink("keyword"),
+ status=Link("status"), assignedto=Link("user"),
+ priority=Link("priority"))
issue.setkey('title')
+
+What you can't do to the schema
+-------------------------------
+
+You must never:
+
+**Remove the users class**
+ This class is the only *required* class in Roundup. Similarly, its
+ username, password and address properties must never be removed.
+
+**Change the type of a property**
+ Property types must *never* be changed - the database simply doesn't take
+ this kind of action into account. Note that you can't just remove a
+ property and re-add it as a new type either. If you wanted to make the
+ assignedto property a Multilink, you'd need to create a new property
+ assignedto_list and remove the old assignedto property.
+
+
+What you can do to the schema
+-----------------------------
+
+Your schema may be changed at any time before or after the tracker has been
+initialised (or used). You may:
+
+**Add new properties to classes, or add whole new classes**
+ This is painless and easy to do - there are generally no repurcussions
+ from adding new information to a tracker's schema.
+
+**Remove properties**
+ Removing properties is a little more tricky - you need to make sure that
+ the property is no longer used in the `web interface`_ *or* by the
+ detectors_.
+
+
+
Classes and Properties - creating a new information store
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+---------------------------------------------------------
-In the instance above, we've defined 7 classes of information:
+In the tracker above, we've defined 7 classes of information:
priority
Defines the possible levels of urgency for issues.
Initially empty, will hold keywords useful for searching issues.
user
- Initially holding the "admin" user, will eventually have an entry for all
- users using roundup.
+ Initially holding the "admin" user, will eventually have an entry
+ for all users using roundup.
msg
- Initially empty, will all e-mail messages sent to or generated by
- roundup.
+ Initially empty, will hold all e-mail messages sent to or
+ generated by roundup.
file
- Initially empty, will all files attached to issues.
+ Initially empty, will hold all files attached to issues.
issue
- Initially emtyp, this is where the issue information is stored.
+ Initially empty, this is where the issue information is stored.
-We define the "priority" and "status" classes to allow two things: reduction in
-the amount of information stored on the issue and more powerful, accurate
-searching of issues by priority and status. By only requiring a link on the
-issue (which is stored as a single number) we reduce the chance that someone
-mis-types a priority or status - or simply makes a new one up.
+We define the "priority" and "status" classes to allow two things:
+reduction in the amount of information stored on the issue and more
+powerful, accurate searching of issues by priority and status. By only
+requiring a link on the issue (which is stored as a single number) we
+reduce the chance that someone mis-types a priority or status - or
+simply makes a new one up.
-Class and Nodes
-:::::::::::::::
-A Class defines a particular class (or type) of data that will be stored in the
-database. A class comprises one or more properties, which given the information
-about the class nodes.
-The actual data entered into the database, using class.create() are called
-nodes. They have a special immutable property called id. We sometimes refer to
-this as the nodeid.
+Class and Items
+~~~~~~~~~~~~~~~
+
+A Class defines a particular class (or type) of data that will be stored
+in the database. A class comprises one or more properties, which gives
+the information about the class items.
+
+The actual data entered into the database, using ``class.create()``, are
+called items. They have a special immutable property called ``'id'``. We
+sometimes refer to this as the *itemid*.
+
Properties
-::::::::::
+~~~~~~~~~~
A Class is comprised of one or more properties of the following types:
- * String properties are for storing arbitrary-length strings.
- * Password properties are for storing encoded arbitrary-length strings. The
- default encoding is defined on the roundup.password.Password class.
- * Date properties store date-and-time stamps. Their values are Timestamp
- objects.
- * A Link property refers to a single other node selected from a specified
- class. The class is part of the property; the value is an integer, the id
- of the chosen node.
- * A Multilink property refers to possibly many nodes in a specified class.
- The value is a list of integers.
+
+* String properties are for storing arbitrary-length strings.
+* Password properties are for storing encoded arbitrary-length strings.
+ The default encoding is defined on the ``roundup.password.Password``
+ class.
+* Date properties store date-and-time stamps. Their values are Timestamp
+ objects.
+* Number properties store numeric values.
+* Boolean properties store on/off, yes/no, true/false values.
+* A Link property refers to a single other item selected from a
+ specified class. The class is part of the property; the value is an
+ integer, the id of the chosen item.
+* A Multilink property refers to possibly many items in a specified
+ class. The value is a list of integers.
+
FileClass
-:::::::::
+~~~~~~~~~
+
+FileClasses save their "content" attribute off in a separate file from
+the rest of the database. This reduces the number of large entries in
+the database, which generally makes databases more efficient, and also
+allows us to use command-line tools to operate on the files. They are
+stored in the files sub-directory of the ``'db'`` directory in your
+tracker.
-FileClasses save their "content" attribute off in a separate file from the rest
-of the database. This reduces the number of large entries in the database,
-which generally makes databases more efficient, and also allows us to use
-command-line tools to operate on the files. They are stored in the files sub-
-directory of the db directory in your instance.
IssueClass
-::::::::::
+~~~~~~~~~~
IssueClasses automatically include the "messages", "files", "nosy", and
"superseder" properties.
-The messages and files properties list the links to the messages and files
-related to the issue. The nosy property is a list of links to users who wish to
-be informed of changes to the issue - they get "CC'ed" e-mails when messages
-are sent to or generated by the issue. The nosy reactor (in the detectors
-directory) handles this action. The superceder link indicates an issue which
-has superceded this one.
-They also have the dynamically generated "creation", "activity" and "creator"
-properties.
-The value of the "creation" property is the date when a node was created, and
-the value of the "activity" property is the date when any property on the node
-was last edited (equivalently, these are the dates on the first and last
-records in the node's journal). The "creator" property holds a link to the user
-that created the issue.
+
+The messages and files properties list the links to the messages and
+files related to the issue. The nosy property is a list of links to
+users who wish to be informed of changes to the issue - they get "CC'ed"
+e-mails when messages are sent to or generated by the issue. The nosy
+reactor (in the ``'detectors'`` directory) handles this action. The
+superseder link indicates an issue which has superseded this one.
+
+They also have the dynamically generated "creation", "activity" and
+"creator" properties.
+
+The value of the "creation" property is the date when an item was
+created, and the value of the "activity" property is the date when any
+property on the item was last edited (equivalently, these are the dates
+on the first and last records in the item's journal). The "creator"
+property holds a link to the user that created the issue.
+
setkey(property)
-::::::::::::::::
+~~~~~~~~~~~~~~~~
-Select a String property of the class to be the key property. The key property
-muse be unique, and allows references to the nodes in the class by the content
-of the key property. That is, we can refer to users by their username, e.g.
-let's say that there's an issue in roundup, issue 23. There's also a user,
-richard who happens to be user 2. To assign an issue to him, we could do either
-of::
+Select a String property of the class to be the key property. The key
+property must be unique, and allows references to the items in the class
+by the content of the key property. That is, we can refer to users by
+their username: for example, let's say that there's an issue in roundup,
+issue 23. There's also a user, richard, who happens to be user 2. To
+assign an issue to him, we could do either of::
- roundup-admin set issue assignedto=2
+ roundup-admin set issue23 assignedto=2
or::
- roundup-admin set issue assignedto=richard
+ roundup-admin set issue23 assignedto=richard
+
+Note, the same thing can be done in the web and e-mail interfaces.
+
+If a class does not have an "order" property, the key is also used to
+sort instances of the class when it is rendered in the user interface.
+(If a class has no "order" property, sorting is by the labelproperty of
+the class. This is computed, in order of precedence, as the key, the
+"name", the "title", or the first property alphabetically.)
-Note, the same thing can be done in the web and e-mail interfaces.
create(information)
-:::::::::::::::::::
+~~~~~~~~~~~~~~~~~~~
+
+Create an item in the database. This is generally used to create items
+in the "definitional" classes like "priority" and "status".
+
+
+Examples of adding to your schema
+---------------------------------
-Create a node in the database. This is generally used to create nodes in the
-"definitional" classes like "priority" and "status".
+TODO
Detectors - adding behaviour to your tracker
---------------------------------------------
+============================================
.. _detectors:
-The detectors in your instance fire before (*auditors*) and after (*reactors*)
-changes to the contents of your database. They are Python modules that sit in
-your instance's ``detectors`` directory. You will have some installed by
-default - have a look. You can write new detectors or modify the existing
-ones. The existing detectors installed for you are:
+Detectors are initialised every time you open your tracker database, so
+you're free to add and remove them any time, even after the database is
+initialised via the "roundup-admin initialise" command.
+
+The detectors in your tracker fire *before* (**auditors**) and *after*
+(**reactors**) changes to the contents of your database. They are Python
+modules that sit in your tracker's ``detectors`` directory. You will
+have some installed by default - have a look. You can write new
+detectors or modify the existing ones. The existing detectors installed
+for you are:
**nosyreaction.py**
- This provides the automatic nosy list maintenance and email sending. The nosy
- reactor (``nosyreaction``) fires when new messages are added to issues.
- The nosy auditor (``updatenosy``) fires when issues are changed and figures
- what changes need to be made to the nosy list (like adding new authors etc)
+ This provides the automatic nosy list maintenance and email sending.
+ The nosy reactor (``nosyreaction``) fires when new messages are added
+ to issues. The nosy auditor (``updatenosy``) fires when issues are
+ changed, and figures out what changes need to be made to the nosy list
+ (such as adding new authors, etc.)
**statusauditor.py**
- This provides the ``chatty`` auditor which changes the issue status from
- ``unread`` or ``closed`` to ``chatting`` if new messages appear. It also
- provides the ``presetunread`` auditor which pre-sets the status to
- ``unread`` on new nodes if the status isn't explicitly defined.
+ This provides the ``chatty`` auditor which changes the issue status
+ from ``unread`` or ``closed`` to ``chatting`` if new messages appear.
+ It also provides the ``presetunread`` auditor which pre-sets the
+ status to ``unread`` on new items if the status isn't explicitly
+ defined.
+**messagesummary.py**
+ Generates the ``summary`` property for new messages based on the message
+ content.
+**userauditor.py**
+ Verifies the content of some of the user fields (email addresses and
+ roles lists).
+
+If you don't want this default behaviour, you're completely free to change
+or remove these detectors.
See the detectors section in the `design document`__ for details of the
interface for detectors.
__ design.html
-Sample additional detectors that have been found useful will appear in the
-``detectors`` directory of the Roundup distribution:
+Sample additional detectors that have been found useful will appear in
+the ``'detectors'`` directory of the Roundup distribution. If you want
+to use one, copy it to the ``'detectors'`` of your tracker instance:
**newissuecopy.py**
This detector sends an email to a team address whenever a new issue is
- created. The address is hard-coded into the detector, so edit it before you
- use it (look for the text 'team@team.host') or you'll get email errors!
+ created. The address is hard-coded into the detector, so edit it
+ before you use it (look for the text 'team@team.host') or you'll get
+ email errors!
+
+ The detector code::
+
+ from roundup import roundupdb
+
+ def newissuecopy(db, cl, nodeid, oldvalues):
+ ''' Copy a message about new issues to a team address.
+ '''
+ # so use all the messages in the create
+ change_note = cl.generateCreateNote(nodeid)
+
+ # send a copy to the nosy list
+ for msgid in cl.get(nodeid, 'messages'):
+ try:
+ # note: last arg must be a list
+ cl.send_message(nodeid, msgid, change_note,
+ ['team@team.host'])
+ except roundupdb.MessageSendError, message:
+ raise roundupdb.DetectorError, message
+
+ def init(db):
+ db.issue.react('create', newissuecopy)
Database Content
-----------------
+================
+
+Note: if you modify the content of definitional classes, you'll most
+ likely need to edit the tracker `detectors`_ to reflect your
+ changes.
+
+Customisation of the special "definitional" classes (eg. status,
+priority, resolution, ...) may be done either before or after the
+tracker is initialised. The actual method of doing so is completely
+different in each case though, so be careful to use the right one.
+
+**Changing content before tracker initialisation**
+ Edit the dbinit module in your tracker to alter the items created in
+ using the ``create()`` methods.
+
+**Changing content after tracker initialisation**
+ As the "admin" user, click on the "class list" link in the web
+ interface to bring up a list of all database classes. Click on the
+ name of the class you wish to change the content of.
+
+ You may also use the ``roundup-admin`` interface's create, set and
+ retire methods to add, alter or remove items from the classes in
+ question.
+
+See "`adding a new field to the classic schema`_" for an example that
+requires database content changes.
+
+
+Access Controls
+===============
+
+A set of Permissions is built into the security module by default:
+
+- Edit (everything)
+- View (everything)
+
+The default interfaces define:
+
+- Web Registration
+- Web Access
+- Web Roles
+- Email Registration
+- Email Access
+
+These are hooked into the default Roles:
+
+- Admin (Edit everything, View everything, Web Roles)
+- User (Web Access, Email Access)
+- Anonymous (Web Registration, Email Registration)
+
+And finally, the "admin" user gets the "Admin" Role, and the "anonymous"
+user gets "Anonymous" assigned when the database is initialised on
+installation. The two default schemas then define:
+
+- Edit issue, View issue (both)
+- Edit file, View file (both)
+- Edit msg, View msg (both)
+- Edit support, View support (extended only)
+
+and assign those Permissions to the "User" Role. Put together, these
+settings appear in the ``open()`` function of the tracker ``dbinit.py``
+(the following is taken from the "minimal" template's ``dbinit.py``)::
+
+ #
+ # SECURITY SETTINGS
+ #
+ # new permissions for this schema
+ for cl in ('user', ):
+ db.security.addPermission(name="Edit", klass=cl,
+ description="User is allowed to edit "+cl)
+ db.security.addPermission(name="View", klass=cl,
+ description="User is allowed to access "+cl)
+
+ # and give the regular users access to the web and email interface
+ p = db.security.getPermission('Web Access')
+ db.security.addPermissionToRole('User', p)
+ p = db.security.getPermission('Email Access')
+ db.security.addPermissionToRole('User', p)
+
+ # May users view other user information? Comment these lines out
+ # if you don't want them to
+ p = db.security.getPermission('View', 'user')
+ db.security.addPermissionToRole('User', p)
+
+ # Assign the appropriate permissions to the anonymous user's
+ # Anonymous role. Choices here are:
+ # - Allow anonymous users to register through the web
+ p = db.security.getPermission('Web Registration')
+ db.security.addPermissionToRole('Anonymous', p)
+ # - Allow anonymous (new) users to register through the email
+ # gateway
+ p = db.security.getPermission('Email Registration')
+ db.security.addPermissionToRole('Anonymous', p)
+
+
+New User Roles
+--------------
+
+New users are assigned the Roles defined in the config file as:
+
+- NEW_WEB_USER_ROLES
+- NEW_EMAIL_USER_ROLES
+
-Note: if you modify the content of definitional classes, you'll most likely
- need to edit the instance `detectors`_ to reflect your changes.
+Changing Access Controls
+------------------------
-Customisation of the special "definitional" classes (eg. status, priority,
-resolution, ...) may be done either before or after the instance is
-initialised. The actual method of doing so is completely different in each
-case though, so be careful to use the right one.
+You may alter the configuration variables to change the Role that new
+web or email users get, for example to not give them access to the web
+interface if they register through email.
+
+You may use the ``roundup-admin`` "``security``" command to display the
+current Role and Permission configuration in your tracker.
+
+
+Adding a new Permission
+~~~~~~~~~~~~~~~~~~~~~~~
-**Changing content before instance initialisation**
- Edit the dbinit module in your instance to alter the nodes created in using
- the create() methods.
+When adding a new Permission, you will need to:
+1. add it to your tracker's dbinit so it is created
+2. enable it for the Roles that should have it (verify with
+ "``roundup-admin security``")
+3. add it to the relevant HTML interface templates
+4. add it to the appropriate xxxPermission methods on in your tracker
+ interfaces module
-**Changing content after instance initialisation**
- Use the roundup-admin interface's create, set and retire methods to add,
- alter or remove nodes from the classes in question.
+Example Scenarios
+~~~~~~~~~~~~~~~~~
+
+**automatic registration of users in the e-mail gateway**
+ By giving the "anonymous" user the "Email Registration" Role, any
+ unidentified user will automatically be registered with the tracker
+ (with no password, so they won't be able to log in through the web
+ until an admin sets their password). Note: this is the default
+ behaviour in the tracker templates that ship with Roundup.
+
+**anonymous access through the e-mail gateway**
+ Give the "anonymous" user the "Email Access" and ("Edit", "issue")
+ Roles but do not not give them the "Email Registration" Role. This
+ means that when an unknown user sends email into the tracker, they're
+ automatically logged in as "anonymous". Since they don't have the
+ "Email Registration" Role, they won't be automatically registered, but
+ since "anonymous" has permission to use the gateway, they'll still be
+ able to submit issues. Note that the Sender information - their email
+ address - will not be available - they're *anonymous*.
+
+**only developers may be assigned issues**
+ Create a new Permission called "Fixer" for the "issue" class. Create a
+ new Role "Developer" which has that Permission, and assign that to the
+ appropriate users. Filter the list of users available in the assignedto
+ list to include only those users. Enforce the Permission with an
+ auditor. See the example
+ `restricting the list of users that are assignable to a task`_.
+
+**only managers may sign off issues as complete**
+ Create a new Permission called "Closer" for the "issue" class. Create a
+ new Role "Manager" which has that Permission, and assign that to the
+ appropriate users. In your web interface, only display the "resolved"
+ issue state option when the user has the "Closer" Permissions. Enforce
+ the Permission with an auditor. This is very similar to the previous
+ example, except that the web interface check would look like::
+
+ <option tal:condition="python:request.user.hasPermission('Closer')"
+ value="resolved">Resolved</option>
+
+**don't give web access to users who register through email**
+ Create a new Role called "Email User" which has all the Permissions of
+ the normal "User" Role minus the "Web Access" Permission. This will
+ allow users to send in emails to the tracker, but not access the web
+ interface.
+
+**let some users edit the details of all users**
+ Create a new Role called "User Admin" which has the Permission for
+ editing users::
+
+ db.security.addRole(name='User Admin', description='Managing users')
+ p = db.security.getPermission('Edit', 'user')
+ db.security.addPermissionToRole('User Admin', p)
+
+ and assign the Role to the users who need the permission.
Web Interface
--------------
-
-The web interface works behind the cgi-bin/roundup.cgi or roundup-server
-scripts. In both cases, the scripts determine which instance is being accessed
-(the first part of the URL path inside the scope of the CGI handler) and pass
-control on to the instance interfaces.Client class which handles the rest of
-the access through its main() method. This means that you can do pretty much
-anything you want as a web interface to your instance.
-Most customisation of the web view can be done by modifying the templates in
-the instance html directory. These are divided into index, item and newitem
-views. The newitem view is optional - the item view will be used if the newitem
-view doesn't exist. The header and footer that wrap the various views give the
-pages an overall look.
-
-Repurcussions of changing the instance schema
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-If you choose to change the `instance schema`_ you will need to ensure the web
-interface knows about it:
-
-1. Index, item and filter pages for the relevant classes may need to have
- properties added or removed,
-2. The default page header relies on the existence of, and some values of
- the priority, status, assignedto and activity classes. If you change any
- of these (specifically if you remove any of the classes or their default
- values) you will need to implement your own pagehead() method in your
- instance's interfaces.py module.
-
-Overall Look - the Header and Footer
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+=============
+
+.. contents::
+ :local:
+ :depth: 1
+
+The web interface is provided by the ``roundup.cgi.client`` module and
+is used by ``roundup.cgi``, ``roundup-server`` and ``ZRoundup``
+(``ZRoundup`` is broken, until further notice). In all cases, we
+determine which tracker is being accessed (the first part of the URL
+path inside the scope of the CGI handler) and pass control on to the
+tracker ``interfaces.Client`` class - which uses the ``Client`` class
+from ``roundup.cgi.client`` - which handles the rest of the access
+through its ``main()`` method. This means that you can do pretty much
+anything you want as a web interface to your tracker.
+
+Repercussions of changing the tracker schema
+---------------------------------------------
+
+If you choose to change the `tracker schema`_ you will need to ensure
+the web interface knows about it:
+
+1. Index, item and search pages for the relevant classes may need to
+ have properties added or removed,
+2. The "page" template may require links to be changed, as might the
+ "home" page's content arguments.
+
+How requests are processed
+--------------------------
+
+The basic processing of a web request proceeds as follows:
+
+1. figure out who we are, defaulting to the "anonymous" user
+2. figure out what the request is for - we call this the "context"
+3. handle any requested action (item edit, search, ...)
+4. render the template requested by the context, resulting in HTML
+ output
+
+In some situations, exceptions occur:
+
+- HTTP Redirect (generally raised by an action)
+- SendFile (generally raised by ``determine_context``)
+ here we serve up a FileClass "content" property
+- SendStaticFile (generally raised by ``determine_context``)
+ here we serve up a file from the tracker "html" directory
+- Unauthorised (generally raised by an action)
+ here the action is cancelled, the request is rendered and an error
+ message is displayed indicating that permission was not granted for
+ the action to take place
+- NotFound (raised wherever it needs to be)
+ this exception percolates up to the CGI interface that called the
+ client
+
+Determining web context
+-----------------------
+
+To determine the "context" of a request, we look at the URL and the
+special request variable ``@template``. The URL path after the tracker
+identifier is examined. Typical URL paths look like:
+
+1. ``/tracker/issue``
+2. ``/tracker/issue1``
+3. ``/tracker/_file/style.css``
+4. ``/cgi-bin/roundup.cgi/tracker/file1``
+5. ``/cgi-bin/roundup.cgi/tracker/file1/kitten.png``
+
+where the "tracker identifier" is "tracker" in the above cases. That means
+we're looking at "issue", "issue1", "_file/style.css", "file1" and
+"file1/kitten.png" in the cases above. The path is generally only one
+entry long - longer paths are handled differently.
+
+a. if there is no path, then we are in the "home" context.
+b. if the path starts with "_file" (as in example 3,
+ "/tracker/_file/style.css"), then the additional path entry,
+ "style.css" specifies the filename of a static file we're to serve up
+ from the tracker "html" directory. Raises a SendStaticFile exception.
+c. if there is something in the path (as in example 1, "issue"), it
+ identifies the tracker class we're to display.
+d. if the path is an item designator (as in examples 2 and 4, "issue1"
+ and "file1"), then we're to display a specific item.
+e. if the path starts with an item designator and is longer than one
+ entry (as in example 5, "file1/kitten.png"), then we're assumed to be
+ handling an item of a ``FileClass``, and the extra path information
+ gives the filename that the client is going to label the download
+ with (i.e. "file1/kitten.png" is nicer to download than "file1").
+ This raises a ``SendFile`` exception.
+
+Both b. and e. stop before we bother to determine the template we're
+going to use. That's because they don't actually use templates.
+
+The template used is specified by the ``@template`` CGI variable, which
+defaults to:
+
+- only classname suplied: "index"
+- full item designator supplied: "item"
+
+
+Performing actions in web requests
+----------------------------------
+
+When a user requests a web page, they may optionally also request for an
+action to take place. As described in `how requests are processed`_, the
+action is performed before the requested page is generated. Actions are
+triggered by using a ``@action`` CGI variable, where the value is one
+of:
+
+**login**
+ Attempt to log a user in.
+
+**logout**
+ Log the user out - make them "anonymous".
+
+**register**
+ Attempt to create a new user based on the contents of the form and then
+ log them in.
+
+**edit**
+ Perform an edit of an item in the database. There are some `special form
+ variables`_ you may use.
+
+**new**
+ Add a new item to the database. You may use the same `special form
+ variables`_ as in the "edit" action.
+
+**retire**
+ Retire the item in the database.
+
+**editCSV**
+ Performs an edit of all of a class' items in one go. See also the
+ *class*.csv templating method which generates the CSV data to be
+ edited, and the ``'_generic.index'`` template which uses both of these
+ features.
+
+**search**
+ Mangle some of the form variables:
+
+ - Set the form ":filter" variable based on the values of the filter
+ variables - if they're set to anything other than "dontcare" then add
+ them to :filter.
+
+ - Also handle the ":queryname" variable and save off the query to the
+ user's query list.
+
+Each of the actions is implemented by a corresponding ``*actionAction*``
+(where "action" is the name of the action) method on the
+``roundup.cgi.Client`` class, which also happens to be available in your
+tracker instance as ``interfaces.Client``. So if you need to define new
+actions, you may add them there (see `defining new web actions`_).
+
+Each action also has a corresponding ``*actionPermission*`` (where
+"action" is the name of the action) method which determines whether the
+action is permissible given the current user. The base permission checks
+are:
+
+**login**
+ Determine whether the user has permission to log in. Base behaviour is
+ to check the user has "Web Access".
+**logout**
+ No permission checks are made.
+**register**
+ Determine whether the user has permission to register. Base behaviour
+ is to check the user has the "Web Registration" Permission.
+**edit**
+ Determine whether the user has permission to edit this item. Base
+ behaviour is to check whether the user can edit this class. If we're
+ editing the "user" class, users are allowed to edit their own details -
+ unless they try to edit the "roles" property, which requires the
+ special Permission "Web Roles".
+**new**
+ Determine whether the user has permission to create (or edit) this
+ item. Base behaviour is to check the user can edit this class. No
+ additional property checks are made. Additionally, new user items may
+ be created if the user has the "Web Registration" Permission.
+**editCSV**
+ Determine whether the user has permission to edit this class. Base
+ behaviour is to check whether the user may edit this class.
+**search**
+ Determine whether the user has permission to search this class. Base
+ behaviour is to check whether the user may view this class.
+
+
+Special form variables
+----------------------
+
+Item properties and their values are edited with html FORM
+variables and their values. You can:
+
+- Change the value of some property of the current item.
+- Create a new item of any class, and edit the new item's
+ properties,
+- Attach newly created items to a multilink property of the
+ current item.
+- Remove items from a multilink property of the current item.
+- Specify that some properties are required for the edit
+ operation to be successful.
+
+In the following, <bracketed> values are variable, "@" may be
+either ":" or "@", and other text "required" is fixed.
+
+Most properties are specified as form variables:
+
+``<propname>``
+ property on the current context item
+
+``<designator>"@"<propname>``
+ property on the indicated item (for editing related information)
+
+Designators name a specific item of a class.
+
+``<classname><N>``
+ Name an existing item of class <classname>.
+
+``<classname>"-"<N>``
+ Name the <N>th new item of class <classname>. If the form
+ submission is successful, a new item of <classname> is
+ created. Within the submitted form, a particular
+ designator of this form always refers to the same new
+ item.
+
+Once we have determined the "propname", we look at it to see
+if it's special:
+
+``@required``
+ The associated form value is a comma-separated list of
+ property names that must be specified when the form is
+ submitted for the edit operation to succeed.
+
+ When the <designator> is missing, the properties are
+ for the current context item. When <designator> is
+ present, they are for the item specified by
+ <designator>.
+
+ The "@required" specifier must come before any of the
+ properties it refers to are assigned in the form.
+
+``@remove@<propname>=id(s)`` or ``@add@<propname>=id(s)``
+ The "@add@" and "@remove@" edit actions apply only to
+ Multilink properties. The form value must be a
+ comma-separate list of keys for the class specified by
+ the simple form variable. The listed items are added
+ to (respectively, removed from) the specified
+ property.
+
+``@link@<propname>=<designator>``
+ If the edit action is "@link@", the simple form
+ variable must specify a Link or Multilink property.
+ The form value is a comma-separated list of
+ designators. The item corresponding to each
+ designator is linked to the property given by simple
+ form variable.
+
+None of the above (ie. just a simple form value)
+ The value of the form variable is converted
+ appropriately, depending on the type of the property.
+
+ For a Link('klass') property, the form value is a
+ single key for 'klass', where the key field is
+ specified in dbinit.py.
+
+ For a Multilink('klass') property, the form value is a
+ comma-separated list of keys for 'klass', where the
+ key field is specified in dbinit.py.
+
+ Note that for simple-form-variables specifiying Link
+ and Multilink properties, the linked-to class must
+ have a key field.
+
+ For a String() property specifying a filename, the
+ file named by the form value is uploaded. This means we
+ try to set additional properties "filename" and "type" (if
+ they are valid for the class). Otherwise, the property
+ is set to the form value.
+
+ For Date(), Interval(), Boolean(), and Number()
+ properties, the form value is converted to the
+ appropriate
+
+Any of the form variables may be prefixed with a classname or
+designator.
+
+Two special form values are supported for backwards compatibility:
+
+@note
+ This is equivalent to::
+
+ @link@messages=msg-1
+ msg-1@content=value
+
+ except that in addition, the "author" and "date" properties of
+ "msg-1" are set to the userid of the submitter, and the current
+ time, respectively.
+
+@file
+ This is equivalent to::
+
+ @link@files=file-1
+ file-1@content=value
+
+ The String content value is handled as described above for file
+ uploads.
+
+If both the "@note" and "@file" form variables are
+specified, the action::
+
+ @link@msg-1@files=file-1
+
+is also performed.
+
+We also check that FileClass items have a "content" property with
+actual content, otherwise we remove them from all_props before
+returning.
+
+
+
+Default templates
+-----------------
+
+Most customisation of the web view can be done by modifying the
+templates in the tracker ``'html'`` directory. There are several types
+of files in there. The *minimal* template includes:
+
+**page.html**
+ This template usually defines the overall look of your tracker. When
+ you view an issue, it appears inside this template. When you view an
+ index, it also appears inside this template. This template defines a
+ macro called "icing" which is used by almost all other templates as a
+ coating for their content, using its "content" slot. It also defines
+ the "head_title" and "body_title" slots to allow setting of the page
+ title.
+**home.html**
+ the default page displayed when no other page is indicated by the user
+**home.classlist.html**
+ a special version of the default page that lists the classes in the
+ tracker
+**classname.item.html**
+ displays an item of the *classname* class
+**classname.index.html**
+ displays a list of *classname* items
+**classname.search.html**
+ displays a search page for *classname* items
+**_generic.index.html**
+ used to display a list of items where there is no
+ ``*classname*.index`` available
+**_generic.help.html**
+ used to display a "class help" page where there is no
+ ``*classname*.help``
+**user.register.html**
+ a special page just for the user class, that renders the registration
+ page
+**style.css.html**
+ a static file that is served up as-is
+
+The *classic* template has a number of additional templates.
+
+Note: Remember that you can create any template extension you want to,
+so if you just want to play around with the templating for new issues,
+you can copy the current "issue.item" template to "issue.test", and then
+access the test template using the "@template" URL argument::
+
+ http://your.tracker.example/tracker/issue?@template=test
+
+and it won't affect your users using the "issue.item" template.
+
+
+How the templates work
+----------------------
+
+
+Basic Templating Actions
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Roundup's templates consist of special attributes on the HTML tags.
+These attributes form the Template Attribute Language, or TAL. The basic
+TAL commands are:
+
+**tal:define="variable expression; variable expression; ..."**
+ Define a new variable that is local to this tag and its contents. For
+ example::
+
+ <html tal:define="title request/description">
+ <head><title tal:content="title"></title></head>
+ </html>
+
+ In this example, the variable "title" is defined as the result of the
+ expression "request/description". The "tal:content" command inside the
+ <html> tag may then use the "title" variable.
+
+**tal:condition="expression"**
+ Only keep this tag and its contents if the expression is true. For
+ example::
+
+ <p tal:condition="python:request.user.hasPermission('View', 'issue')">
+ Display some issue information.
+ </p>
+
+ In the example, the <p> tag and its contents are only displayed if
+ the user has the "View" permission for issues. We consider the number
+ zero, a blank string, an empty list, and the built-in variable
+ nothing to be false values. Nearly every other value is true,
+ including non-zero numbers, and strings with anything in them (even
+ spaces!).
+
+**tal:repeat="variable expression"**
+ Repeat this tag and its contents for each element of the sequence
+ that the expression returns, defining a new local variable and a
+ special "repeat" variable for each element. For example::
+
+ <tr tal:repeat="u user/list">
+ <td tal:content="u/id"></td>
+ <td tal:content="u/username"></td>
+ <td tal:content="u/realname"></td>
+ </tr>
+
+ The example would iterate over the sequence of users returned by
+ "user/list" and define the local variable "u" for each entry.
+
+**tal:replace="expression"**
+ Replace this tag with the result of the expression. For example::
+
+ <span tal:replace="request/user/realname" />
+
+ The example would replace the <span> tag and its contents with the
+ user's realname. If the user's realname was "Bruce", then the
+ resultant output would be "Bruce".
+
+**tal:content="expression"**
+ Replace the contents of this tag with the result of the expression.
+ For example::
+
+ <span tal:content="request/user/realname">user's name appears here
+ </span>
+
+ The example would replace the contents of the <span> tag with the
+ user's realname. If the user's realname was "Bruce" then the
+ resultant output would be "<span>Bruce</span>".
+
+**tal:attributes="attribute expression; attribute expression; ..."**
+ Set attributes on this tag to the results of expressions. For
+ example::
+
+ <a tal:attributes="href string:user${request/user/id}">My Details</a>
+
+ In the example, the "href" attribute of the <a> tag is set to the
+ value of the "string:user${request/user/id}" expression, which will
+ be something like "user123".
+
+**tal:omit-tag="expression"**
+ Remove this tag (but not its contents) if the expression is true. For
+ example::
+
+ <span tal:omit-tag="python:1">Hello, world!</span>
+
+ would result in output of::
+
+ Hello, world!
+
+Note that the commands on a given tag are evaulated in the order above,
+so *define* comes before *condition*, and so on.
+
+Additionally, you may include tags such as <tal:block>, which are
+removed from output. Its content is kept, but the tag itself is not (so
+don't go using any "tal:attributes" commands on it). This is useful for
+making arbitrary blocks of HTML conditional or repeatable (very handy
+for repeating multiple table rows, which would othewise require an
+illegal tag placement to effect the repeat).
+
+
+Templating Expressions
+~~~~~~~~~~~~~~~~~~~~~~
+
+The expressions you may use in the attribute values may be one of the
+following forms:
+
+**Path Expressions** - eg. ``item/status/checklist``
+ These are object attribute / item accesses. Roughly speaking, the
+ path ``item/status/checklist`` is broken into parts ``item``,
+ ``status`` and ``checklist``. The ``item`` part is the root of the
+ expression. We then look for a ``status`` attribute on ``item``, or
+ failing that, a ``status`` item (as in ``item['status']``). If that
+ fails, the path expression fails. When we get to the end, the object
+ we're left with is evaluated to get a string - if it is a method, it
+ is called; if it is an object, it is stringified. Path expressions
+ may have an optional ``path:`` prefix, but they are the default
+ expression type, so it's not necessary.
+
+ If an expression evaluates to ``default``, then the expression is
+ "cancelled" - whatever HTML already exists in the template will
+ remain (tag content in the case of ``tal:content``, attributes in the
+ case of ``tal:attributes``).
+
+ If an expression evaluates to ``nothing`` then the target of the
+ expression is removed (tag content in the case of ``tal:content``,
+ attributes in the case of ``tal:attributes`` and the tag itself in
+ the case of ``tal:replace``).
+
+ If an element in the path may not exist, then you can use the ``|``
+ operator in the expression to provide an alternative. So, the
+ expression ``request/form/foo/value | default`` would simply leave
+ the current HTML in place if the "foo" form variable doesn't exist.
+
+ You may use the python function ``path``, as in
+ ``path("item/status")``, to embed path expressions in Python
+ expressions.
+
+**String Expressions** - eg. ``string:hello ${user/name}``
+ These expressions are simple string interpolations - though they can
+ be just plain strings with no interpolation if you want. The
+ expression in the ``${ ... }`` is just a path expression as above.
+
+**Python Expressions** - eg. ``python: 1+1``
+ These expressions give the full power of Python. All the "root level"
+ variables are available, so ``python:item.status.checklist()`` would
+ be equivalent to ``item/status/checklist``, assuming that
+ ``checklist`` is a method.
+
+Modifiers:
+
+**structure** - eg. ``structure python:msg.content.plain(hyperlink=1)``
+ The result of expressions are normally *escaped* to be safe for HTML
+ display (all "<", ">" and "&" are turned into special entities). The
+ ``structure`` expression modifier turns off this escaping - the
+ result of the expression is now assumed to be HTML, which is passed
+ to the web browser for rendering.
+
+**not:** - eg. ``not:python:1=1``
+ This simply inverts the logical true/false value of another
+ expression.
+
+
+Template Macros
+~~~~~~~~~~~~~~~
+
+Macros are used in Roundup to save us from repeating the same common
+page stuctures over and over. The most common (and probably only) macro
+you'll use is the "icing" macro defined in the "page" template.
+
+Macros are generated and used inside your templates using special
+attributes similar to the `basic templating actions`_. In this case,
+though, the attributes belong to the Macro Expansion Template Attribute
+Language, or METAL. The macro commands are:
+
+**metal:define-macro="macro name"**
+ Define that the tag and its contents are now a macro that may be
+ inserted into other templates using the *use-macro* command. For
+ example::
+
+ <html metal:define-macro="page">
+ ...
+ </html>
+
+ defines a macro called "page" using the ``<html>`` tag and its
+ contents. Once defined, macros are stored on the template they're
+ defined on in the ``macros`` attribute. You can access them later on
+ through the ``templates`` variable, eg. the most common
+ ``templates/page/macros/icing`` to access the "page" macro of the
+ "page" template.
+
+**metal:use-macro="path expression"**
+ Use a macro, which is identified by the path expression (see above).
+ This will replace the current tag with the identified macro contents.
+ For example::
+
+ <tal:block metal:use-macro="templates/page/macros/icing">
+ ...
+ </tal:block>
+
+ will replace the tag and its contents with the "page" macro of the
+ "page" template.
+
+**metal:define-slot="slot name"** and **metal:fill-slot="slot name"**
+ To define *dynamic* parts of the macro, you define "slots" which may
+ be filled when the macro is used with a *use-macro* command. For
+ example, the ``templates/page/macros/icing`` macro defines a slot like
+ so::
+
+ <title metal:define-slot="head_title">title goes here</title>
+
+ In your *use-macro* command, you may now use a *fill-slot* command
+ like this::
+
+ <title metal:fill-slot="head_title">My Title</title>
+
+ where the tag that fills the slot completely replaces the one defined
+ as the slot in the macro.
+
+Note that you may not mix METAL and TAL commands on the same tag, but
+TAL commands may be used freely inside METAL-using tags (so your
+*fill-slots* tags may have all manner of TAL inside them).
+
+
+Information available to templates
+----------------------------------
+
+Note: this is implemented by
+``roundup.cgi.templating.RoundupPageTemplate``
+
+The following variables are available to templates.
+
+**context**
+ The current context. This is either None, a `hyperdb class wrapper`_
+ or a `hyperdb item wrapper`_
+**request**
+ Includes information about the current request, including:
+ - the current index information (``filterspec``, ``filter`` args,
+ ``properties``, etc) parsed out of the form.
+ - methods for easy filterspec link generation
+ - *user*, the current user item as an HTMLItem instance
+ - *form*
+ The current CGI form information as a mapping of form argument name
+ to value
+**config**
+ This variable holds all the values defined in the tracker config.py
+ file (eg. TRACKER_NAME, etc.)
+**db**
+ The current database, used to access arbitrary database items.
+**templates**
+ Access to all the tracker templates by name. Used mainly in
+ *use-macro* commands.
+**utils**
+ This variable makes available some utility functions like batching.
+**nothing**
+ This is a special variable - if an expression evaluates to this, then
+ the tag (in the case of a ``tal:replace``), its contents (in the case
+ of ``tal:content``) or some attributes (in the case of
+ ``tal:attributes``) will not appear in the the output. So, for
+ example::
+
+ <span tal:attributes="class nothing">Hello, World!</span>
+
+ would result in::
+
+ <span>Hello, World!</span>
+
+**default**
+ Also a special variable - if an expression evaluates to this, then the
+ existing HTML in the template will not be replaced or removed, it will
+ remain. So::
+
+ <span tal:replace="default">Hello, World!</span>
+
+ would result in::
+
+ <span>Hello, World!</span>
+
+
+The context variable
+~~~~~~~~~~~~~~~~~~~~
+
+The *context* variable is one of three things based on the current
+context (see `determining web context`_ for how we figure this out):
+
+1. if we're looking at a "home" page, then it's None
+2. if we're looking at a specific hyperdb class, it's a
+ `hyperdb class wrapper`_.
+3. if we're looking at a specific hyperdb item, it's a
+ `hyperdb item wrapper`_.
+
+If the context is not None, we can access the properties of the class or
+item. The only real difference between cases 2 and 3 above are:
+
+1. the properties may have a real value behind them, and this will
+ appear if the property is displayed through ``context/property`` or
+ ``context/property/field``.
+2. the context's "id" property will be a false value in the second case,
+ but a real, or true value in the third. Thus we can determine whether
+ we're looking at a real item from the hyperdb by testing
+ "context/id".
+
+Hyperdb class wrapper
+:::::::::::::::::::::
+
+Note: this is implemented by the ``roundup.cgi.templating.HTMLClass``
+class.
+
+This wrapper object provides access to a hyperb class. It is used
+primarily in both index view and new item views, but it's also usable
+anywhere else that you wish to access information about a class, or the
+items of a class, when you don't have a specific item of that class in
+mind.
+
+We allow access to properties. There will be no "id" property. The value
+accessed through the property will be the current value of the same name
+from the CGI form.
+
+There are several methods available on these wrapper objects:
+
+=========== =============================================================
+Method Description
+=========== =============================================================
+properties return a `hyperdb property wrapper`_ for all of this class's
+ properties.
+list lists all of the active (not retired) items in the class.
+csv return the items of this class as a chunk of CSV text.
+propnames lists the names of the properties of this class.
+filter lists of items from this class, filtered and sorted by the
+ current *request* filterspec/filter/sort/group args
+classhelp display a link to a javascript popup containing this class'
+ "help" template.
+submit generate a submit button (and action hidden element)
+renderWith render this class with the given template.
+history returns 'New node - no history' :)
+is_edit_ok is the user allowed to Edit the current class?
+is_view_ok is the user allowed to View the current class?
+=========== =============================================================
+
+Note that if you have a property of the same name as one of the above
+methods, you'll need to access it using a python "item access"
+expression. For example::
+
+ python:context['list']
+
+will access the "list" property, rather than the list method.
+
+
+Hyperdb item wrapper
+::::::::::::::::::::
+
+Note: this is implemented by the ``roundup.cgi.templating.HTMLItem``
+class.
+
+This wrapper object provides access to a hyperb item.
+
+We allow access to properties. There will be no "id" property. The value
+accessed through the property will be the current value of the same name
+from the CGI form.
+
+There are several methods available on these wrapper objects:
+
+=============== ========================================================
+Method Description
+=============== ========================================================
+submit generate a submit button (and action hidden element)
+journal return the journal of the current item (**not
+ implemented**)
+history render the journal of the current item as HTML
+renderQueryForm specific to the "query" class - render the search form
+ for the query
+hasPermission specific to the "user" class - determine whether the
+ user has a Permission
+is_edit_ok is the user allowed to Edit the current item?
+is_view_ok is the user allowed to View the current item?
+=============== ========================================================
+
+Note that if you have a property of the same name as one of the above
+methods, you'll need to access it using a python "item access"
+expression. For example::
+
+ python:context['journal']
+
+will access the "journal" property, rather than the journal method.
+
+
+Hyperdb property wrapper
+::::::::::::::::::::::::
+
+Note: this is implemented by subclasses of the
+``roundup.cgi.templating.HTMLProperty`` class (``HTMLStringProperty``,
+``HTMLNumberProperty``, and so on).
+
+This wrapper object provides access to a single property of a class. Its
+value may be either:
+
+1. if accessed through a `hyperdb item wrapper`_, then it's a value from
+ the hyperdb
+2. if access through a `hyperdb class wrapper`_, then it's a value from
+ the CGI form
+
+
+The property wrapper has some useful attributes:
+
+=============== ========================================================
+Attribute Description
+=============== ========================================================
+_name the name of the property
+_value the value of the property if any - this is the actual
+ value retrieved from the hyperdb for this property
+=============== ========================================================
+
+There are several methods available on these wrapper objects:
+
+=========== ================================================================
+Method Description
+=========== ================================================================
+plain render a "plain" representation of the property. This method
+ may take two arguments:
+
+ escape
+ If true, escape the text so it is HTML safe (default: no). The
+ reason this defaults to off is that text is usually escaped
+ at a later stage by the TAL commands, unless the "structure"
+ option is used in the template. The following ``tal:content``
+ expressions are all equivalent::
+
+ "structure python:msg.content.plain(escape=1)"
+ "python:msg.content.plain()"
+ "msg/content/plain"
+ "msg/content"
+
+ Usually you'll only want to use the escape option in a
+ complex expression.
+
+ hyperlink
+ If true, turn URLs, email addresses and hyperdb item
+ designators in the text into hyperlinks (default: no). Note
+ that you'll need to use the "structure" TAL option if you
+ want to use this ``tal:content`` expression::
+
+ "structure python:msg.content.plain(hyperlink=1)"
+
+ Note also that the text is automatically HTML-escaped before
+ the hyperlinking transformation.
+hyperlinked The same as msg.content.plain(hyperlink=1), but nicer::
+
+ "structure msg/content/hyperlinked"
+
+field render an appropriate form edit field for the property - for
+ most types this is a text entry box, but for Booleans it's a
+ tri-state yes/no/neither selection.
+stext only on String properties - render the value of the property
+ as StructuredText (requires the StructureText module to be
+ installed separately)
+multiline only on String properties - render a multiline form edit
+ field for the property
+email only on String properties - render the value of the property
+ as an obscured email address
+confirm only on Password properties - render a second form edit field
+ for the property, used for confirmation that the user typed
+ the password correctly. Generates a field with name
+ "name:confirm".
+now only on Date properties - return the current date as a new
+ property
+reldate only on Date properties - render the interval between the date
+ and now
+local only on Date properties - return this date as a new property
+ with some timezone offset
+pretty only on Interval properties - render the interval in a pretty
+ format (eg. "yesterday")
+menu only on Link and Multilink properties - render a form select
+ list for this property
+reverse only on Multilink properties - produce a list of the linked
+ items in reverse order
+=========== ================================================================
+
+
+The request variable
+~~~~~~~~~~~~~~~~~~~~
+
+Note: this is implemented by the ``roundup.cgi.templating.HTMLRequest``
+class.
+
+The request variable is packed with information about the current
+request.
+
+.. taken from ``roundup.cgi.templating.HTMLRequest`` docstring
+
+=========== ============================================================
+Variable Holds
+=========== ============================================================
+form the CGI form as a cgi.FieldStorage
+env the CGI environment variables
+base the base URL for this tracker
+user a HTMLUser instance for this user
+classname the current classname (possibly None)
+template the current template (suffix, also possibly None)
+form the current CGI form variables in a FieldStorage
+=========== ============================================================
+
+**Index page specific variables (indexing arguments)**
+
+=========== ============================================================
+Variable Holds
+=========== ============================================================
+columns dictionary of the columns to display in an index page
+show a convenience access to columns - request/show/colname will
+ be true if the columns should be displayed, false otherwise
+sort index sort column (direction, column name)
+group index grouping property (direction, column name)
+filter properties to filter the index on
+filterspec values to filter the index on
+search_text text to perform a full-text search on for an index
+=========== ============================================================
+
+There are several methods available on the request variable:
+
+=============== ========================================================
+Method Description
+=============== ========================================================
+description render a description of the request - handle for the
+ page title
+indexargs_form render the current index args as form elements
+indexargs_url render the current index args as a URL
+base_javascript render some javascript that is used by other components
+ of the templating
+batch run the current index args through a filter and return a
+ list of items (see `hyperdb item wrapper`_, and
+ `batching`_)
+=============== ========================================================
+
+The form variable
+:::::::::::::::::
+
+The form variable is a bit special because it's actually a python
+FieldStorage object. That means that you have two ways to access its
+contents. For example, to look up the CGI form value for the variable
+"name", use the path expression::
+
+ request/form/name/value
+
+or the python expression::
+
+ python:request.form['name'].value
+
+Note the "item" access used in the python case, and also note the
+explicit "value" attribute we have to access. That's because the form
+variables are stored as MiniFieldStorages. If there's more than one
+"name" value in the form, then the above will break since
+``request/form/name`` is actually a *list* of MiniFieldStorages. So it's
+best to know beforehand what you're dealing with.
+
+
+The db variable
+~~~~~~~~~~~~~~~
+
+Note: this is implemented by the ``roundup.cgi.templating.HTMLDatabase``
+class.
+
+Allows access to all hyperdb classes as attributes of this variable. If
+you want access to the "user" class, for example, you would use::
+
+ db/user
+ python:db.user
+
+Also, the current id of the current user is available as
+``db.getuid()``. This isn't so useful in templates (where you have
+``request/user``), but it can be useful in detectors or interfaces.
-The header and footer are generated by Python code. The default code is in
-roundup.cgi_client.Class. This class is mixed-in to your instance through the
-instance's interfaces module. This means you can override the header and
-footer with your own code. This allows you to use a sidebar navigation scheme,
-for example.
+The access results in a `hyperdb class wrapper`_.
+The templates variable
+~~~~~~~~~~~~~~~~~~~~~~
+
+Note: this is implemented by the ``roundup.cgi.templating.Templates``
+class.
+
+This variable doesn't have any useful methods defined. It supports being
+used in expressions to access the templates, and consequently the
+template macros. You may access the templates using the following path
+expression::
+
+ templates/name
+
+or the python expression::
+
+ templates[name]
+
+where "name" is the name of the template you wish to access. The
+template has one useful attribute, namely "macros". To access a specific
+macro (called "macro_name"), use the path expression::
+
+ templates/name/macros/macro_name
+
+or the python expression::
+
+ templates[name].macros[macro_name]
+
+
+The utils variable
+~~~~~~~~~~~~~~~~~~
+
+Note: this is implemented by the
+``roundup.cgi.templating.TemplatingUtils`` class, but it may be extended
+as described below.
+
+=============== ========================================================
+Method Description
+=============== ========================================================
+Batch return a batch object using the supplied list
+=============== ========================================================
+
+You may add additional utility methods by writing them in your tracker
+``interfaces.py`` module's ``TemplatingUtils`` class. See `adding a time
+log to your issues`_ for an example. The TemplatingUtils class itself
+will have a single attribute, ``client``, which may be used to access
+the ``client.db`` when you need to perform arbitrary database queries.
+
+Batching
+::::::::
+
+Use Batch to turn a list of items, or item ids of a given class, into a
+series of batches. Its usage is::
+
+ python:utils.Batch(sequence, size, start, end=0, orphan=0,
+ overlap=0)
+
+or, to get the current index batch::
+
+ request/batch
+
+The parameters are:
+
+========= ==============================================================
+Parameter Usage
+========= ==============================================================
+sequence a list of HTMLItems
+size how big to make the sequence.
+start where to start (0-indexed) in the sequence.
+end where to end (0-indexed) in the sequence.
+orphan if the next batch would contain less items than this value,
+ then it is combined with this batch
+overlap the number of items shared between adjacent batches
+========= ==============================================================
+
+All of the parameters are assigned as attributes on the batch object. In
+addition, it has several more attributes:
+
+=============== ========================================================
+Attribute Description
+=============== ========================================================
+start indicates the start index of the batch. *Note: unlike
+ the argument, is a 1-based index (I know, lame)*
+first indicates the start index of the batch *as a 0-based
+ index*
+length the actual number of elements in the batch
+sequence_length the length of the original, unbatched, sequence.
+=============== ========================================================
+
+And several methods:
+
+=============== ========================================================
+Method Description
+=============== ========================================================
+previous returns a new Batch with the previous batch settings
+next returns a new Batch with the next batch settings
+propchanged detect if the named property changed on the current item
+ when compared to the last item
+=============== ========================================================
+
+An example of batching::
+
+ <table class="otherinfo">
+ <tr><th colspan="4" class="header">Existing Keywords</th></tr>
+ <tr tal:define="keywords db/keyword/list"
+ tal:repeat="start python:range(0, len(keywords), 4)">
+ <td tal:define="batch python:utils.Batch(keywords, 4, start)"
+ tal:repeat="keyword batch" tal:content="keyword/name">
+ keyword here</td>
+ </tr>
+ </table>
+
+... which will produce a table with four columns containing the items of
+the "keyword" class (well, their "name" anyway).
+
Displaying Properties
-~~~~~~~~~~~~~~~~~~~~~
+---------------------
-Properties appear in the user interface in three contexts: in indices, in
-editors, and as filters. For each type of property, there are several display
-possibilities. For example, in an index view, a string property may just be
-printed as a plain string, but in an editor view, that property should be
-displayed in an editable field.
-
-The display of a property is handled by functions in the htmltemplate module.
-Displayer functions are triggered by <display> tags in templates. The call
-attribute of the tag provides a Python expression for calling the displayer
-function. The three standard arguments are inserted in front of the arguments
-given. For example, the occurrence of::
-
- <display call="plain('status')">
-
-in a template triggers a call the "plain" function. The displayer functions can
-accept extra arguments to further specify details about the widgets that should
-be generated. By defining new displayer functions, the user interface can be
-highly customized.
-
-+-----------------------------------------------------------------------------+
-|The displayer functions are |
-+---------+-------------------------------------------------------------------+
-|plain |Display a String property directly. |
-| |Display a Date property in a specified time zone with an option to |
-| |omit the time from the date stamp. |
-| |For a Link or Multilink property, display the key strings of the |
-| |linked nodes (or the ids if the linked class has no key property). |
-| |Options: |
-| |escape (boolean) - HTML-escape the resulting text. |
-+---------+-------------------------------------------------------------------+
-|field |Display a property like the plain displayer above, but in a form |
-| |field to be edited. Strings, Dates and Intervals use TEXT fields, |
-| |Links use SELECT fields and Multilinks use SELECT MULTIPLE fields. |
-| |Options: |
-| |size (number) - width of TEXT fields. |
-| |height (number) - number of nows in SELECT MULTIPLE tags. |
-| |showid (boolean) - true includes the id of linked nodes in the |
-| |SELECT MULTIPLE fields. |
-+---------+-------------------------------------------------------------------+
-|menu |For a Links and Multilinks, display the same field as would be |
-| |generated using field. |
-+---------+-------------------------------------------------------------------+
-|link |For a Link or Multilink property, display the names of the linked |
-| |nodes, hyperlinked to the item views on those nodes. |
-| |For other properties, link to this node with the property as the |
-| |text. |
-| |Options: |
-| |property (property name) - the property to use in the second case. |
-| |showid - use the linked node id as the link text (linked node |
-| |"value" will be set as a tooltip) |
-+---------+-------------------------------------------------------------------+
-|count |For a Multilink property, display a count of the number of links in|
-| |the list. |
-| |Arguments: |
-| |property (property name) - the property to use. |
-+---------+-------------------------------------------------------------------+
-|reldate |Display a Date property in terms of an interval relative to the |
-| |current date (e.g. "+ 3w", "- 2d"). |
-| |Arguments: |
-| |property (property name) - the property to use. |
-| |Options: |
-| |pretty (boolean) - display the relative date in an English form. |
-+---------+-------------------------------------------------------------------+
-|download |For a Link or Multilink property, display the names of the linked |
-| |nodes, hyperlinked to the item views on those nodes. |
-| |For other properties, link to this node with the property as the |
-| |text. |
-| |In all cases, append the name (key property) of the item to the |
-| |path so it is the name of the file being downloaded. |
-| |Arguments: |
-| |property (property name) - the property to use. |
-+---------+-------------------------------------------------------------------+
-|checklist|For a Link or Multilink property, display checkboxes for the |
-| |available choices to permit filtering. |
-| |Arguments: |
-| |property (property name) - the property to use. |
-+---------+-------------------------------------------------------------------+
-|note |Display the special notes field, which is a text area for entering |
-| |a note to go along with a change. |
-+---------+-------------------------------------------------------------------+
-|list |List the nodes specified by property using the standard index for |
-| |the class. |
-| |Arguments: |
-| |property (property name) - the property to use. |
-+---------+-------------------------------------------------------------------+
-|history |List the history of the item. |
-+---------+-------------------------------------------------------------------+
-|submit |Add a submit button for the item. |
-+---------+-------------------------------------------------------------------+
+Properties appear in the user interface in three contexts: in indices,
+in editors, and as search arguments. For each type of property, there
+are several display possibilities. For example, in an index view, a
+string property may just be printed as a plain string, but in an editor
+view, that property may be displayed in an editable field.
Index Views
-~~~~~~~~~~~
+-----------
+
+This is one of the class context views. It is also the default view for
+classes. The template used is "*classname*.index".
-An index view contains two sections: a filter section and an index section. The
-filter section provides some widgets for selecting which items appear in the
-index. The index section is a table of items.
Index View Specifiers
-:::::::::::::::::::::
+~~~~~~~~~~~~~~~~~~~~~
-An index view specifier (URL fragment) looks like this (whitespace has been
-added for clarity)::
+An index view specifier (URL fragment) looks like this (whitespace has
+been added for clarity)::
/issue?status=unread,in-progress,resolved&
- topic=security,ui&
- :group=+priority&
- :sort=-activity&
- :filters=status,topic&
- :columns=title,status,fixer
-
-The index view is determined by two parts of the specifier: the layout part and
-the filter part. The layout part consists of the query parameters that begin
-with colons, and it determines the way that the properties of selected nodes
-are displayed. The filter part consists of all the other query parameters, and
-it determines the criteria by which nodes are selected for display.
-The filter part is interactively manipulated with the form widgets displayed in
-the filter section. The layout part is interactively manipulated by clicking on
-the column headings in the table.
-
-The filter part selects the union of the sets of items with values matching any
-specified Link properties and the intersection of the sets of items with values
-matching any specified Multilink properties.
-
-The example specifies an index of "issue" nodes. Only items with a "status" of
-either "unread" or "in-progres" or "resolved" are displayed, and only items
-with "topic" values including both "security" and "ui" are displayed. The items
-are grouped by priority, arranged in ascending order; and within groups, sorted
-by activity, arranged in descending order. The filter section shows filters for
-the "status" and "topic" properties, and the table includes columns for the
-"title", "status", and "fixer" properties.
-
-Associated with each item class is a default layout specifier. The layout
-specifier in the above example is the default layout to be provided with the
-default bug-tracker schema described above in section 4.4.
-
-Filter Section
-::::::::::::::
-
-The template for a filter section provides the filtering widgets at the top of
-the index view. Fragments enclosed in <property>...</property> tags are
-included or omitted depending on whether the view specifier requests a filter
-for a particular property.
-
-A property must appear in the filter template for it to be available as a
-filter.
-
-Here's a simple example of a filter template.::
-
- <property name=status>
- <display call="checklist('status')">
- </property>
- <br>
- <property name=priority>
- <display call="checklist('priority')">
- </property>
- <br>
- <property name=fixer>
- <display call="menu('fixer')">
- </property>
-
-The standard index generation code appends a section to the index pages which
-allows selection of the filters - from those which are defined in the filter
-template.
+ topic=security,ui&
+ :group=+priority&
+ :sort==activity&
+ :filters=status,topic&
+ :columns=title,status,fixer
+
+The index view is determined by two parts of the specifier: the layout
+part and the filter part. The layout part consists of the query
+parameters that begin with colons, and it determines the way that the
+properties of selected items are displayed. The filter part consists of
+all the other query parameters, and it determines the criteria by which
+items are selected for display. The filter part is interactively
+manipulated with the form widgets displayed in the filter section. The
+layout part is interactively manipulated by clicking on the column
+headings in the table.
+
+The filter part selects the union of the sets of items with values
+matching any specified Link properties and the intersection of the sets
+of items with values matching any specified Multilink properties.
+
+The example specifies an index of "issue" items. Only items with a
+"status" of either "unread" or "in-progress" or "resolved" are
+displayed, and only items with "topic" values including both "security"
+and "ui" are displayed. The items are grouped by priority, arranged in
+ascending order; and within groups, sorted by activity, arranged in
+descending order. The filter section shows filters for the "status" and
+"topic" properties, and the table includes columns for the "title",
+"status", and "fixer" properties.
+
+Searching Views
+---------------
-Index Section
-:::::::::::::
+Note: if you add a new column to the ``:columns`` form variable
+ potentials then you will need to add the column to the appropriate
+ `index views`_ template so that it is actually displayed.
+
+This is one of the class context views. The template used is typically
+"*classname*.search". The form on this page should have "search" as its
+``@action`` variable. The "search" action:
+
+- sets up additional filtering, as well as performing indexed text
+ searching
+- sets the ``:filter`` variable correctly
+- saves the query off if ``:query_name`` is set.
+
+The search page should lay out any fields that you wish to allow the
+user to search on. If your schema contains a large number of properties,
+you should be wary of making all of those properties available for
+searching, as this can cause confusion. If the additional properties are
+Strings, consider having their value indexed, and then they will be
+searchable using the full text indexed search. This is both faster, and
+more useful for the end user.
+
+The two special form values on search pages which are handled by the
+"search" action are:
+
+:search_text
+ Text with which to perform a search of the text index. Results from
+ that search will be used to limit the results of other filters (using
+ an intersection operation)
+:query_name
+ If supplied, the search parameters (including :search_text) will be
+ saved off as a the query item and registered against the user's
+ queries property. Note that the *classic* template schema has this
+ ability, but the *minimal* template schema does not.
-The template for an index section describes one row of the index table.
-Fragments enclosed in <property>...</property> tags are included or omitted
-depending on whether the view specifier requests a column for a particular
-property. The table cells should contain <display> tags to display the values
-of the item's properties.
-Here's a simple example of an index template.::
+Item Views
+----------
- <tr>
- <property name=title>
- <td><display call="plain('title', max=50)"></td>
- </property>
- <property name=status>
- <td><display call="plain('status')"></td>
- </property>
- <property name=fixer>
- <td><display call="plain('fixer')"></td>
- </property>
- </tr>
+The basic view of a hyperdb item is provided by the "*classname*.item"
+template. It generally has three sections; an "editor", a "spool" and a
+"history" section.
-Sorting
-:::::::
-String and Date values are sorted in the natural way. Link properties are
-sorted according to the value of the "order" property on the linked nodes if it
-is present; or otherwise on the key string of the linked nodes; or finally on
-the node ids. Multilink properties are sorted according to how many links are
-present.
+Editor Section
+~~~~~~~~~~~~~~
+
+The editor section is used to manipulate the item - it may be a static
+display if the user doesn't have permission to edit the item.
+
+Here's an example of a basic editor template (this is the default
+"classic" template issue item edit form - from the "issue.item.html"
+template)::
+
+ <table class="form">
+ <tr>
+ <th>Title</th>
+ <td colspan="3" tal:content="structure python:context.title.field(size=60)">title</td>
+ </tr>
+
+ <tr>
+ <th>Priority</th>
+ <td tal:content="structure context/priority/menu">priority</td>
+ <th>Status</th>
+ <td tal:content="structure context/status/menu">status</td>
+ </tr>
+
+ <tr>
+ <th>Superseder</th>
+ <td>
+ <span tal:replace="structure python:context.superseder.field(showid=1, size=20)" />
+ <span tal:replace="structure python:db.issue.classhelp('id,title')" />
+ <span tal:condition="context/superseder">
+ <br>View: <span tal:replace="structure python:context.superseder.link(showid=1)" />
+ </span>
+ </td>
+ <th>Nosy List</th>
+ <td>
+ <span tal:replace="structure context/nosy/field" />
+ <span tal:replace="structure python:db.user.classhelp('username,realname,address,phone')" />
+ </td>
+ </tr>
+
+ <tr>
+ <th>Assigned To</th>
+ <td tal:content="structure context/assignedto/menu">
+ assignedto menu
+ </td>
+ <td> </td>
+ <td> </td>
+ </tr>
+
+ <tr>
+ <th>Change Note</th>
+ <td colspan="3">
+ <textarea name=":note" wrap="hard" rows="5" cols="60"></textarea>
+ </td>
+ </tr>
+
+ <tr>
+ <th>File</th>
+ <td colspan="3"><input type="file" name=":file" size="40"></td>
+ </tr>
+
+ <tr>
+ <td> </td>
+ <td colspan="3" tal:content="structure context/submit">
+ submit button will go here
+ </td>
+ </tr>
+ </table>
-Item Views
-~~~~~~~~~~
-An item view contains an editor section and a spool section. At the top of an
-item view, links to superseding and superseded items are always displayed.
+When a change is submitted, the system automatically generates a message
+describing the changed properties. As shown in the example, the editor
+template can use the ":note" and ":file" fields, which are added to the
+standard changenote message generated by Roundup.
-Editor Section
-::::::::::::::
-The editor section is generated from a template containing <display> tags to
-insert the appropriate widgets for editing properties.
+Form values
+:::::::::::
-Here's an example of a basic editor template.::
+We have a number of ways to pull properties out of the form in order to
+meet the various needs of:
+
+1. editing the current item (perhaps an issue item)
+2. editing information related to the current item (eg. messages or
+ attached files)
+3. creating new information to be linked to the current item (eg. time
+ spent on an issue)
+
+In the following, ``<bracketed>`` values are variable, ":" may be one of
+":" or "@", and other text ("required") is fixed.
+
+Properties are specified as form variables:
+
+``<propname>``
+ property on the current context item
+
+``<designator>:<propname>``
+ property on the indicated item (for editing related information)
+
+``<classname>-<N>:<propname>``
+ property on the Nth new item of classname (generally for creating new
+ items to attach to the current item)
+
+Once we have determined the "propname", we check to see if it is one of
+the special form values:
+
+``@required``
+ The named property values must be supplied or a ValueError will be
+ raised.
+
+``@remove@<propname>=id(s)``
+ The ids will be removed from the multilink property.
+
+``:add:<propname>=id(s)``
+ The ids will be added to the multilink property.
+
+``:link:<propname>=<designator>``
+ Used to add a link to new items created during edit. These are
+ collected and returned in ``all_links``. This will result in an
+ additional linking operation (either Link set or Multilink append)
+ after the edit/create is done using ``all_props`` in ``_editnodes``.
+ The <propname> on the current item will be set/appended the id of the
+ newly created item of class <designator> (where <designator> must be
+ <classname>-<N>).
+
+Any of the form variables may be prefixed with a classname or
+designator.
+
+Two special form values are supported for backwards compatibility:
+
+``:note``
+ create a message (with content, author and date), linked to the
+ context item. This is ALWAYS designated "msg-1".
+``:file``
+ create a file, attached to the current item and any message created by
+ :note. This is ALWAYS designated "file-1".
+
+
+Spool Section
+~~~~~~~~~~~~~
+
+The spool section lists related information like the messages and files
+of an issue.
+
+TODO
+
+
+History Section
+~~~~~~~~~~~~~~~
+
+The final section displayed is the history of the item - its database
+journal. This is generally generated with the template::
+
+ <tal:block tal:replace="structure context/history" />
+
+*To be done:*
+
+*The actual history entries of the item may be accessed for manual
+templating through the "journal" method of the item*::
+
+ <tal:block tal:repeat="entry context/journal">
+ a journal entry
+ </tal:block>
+
+*where each journal entry is an HTMLJournalEntry.*
+
+Defining new web actions
+------------------------
+
+You may define new actions to be triggered by the ``@action`` form
+variable. These are added to the tracker ``interfaces.py`` as methods on
+the ``Client`` class.
+
+Adding action methods takes three steps; first you `define the new
+action method`_, then you `register the action method`_ with the cgi
+interface so it may be triggered by the ``@action`` form variable.
+Finally you `use the new action`_ in your HTML form.
+
+See "`setting up a "wizard" (or "druid") for controlled adding of
+issues`_" for an example.
+
+
+Define the new action method
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The action methods have the following interface::
+
+ def myActionMethod(self):
+ ''' Perform some action. No return value is required.
+ '''
+
+The *self* argument is an instance of your tracker ``instance.Client``
+class - thus it's mostly implemented by ``roundup.cgi.Client``. See the
+docstring of that class for details of what it can do.
+
+The method will typically check the ``self.form`` variable's contents.
+It may then:
+
+- add information to ``self.ok_message`` or ``self.error_message``
+- change the ``self.template`` variable to alter what the user will see
+ next
+- raise Unauthorised, SendStaticFile, SendFile, NotFound or Redirect
+ exceptions
+
+
+Register the action method
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The method is now written, but isn't available to the user until you add
+it to the `instance.Client`` class ``actions`` variable, like so::
+
+ actions = client.Class.actions + (
+ ('myaction', 'myActionMethod'),
+ )
+
+This maps the action name "myaction" to the action method we defined.
+
+
+Use the new action
+~~~~~~~~~~~~~~~~~~
+
+In your HTML form, add a hidden form element like so::
+
+ <input type="hidden" name="@action" value="myaction">
+
+where "myaction" is the name you registered in the previous step.
+
+
+Examples
+========
+
+.. contents::
+ :local:
+ :depth: 1
+
+
+Adding a new field to the classic schema
+----------------------------------------
+
+This example shows how to add a new constrained property (i.e. a
+selection of distinct values) to your tracker.
+
+
+Introduction
+~~~~~~~~~~~~
+
+To make the classic schema of roundup useful as a TODO tracking system
+for a group of systems administrators, it needed an extra data field per
+issue: a category.
+
+This would let sysadmins quickly list all TODOs in their particular area
+of interest without having to do complex queries, and without relying on
+the spelling capabilities of other sysadmins (a losing proposition at
+best).
+
+
+Adding a field to the database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is the easiest part of the change. The category would just be a
+plain string, nothing fancy. To change what is in the database you need
+to add some lines to the ``open()`` function in ``dbinit.py``. Under the
+comment::
+
+ # add any additional database schema configuration here
+
+add::
+
+ category = Class(db, "category", name=String())
+ category.setkey("name")
+
+Here we are setting up a chunk of the database which we are calling
+"category". It contains a string, which we are refering to as "name" for
+lack of a more imaginative title. (Since "name" is one of the properties
+that Roundup looks for on items if you do not set a key for them, it's
+probably a good idea to stick with it for new classes if at all
+appropriate.) Then we are setting the key of this chunk of the database
+to be that "name". This is equivalent to an index for database types.
+This also means that there can only be one category with a given name.
+
+Adding the above lines allows us to create categories, but they're not
+tied to the issues that we are going to be creating. It's just a list of
+categories off on its own, which isn't much use. We need to link it in
+with the issues. To do that, find the lines in the ``open()`` function
+in ``dbinit.py`` which set up the "issue" class, and then add a link to
+the category::
+
+ issue = IssueClass(db, "issue", ... ,
+ category=Multilink("category"), ... )
+
+The ``Multilink()`` means that each issue can have many categories. If
+you were adding something with a one-to-one relationship to issues (such
+as the "assignedto" property), use ``Link()`` instead.
+
+That is all you need to do to change the schema. The rest of the effort
+is fiddling around so you can actually use the new category.
+
+
+Populating the new category class
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you haven't initialised the database with the roundup-admin
+"initialise" command, then you can add the following to the tracker
+``dbinit.py`` in the ``init()`` function under the comment::
+
+ # add any additional database create steps here - but only if you
+ # haven't initialised the database with the admin "initialise" command
+
+Add::
+
+ category = db.getclass('category')
+ category.create(name="scipy", order="1")
+ category.create(name="chaco", order="2")
+ category.create(name="weave", order="3")
+
+If the database has already been initalised, then you need to use the
+``roundup-admin`` tool::
+
+ % roundup-admin -i <tracker home>
+ Roundup <version> ready for input.
+ Type "help" for help.
+ roundup> create category name=scipy order=1
+ 1
+ roundup> create category name=chaco order=1
+ 2
+ roundup> create category name=weave order=1
+ 3
+ roundup> exit...
+ There are unsaved changes. Commit them (y/N)? y
+
+TODO: explain why order=1 in each case. Also, does key get set to "name"
+automatically when added via roundup-admin?
+
+
+Setting up security on the new objects
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+By default only the admin user can look at and change objects. This
+doesn't suit us, as we want any user to be able to create new categories
+as required, and obviously everyone needs to be able to view the
+categories of issues for it to be useful.
+
+We therefore need to change the security of the category objects. This
+is also done in the ``open()`` function of ``dbinit.py``.
+
+There are currently two loops which set up permissions and then assign
+them to various roles. Simply add the new "category" to both lists::
+
+ # new permissions for this schema
+ for cl in 'issue', 'file', 'msg', 'user', 'category':
+ db.security.addPermission(name="Edit", klass=cl,
+ description="User is allowed to edit "+cl)
+ db.security.addPermission(name="View", klass=cl,
+ description="User is allowed to access "+cl)
+
+ # Assign the access and edit permissions for issue, file and message
+ # to regular users now
+ for cl in 'issue', 'file', 'msg', 'category':
+ p = db.security.getPermission('View', cl)
+ db.security.addPermissionToRole('User', p)
+ p = db.security.getPermission('Edit', cl)
+ db.security.addPermissionToRole('User', p)
+
+So you are in effect doing the following (with 'cl' substituted by its
+value)::
+
+ db.security.addPermission(name="Edit", klass='category',
+ description="User is allowed to edit "+'category')
+ db.security.addPermission(name="View", klass='category',
+ description="User is allowed to access "+'category')
+
+which is creating two permission types; that of editing and viewing
+"category" objects respectively. Then the following lines assign those
+new permissions to the "User" role, so that normal users can view and
+edit "category" objects::
+
+ p = db.security.getPermission('View', 'category')
+ db.security.addPermissionToRole('User', p)
+
+ p = db.security.getPermission('Edit', 'category')
+ db.security.addPermissionToRole('User', p)
+
+This is all the work that needs to be done for the database. It will
+store categories, and let users view and edit them. Now on to the
+interface stuff.
+
+
+Changing the web left hand frame
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We need to give the users the ability to create new categories, and the
+place to put the link to this functionality is in the left hand function
+bar, under the "Issues" area. The file that defines how this area looks
+is ``html/page``, which is what we are going to be editing next.
+
+If you look at this file you can see that it contains a lot of
+"classblock" sections which are chunks of HTML that will be included or
+excluded in the output depending on whether the condition in the
+classblock is met. Under the end of the classblock for issue is where we
+are going to add the category code::
+
+ <p class="classblock"
+ tal:condition="python:request.user.hasPermission('View', 'category')">
+ <b>Categories</b><br>
+ <a tal:condition="python:request.user.hasPermission('Edit', 'category')"
+ href="category?@template=item">New Category<br></a>
+ </p>
+
+The first two lines is the classblock definition, which sets up a
+condition that only users who have "View" permission for the "category"
+object will have this section included in their output. Next comes a
+plain "Categories" header in bold. Everyone who can view categories will
+get that.
+
+Next comes the link to the editing area of categories. This link will
+only appear if the condition - that the user has "Edit" permissions for
+the "category" objects - is matched. If they do have permission then
+they will get a link to another page which will let the user add new
+categories.
+
+Note that if you have permission to *view* but not to *edit* categories,
+then all you will see is a "Categories" header with nothing underneath
+it. This is obviously not very good interface design, but will do for
+now. I just claim that it is so I can add more links in this section
+later on. However to fix the problem you could change the condition in
+the classblock statement, so that only users with "Edit" permission
+would see the "Categories" stuff.
+
+
+Setting up a page to edit categories
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We defined code in the previous section which let users with the
+appropriate permissions see a link to a page which would let them edit
+conditions. Now we have to write that page.
+
+The link was for the *item* template of the *category* object. This
+translates into Roundup looking for a file called ``category.item.html``
+in the ``html`` tracker directory. This is the file that we are going to
+write now.
+
+First we add an info tag in a comment which doesn't affect the outcome
+of the code at all, but is useful for debugging. If you load a page in a
+browser and look at the page source, you can see which sections come
+from which files by looking for these comments::
+
+ <!-- category.item -->
+
+Next we need to add in the METAL macro stuff so we get the normal page
+trappings::
+
+ <tal:block metal:use-macro="templates/page/macros/icing">
+ <title metal:fill-slot="head_title">Category editing</title>
+ <td class="page-header-top" metal:fill-slot="body_title">
+ <h2>Category editing</h2>
+ </td>
+ <td class="content" metal:fill-slot="content">
+
+Next we need to setup up a standard HTML form, which is the whole
+purpose of this file. We link to some handy javascript which sends the
+form through only once. This is to stop users hitting the send button
+multiple times when they are impatient and thus having the form sent
+multiple times::
+
+ <form method="POST" onSubmit="return submit_once()"
+ enctype="multipart/form-data">
+
+Next we define some code which sets up the minimum list of fields that
+we require the user to enter. There will be only one field - "name" - so
+they better put something in it, otherwise the whole form is pointless::
+
+ <input type="hidden" name="@required" value="name">
+
+To get everything to line up properly we will put everything in a table,
+and put a nice big header on it so the user has an idea what is
+happening::
+
+ <table class="form">
+ <tr><th class="header" colspan="2">Category</th></tr>
+
+Next, we need the field into which the user is going to enter the new
+category. The "context.name.field(size=60)" bit tells Roundup to
+generate a normal HTML field of size 60, and the contents of that field
+will be the "name" variable of the current context (which is
+"category"). The upshot of this is that when the user types something in
+to the form, a new category will be created with that name::
+
+ <tr>
+ <th>Name</th>
+ <td tal:content="structure python:context.name.field(size=60)">
+ name</td>
+ </tr>
+
+Then a submit button so that the user can submit the new category::
+
+ <tr>
+ <td> </td>
+ <td colspan="3" tal:content="structure context/submit">
+ submit button will go here
+ </td>
+ </tr>
+
+Finally we finish off the tags we used at the start to do the METAL
+stuff::
+
+ </td>
+ </tal:block>
+
+So putting it all together, and closing the table and form we get::
+
+ <!-- category.item -->
+ <tal:block metal:use-macro="templates/page/macros/icing">
+ <title metal:fill-slot="head_title">Category editing</title>
+ <td class="page-header-top" metal:fill-slot="body_title">
+ <h2>Category editing</h2>
+ </td>
+ <td class="content" metal:fill-slot="content">
+ <form method="POST" onSubmit="return submit_once()"
+ enctype="multipart/form-data">
+
+ <table class="form">
+ <tr><th class="header" colspan="2">Category</th></tr>
- <table>
- <tr>
- <td colspan=2>
- <display call="field('title', size=60)">
- </td>
- </tr>
- <tr>
- <td>
- <display call="field('fixer', size=30)">
- </td>
- <td>
- <display call="menu('status')>
- </td>
- </tr>
<tr>
- <td>
- <display call="field('nosy', size=30)">
- </td>
- <td>
- <display call="menu('priority')>
- </td>
+ <th>Name</th>
+ <td tal:content="structure python:context.name.field(size=60)">
+ name</td>
</tr>
+
<tr>
- <td colspan=2>
- <display call="note()">
- </td>
+ <td>
+
+ <input type="hidden" name="@required" value="name">
+ </td>
+ <td colspan="3" tal:content="structure context/submit">
+ submit button will go here
+ </td>
</tr>
- </table>
+ </table>
+ </form>
+ </td>
+ </tal:block>
-As shown in the example, the editor template can also request the display of a
-"note" field, which is a text area for entering a note to go along with a
-change.
+This is quite a lot to just ask the user one simple question, but there
+is a lot of setup for basically one line (the form line) to do its work.
+To add another field to "category" would involve one more line (well,
+maybe a few extra to get the formatting correct).
-The <property> tag used in the index may also be used here - it checks to see
-if the nominated Multilink property has any entries. This can be used to
-eliminate sections of the editor section if the property has no entries::
- <td class="form-text">
- <display call="field('superseder', size=40, showid=1)">
- <display call="classhelp('issue', 'id,title', label='list', width=500)">
- <property name="superseder">
- <br>View: <display call="link('superseder', showid=1)">
- </property>
- </td>
+Adding the category to the issue
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The "View: " part with the links will only display if the superseder property
-has values.
+We now have the ability to create issues to our heart's content, but
+that is pointless unless we can assign categories to issues. Just like
+the ``html/category.item.html`` file was used to define how to add a new
+category, the ``html/issue.item.html`` is used to define how a new issue
+is created.
-When a change is submitted, the system automatically generates a message
-describing the changed properties.
+Just like ``category.issue.html`` this file defines a form which has a
+table to lay things out. It doesn't matter where in the table we add new
+stuff, it is entirely up to your sense of aesthetics::
-If a note is given in the "note" field, the note is appended to the
-description. The message is then added to the item's message spool (thus
-triggering the standard detector to react by sending out this message to the
-nosy list).
+ <th>Category</th>
+ <td><span tal:replace="structure context/category/field" />
+ <span tal:replace="structure db/category/classhelp" />
+ </td>
-The message also displays all of the property values on the item and indicates
-which ones have changed. An example of such a message might be this::
+First, we define a nice header so that the user knows what the next
+section is, then the middle line does what we are most interested in.
+This ``context/category/field`` gets replaced by a field which contains
+the category in the current context (the current context being the new
+issue).
- Polly's taken a turn for the worse - this is now really important!
- -----
- title: Polly Parrot is dead
- priority: critical
- status: unread -> in-progress
- fixer: terry
- keywords: parrot,plumage,perch,nailed,dead
+The classhelp lines generate a link (labelled "list") to a popup window
+which contains the list of currently known categories.
-Spool Section
-:::::::::::::
-The spool section lists messages in the item's "messages" property. The index
-of messages displays the "date", "author", and "summary" properties on the
-message nodes, and selecting a message takes you to its content.
+Searching on categories
+~~~~~~~~~~~~~~~~~~~~~~~
-The <property> tag used in the index may also be used here - it checks to see
-if the nominated Multilink property has any entries. This can be used to
-eliminate sections of the spool section if the property has no entries::
+We can add categories, and create issues with categories. The next
+obvious thing that we would like to be able to do, would be to search
+for issues based on their category, so that, for example, anyone working
+on the web server could look at all issues in the category "Web".
- <property name="files">
- <tr class="strong-header">
- <td><b>Files</b></td>
- </tr>
+If you look for "Search Issues" in the 'html/page.html' file, you will
+find that it looks something like
+``<a href="issue?@template=search">Search Issues</a>``. This shows us
+that when you click on "Search Issues" it will be looking for a
+``issue.search.html`` file to display. So that is the file that we will
+change.
- <tr>
- <td><display call="list('files')"></td>
- </tr>
- </property>
+If you look at this file it should be starting to seem familiar, although it
+does use some new macros. You can add the new category search code anywhere you
+like within that form::
+
+ <tr tal:define="name string:category;
+ db_klass string:category;
+ db_content string:name;">
+ <th>Priority:</th>
+ <td metal:use-macro="search_select"></td>
+ <td metal:use-macro="column_input"></td>
+ <td metal:use-macro="sort_input"></td>
+ <td metal:use-macro="group_input"></td>
+ </tr>
+The definitions in the <tr> opening tag are used by the macros:
-Access Controls
----------------
+- search_select expands to a drop-down box with all categories using db_klass
+ and db_content.
+- column_input expands to a checkbox for selecting what columns should be
+ displayed.
+- sort_input expands to a radio button for selecting what property should be
+ sorted on.
+- group_input expands to a radio button for selecting what property should be
+ group on.
+
+The category search code above would expand to the following::
+
+ <tr>
+ <th>Category:</th>
+ <td>
+ <select name="category">
+ <option value="">don't care</option>
+ <option value="">------------</option>
+ <option value="1">scipy</option>
+ <option value="2">chaco</option>
+ <option value="3">weave</option>
+ </select>
+ </td>
+ <td><input type="checkbox" name=":columns" value="category"></td>
+ <td><input type="radio" name=":sort" value="category"></td>
+ <td><input type="radio" name=":group" value="category"></td>
+ </tr>
+
+Adding category to the default view
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We can now add categories, add issues with categories, and search for
+issues based on categories. This is everything that we need to do;
+however, there is some more icing that we would like. I think the
+category of an issue is important enough that it should be displayed by
+default when listing all the issues.
+
+Unfortunately, this is a bit less obvious than the previous steps. The
+code defining how the issues look is in ``html/issue.index.html``. This
+is a large table with a form down at the bottom for redisplaying and so
+forth.
+
+Firstly we need to add an appropriate header to the start of the table::
+
+ <th tal:condition="request/show/category">Category</th>
+
+The *condition* part of this statement is to avoid displaying the
+Category column if the user has selected not to see it.
+
+The rest of the table is a loop which will go through every issue that
+matches the display criteria. The loop variable is "i" - which means
+that every issue gets assigned to "i" in turn.
+
+The new part of code to display the category will look like this::
+
+ <td tal:condition="request/show/category"
+ tal:content="i/category"></td>
+
+The condition is the same as above: only display the condition when the
+user hasn't asked for it to be hidden. The next part is to set the
+content of the cell to be the category part of "i" - the current issue.
+
+Finally we have to edit ``html/page.html`` again. This time, we need to
+tell it that when the user clicks on "Unasigned Issues" or "All Issues",
+the category column should be included in the resulting list. If you
+scroll down the page file, you can see the links with lots of options.
+The option that we are interested in is the ``:columns=`` one which
+tells roundup which fields of the issue to display. Simply add
+"category" to that list and it all should work.
+
+
+Adding in state transition control
+----------------------------------
+
+Sometimes tracker admins want to control the states that users may move
+issues to. You can do this by following these steps:
+
+1. make "status" a required variable. This is achieved by adding the
+ following to the top of the form in the ``issue.item.html``
+ template::
+
+ <input type="hidden" name="@required" value="status">
+
+ this will force users to select a status.
+
+2. add a Multilink property to the status class::
+
+ stat = Class(db, "status", ... , transitions=Multilink('status'),
+ ...)
+
+ and then edit the statuses already created, either:
+
+ a. through the web using the class list -> status class editor, or
+ b. using the roundup-admin "set" command.
+
+3. add an auditor module ``checktransition.py`` in your tracker's
+ ``detectors`` directory, for example::
+
+ def checktransition(db, cl, nodeid, newvalues):
+ ''' Check that the desired transition is valid for the "status"
+ property.
+ '''
+ if not newvalues.has_key('status'):
+ return
+ current = cl.get(nodeid, 'status')
+ new = newvalues['status']
+ if new == current:
+ return
+ ok = db.status.get(current, 'transitions')
+ if new not in ok:
+ raise ValueError, 'Status not allowed to move from "%s" to "%s"'%(
+ db.status.get(current, 'name'), db.status.get(new, 'name'))
+
+ def init(db):
+ db.issue.audit('set', checktransition)
+
+4. in the ``issue.item.html`` template, change the status editing bit
+ from::
+
+ <th>Status</th>
+ <td tal:content="structure context/status/menu">status</td>
+
+ to::
+
+ <th>Status</th>
+ <td>
+ <select tal:condition="context/id" name="status">
+ <tal:block tal:define="ok context/status/transitions"
+ tal:repeat="state db/status/list">
+ <option tal:condition="python:state.id in ok"
+ tal:attributes="
+ value state/id;
+ selected python:state.id == context.status.id"
+ tal:content="state/name"></option>
+ </tal:block>
+ </select>
+ <tal:block tal:condition="not:context/id"
+ tal:replace="structure context/status/menu" />
+ </td>
+
+ which displays only the allowed status to transition to.
+
+
+Displaying only message summaries in the issue display
+------------------------------------------------------
+
+Alter the issue.item template section for messages to::
+
+ <table class="messages" tal:condition="context/messages">
+ <tr><th colspan="5" class="header">Messages</th></tr>
+ <tr tal:repeat="msg context/messages">
+ <td><a tal:attributes="href string:msg${msg/id}"
+ tal:content="string:msg${msg/id}"></a></td>
+ <td tal:content="msg/author">author</td>
+ <td class="date" tal:content="msg/date/pretty">date</td>
+ <td tal:content="msg/summary">summary</td>
+ <td>
+ <a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">
+ remove</a>
+ </td>
+ </tr>
+ </table>
+
+Restricting the list of users that are assignable to a task
+-----------------------------------------------------------
+
+1. In your tracker's "dbinit.py", create a new Role, say "Developer"::
+
+ db.security.addRole(name='Developer', description='A developer')
+
+2. Just after that, create a new Permission, say "Fixer", specific to
+ "issue"::
+
+ p = db.security.addPermission(name='Fixer', klass='issue',
+ description='User is allowed to be assigned to fix issues')
+
+3. Then assign the new Permission to your "Developer" Role::
+
+ db.security.addPermissionToRole('Developer', p)
+
+4. In the issue item edit page ("html/issue.item.html" in your tracker
+ directory), use the new Permission in restricting the "assignedto"
+ list::
+
+ <select name="assignedto">
+ <option value="-1">- no selection -</option>
+ <tal:block tal:repeat="user db/user/list">
+ <option tal:condition="python:user.hasPermission(
+ 'Fixer', context._classname)"
+ tal:attributes="
+ value user/id;
+ selected python:user.id == context.assignedto"
+ tal:content="user/realname"></option>
+ </tal:block>
+ </select>
+
+For extra security, you may wish to setup an auditor to enforce the
+Permission requirement (install this as "assignedtoFixer.py" in your
+tracker "detectors" directory)::
+
+ def assignedtoMustBeFixer(db, cl, nodeid, newvalues):
+ ''' Ensure the assignedto value in newvalues is a used with the
+ Fixer Permission
+ '''
+ if not newvalues.has_key('assignedto'):
+ # don't care
+ return
+
+ # get the userid
+ userid = newvalues['assignedto']
+ if not db.security.hasPermission('Fixer', userid, cl.classname):
+ raise ValueError, 'You do not have permission to edit %s'%cl.classname
+
+ def init(db):
+ db.issue.audit('set', assignedtoMustBeFixer)
+ db.issue.audit('create', assignedtoMustBeFixer)
+
+So now, if an edit action attempts to set "assignedto" to a user that
+doesn't have the "Fixer" Permission, the error will be raised.
+
+
+Setting up a "wizard" (or "druid") for controlled adding of issues
+------------------------------------------------------------------
+
+1. Set up the page templates you wish to use for data input. My wizard
+ is going to be a two-step process: first figuring out what category
+ of issue the user is submitting, and then getting details specific to
+ that category. The first page includes a table of help, explaining
+ what the category names mean, and then the core of the form::
+
+ <form method="POST" onSubmit="return submit_once()"
+ enctype="multipart/form-data">
+ <input type="hidden" name="@template" value="add_page1">
+ <input type="hidden" name="@action" value="page1submit">
+
+ <strong>Category:</strong>
+ <tal:block tal:replace="structure context/category/menu" />
+ <input type="submit" value="Continue">
+ </form>
+
+ The next page has the usual issue entry information, with the
+ addition of the following form fragments::
+
+ <form method="POST" onSubmit="return submit_once()"
+ enctype="multipart/form-data"
+ tal:condition="context/is_edit_ok"
+ tal:define="cat request/form/category/value">
+
+ <input type="hidden" name="@template" value="add_page2">
+ <input type="hidden" name="@required" value="title">
+ <input type="hidden" name="category" tal:attributes="value cat">
+ .
+ .
+ .
+ </form>
+
+ Note that later in the form, I test the value of "cat" include form
+ elements that are appropriate. For example::
+
+ <tal:block tal:condition="python:cat in '6 10 13 14 15 16 17'.split()">
+ <tr>
+ <th>Operating System</th>
+ <td tal:content="structure context/os/field"></td>
+ </tr>
+ <tr>
+ <th>Web Browser</th>
+ <td tal:content="structure context/browser/field"></td>
+ </tr>
+ </tal:block>
-A set of Permissions are built in to the security module by default:
+ ... the above section will only be displayed if the category is one
+ of 6, 10, 13, 14, 15, 16 or 17.
-- Edit (everything)
-- View (everything)
+3. Determine what actions need to be taken between the pages - these are
+ usually to validate user choices and determine what page is next. Now
+ encode those actions in methods on the ``interfaces.Client`` class
+ and insert hooks to those actions in the "actions" attribute on that
+ class, like so::
-The default interfaces define:
+ actions = client.Client.actions + (
+ ('page1_submit', 'page1SubmitAction'),
+ )
-- Web Registration
-- Web Access
-- Web Roles
-- Email Registration
-- Email Access
+ def page1SubmitAction(self):
+ ''' Verify that the user has selected a category, and then move
+ on to page 2.
+ '''
+ category = self.form['category'].value
+ if category == '-1':
+ self.error_message.append('You must select a category of report')
+ return
+ # everything's ok, move on to the next page
+ self.template = 'add_page2'
-These are hooked into the default Roles:
+4. Use the usual "new" action as the ``@action`` on the final page, and
+ you're done (the standard context/submit method can do this for you).
-- Admin (Edit everything, View everything, Web Roles)
-- User (Web Access, Email Access)
-- Anonymous (Web Registration, Email Registration)
-And finally, the "admin" user gets the "Admin" Role, and the "anonymous" user
-gets the "Anonymous" assigned when the database is initialised on installation.
-The two default schemas then define:
+Using an external password validation source
+--------------------------------------------
-- Edit issue, View issue (both)
-- Edit file, View file (both)
-- Edit msg, View msg (both)
-- Edit support, View support (extended only)
+We have a centrally-managed password changing system for our users. This
+results in a UN*X passwd-style file that we use for verification of
+users. Entries in the file consist of ``name:password`` where the
+password is encrypted using the standard UN*X ``crypt()`` function (see
+the ``crypt`` module in your Python distribution). An example entry
+would be::
-and assign those Permissions to the "User" Role. New users are assigned the
-Roles defined in the config file as:
+ admin:aamrgyQfDFSHw
-- NEW_WEB_USER_ROLES
-- NEW_EMAIL_USER_ROLES
+Each user of Roundup must still have their information stored in the
+Roundup database - we just use the passwd file to check their password.
+To do this, we add the following code to our ``Client`` class in the
+tracker home ``interfaces.py`` module::
-You may alter the configuration variables to change the Role that new web or
-email users get, for example to not give them access to the web interface if
-they register through email.
+ def verifyPassword(self, userid, password):
+ # get the user's username
+ username = self.db.user.get(userid, 'username')
-You may use the ``roundup-admin`` "``security``" command to display the
-current Role and Permission configuration in your instance.
+ # the passwords are stored in the "passwd.txt" file in the
+ # tracker home
+ file = os.path.join(self.db.config.TRACKER_HOME, 'passwd.txt')
-Adding a new Permission
-~~~~~~~~~~~~~~~~~~~~~~~
+ # see if we can find a match
+ for ent in [line.strip().split(':') for line in
+ open(file).readlines()]:
+ if ent[0] == username:
+ return crypt.crypt(password, ent[1][:2]) == ent[1]
-When adding a new Permission, you will need to:
+ # user doesn't exist in the file
+ return 0
-1. add it to your instance's dbinit so it is created
-2. enable it for the Roles that should have it (verify with
- "``roundup-admin security``")
-3. add it to the relevant HTML interface templates
-4. add it to the appropriate xxxPermission methods on in your instance
- interfaces module
+What this does is look through the file, line by line, looking for a
+name that matches.
+
+We also remove the redundant password fields from the ``user.item``
+template.
+Adding a "vacation" flag to users for stopping nosy messages
+------------------------------------------------------------
------------------
+When users go on vacation and set up vacation email bouncing, you'll
+start to see a lot of messages come back through Roundup "Fred is on
+vacation". Not very useful, and relatively easy to stop.
+
+1. add a "vacation" flag to your users::
+
+ user = Class(db, "user",
+ username=String(), password=Password(),
+ address=String(), realname=String(),
+ phone=String(), organisation=String(),
+ alternate_addresses=String(),
+ roles=String(), queries=Multilink("query"),
+ vacation=Boolean())
+
+2. So that users may edit the vacation flags, add something like the
+ following to your ``user.item`` template::
+
+ <tr>
+ <th>On Vacation</th>
+ <td tal:content="structure context/vacation/field">vacation</td>
+ </tr>
+
+3. edit your detector ``nosyreactor.py`` so that the ``nosyreaction()``
+ consists of::
+
+ def nosyreaction(db, cl, nodeid, oldvalues):
+ # send a copy of all new messages to the nosy list
+ for msgid in determineNewMessages(cl, nodeid, oldvalues):
+ try:
+ users = db.user
+ messages = db.msg
+
+ # figure the recipient ids
+ sendto = []
+ r = {}
+ recipients = messages.get(msgid, 'recipients')
+ for recipid in messages.get(msgid, 'recipients'):
+ r[recipid] = 1
+
+ # figure the author's id, and indicate they've received
+ # the message
+ authid = messages.get(msgid, 'author')
+
+ # possibly send the message to the author, as long as
+ # they aren't anonymous
+ if (db.config.MESSAGES_TO_AUTHOR == 'yes' and
+ users.get(authid, 'username') != 'anonymous'):
+ sendto.append(authid)
+ r[authid] = 1
+
+ # now figure the nosy people who weren't recipients
+ nosy = cl.get(nodeid, 'nosy')
+ for nosyid in nosy:
+ # Don't send nosy mail to the anonymous user (that
+ # user shouldn't appear in the nosy list, but just
+ # in case they do...)
+ if users.get(nosyid, 'username') == 'anonymous':
+ continue
+ # make sure they haven't seen the message already
+ if not r.has_key(nosyid):
+ # send it to them
+ sendto.append(nosyid)
+ recipients.append(nosyid)
+
+ # generate a change note
+ if oldvalues:
+ note = cl.generateChangeNote(nodeid, oldvalues)
+ else:
+ note = cl.generateCreateNote(nodeid)
+
+ # we have new recipients
+ if sendto:
+ # filter out the people on vacation
+ sendto = [i for i in sendto
+ if not users.get(i, 'vacation', 0)]
+
+ # map userids to addresses
+ sendto = [users.get(i, 'address') for i in sendto]
+
+ # update the message's recipients list
+ messages.set(msgid, recipients=recipients)
+
+ # send the message
+ cl.send_message(nodeid, msgid, note, sendto)
+ except roundupdb.MessageSendError, message:
+ raise roundupdb.DetectorError, message
+
+ Note that this is the standard nosy reaction code, with the small
+ addition of::
+
+ # filter out the people on vacation
+ sendto = [i for i in sendto if not users.get(i, 'vacation', 0)]
+
+ which filters out the users that have the vacation flag set to true.
+
+
+Adding a time log to your issues
+--------------------------------
+
+We want to log the dates and amount of time spent working on issues, and
+be able to give a summary of the total time spent on a particular issue.
+
+1. Add a new class to your tracker ``dbinit.py``::
+
+ # storage for time logging
+ timelog = Class(db, "timelog", period=Interval())
+
+ Note that we automatically get the date of the time log entry
+ creation through the standard property "creation".
+
+2. Link to the new class from your issue class (again, in
+ ``dbinit.py``)::
+
+ issue = IssueClass(db, "issue",
+ assignedto=Link("user"), topic=Multilink("keyword"),
+ priority=Link("priority"), status=Link("status"),
+ times=Multilink("timelog"))
+
+ the "times" property is the new link to the "timelog" class.
+
+3. We'll need to let people add in times to the issue, so in the web
+ interface we'll have a new entry field. This is a special field
+ because unlike the other fields in the issue.item template, it
+ affects a different item (a timelog item) and not the template's
+ item, an issue. We have a special syntax for form fields that affect
+ items other than the template default item (see the cgi
+ documentation on `special form variables`_). In particular, we add a
+ field to capture a new timelog item's perdiod::
+
+ <tr>
+ <th>Time Log</th>
+ <td colspan=3><input type="text" name="timelog-1@period" />
+ <br />(enter as '3y 1m 4d 2:40:02' or parts thereof)
+ </td>
+ </tr>
+
+ and another hidden field that links that new timelog item (new
+ because it's marked as having id "-1") to the issue item. It looks
+ like this::
+
+ <input type="hidden" name="@link@times" value="timelog-1" />
+
+ On submission, the "-1" timelog item will be created and assigned a
+ real item id. The "times" property of the issue will have the new id
+ added to it.
+
+4. We want to display a total of the time log times that have been
+ accumulated for an issue. To do this, we'll need to actually write
+ some Python code, since it's beyond the scope of PageTemplates to
+ perform such calculations. We do this by adding a method to the
+ TemplatingUtils class in our tracker ``interfaces.py`` module::
+
+ class TemplatingUtils:
+ ''' Methods implemented on this class will be available to HTML
+ templates through the 'utils' variable.
+ '''
+ def totalTimeSpent(self, times):
+ ''' Call me with a list of timelog items (which have an
+ Interval "period" property)
+ '''
+ total = Interval('')
+ for time in times:
+ total += time.period._value
+ return total
+
+ Replace the ``pass`` line as we did in step 4 above with the Client
+ class. As indicated in the docstrings, we will be able to access the
+ ``totalTimeSpent`` method via the ``utils`` variable in our
+ templates.
+
+5. Display the time log for an issue::
+
+ <table class="otherinfo" tal:condition="context/times">
+ <tr><th colspan="3" class="header">Time Log
+ <tal:block
+ tal:replace="python:utils.totalTimeSpent(context.times)" />
+ </th></tr>
+ <tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
+ <tr tal:repeat="time context/times">
+ <td tal:content="time/creation"></td>
+ <td tal:content="time/period"></td>
+ <td tal:content="time/creator"></td>
+ </tr>
+ </table>
+
+ I put this just above the Messages log in my issue display. Note our
+ use of the ``totalTimeSpent`` method which will total up the times
+ for the issue and return a new Interval. That will be automatically
+ displayed in the template as text like "+ 1y 2:40" (1 year, 2 hours
+ and 40 minutes).
+
+8. If you're using a persistent web server - roundup-server or
+ mod_python for example - then you'll need to restart that to pick up
+ the code changes. When that's done, you'll be able to use the new
+ time logging interface.
+
+Using a UN*X passwd file as the user database
+---------------------------------------------
+
+On some systems the primary store of users is the UN*X passwd file. It
+holds information on users such as their username, real name, password
+and primary user group.
+
+Roundup can use this store as its primary source of user information,
+but it needs additional information too - email address(es), roundup
+Roles, vacation flags, roundup hyperdb item ids, etc. Also, "retired"
+users must still exist in the user database, unlike some passwd files in
+which the users are removed when they no longer have access to a system.
+
+To make use of the passwd file, we therefore synchronise between the two
+user stores. We also use the passwd file to validate the user logins, as
+described in the previous example, `using an external password
+validation source`_. We keep the users lists in sync using a fairly
+simple script that runs once a day, or several times an hour if more
+immediate access is needed. In short, it:
+
+1. parses the passwd file, finding usernames, passwords and real names,
+2. compares that list to the current roundup user list:
+
+ a. entries no longer in the passwd file are *retired*
+ b. entries with mismatching real names are *updated*
+ c. entries only exist in the passwd file are *created*
+
+3. send an email to administrators to let them know what's been done.
+
+The retiring and updating are simple operations, requiring only a call
+to ``retire()`` or ``set()``. The creation operation requires more
+information though - the user's email address and their roundup Roles.
+We're going to assume that the user's email address is the same as their
+login name, so we just append the domain name to that. The Roles are
+determined using the passwd group identifier - mapping their UN*X group
+to an appropriate set of Roles.
+
+The script to perform all this, broken up into its main components, is
+as follows. Firstly, we import the necessary modules and open the
+tracker we're to work on::
+
+ import sys, os, smtplib
+ from roundup import instance, date
+
+ # open the tracker
+ tracker_home = sys.argv[1]
+ tracker = instance.open(tracker_home)
+
+Next we read in the *passwd* file from the tracker home::
+
+ # read in the users
+ file = os.path.join(tracker_home, 'users.passwd')
+ users = [x.strip().split(':') for x in open(file).readlines()]
+
+Handle special users (those to ignore in the file, and those who don't
+appear in the file)::
+
+ # users to not keep ever, pre-load with the users I know aren't
+ # "real" users
+ ignore = ['ekmmon', 'bfast', 'csrmail']
+
+ # users to keep - pre-load with the roundup-specific users
+ keep = ['comment_pool', 'network_pool', 'admin', 'dev-team',
+ 'cs_pool', 'anonymous', 'system_pool', 'automated']
+
+Now we map the UN*X group numbers to the Roles that users should have::
+
+ roles = {
+ '501': 'User,Tech', # tech
+ '502': 'User', # finance
+ '503': 'User,CSR', # customer service reps
+ '504': 'User', # sales
+ '505': 'User', # marketing
+ }
+
+Now we do all the work. Note that the body of the script (where we have
+the tracker database open) is wrapped in a ``try`` / ``finally`` clause,
+so that we always close the database cleanly when we're finished. So, we
+now do all the work::
+
+ # open the database
+ db = tracker.open('admin')
+ try:
+ # store away messages to send to the tracker admins
+ msg = []
+
+ # loop over the users list read in from the passwd file
+ for user,passw,uid,gid,real,home,shell in users:
+ if user in ignore:
+ # this user shouldn't appear in our tracker
+ continue
+ keep.append(user)
+ try:
+ # see if the user exists in the tracker
+ uid = db.user.lookup(user)
+
+ # yes, they do - now check the real name for correctness
+ if real != db.user.get(uid, 'realname'):
+ db.user.set(uid, realname=real)
+ msg.append('FIX %s - %s'%(user, real))
+ except KeyError:
+ # nope, the user doesn't exist
+ db.user.create(username=user, realname=real,
+ address='%s@ekit-inc.com'%user, roles=roles[gid])
+ msg.append('ADD %s - %s (%s)'%(user, real, roles[gid]))
+
+ # now check that all the users in the tracker are also in our
+ # "keep" list - retire those who aren't
+ for uid in db.user.list():
+ user = db.user.get(uid, 'username')
+ if user not in keep:
+ db.user.retire(uid)
+ msg.append('RET %s'%user)
+
+ # if we did work, then send email to the tracker admins
+ if msg:
+ # create the email
+ msg = '''Subject: %s user database maintenance
+
+ %s
+ '''%(db.config.TRACKER_NAME, '\n'.join(msg))
+
+ # send the email
+ smtp = smtplib.SMTP(db.config.MAILHOST)
+ addr = db.config.ADMIN_EMAIL
+ smtp.sendmail(addr, addr, msg)
+
+ # now we're done - commit the changes
+ db.commit()
+ finally:
+ # always close the database cleanly
+ db.close()
+
+And that's it!
+
+
+Using an LDAP database for user information
+-------------------------------------------
+
+A script that reads users from an LDAP store using
+http://python-ldap.sf.net/ and then compares the list to the users in the
+roundup user database would be pretty easy to write. You'd then have it run
+once an hour / day (or on demand if you can work that into your LDAP store
+workflow). See the example `Using a UN*X passwd file as the user database`_
+for more information about doing this.
+
+To authenticate off the LDAP store (rather than using the passwords in the
+roundup user database) you'd use the same python-ldap module inside an
+extension to the cgi interface. You'd do this by adding a method called
+"verifyPassword" to the Client class in your tracker's interfaces.py
+module. The method is implemented by default as::
+
+ def verifyPassword(self, userid, password):
+ ''' Verify the password that the user has supplied
+ '''
+ stored = self.db.user.get(self.userid, 'password')
+ if password == stored:
+ return 1
+ if not password and not stored:
+ return 1
+ return 0
+
+So you could reimplement this as something like::
+
+ def verifyPassword(self, userid, password):
+ ''' Verify the password that the user has supplied
+ '''
+ # look up some unique LDAP information about the user
+ username = self.db.user.get(self.userid, 'username')
+ # now verify the password supplied against the LDAP store
+
+
+Enabling display of either message summaries or the entire messages
+-------------------------------------------------------------------
+
+This is pretty simple - all we need to do is copy the code from the
+example `displaying only message summaries in the issue display`_ into
+our template alongside the summary display, and then introduce a switch
+that shows either one or the other. We'll use a new form variable,
+``@whole_messages`` to achieve this::
+
+ <table class="messages" tal:condition="context/messages">
+ <tal:block tal:condition="not:request/form/@whole_messages/value | python:0">
+ <tr><th colspan="3" class="header">Messages</th>
+ <th colspan="2" class="header">
+ <a href="?@whole_messages=yes">show entire messages</a>
+ </th>
+ </tr>
+ <tr tal:repeat="msg context/messages">
+ <td><a tal:attributes="href string:msg${msg/id}"
+ tal:content="string:msg${msg/id}"></a></td>
+ <td tal:content="msg/author">author</td>
+ <td class="date" tal:content="msg/date/pretty">date</td>
+ <td tal:content="msg/summary">summary</td>
+ <td>
+ <a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>
+ </td>
+ </tr>
+ </tal:block>
+
+ <tal:block tal:condition="request/form/@whole_messages/value | python:0">
+ <tr><th colspan="2" class="header">Messages</th>
+ <th class="header">
+ <a href="?@whole_messages=">show only summaries</a>
+ </th>
+ </tr>
+ <tal:block tal:repeat="msg context/messages">
+ <tr>
+ <th tal:content="msg/author">author</th>
+ <th class="date" tal:content="msg/date/pretty">date</th>
+ <th style="text-align: right">
+ (<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>)
+ </th>
+ </tr>
+ <tr><td colspan="3" tal:content="msg/content"></td></tr>
+ </tal:block>
+ </tal:block>
+ </table>
+
+
+Blocking issues that depend on other issues
+-------------------------------------------
+
+We needed the ability to mark certain issues as "blockers" - that is,
+they can't be resolved until another issue (the blocker) they rely on is
+resolved. To achieve this:
+
+1. Create a new property on the issue Class,
+ ``blockers=Multilink("issue")``. Edit your tracker's dbinit.py file.
+ Where the "issue" class is defined, something like::
+
+ issue = IssueClass(db, "issue",
+ assignedto=Link("user"), topic=Multilink("keyword"),
+ priority=Link("priority"), status=Link("status"))
+
+ add the blockers entry like so::
+
+ issue = IssueClass(db, "issue",
+ blockers=Multilink("issue"),
+ assignedto=Link("user"), topic=Multilink("keyword"),
+ priority=Link("priority"), status=Link("status"))
+
+2. Add the new "blockers" property to the issue.item edit page, using
+ something like::
+
+ <th>Waiting On</th>
+ <td>
+ <span tal:replace="structure python:context.blockers.field(showid=1,
+ size=20)" />
+ <span tal:replace="structure python:db.issue.classhelp('id,title')" />
+ <span tal:condition="context/blockers"
+ tal:repeat="blk context/blockers">
+ <br>View: <a tal:attributes="href string:issue${blk/id}"
+ tal:content="blk/id"></a>
+ </span>
+
+ You'll need to fiddle with your item page layout to find an
+ appropriate place to put it - I'll leave that fun part up to you.
+ Just make sure it appears in the first table, possibly somewhere near
+ the "superseders" field.
+
+3. Create a new detector module (attached) which enforces the rules:
+
+ - issues may not be resolved if they have blockers
+ - when a blocker is resolved, it's removed from issues it blocks
+
+ The contents of the detector should be something like this::
+
+ def blockresolution(db, cl, nodeid, newvalues):
+ ''' If the issue has blockers, don't allow it to be resolved.
+ '''
+ if nodeid is None:
+ blockers = []
+ else:
+ blockers = cl.get(nodeid, 'blockers')
+ blockers = newvalues.get('blockers', blockers)
+
+ # don't do anything if there's no blockers or the status hasn't
+ # changed
+ if not blockers or not newvalues.has_key('status'):
+ return
+
+ # get the resolved state ID
+ resolved_id = db.status.lookup('resolved')
+
+ # format the info
+ u = db.config.TRACKER_WEB
+ s = ', '.join(['<a href="%sissue%s">%s</a>'%(
+ u,id,id) for id in blockers])
+ if len(blockers) == 1:
+ s = 'issue %s is'%s
+ else:
+ s = 'issues %s are'%s
+
+ # ok, see if we're trying to resolve
+ if newvalues['status'] == resolved_id:
+ raise ValueError, "This issue can't be resolved until %s resolved."%s
+
+ def resolveblockers(db, cl, nodeid, newvalues):
+ ''' When we resolve an issue that's a blocker, remove it from the
+ blockers list of the issue(s) it blocks.
+ '''
+ if not newvalues.has_key('status'):
+ return
+
+ # get the resolved state ID
+ resolved_id = db.status.lookup('resolved')
+
+ # interesting?
+ if newvalues['status'] != resolved_id:
+ return
+
+ # yes - find all the blocked issues, if any, and remove me from
+ # their blockers list
+ issues = cl.find(blockers=nodeid)
+ for issueid in issues:
+ blockers = cl.get(issueid, 'blockers')
+ if nodeid in blockers:
+ blockers.remove(nodeid)
+ cl.set(issueid, blockers=blockers)
+
+
+ def init(db):
+ # might, in an obscure situation, happen in a create
+ db.issue.audit('create', blockresolution)
+ db.issue.audit('set', blockresolution)
+
+ # can only happen on a set
+ db.issue.react('set', resolveblockers)
+
+ Put the above code in a file called "blockers.py" in your tracker's
+ "detectors" directory.
+
+4. Finally, and this is an optional step, modify the tracker web page
+ URLs so they filter out issues with any blockers. You do this by
+ adding an additional filter on "blockers" for the value "-1". For
+ example, the existing "Show All" link in the "page" template (in the
+ tracker's "html" directory) looks like this::
+
+ <a href="issue?:sort=-activity&:group=priority&:filter=status&:columns=id,activity,title,creator,assignedto,status&status=-1,1,2,3,4,5,6,7">Show All</a><br>
+
+ modify it to add the "blockers" info to the URL (note, both the
+ ":filter" *and* "blockers" values must be specified)::
+
+ <a href="issue?:sort=-activity&:group=priority&:filter=status,blockers&blockers=-1&:columns=id,activity,title,creator,assignedto,status&status=-1,1,2,3,4,5,6,7">Show All</a><br>
+
+That's it. You should now be able to set blockers on your issues. Note
+that if you want to know whether an issue has any other issues dependent
+on it (i.e. it's in their blockers list) you can look at the journal
+history at the bottom of the issue page - look for a "link" event to
+another issue's "blockers" property.
+
+Add users to the nosy list based on the topic
+---------------------------------------------
+
+We need the ability to automatically add users to the nosy list based
+on the occurence of a topic. Every user should be allowed to edit his
+own list of topics for which he wants to be added to the nosy list.
+
+Below will be showed that such a change can be performed with only
+minimal understanding of the roundup system, but with clever use
+of Copy and Paste.
+
+This requires three changes to the tracker: a change in the database to
+allow per-user recording of the lists of topics for which he wants to
+be put on the nosy list, a change in the user view allowing to edit
+this list of topics, and addition of an auditor which updates the nosy
+list when a topic is set.
+
+Adding the nosy topic list
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The change in the database to make is that for any user there should be
+a list of topics for which he wants to be put on the nosy list. Adding
+a ``Multilink`` of ``keyword`` seem to fullfill this (note that within
+the code topics are called ``keywords``.) As such, all what has to be
+done is to add a new field to the definition of ``user`` within the
+file ``dbinit.py``. We will call this new field ``nosy_keywords``, and
+the updated definition of user will be::
+
+ user = Class(db, "user",
+ username=String(), password=Password(),
+ address=String(), realname=String(),
+ phone=String(), organisation=String(),
+ alternate_addresses=String(),
+ queries=Multilink('query'), roles=String(),
+ timezone=String(),
+ nosy_keywords=Multilink('keyword'))
+
+Changing the user view to allow changing the nosy topic list
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We want any user to be able to change the list of topics for which
+he will by default be added to the nosy list. We choose to add this
+to the user view, as is generated by the file ``html/user.item.html``.
+We easily can
+see that the topic field in the issue view has very similar editting
+requirements as our nosy topics, both being a list of topics. As
+such, we search for Topics in ``issue.item.html``, and extract the
+associated parts from there. We add this to ``user.item.html`` at the
+bottom of the list of viewed items (i.e. just below the 'Alternate
+E-mail addresses' in the classic template)::
+
+ <tr>
+ <th>Nosy Topics</th>
+ <td>
+ <span tal:replace="structure context/nosy_keywords/field" />
+ <span tal:replace="structure python:db.keyword.classhelp(property='nosy_keywords')" />
+ </td>
+ </tr>
+
+
+Addition of an auditor to update the nosy list
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The more difficult part is the addition of the logic to actually
+at the users to the nosy list when it is required.
+The choice is made to perform this action when the topics on an
+item are set, including when an item is created.
+Here we choose to start out with a copy of the
+``detectors/nosyreaction.py`` detector, which we copy to the file
+``detectors/nosy_keyword_reaction.py``.
+This looks like a good start as it also adds users
+to the nosy list. A look through the code reveals that the
+``nosyreaction`` function actually is sending the e-mail, which
+we do not need. As such, we can change the init function to::
+
+ def init(db):
+ db.issue.audit('create', update_kw_nosy)
+ db.issue.audit('set', update_kw_nosy)
+
+After that we rename the ``updatenosy`` function to ``update_kw_nosy``.
+The first two blocks of code in that function relate to settings
+``current`` to a combination of the old and new nosy lists. This
+functionality is left in the new auditor. The following block of
+code, which in ``updatenosy`` handled adding the assignedto user(s)
+to the nosy list, should be replaced by a block of code to add the
+interested users to the nosy list. We choose here to loop over all
+new topics, than loop over all users,
+and assign the user to the nosy list when the topic in the user's
+nosy_keywords. The next part in ``updatenosy``, adding the author
+and/or recipients of a message to the nosy list, obviously is not
+relevant here and thus is deleted from the new auditor. The last
+part, copying the new nosy list to newvalues, does not have to be changed.
+This brings the following function::
+
+ def update_kw_nosy(db, cl, nodeid, newvalues):
+ '''Update the nosy list for changes to the topics
+ '''
+ # nodeid will be None if this is a new node
+ current = {}
+ if nodeid is None:
+ ok = ('new', 'yes')
+ else:
+ ok = ('yes',)
+ # old node, get the current values from the node if they haven't
+ # changed
+ if not newvalues.has_key('nosy'):
+ nosy = cl.get(nodeid, 'nosy')
+ for value in nosy:
+ if not current.has_key(value):
+ current[value] = 1
+
+ # if the nosy list changed in this transaction, init from the new value
+ if newvalues.has_key('nosy'):
+ nosy = newvalues.get('nosy', [])
+ for value in nosy:
+ if not db.hasnode('user', value):
+ continue
+ if not current.has_key(value):
+ current[value] = 1
+
+ # add users with topic in nosy_keywords to the nosy list
+ if newvalues.has_key('topic') and newvalues['topic'] is not None:
+ topic_ids = newvalues['topic']
+ for topic in topic_ids:
+ # loop over all users,
+ # and assign user to nosy when topic in nosy_keywords
+ for user_id in db.user.list():
+ nosy_kw = db.user.get(user_id, "nosy_keywords")
+ found = 0
+ for kw in nosy_kw:
+ if kw == topic:
+ found = 1
+ if found:
+ current[user_id] = 1
+
+ # that's it, save off the new nosy list
+ newvalues['nosy'] = current.keys()
+
+and these two function are the only ones needed in the file.
+
+TODO: update this example to use the find() Class method.
+
+Caveats
+~~~~~~~
+
+A few problems with the design here can be noted:
+
+Multiple additions
+ When a user, after automatic selection, is manually removed
+ from the nosy list, he again is added to the nosy list when the
+ topic list of the issue is updated. A better design might be
+ to only check which topics are new compared to the old list
+ of topics, and only add users when they have indicated
+ interest on a new topic.
+
+ The code could also be changed to only trigger on the create() event,
+ rather than also on the set() event, thus only setting the nosy list
+ when the issue is created.
+
+Scalability
+ In the auditor there is a loop over all users. For a site with
+ only few users this will pose no serious problem, however, with
+ many users this will be a serious performance bottleneck.
+ A way out will be to link from the topics to the users which
+ selected these topics a nosy topics. This will eliminate the
+ loop over all users.
+
+
+Adding action links to the index page
+-------------------------------------
+
+Add a column to the item.index.html template.
+
+Resolving the issue::
+
+ <a tal:attributes="href
+ string:issue${i/id}?:status=resolved&:action=edit">resolve</a>
+
+"Take" the issue::
+
+ <a tal:attributes="href
+ string:issue${i/id}?:assignedto=${request/user/id}&:action=edit">take</a>
+
+... and so on
+
+-------------------
Back to `Table of Contents`_
.. _`Table of Contents`: index.html
+.. _`design documentation`: design.html