When debugging mail (debug = <filename> setting in [mail] section of

[roundup.git] / doc / whatsnew-0.7.txt

=========================
What's New in Roundup 0.7
=========================

For those completely new to Roundup, you might want to look over the very
terse features__ page.

__ features.html

.. contents::

Instant-Gratification script even more gratifying
=================================================

The immensely popular ``python demo.py`` instant-gratification script has
been extended to allow you to choose the backend to use with the demo. To
select the "sqlite" backend (assuming it is available) you use::

  python demo.py sqlite nuke

This will nuke any existing demo and reinitialise it with the sqlite
backend. Remember folks, if you want to restart the demo at a later point,
you just need to type::

  python demo.py

without the "sqlite nuke" part, or you'll clear out the demo again. The
backend names are:

  anydbm bsddb bsddb3 sqlite metakit mysql postgresql

You will need support modules installed for all except the first two. If
you're not sure whether you have support, run::

  python run_tests.py

and if you see a line saying "Including XXXX tests" where XXXX is the
backend you wish to try, then you're on your way. The mysql and postgresql
require their test environments to be set up. Read their respective
documents in the "doc" directory to do that.


Web Interface
=============

Saving and sharing of user queries
----------------------------------

Due to popular demand, the user query saving mechanisms have been
overhauled.

As before, you may save queries in the tracker by giving the query a
name. Each user may only have one query with a given name - if a
subsequent search is performed with the same query name supplied, then
it will edit the existing query of the same name.

Queries may be marked as "private". These queries are only visible to the
user that created them. If they're not marked "private" then all other
users may include the query in their list of "Your Queries". Marking it as
private at a later date does not affect users already using the query, nor
does deleting the query.

If a user subsequently creates or edits a public query, a new personal
version of that query is made, with the same editing rules as described
above.

You *are not required* to make these changes in your tracker. You only
need to make them if you wish to use the new query editing features. It's
highly recommended, as the effort is minimal.

1. You will need to edit your tracker's ``dbinit.py`` to change the way
   queries are stored. Change the lines::

      query = Class(db, "query",
                      klass=String(),     name=String(),
                      url=String())
      query.setkey("name")

   to::

      query = Class(db, "query",
                      klass=String(),     name=String(),
                      url=String(),       private_for=Link('user'))

   That is, add the "private_for" property, and remove the line that says
   ``query.setkey("name")``.

2. You will also need to copy the ``query.edit.html`` template page from the
   ``templates/classic/html/`` directory of the source to your tracker's
   ``html`` directory.

3. Once you've done that, edit the tracker's ``page.html`` template to
   change::

    <td rowspan="2" valign="top" class="sidebar">
     <p class="classblock" tal:condition="request/user/queries">
      <b>Your Queries</b><br>
      <tal:block tal:repeat="qs request/user/queries">

   to::

    <td rowspan="2" valign="top" class="sidebar">
     <p class="classblock">
      <b>Your Queries</b> (<a href="query?@template=edit">edit</a>)<br>
      <tal:block tal:repeat="qs request/user/queries">

   That is, you're removing the ``tal:condition`` and adding a link to the
   new edit page.

4. You might also wish to remove the redundant query editing section from the
   ``user.item.html`` page.

ZRoundup reinstated
-------------------

The Zope interface, ZRoundup, lives again!

See the `upgrading documentation`__ if you wish to use it.

__ upgrading.html#zroundup-changes


Simple support for collision detection
--------------------------------------

Item edit pages that use the ``context/submit`` function to generate their
submit buttons now automatically include a datestamp in the form. This
datestamp is compared to the "activity" property of the item when the form
is submitted. If the "actvity" property is younger than the datestamp in
the form submission, then someone else has edited the item, and a page
indicating this is displayed to the user.


Extending the cgi interface
---------------------------

Before 0.7.0 adding or extending web actions was done by overriding or adding
methods on the Client class. Though this approach still works to provide
backwards compatibility, it is recommended you upgrade to the new approach, as
described in the `Defining new web actions`__ section of the customization
documentation. You might also want to take a look at the `Using an external
password validation source`__ example.

__ customizing.html#defining-new-web-actions
__ customizing.html#using-an-external-password-validation-source

Actions may also return the content that should return to the user, which
causes the web interface to skip the normal template formatting step.
This could be used to return an image to the user instead of HTML. Be sure
to set the correct content-type header though! The default is still
text/html. This is done with::

   self.client.setHeader('Content-Type', 'image/png')

if you were returning a PNG image.


Roundup server 
--------------

The roundup-server web interface now supports setgid and running on port
< 1024.

It also forks to handle new connections, which means that trackers using
the postgresql or mysql backends will be able to have multiple users
accessing the tracker simultaneously.


HTML templating made easier
---------------------------

All HTML templating functions perform checks for permissions required to
display or edit the data they are manipulating. The simplest case is
editing an issue title. Including the expression::

   context/title/field

will present the user with an edit field if they have Edit Permission. If
not, then they will be presented with a static display if they have View
Permission. If they don't even have View Permission, then an error message
is raised, preventing the display of the page, indicating that they don't
have permission to view the information.

This removes the need for the template to perform those checks, which was
just plain messy.

Some new permissions will need to be created in your trackers to cope with
this change, as outlined in the `upgrading documentation`__.

__ upgrading.html#permission-assignments


Standards changes
-----------------

The HTTP Content-Length header when we serve up files, either
static ones from the "html" folder or file content from the database.

We also handle If-Modified-Since and supply Last-Modified for both types
of file too.

The HTML generated in the classic tracker is now HTML4 (or optionally
XHTML) compliant. The ``config.py`` variable "HTML_VERSION" is used to
control this behaviour.

The stylesheet includes printer settings now too, so printed pages
don't include the sidebar.


Quoting of URLs and HTML
------------------------

Templates that wish to offer file downloads may now use a new
``download_url`` method::

 <tr tal:repeat="file context/files">
  <td>
   <a tal:attributes="href file/download_url"
      tal:content="file/name">dld link</a>
  </td>
 ...

The ``download_url`` method looks up the file's "id" and "name" and
generates a correctly-quoted URL.

Additionally, users wishing to URL- or HTML- quote text in their templates
may use the new ``utils.url_quote(url)`` and ``utils.html_quote(html)``
methods.


CSV download of search results
------------------------------

A new CGI action, ``export_csv`` has been added which exports a given
index page query as a comma-separated-value file.

To use this new action, just add a link to your ``issue.index.html``
page::

  <a tal:attributes="href python:request.indexargs_url('issue',
            {'@action':'export_csv'})">Download as CSV</a>

You may use this for other classes by adding it to their index page and
changing the ``'issue'`` part of the expression to the new class' name.


Other changes
-------------

- we serve up a favicon now
- the page titles have the tracker name at the end of the text instead
  of the start
- added url_quote and html_quote methods to the utils object
- added isset method to HTMLProperty
- added search_checkboxes as an option for the search form


Email Interface
===============

Better handling of some email headers
-------------------------------------

We ignore messages with the header "Precedence: bulk".

If a Resent-From: header is present, it is used in preference to the From:
header when determining the author of the message. Useful for redirecting
error messages from automated systems.


Email character set
-------------------

The default character set for sending email is UTF-8 (ie. Unicode). If you
have users whose email clients can't handle UTF-8 (eg. Eudora) then you
will need to edit the new config.py variable ``EMAIL_CHARSET``.


Dispatcher configuration
------------------------

A new config option has been added that specifies the email address of
a "dispatcher" role.  This email address acts as a central sentinel for
issues coming into the system. You can configure it so that all e-mail
error messages get bounced to them, them and the user in question, or
just the user (default).

To toggle these switches, add the "DISPATCHER_EMAIL" and
"ERROR_MESSAGES_TO" configuration values to your tracker's ``config.py``.
See the `customisation documentation`_ for how to use them.


More flexible message generation
--------------------------------

The code for generating email messages in Roundup has been refactored. A
new module, ``roundup.mailer`` contains most of the nuts-n-bolts required
to generate email messages from Roundup.

In addition, the ``IssueClass`` methods ``nosymessage()`` and
``send_message()`` have both been altered so that they don't require the
message id parameter. This means that change notes with no associated
change message may now be generated much more easily.

The roundupdb nosymessage() method also accepts a ``bcc`` argument which
specifies additional userids to send the message to that will not be
included in the To: header of the message.


Registration confirmation by email
----------------------------------

Users may now reply to their registration confirmation email, and the
roundup mail gateway will complete their registration.


``roundup-mailgw`` now supports IMAP
------------------------------------

To retrieve from an IMAP mailbox, use a *cron* entry similar to the
POP one::

  0,10,20,30,40,50 * * * * /usr/local/bin/roundup-mailgw /opt/roundup/trackers/support imap <imap_spec>

where imap_spec is "``username:password@server``" that specifies the roundup
submission user's IMAP account name, password and server. You may
optionally include a mailbox to use other than the default ``INBOX`` with
"``imap username:password@server mailbox``".

If you have a secure (ie. HTTPS) IMAP server then you may use ``imaps``
in place of ``imap`` in the command to use a secure connection.


Database configuration
======================

Postgresql added as a backend option
------------------------------------

Trackers may now use the postgresql RDBMS as a database store.

Postgresql is a good choice if you expect your tracker to grow very large,
and are expecting many users.


API change
----------

The Database.curuserid attribute was removed. Any code referencing this
attribute should be replaced with a call to Database.getuid().


New configuration options
-------------------------

- Added DEFAULT_TIMEZONE which allows the tracker to have a different
  default to UTC when users don't specify their own preference.

- Added EMAIL_CHARSET (in 0.6.6, but worth mentioning here) which hard-codes
  the character set to be used when sending email from Roundup. This works
  around some email clients' inability to cope well with UTF-8 (the
  default).

- ERROR_MESSAGES_TO and DISPATCHER_EMAIL as described above in `Dispatcher
  configuration`_.


Typed columns in RDBMS backends
-------------------------------

The SQLite, MySQL and Postgresql backends now create tables with
appropriate column datatypes (not just varchar).

Your database will be automatically migrated to use the new schemas, but
it will take time. It's probably a good idea to make sure you do this as
part of the upgrade when users are not expected to be using the system.


Permission setup
----------------

0.7 automatically sets up the Edit and View Permissions for all classes,
thus you don't need to do so. Feel free to remove the code::

    # Add new Permissions for this schema
    for cl in 'issue', 'file', 'msg', 'user', 'query', 'keyword':
        db.security.addPermission(name="Edit", klass=cl,
            description="User is allowed to edit "+cl)
        db.security.addPermission(name="View", klass=cl,
            description="User is allowed to access "+cl)

from your ``dbinit.py``.


New "actor" property
--------------------

Roundup's database has a new per-item property "actor" which reflects the
user performing the last "actvitiy". See the classic template for ways to
integrate this new property into your interface.

The property will be automatically added to your existing database.


New Reject exception for Auditors
---------------------------------

An auditor may raise this exception when the current create or set
operation should be stopped.

It is up to the specific interface invoking the create or set to
handle this exception sanely. For example:

- mailgw will trap and ignore Reject for file attachments and messages
- cgi will trap and present the exception in a nice format


New auditor fixes Outlook bug
-----------------------------

The new optional auditor ``detectors/emailauditor.py`` fires whenever a
new file entity is created.

If the file is of type message/rfc822, we tack on the extension .mht.

The reason for this is that Microsoft Internet Explorer will not open
things with a .eml attachment, as they deem it 'unsafe'. Worse yet,
they'll just give you an incomprehensible error message. For more 
information, please see: 

http://support.microsoft.com/default.aspx?scid=kb;EN-US;825803

Their suggested work around is (excerpt):

 WORKAROUND

 To work around this behavior, rename the .EML file that the URL
 links to so that it has a .MHT file name extension, and then update
 the URL to reflect the change to the file name. To do this:

 1. In Windows Explorer, locate and then select the .EML file that
    the URL links.
 2. Right-click the .EML file, and then click Rename.
 3. Change the file name so that the .EML file uses a .MHT file name
    extension, and then press ENTER.
 4. Updated the URL that links to the file to reflect the new file
    name extension.


New script for copying users
----------------------------

A new script, ``scripts/copy-user.py``, will copy users from one tracker
to another.  Example usage::

    copy-user.py /roundup/tracker1 /roundup/tracker2 `seq 3 10` 14 16

which copies users 3, 4, 5, 6, 7, 8, 9, 10, 14 and 16.


Other improvements
------------------

- All RDBMS backends now have indexes automatically created on critical
  table columns.

- Additionally, the RDBMS backends also implement their own session,
  one-time-key and full-text indexing stores. These were previously external
  dbm stores. This change allows control of locking the database to be
  completely handed over to the RDBMS.

- Date values capture fractions of seconds now. Note that the MySQL backend
  is not capable of storing this precision though, so it will be lost for
  users of that backend.

- The roundup-admin "export" and "import" commands now handle the database
  journals too. This means that exports from previous versions of Roundup
  will not work under 0.7!


.. _`customisation documentation`: customizing.html