bye bye gadfly - you served your purpose well (sf bug 701127)

[roundup.git] / doc / installation.txt

==================
Installing Roundup
==================

:Version: $Revision: 1.40 $

.. contents::


Overview
========

Broken out separately, there are several conceptual pieces to a
Roundup installation:

Roundup trackers
 Trackers consist of issues (be they bug reports or otherwise), tracker
 configuration file(s), web HTML files etc. Roundup trackers are initialised
 with a "Template" which defines the fields usable/assignable on a
 per-issue basis.  Descriptions of the provided templates are given in
 `choosing your template`_.

Roundup support code
 Installed into your Python install's lib directory

Roundup scripts
 These include the email gateway, the roundup
 HTTP server, the roundup administration command-line interface, etc.


Prerequisites
=============

Python 2.1.1 or newer with a functioning anydbm or bsddb module. Download the
latest version from http://www.python.org/. It is highly recommended that
users install the latest patch version of python - 2.1.3 or 2.2.1 - as these
contain many fixes to serious bugs.

If you want to use Berkeley DB bsddb3 with Roundup, use version 3.3.0 or
later. Download the latest version from http://pybsddb.sourceforge.net/.

If you're on windows, you will either need to be using the ActiveState python
distribution (at http://www.activestate.com/Products/ActivePython/), or you'll
have to install the win32all package separately (get it from
http://starship.python.net/crew/mhammond/win32/).


Getting Roundup
===============

Download the latest version from http://roundup.sf.net/.

Testing your Python
-------------------

Once you've unpacked roundup's source, run ``python ./run_tests`` in the
source directory and make sure there are no errors.
If there are errors, please let us know!

If the above fails, you may be using the wrong version of python. Try
``python2 ./run_tests``. If that works, you will need to substitute
``python2`` for ``python`` in all further commands you use in relation to
Roundup -- from installation and scripts.


Installation
============

Set aside 15-30 minutes. Please make sure you're using a supported version of
Python -- see `testing your python`_. There's four steps to follow in your
installation:

1. `basic installation steps`_ that all installers must follow
2. then optionally `configure a web interface`_
3. and optionally `configure an email interface`_
4. `shared environment steps`_ to take if you're installing on a shared
   UNIX machine and want to restrict local access to roundup

Most users will only need to follow the first step, since the environment will
be a trusted one.


Basic Installation Steps
------------------------

1. To install the Roundup support code into your Python tree and
   Roundup scripts into /usr/local/bin (substitute that path for whatever is
   appropriate on your system). You need to have write permissions
   for these locations, eg. being root on unix::

       python setup.py install

   If you would like to place the Roundup scripts in a directory other
   than ``/usr/local/bin``, then specify the preferred location with
   ``--install-script``. For example, to install them in
   ``/opt/roundup/bin``::

       python setup.py install --install-scripts=/opt/roundup/bin

   You can also use the ``--prefix`` option to use a completely different
   base directory, if you do not want to use administrator rights. If you
   choose to do this, take note of the message at the end of installation
   and modify the python path accordingly.

2. To create a Roundup tracker (necessary to do before you can
   use the software in any real fashion):

   a. (Optional) If you intend to keep your roundup trackers
      under one top level directory which does not exist yet,
      you should create that directory now.  Example::

         mkdir /opt/roundup/trackers

   b. Either add the Roundup script location to your ``PATH``
      environment variable or specify the full path to
      the command in the next step.

   c. Install a new tracker with the command ``roundup-admin install``.
      You will be asked a series of questions.  Descriptions of the provided
      templates can be found in `choosing your template`_ below.  Descriptions
      of the available backends can be found in `choosing your backend`_
      below.  The questions will be something like (you may have more
      templates or backends available)::

          Enter tracker home: /opt/roundup/trackers/support
          Templates: classic
          Select template [classic]: classic
          Back ends: anydbm, bsddb
          Select backend [anydbm]: anydbm

      You will now be directed to edit the tracker configuration and
      initial schema.  At a minimum, you must set ``MAILHOST``,
      ``TRACKER_WEB``, ``MAIL_DOMAIN`` and ``ADMIN_EMAIL``. Note that the
      configuration file uses Python syntax, so almost every value must be
      ``'quoted'`` using single or double quotes. If you get stuck, and get
      configuration file errors, then see the `tracker configuration`_ section
      of the `customisation documentation`_.

      If you just want to get set up to test things quickly, you can even
      just set the TRACKER_WEB variable to::

         TRACKER_WEB = 'http://localhost:8080/support/'

      The URL *must* end in a '/', or your web interface *will not work*.
      See `Customising Roundup`_ for details on configuration and schema
      changes. Note that you may change any of the configuration after
      you've initialised the tracker - it's just better to have valid values
      for this stuff now.

   d. Initialise the tracker database with ``roundup-admin initialise``.
      You will need to supply an admin password at this step. You will be
      prompted::

          Admin Password:
                 Confirm:

      Once this is done, the tracker has been created.

3. At this point, your tracker is set up, but doesn't have a nice user
   interface. To set that up, we need to `configure a web interface`_ and
   optionally `configure an email interface`_. If you want to try your
   new tracker out, assuming ``TRACKER_WEB`` is set to
   ``'http://localhost:8080/support/'``, run::

     roundup-server -p 8080 support=/opt/roundup/trackers/support

   then direct your web browser at:

     http://locahost:8080/support/

   and you should see the tracker interface.


Choosing Your Template
----------------------

Classic Template
~~~~~~~~~~~~~~~~

The classic template is the one defined in the `Roundup Specification`_. It
holds issues which have priorities and statuses. Each issue may also have a
set of messages which are disseminated to the issue's list of nosy users.

Minimal Template
~~~~~~~~~~~~~~~~

The minimal template has the minimum setup required for a tracker
installation. That is, it has the configuration files, defines a user database
and the basic HTML interface to that. It's a completely clean slate for you to
create your tracker on.


Choosing Your Backend
---------------------

The actual storage of Roundup tracker information is handled by backends.
There's several to choose from, each with benefits and limitations:

**anydbm**
  This backend is guaranteed to work on any system that Python runs on. It
  will generally choose the best dbm backend that is available on your system
  (from the list dbhash, gdbm, dbm, dumbdbm). It is the least scaleable of all
  backends, but performs well enough for a smallish tracker (a couple of
  thousand issues, under fifty users, ...).
**bsddb**
  This effectively the same as anydbm, but uses the bsddb backend. This allows
  it to gain some performance and scaling benefits.
**bsddb3**
  Again, this effectively the same as anydbm, but uses the bsddb3 backend.
  This allows it to gain some performance and scaling benefits.
**sqlite**
  This uses the SQLite_ embedded RDBMS to provide a fast, scaleable backend.
  There are no limitations, and it's much faster and more scaleable than the
  dbm backends.
**mysql**
  Backend for popular RDBMS MySQL. This backend uses MySQL's Berkeley DB
  (transaction safe) table format, so with small amount of data it may be a
  bit slower than *dbm backends (due to mysql layer's overhead). However with
  lots of data mysql may perform better (due to query optimisation). No actual
  serious benchmarks were made though. For more info on installation of this
  backend see doc/mysql.txt. 
**metakit**
  This backend is implemented over the metakit_ storage system, using Mk4Py as
  the interface. It scales much better than the dbm backends.

Note: you may set your tracker up with the anydbm backend (which is guaranteed
to be available) and switch to one of the other backends at any time using the
instructions in the `maintenance documentation`_.


Configure a Web Interface
-------------------------

There are three web interfaces to choose from:

1. `web server cgi-bin`_
2. `stand-alone web server`_
3. `Zope product - ZRoundup`_

You may need to give the web server user permission to access the tracker home
- see the `shared environment steps`_ for information.


Web Server cgi-bin
~~~~~~~~~~~~~~~~~~

A benefit of using the cgi-bin approach is that it's the easiest way to
restrict access to your tracker to only use HTTPS. Access will be slower
than through the `stand-alone web server`_ though.

Copy the ``cgi-bin/roundup.cgi`` file to your web server's ``cgi-bin``
directory. You will need to configure it to tell it where your tracker home
is. You can do this either:

through an environment variable
 set the variable TRACKER_HOMES to be a colon (":") separated list of
 name=home pairs (if you're using apache, the SetEnv directive can do this)
directly in the ``roundup.cgi`` file itself
 add your instance to the TRACKER_HOMES variable as ``'name': 'home'``

The "name" part of the configuration will appear in the URL and identifies the
tracker (so you may have more than one tracker per cgi-bin script). Make sure
there are no spaces or other illegal characters in it (to be safe, stick to
letters and numbers). The "name" forms part of the URL that appears in the
tracker config TRACKER_WEB variable, so make sure they match. The "home"
part of the configuration is the tracker home directory.

Stand-alone Web Server
~~~~~~~~~~~~~~~~~~~~~~

This approach will give you the fastest of the three web interfaces. You may
investigate using ProxyPass or similar configuration in apache to have your
tracker accessed through the same URL as other systems.

The stand-alone web server is started with the command ``roundup-server``. It
has several options - display them with ``roundup-server -h``.

The tracker home configuration is similar to the cgi-bin - you may either edit
the script to change the TRACKER_HOMES variable or you may supply the
name=home values on the command-line after all the other options.

To make the server run in the background, use the "-d" option, specifying the
name of a file to write the server process id (pid) to.


Zope Product - ZRoundup
~~~~~~~~~~~~~~~~~~~~~~~

ZRoundup installs as a regular Zope product. Copy the ZRoundup directory to
your Products directory either in INSTANCE_HOME/Products or the Zope
code tree lib/python/Products.

When you next (re)start up Zope, you will be able to add a ZRoundup object
that interfaces to your new tracker.


Configure an Email Interface
----------------------------

If you don't want to use the email component of Roundup, then remove the
"``nosyreaction.py``" module from your tracker "``detectors``" directory.

There are three supported ways to get emailed issues into the
Roundup tracker.  You should pick ONE of the following, all
of which will continue my example setup from above:

As a mail alias pipe process 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Set up a mail alias called "issue_tracker" as (include the quote marks):
"``|/usr/bin/python /usr/local/bin/roundup-mailgw <tracker_home>``"

In some installations (e.g. RedHat 6.2 I think) you'll need to set up smrsh so
sendmail will accept the pipe command. In that case, symlink
``/etc/smrsh/roundup-mailgw`` to "``/usr/local/bin/roundup-mailgw``" and change
the command to::

    |roundup-mailgw /opt/roundup/trackers/support
 
To test the mail gateway on unix systems, try::

    echo test |mail -s '[issue] test' support@YOUR_DOMAIN_HERE

As a regular job using a mailbox source
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Set ``roundup-mailgw`` up to run every 10 minutes or so. For example::

  10 * * * * /usr/local/bin/roundup-mailgw /opt/roundup/trackers/support mailbox <mail_spool_file>

Where the ``mail_spool_file`` argument is the location of the roundup submission
user's mail spool. On most systems, the spool for a user "issue_tracker"
will be "``/var/mail/issue_tracker``".

As a regular job using a POP source
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To retrieve from a POP mailbox, use a *cron* entry to the mailbox one::

  10 * * * * /usr/local/bin/roundup-mailgw /opt/roundup/trackers/support pop <pop_spec>

where pop_spec is "``username:password@server``" that specifies the roundup
submission user's POP account name, password and server.

On windows, you would set up the command using the windows scheduler.


Shared Environment Steps
------------------------

Each tracker ideally should have its own UNIX group, so create
a UNIX group (edit ``/etc/group`` or your appropriate NIS map if
you're using NIS).  To continue with my examples so far, I would
create the UNIX group 'support', although the name of the UNIX
group does not have to be the same as the tracker name.  To this
'support' group I then add all of the UNIX usernames who will be
working with this Roundup tracker.  In addition to 'real' users,
the Roundup email gateway will need to have permissions to this
area as well, so add the user your mail service runs as to the
group.  The UNIX group might then look like::

     support:*:1002:jblaine,samh,geezer,mail

If you intend to use the web interface (as most people do), you
should also add the username your web server runs as to the group.
My group now looks like this::

     support:*:1002:jblaine,samh,geezer,mail,apache

The tracker "db" directory should be chmod'ed g+sw so that the group can
write to the database, and any new files created in the database will be owned
by the group.

An alternative to the above is to create a new user who has the sole
responsibility of running roundup. This user:

1. runs the CGI interface daemon
2. runs regular polls for email
3. runs regular checks (using cron) to ensure the daemon is up
4. optionally has no login password so that nobody but the "root" user
   may actually login and play with the roundup setup.


Maintenance
===========

Read the separate `maintenance documentation`_ for information about how to
perform common maintenance tasks with Roundup.


Upgrading
=========

Read the separate `upgrading document`_, which describes the steps needed to
upgrade existing tracker trackers for each version of Roundup that is
released.


Further Reading
===============

If you intend to use Roundup with anything other than the defualt
templates, if you would like to hack on Roundup, or if you would
like implementation details, you should read `Customising Roundup`_.


Platform-Specific Notes
=======================

Sendmail smrsh
--------------

If you use Sendmail's ``smrsh`` mechanism, you will need to tell
smrsh that roundup-mailgw is a valid/trusted mail handler
before it will work.

This is usually done via the following 2 steps:

1. make a symlink in ``/etc/smrsh`` called ``roundup-mailgw``
   which points to the full path of your actual ``roundup-mailgw``
   script.

2. change your alias to ``"|roundup-mailgw <tracker_home>"``


Linux
-----

Python 2.1.1 as shipped with SuSE7.3 might be missing module
``_weakref``.

-------------------------------------------------------------------------------

Back to `Table of Contents`_

Next: `User Guide`_

.. _`table of contents`: index.html
.. _`user guide`: user_guide.html
.. _`roundup specification`: spec.html
.. _`tracker configuration`: customizing.html#tracker-configuration
.. _`customisation documentation`: customizing.html
.. _`customising roundup`: customizing.html
.. _`upgrading document`: upgrading.html
.. _`maintenance documentation`: maintenance.html
.. _sqlite: http://www.hwaci.com/sw/sqlite/
.. _metakit: http://www.equi4.com/metakit/