Wrote most of the upgrading documentation (please read!)

Removed the hard coded template path
Removed unused config vars from instance_config

git-svn-id: http://svn.roundup-tracker.org/svnroot/roundup/trunk@1025 57a73879-2fb5-44c3-a270-3262357dd7e2

diff --git a/doc/upgrading.txt b/doc/upgrading.txt
index 0e10ad316d15e75ac07114b81c7a502bc4f0fc3f..21074a2fec6483f7a46e33d813d5d1013520b494 100644 (file)
--- a/doc/upgrading.txt
+++ b/doc/upgrading.txt
@@ -10,18 +10,340 @@ accordingly.
 Migrating from 0.4.x to 0.5.0
 =============================
 
+This has been a fairly major revision of Roundup:
+
+1. Brand new, much more powerful, flexible, tasty and nutritious templating.
+   Unfortunately, this means all your current templates are useless. Please
+   don't hesitate to ask on roundup-users for help (or complete conversions
+   if you're completely stuck)!
+2. The database backed got a lot more flexible, allowing Metakit and SQL
+   databases! The only SQL database implemented at present is gadfly, but
+   others shouldn't be a whole lot more work.
+3. A brand new, highly flexible and much more robust security system including
+   a system of Permissions, Roles and Role assignments to users. You may now
+   define your own Permissions that may be checked in CGI transactions.
+4. Journalling has been made less storage-hungry, so has been turned on
+   by default *except* for author, recipient and nosy link/unlink events. You
+   are advised to turn it off in your instances too.
+5. Because of the above changes, the instance configuration has seen some
+   major changes. See below for the details.
+
+Please, *back up your database* before you start the migration process. This
+is as simple as copying the "db" directory and all its contents from your
+instance to somewhere safe.
+
+
 0.5.0 Configuration
 -------------------
 
-TODO: mention stuff about indexing
-TODO: mention that the dbinit needs the db.post_init() method call for
-reindexing
-TODO: dbinit now imports classes from selct_db
-TODO: select_db needs fixing to include Class, FileClass and IssueClass
-TODO: migration of security settings
-TODO: nosy reactor has been updated
-TODO: user.item template html needs updating for new security
-TODO: maybe something about export/import?
+Due to the new templating having a top-level ``page`` that defines links for
+searching, indexes, adding items etc, the following variables are no longer
+used:
+
+- HEADER_INDEX_LINKS
+- HEADER_ADD_LINKS
+- HEADER_SEARCH_LINKS
+- SEARCH_FILTERS
+- DEFAULT_INDEX
+- UNASSIGNED_INDEX
+- USER_INDEX
+- ISSUE_FILTER
+
+The new security implementation will require additions to the dbinit module,
+but also removes the need for the following instance config variables:
+
+- ANONYMOUS_ACCESS
+- ANONYMOUS_REGISTER
+
+but requires two new variables which define the Roles assigned to users who
+register through the web and e-mail interfaces:
+
+- NEW_WEB_USER_ROLES
+- NEW_EMAIL_USER_ROLES
+
+in both cases, 'User' is a good initial setting. To emulate
+``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the
+"Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web
+Registration" and/or the "Email Registration" Permission from the "Anonymous"
+Role. See the section on customising security in the `customisation
+documentation`_ for more information.
+
+
+0.5.0 Schema Specification
+--------------------------
+
+0.5.0 Database backend changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Your select_db module in your instance has changed a fair bit. Where it used
+to contain::
+
+ # WARNING: DO NOT EDIT THIS FILE!!!
+ from roundup.backends.back_anydbm import Database
+
+it must now contain::
+
+ # WARNING: DO NOT EDIT THIS FILE!!!
+ from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass
+
+Note the addition of the Class, FileClass, IssueClass imports. These are very
+important, as they're going to make the next change work too. You now need to
+modify the top of the dbinit module in your instance from::
+
+ import instance_config
+ from roundup import roundupdb
+ from select_db import Database
+
+ from roundup.roundupdb import Class, FileClass
+
+ class Database(roundupdb.Database, select_db.Database):
+     ''' Creates a hybrid database from:
+          . the selected database back-end from select_db
+          . the roundup extensions from roundupdb
+     '''
+     pass
+
+ class IssueClass(roundupdb.IssueClass):
+     ''' issues need the email information
+     '''
+     pass
+
+
+
+to just::
+
+ import instance_config
+ from select_db import Database, Class, FileClass, IssueClass
+
+Yes, remove the Database and IssueClass definitions and those other imports.
+They're not needed any more!
+
+
+0.5.0 Journalling changes
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Journalling has been optimised for storage. Journalling of links has been
+turned back on by default. If your tracker has a large user base, you may wish
+to turn off journalling of nosy list, message author and message recipient
+link and unlink events. You do this by adding ``do_journal='no'`` to the Class
+initialisation in your dbinit. For example, your *msg* class initialisation
+probably looks like this::
+
+    msg = FileClass(db, "msg",
+                    author=Link("user"), recipients=Multilink("user"),
+                    date=Date(),         summary=String(),
+                    files=Multilink("file"),
+                    messageid=String(),  inreplyto=String())
+
+to turn off journalling of author and recipient link events, add
+``do_journal='no'`` to the ``author=Link("user")`` part of the statement,
+like so::
+
+    msg = FileClass(db, "msg",
+                    author=Link("user", do_journal='no'),
+                    recipients=Multilink("user", do_journal='no'),
+                    date=Date(),         summary=String(),
+                    files=Multilink("file"),
+                    messageid=String(),  inreplyto=String())
+
+Nosy list link event journalling is actually turned off by default now. If you
+want to turn it onn, change to your issue class' nosy list, change its
+definition from::
+
+    issue = IssueClass(db, "issue",
+                    assignedto=Link("user"), topic=Multilink("keyword"),
+                    priority=Link("priority"), status=Link("status"))
+
+to::
+
+    issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'),
+                    assignedto=Link("user"), topic=Multilink("keyword"),
+                    priority=Link("priority"), status=Link("status"))
+
+noting that your definition of the nosy Multilink will override the normal one.
+
+0.5.0 User schema changes
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Users have two more properties, "queries" and "roles". You'll have something
+like this in your dbinit module now::
+
+    user = Class(db, "user",
+                    username=String(),   password=Password(),
+                    address=String(),    realname=String(),
+                    phone=String(),      organisation=String(),
+                    alternate_addresses=String())
+    user.setkey("username")
+
+and you'll need to add the new properties and the new "query" class to it
+like so::
+
+    query = Class(db, "query",
+                    klass=String(),     name=String(),
+                    url=String())
+    query.setkey("name")
+
+    # Note: roles is a comma-separated string of Role names
+    user = Class(db, "user",
+                    username=String(),   password=Password(),
+                    address=String(),    realname=String(),
+                    phone=String(),      organisation=String(),
+                    alternate_addresses=String(),
+                    queries=Multilink('query'), roles=String())
+    user.setkey("username")
+
+The "queries" property is used to store off the user's favourite database
+queries. The "roles" property is explained below in `0.5.0 Security
+Settings`_.
+
+
+0.5.0 Security Settings
+~~~~~~~~~~~~~~~~~~~~~~~
+
+See the `security documentation`_ for an explanation of how the new security
+system works. In a nutshell though, the security is handled as a four step
+process:
+
+1. Permissions are defined as having a name and optionally a hyperdb class
+   they're specific to,
+2. Roles are defined that have one or more Permissions,
+3. Users are assigned Roles in their "roles" property, and finally
+4. Roundup checks that users have appropriate Permissions at appropriate times
+   (like editing issues).
+
+Your instance dbinit module's *open* function now has to define any
+Permissions that are specific to your instance, and also the assignment
+of Permissions to Roles. At the moment, your open function
+ends with::
+
+    import detectors
+    detectors.init(db)
+
+    return db
+
+and what we need to do is insert some commands that will set up the security
+parameters. Right above the ``import detectors`` line, you'll want to insert
+these lines::
+
+    #
+    # SECURITY SETTINGS
+    #
+    # new permissions for this schema
+    for cl in 'issue', 'file', 'msg', 'user':
+        db.security.addPermission(name="Edit", klass=cl,
+            description="User is allowed to edit "+cl)
+        db.security.addPermission(name="View", klass=cl,
+            description="User is allowed to access "+cl)
+
+    # Assign the access and edit permissions for issue, file and message
+    # to regular users now
+    for cl in 'issue', 'file', 'msg':
+        p = db.security.getPermission('View', cl)
+        db.security.addPermissionToRole('User', p)
+        p = db.security.getPermission('Edit', cl)
+        db.security.addPermissionToRole('User', p)
+    # and give the regular users access to the web and email interface
+    p = db.security.getPermission('Web Access')
+    db.security.addPermissionToRole('User', p)
+    p = db.security.getPermission('Email Access')
+    db.security.addPermissionToRole('User', p)
+
+    # May users view other user information? Comment these lines out
+    # if you don't want them to
+    p = db.security.getPermission('View', 'user')
+    db.security.addPermissionToRole('User', p)
+
+    # Assign the appropriate permissions to the anonymous user's Anonymous
+    # Role. Choices here are:
+    # - Allow anonymous users to register through the web
+    p = db.security.getPermission('Web Registration')
+    db.security.addPermissionToRole('Anonymous', p)
+    # - Allow anonymous (new) users to register through the email gateway
+    p = db.security.getPermission('Email Registration')
+    db.security.addPermissionToRole('Anonymous', p)
+    # - Allow anonymous users access to the "issue" class of data
+    #   Note: this also grants access to related information like files,
+    #         messages, statuses etc that are linked to issues
+    #p = db.security.getPermission('View', 'issue')
+    #db.security.addPermissionToRole('Anonymous', p)
+    # - Allow anonymous users access to edit the "issue" class of data
+    #   Note: this also grants access to create related information like
+    #         files and messages etc that are linked to issues
+    #p = db.security.getPermission('Edit', 'issue')
+    #db.security.addPermissionToRole('Anonymous', p)
+
+    # oh, g'wan, let anonymous access the web interface too
+    p = db.security.getPermission('Web Access')
+    db.security.addPermissionToRole('Anonymous', p)
+
+Note in the comments there the places where you might change the permissions
+to restrict users or grant users more access. If you've created additional
+classes that users should be able to edit and view, then you should add them
+to the "new permissions for this schema" section at the start of the security
+block. Then add them to the "Assign the access and edit permissions" section
+too, so people actually have the new Permission you've created.
+
+One final change is needed that finishes off the security system's
+initialisation. We need to add a call to ``db.post_init()`` at the end of the
+dbinit open() function. Add it like this::
+
+    import detectors
+    detectors.init(db)
+
+    # schema is set up - run any post-initialisation
+    db.post_init()
+    return db
+
+You may verify the setup of Permissions and Roles using the new
+"``roundup-admin security``" command.
+
+
+0.5.0 CGI interface changes
+---------------------------
+
+The CGI interface code was completely reorganised and largely rewritten. The
+end result is that this section of your instance interfaces module will need
+changing from::
+
+ from roundup import mailgw
+ from roundup.cgi import client
+ 
+ class Client(client.Client): 
+     ''' derives basic CGI implementation from the standard module,
+         with any specific extensions
+     '''
+     pass
+
+to::
+
+ from roundup import cgi_client, mailgw
+ from roundup.i18n import _
+ 
+ class Client(cgi_client.Client):
+     ''' derives basic CGI implementation from the standard module,
+         with any specific extensions
+     '''
+     pass
+
+0.5.0 HTML templating
+---------------------
+
+You'll want to make a backup of your current instance html directory. You
+should then copy the html directory from the Roundup source template that you
+used to create your instance, and modify it according to your local schema
+changes.
+
+If you need help with the new templating system, please ask questions on the
+roundup-users mailing list (available through the roundup project page on
+sourceforge, http://roundup.sf.net/)
+
+
+0.5.0 Detectors
+---------------
+
+The nosy reactor has been updated to handle the instance not having an
+"assignedto" property on issues. You may want to copy it into your instance's
+detectors directory. Chances are you've already fixed it though :)
 
 
 Migrating from 0.4.1 to 0.4.2
@@ -48,7 +370,7 @@ and::
 
  <roundup source>/roundup/templates/extended/instance_config.py
 
-and the documentation in customizing_ for information on how they're used.
+and the `customisation documentation`_ for information on how they're used.
 
 
 0.4.2 Changes to detectors
@@ -345,4 +667,5 @@ distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the
 copy.
 
 
-.. _customizing: customizing.html
+.. _`customisation documentation`: customizing.html
+.. _`security documentation`: security.html

diff --git a/roundup/cgi/client.py b/roundup/cgi/client.py
index ff234f95533f5f8d0c54574e248cc88f57854bc2..9683e6724c589bd8f91c2f9a65309ec3acaf9790 100644 (file)
--- a/roundup/cgi/client.py
+++ b/roundup/cgi/client.py
@@ -1,10 +1,10 @@
-# $Id: client.py,v 1.5 2002-09-01 23:57:53 richard Exp $
+# $Id: client.py,v 1.6 2002-09-02 07:46:55 richard Exp $
 
 __doc__ = """
 WWW request handler (also used in the stand-alone server).
 """
 
-import os, cgi, StringIO, urlparse, re, traceback, mimetypes, urllib
+import os, os.path, cgi, StringIO, urlparse, re, traceback, mimetypes, urllib
 import binascii, Cookie, time, random
 
 from roundup import roundupdb, date, hyperdb, password
@@ -275,7 +275,7 @@ class Client:
         # we just want to serve up the file named
         mt = mimetypes.guess_type(str(file))[0]
         self.header({'Content-Type': mt})
-        self.write(open('/tmp/test/html/%s'%file).read())
+        self.write(open(os.path.join(self.instance.TEMPLATES, file)).read())
 
     def template(self, name, **kwargs):
         ''' Return a PageTemplate for the named page
@@ -283,7 +283,7 @@ class Client:
         pt = RoundupPageTemplate(self)
         # make errors nicer
         pt.id = name
-        pt.write(open('/tmp/test/html/%s'%name).read())
+        pt.write(open(os.path.join(self.instance.TEMPLATES, name)).read())
         # XXX handle PT rendering errors here nicely
         try:
             return pt.render(**kwargs)

diff --git a/roundup/cgi/templating.py b/roundup/cgi/templating.py
index 54a592b6341fa30cc21d1eeffc55e4c801edaa48..8f4bf32892c44dfd2d4b77e95f89791ec3bde3cc 100644 (file)
--- a/roundup/cgi/templating.py
+++ b/roundup/cgi/templating.py
@@ -1,4 +1,4 @@
-import sys, cgi, urllib, os, re
+import sys, cgi, urllib, os, re, os.path
 
 from roundup import hyperdb, date
 from roundup.i18n import _
@@ -232,7 +232,7 @@ class HTMLClass:
 
         # use the specified template
         name = self.classname + '.' + name
-        pt.write(open('/tmp/test/html/%s'%name).read())
+        pt.write(open(os.path.join(self.db.config.TEMPLATES, name)).read())
         pt.id = name
 
         # XXX handle PT rendering errors here nicely

diff --git a/roundup/templates/classic/instance_config.py b/roundup/templates/classic/instance_config.py
index 5996d59b9c535ea32d9fbca117cdf5ce29c94e07..a44669b332da31095d9277dde7beff327435e521 100644 (file)
--- a/roundup/templates/classic/instance_config.py
+++ b/roundup/templates/classic/instance_config.py
@@ -15,7 +15,7 @@
 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
 # 
-# $Id: instance_config.py,v 1.20 2002-08-16 04:28:41 richard Exp $
+# $Id: instance_config.py,v 1.21 2002-09-02 07:46:55 richard Exp $
 
 MAIL_DOMAIN=MAILHOST=HTTP_HOST=None
 HTTP_PORT=0
@@ -97,77 +97,11 @@ EMAIL_LEAVE_BODY_UNCHANGED = 'no'   # either 'yes' or 'no'
 MAIL_DEFAULT_CLASS = 'issue'   # use "issue" class by default
 #MAIL_DEFAULT_CLASS = ''        # disable (or just comment the var out)
 
-# Define what index links are available in the header, and what their
-# labels are. Each key is used to look up one of the index specifications
-# below - so 'DEFAULT' will use 'DEFAULT_INDEX'.
-# Where the FILTERSPEC has 'assignedto' with a value of None, it will be
-# replaced by the id of the logged-in user.
-HEADER_INDEX_LINKS = ['DEFAULT', 'UNASSIGNED', 'USER']
-
-# list the classes that users are able to add nodes to
-HEADER_ADD_LINKS = ['issue']
-
-# list the classes that users can search
-HEADER_SEARCH_LINKS = ['issue']
-
-# list search filters per class
-SEARCH_FILTERS = ['ISSUE_FILTER', 'SUPPORT_FILTER']
-
-# Now the DEFAULT display specification. TODO: describe format
-DEFAULT_INDEX = {
-  'LABEL': 'All Issues',
-  'CLASS': 'issue',
-  'SORT': ['-activity'],
-  'GROUP': ['priority'],
-  'FILTER': ['status'],
-  'COLUMNS': ['id','activity','title','creator','assignedto'],
-  'FILTERSPEC': {
-    'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
-  },
-}
-
-# The "unsassigned issues" index
-UNASSIGNED_INDEX = {
-  'LABEL': 'Unassigned Issues',
-  'CLASS': 'issue',
-  'SORT': ['-activity'],
-  'GROUP': ['priority'],
-  'FILTER': ['status', 'assignedto'],
-  'COLUMNS': ['id','activity','title','creator','status'],
-  'FILTERSPEC': {
-    'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
-    'assignedto': ['-1'],
-  },
-}
-
-# The "my issues" index -- note that the user's id will replace the
-# 'CURRENT USER' value of the "assignedto" filterspec
-USER_INDEX = {
-  'LABEL': 'My Issues',
-  'CLASS': 'issue',
-  'SORT': ['-activity'],
-  'GROUP': ['priority'],
-  'FILTER': ['status', 'assignedto'],
-  'COLUMNS': ['id','activity','title','creator','status'],
-  'FILTERSPEC': {
-    'status': ['-1', '1', '2', '3', '4', '5', '6', '7'],
-    'assignedto': 'CURRENT USER',
-  },
-}
-
-ISSUE_FILTER = {
-  'CLASS': 'issue',
-  'FILTER': ['status', 'priority', 'assignedto', 'creator']
-}
-
-SUPPORT_FILTER = {
-  'CLASS': 'issue',
-  'FILTER': ['status', 'priority', 'assignedto', 'creator']
-}
-
-
 #
 # $Log: not supported by cvs2svn $
+# Revision 1.20  2002/08/16 04:28:41  richard
+# removed old, unused config vars
+#
 # Revision 1.19  2002/07/26 08:26:59  richard
 # Very close now. The cgi and mailgw now use the new security API. The two
 # templates have been migrated to that setup. Lots of unit tests. Still some