1 ======================================
2 Upgrading to newer versions of Roundup
3 ======================================
5 Please read each section carefully and edit your tracker home files
6 accordingly.
8 .. contents::
10 Migrating from 0.4.x to 0.5.0
11 =============================
13 This has been a fairly major revision of Roundup:
15 1. Brand new, much more powerful, flexible, tasty and nutritious templating.
16 Unfortunately, this means all your current templates are useless. Hopefully
17 the new documentation and examples will be enough to help you make the
18 transition. Please don't hesitate to ask on roundup-users for help (or
19 complete conversions if you're completely stuck)!
20 2. The database backed got a lot more flexible, allowing Metakit and SQL
21 databases! The only decent SQL database implemented at present is sqlite,
22 but others shouldn't be a whole lot more work.
23 3. A brand new, highly flexible and much more robust security system including
24 a system of Permissions, Roles and Role assignments to users. You may now
25 define your own Permissions that may be checked in CGI transactions.
26 4. Journalling has been made less storage-hungry, so has been turned on
27 by default *except* for author, recipient and nosy link/unlink events. You
28 are advised to turn it off in your trackers too.
29 5. We've changed the terminology from "instance" to "tracker", to ease the
30 learning curve/impact for new users.
31 6. Because of the above changes, the tracker configuration has seen some
32 major changes. See below for the details.
34 Please, **back up your database** before you start the migration process. This
35 is as simple as copying the "db" directory and all its contents from your
36 tracker to somewhere safe.
39 0.5.0 Configuration
40 -------------------
42 First up, rename your ``instance_config.py`` file to just ``config.py``.
44 Then edit your tracker's ``__init__.py`` module. It'll currently look
45 like this::
47 from instance_config import *
48 try:
49 from dbinit import *
50 except ImportError:
51 pass # in installdir (probably :)
52 from interfaces import *
54 and it needs to be::
56 import config
57 from dbinit import open, init
58 from interfaces import Client, MailGW
60 Due to the new templating having a top-level ``page`` that defines links for
61 searching, indexes, adding items etc, the following variables are no longer
62 used:
64 - HEADER_INDEX_LINKS
65 - HEADER_ADD_LINKS
66 - HEADER_SEARCH_LINKS
67 - SEARCH_FILTERS
68 - DEFAULT_INDEX
69 - UNASSIGNED_INDEX
70 - USER_INDEX
71 - ISSUE_FILTER
73 The new security implementation will require additions to the dbinit module,
74 but also removes the need for the following tracker config variables:
76 - ANONYMOUS_ACCESS
77 - ANONYMOUS_REGISTER
79 but requires two new variables which define the Roles assigned to users who
80 register through the web and e-mail interfaces:
82 - NEW_WEB_USER_ROLES
83 - NEW_EMAIL_USER_ROLES
85 in both cases, 'User' is a good initial setting. To emulate
86 ``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the
87 "Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web
88 Registration" and/or the "Email Registration" Permission from the "Anonymous"
89 Role. See the section on customising security in the `customisation
90 documentation`_ for more information.
92 Finally, the following config variables have been renamed to make more sense:
94 - INSTANCE_HOME -> TRACKER_HOME
95 - INSTANCE_NAME -> TRACKER_NAME
96 - ISSUE_TRACKER_WEB -> TRACKER_WEB
97 - ISSUE_TRACKER_EMAIL -> TRACKER_EMAIL
100 0.5.0 Schema Specification
101 --------------------------
103 0.5.0 Database backend changes
104 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
106 Your select_db module in your tracker has changed a fair bit. Where it used
107 to contain::
109 # WARNING: DO NOT EDIT THIS FILE!!!
110 from roundup.backends.back_anydbm import Database
112 it must now contain::
114 # WARNING: DO NOT EDIT THIS FILE!!!
115 from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass
117 Yes, I realise the irony of the "DO NOT EDIT THIS FILE" statement :)
118 Note the addition of the Class, FileClass, IssueClass imports. These are very
119 important, as they're going to make the next change work too. You now need to
120 modify the top of the dbinit module in your tracker from::
122 import instance_config
123 from roundup import roundupdb
124 from select_db import Database
126 from roundup.roundupdb import Class, FileClass
128 class Database(roundupdb.Database, select_db.Database):
129 ''' Creates a hybrid database from:
130 . the selected database back-end from select_db
131 . the roundup extensions from roundupdb
132 '''
133 pass
135 class IssueClass(roundupdb.IssueClass):
136 ''' issues need the email information
137 '''
138 pass
140 to::
142 import config
143 from select_db import Database, Class, FileClass, IssueClass
145 Yes, remove the Database and IssueClass definitions and those other imports.
146 They're not needed any more!
148 Look for places in dbinit.py where ``instance_config`` is used too, and
149 rename them ``config``.
152 0.5.0 Journalling changes
153 ~~~~~~~~~~~~~~~~~~~~~~~~~
155 Journalling has been optimised for storage. Journalling of links has been
156 turned back on by default. If your tracker has a large user base, you may wish
157 to turn off journalling of nosy list, message author and message recipient
158 link and unlink events. You do this by adding ``do_journal='no'`` to the Class
159 initialisation in your dbinit. For example, your *msg* class initialisation
160 probably looks like this::
162 msg = FileClass(db, "msg",
163 author=Link("user"), recipients=Multilink("user"),
164 date=Date(), summary=String(),
165 files=Multilink("file"),
166 messageid=String(), inreplyto=String())
168 to turn off journalling of author and recipient link events, add
169 ``do_journal='no'`` to the ``author=Link("user")`` part of the statement,
170 like so::
172 msg = FileClass(db, "msg",
173 author=Link("user", do_journal='no'),
174 recipients=Multilink("user", do_journal='no'),
175 date=Date(), summary=String(),
176 files=Multilink("file"),
177 messageid=String(), inreplyto=String())
179 Nosy list link event journalling is actually turned off by default now. If you
180 want to turn it on, change to your issue class' nosy list, change its
181 definition from::
183 issue = IssueClass(db, "issue",
184 assignedto=Link("user"), topic=Multilink("keyword"),
185 priority=Link("priority"), status=Link("status"))
187 to::
189 issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'),
190 assignedto=Link("user"), topic=Multilink("keyword"),
191 priority=Link("priority"), status=Link("status"))
193 noting that your definition of the nosy Multilink will override the normal one.
196 0.5.0 User schema changes
197 ~~~~~~~~~~~~~~~~~~~~~~~~~
199 Users have two more properties, "queries" and "roles". You'll have something
200 like this in your dbinit module now::
202 user = Class(db, "user",
203 username=String(), password=Password(),
204 address=String(), realname=String(),
205 phone=String(), organisation=String(),
206 alternate_addresses=String())
207 user.setkey("username")
209 and you'll need to add the new properties and the new "query" class to it
210 like so::
212 query = Class(db, "query",
213 klass=String(), name=String(),
214 url=String())
215 query.setkey("name")
217 # Note: roles is a comma-separated string of Role names
218 user = Class(db, "user",
219 username=String(), password=Password(),
220 address=String(), realname=String(),
221 phone=String(), organisation=String(),
222 alternate_addresses=String(),
223 queries=Multilink('query'), roles=String())
224 user.setkey("username")
226 The "queries" property is used to store off the user's favourite database
227 queries. The "roles" property is explained below in `0.5.0 Security
228 Settings`_.
231 0.5.0 Security Settings
232 ~~~~~~~~~~~~~~~~~~~~~~~
234 See the `security documentation`_ for an explanation of how the new security
235 system works. In a nutshell though, the security is handled as a four step
236 process:
238 1. Permissions are defined as having a name and optionally a hyperdb class
239 they're specific to,
240 2. Roles are defined that have one or more Permissions,
241 3. Users are assigned Roles in their "roles" property, and finally
242 4. Roundup checks that users have appropriate Permissions at appropriate times
243 (like editing issues).
245 Your tracker dbinit module's *open* function now has to define any
246 Permissions that are specific to your tracker, and also the assignment
247 of Permissions to Roles. At the moment, your open function
248 ends with::
250 import detectors
251 detectors.init(db)
253 return db
255 and what we need to do is insert some commands that will set up the security
256 parameters. Right above the ``import detectors`` line, you'll want to insert
257 these lines::
259 #
260 # SECURITY SETTINGS
261 #
262 # new permissions for this schema
263 for cl in 'issue', 'file', 'msg', 'user':
264 db.security.addPermission(name="Edit", klass=cl,
265 description="User is allowed to edit "+cl)
266 db.security.addPermission(name="View", klass=cl,
267 description="User is allowed to access "+cl)
269 # Assign the access and edit permissions for issue, file and message
270 # to regular users now
271 for cl in 'issue', 'file', 'msg':
272 p = db.security.getPermission('View', cl)
273 db.security.addPermissionToRole('User', p)
274 p = db.security.getPermission('Edit', cl)
275 db.security.addPermissionToRole('User', p)
276 # and give the regular users access to the web and email interface
277 p = db.security.getPermission('Web Access')
278 db.security.addPermissionToRole('User', p)
279 p = db.security.getPermission('Email Access')
280 db.security.addPermissionToRole('User', p)
282 # May users view other user information? Comment these lines out
283 # if you don't want them to
284 p = db.security.getPermission('View', 'user')
285 db.security.addPermissionToRole('User', p)
287 # Assign the appropriate permissions to the anonymous user's Anonymous
288 # Role. Choices here are:
289 # - Allow anonymous users to register through the web
290 p = db.security.getPermission('Web Registration')
291 db.security.addPermissionToRole('Anonymous', p)
292 # - Allow anonymous (new) users to register through the email gateway
293 p = db.security.getPermission('Email Registration')
294 db.security.addPermissionToRole('Anonymous', p)
295 # - Allow anonymous users access to the "issue" class of data
296 # Note: this also grants access to related information like files,
297 # messages, statuses etc that are linked to issues
298 #p = db.security.getPermission('View', 'issue')
299 #db.security.addPermissionToRole('Anonymous', p)
300 # - Allow anonymous users access to edit the "issue" class of data
301 # Note: this also grants access to create related information like
302 # files and messages etc that are linked to issues
303 #p = db.security.getPermission('Edit', 'issue')
304 #db.security.addPermissionToRole('Anonymous', p)
306 # oh, g'wan, let anonymous access the web interface too
307 p = db.security.getPermission('Web Access')
308 db.security.addPermissionToRole('Anonymous', p)
310 Note in the comments there the places where you might change the permissions
311 to restrict users or grant users more access. If you've created additional
312 classes that users should be able to edit and view, then you should add them
313 to the "new permissions for this schema" section at the start of the security
314 block. Then add them to the "Assign the access and edit permissions" section
315 too, so people actually have the new Permission you've created.
317 One final change is needed that finishes off the security system's
318 initialisation. We need to add a call to ``db.post_init()`` at the end of the
319 dbinit open() function. Add it like this::
321 import detectors
322 detectors.init(db)
324 # schema is set up - run any post-initialisation
325 db.post_init()
326 return db
328 You may verify the setup of Permissions and Roles using the new
329 "``roundup-admin security``" command.
332 0.5.0 User changes
333 ~~~~~~~~~~~~~~~~~~
335 To support all those schema changes, you'll need to massage your user database
336 a little too, to:
338 1. make sure there's an "anonymous" user - this user is mandatory now and is
339 the one that unknown users are logged in as.
340 2. make sure all users have at least one Role.
342 If you don't have the "anonymous" user, create it now with the command::
344 roundup-admin create user username=anonymous roles=Anonymous
346 making sure the capitalisation is the same as above. Once you've done that,
347 you'll need to set the roles property on all users to a reasonable default.
348 The admin user should get "Admin", the anonymous user "Anonymous"
349 and all other users "User". The ``fixroles.py`` script in the tools directory
350 will do this. Run it like so (where python is your python 2+ binary)::
352 python tools/fixroles.py -i <tracker home>
356 0.5.0 CGI interface changes
357 ---------------------------
359 The CGI interface code was completely reorganised and largely rewritten. The
360 end result is that this section of your tracker interfaces module will need
361 changing from::
363 from roundup import mailgw
364 from roundup.cgi import client
366 class Client(client.Client):
367 ''' derives basic CGI implementation from the standard module,
368 with any specific extensions
369 '''
370 pass
372 to::
374 from roundup import cgi_client, mailgw
375 from roundup.i18n import _
377 class Client(cgi_client.Client):
378 ''' derives basic CGI implementation from the standard module,
379 with any specific extensions
380 '''
381 pass
383 You will also need to install the new version of roundup.cgi from the source
384 cgi-bin directory if you're using it.
387 0.5.0 HTML templating
388 ---------------------
390 You'll want to make a backup of your current tracker html directory. You
391 should then copy the html directory from the Roundup source "classic" template
392 and modify it according to your local schema changes.
394 If you need help with the new templating system, please ask questions on the
395 roundup-users mailing list (available through the roundup project page on
396 sourceforge, http://roundup.sf.net/)
399 0.5.0 Detectors
400 ---------------
402 The nosy reactor has been updated to handle the tracker not having an
403 "assignedto" property on issues. You may want to copy it into your tracker's
404 detectors directory. Chances are you've already fixed it though :)
407 Migrating from 0.4.1 to 0.4.2
408 =============================
410 0.4.2 Configuration
411 -------------------
412 The USER_INDEX definition introduced in 0.4.1 was too restrictive in its
413 allowing replacement of 'assignedto' with the user's userid. Users must change
414 the None value of 'assignedto' to 'CURRENT USER' (the string, in quotes) for
415 the replacement behaviour to occur now.
417 The new configuration variables are:
419 - EMAIL_KEEP_QUOTED_TEXT
420 - EMAIL_LEAVE_BODY_UNCHANGED
421 - ADD_RECIPIENTS_TO_NOSY
423 See the sample configuration files in::
425 <roundup source>/roundup/templates/classic/instance_config.py
427 and::
429 <roundup source>/roundup/templates/extended/instance_config.py
431 and the `customisation documentation`_ for information on how they're used.
434 0.4.2 Changes to detectors
435 --------------------------
436 You will need to copy the detectors from the distribution into your instance
437 home "detectors" directory. If you used the classic schema, the detectors
438 are in::
440 <roundup source>/roundup/templates/classic/detectors/
442 If you used the extended schema, the detectors are in::
444 <roundup source>/roundup/templates/extended/detectors/
446 The change means that schema-specific code has been removed from the
447 mail gateway and cgi interface and made into auditors:
449 - nosyreactor.py has now got an updatenosy auditor which updates the nosy
450 list with author, recipient and assignedto information.
451 - statusauditor.py makes the unread or resolved -> chatting changes and
452 presets the status of an issue to unread.
454 There's also a bug or two fixed in the nosyreactor code.
456 0.4.2 HTML templating changes
457 -----------------------------
458 The link() htmltemplate function now has a "showid" option for links and
459 multilinks. When true, it only displays the linked item id as the anchor
460 text. The link value is displayed as a tooltip using the title anchor
461 attribute. To use in eg. the superseder field, have something like this::
463 <td>
464 <display call="field('superseder', showid=1)">
465 <display call="classhelp('issue', 'id,title', label='list', width=500)">
466 <property name="superseder">
467 <br>View: <display call="link('superseder', showid=1)">
468 </property>
469 </td>
471 The stylesheets have been cleaned up too. You may want to use the newer
472 versions in::
474 <roundup source>/roundup/templates/<template>/html/default.css
478 Migrating from 0.4.0 to 0.4.1
479 =============================
481 0.4.1 Files storage
482 -------------------
484 Messages and files from newly created issues will be put into subdierectories
485 in thousands e.g. msg123 will be put into files/msg/0/msg123, file2003
486 will go into files/file/2/file2003. Previous messages are still found, but
487 could be put into this structure.
489 0.4.1 Configuration
490 -------------------
492 To allow more fine-grained access control, the variable used to check
493 permission to auto-register users in the mail gateway is now called
494 ANONYMOUS_REGISTER_MAIL rather than overloading ANONYMOUS_REGISTER. If the
495 variable doesn't exist, then ANONYMOUS_REGISTER is tested as before.
497 Configuring the links in the web header is now easier too. The following
498 variables have been added to the classic instance_config.py::
500 HEADER_INDEX_LINKS - defines the "index" links to be made available
501 HEADER_ADD_LINKS - defines the "add" links
502 DEFAULT_INDEX - specifies the index view for DEFAULT
503 UNASSIGNED_INDEX - specifies the index view for UNASSIGNED
504 USER_INDEX - specifies the index view for USER
506 See the <roundup source>/roundup/templates/classic/instance_config.py for more
507 information - including how the variables are to be set up. Most users will
508 just be able to copy the variables from the source to their instance home. If
509 you've modified the header by changing the source of the interfaces.py file in
510 the instance home, you'll need to remove that customisation and move it into
511 the appropriate variables in instance_config.py.
513 The extended schema has similar variables added too - see the source for more
514 info.
516 0.4.1 Alternate E-Mail Addresses
517 --------------------------------
519 If you add the property "alternate_addresses" to your user class, your users
520 will be able to register alternate email addresses that they may use to
521 communicate with roundup as. All email from roundup will continue to be sent
522 to their primary address.
524 If you have not edited the dbinit.py file in your instance home directory,
525 you may simply copy the new dbinit.py file from the core code. If you used
526 the classic schema, the interfaces file is in::
528 <roundup source>/roundup/templates/classic/dbinit.py
530 If you used the extended schema, the file is in::
532 <roundup source>/roundup/templates/extended/dbinit.py
534 If you have modified your dbinit.py file, you need to edit the dbinit.py
535 file in your instance home directory. Find the lines which define the user
536 class::
538 user = Class(db, "msg",
539 username=String(), password=Password(),
540 address=String(), realname=String(),
541 phone=String(), organisation=String(),
542 alternate_addresses=String())
544 You will also want to add the property to the user's details page. The
545 template for this is the "user.item" file in your instance home "html"
546 directory. Similar to above, you may copy the file from the roundup source if
547 you haven't modified it. Otherwise, add the following to the template::
549 <display call="multiline('alternate_addresses')">
551 with appropriate labelling etc. See the standard template for an idea.
555 Migrating from 0.3.x to 0.4.0
556 =============================
558 0.4.0 Message-ID and In-Reply-To addition
559 -----------------------------------------
560 0.4.0 adds the tracking of messages by message-id and allows threading
561 using in-reply-to. Most e-mail clients support threading using this
562 feature, and we hope to add support for it to the web gateway. If you
563 have not edited the dbinit.py file in your instance home directory, you may
564 simply copy the new dbinit.py file from the core code. If you used the
565 classic schema, the interfaces file is in::
567 <roundup source>/roundup/templates/classic/dbinit.py
569 If you used the extended schema, the file is in::
571 <roundup source>/roundup/templates/extended/dbinit.py
573 If you have modified your dbinit.py file, you need to edit the dbinit.py
574 file in your instance home directory. Find the lines which define the msg
575 class::
577 msg = FileClass(db, "msg",
578 author=Link("user"), recipients=Multilink("user"),
579 date=Date(), summary=String(),
580 files=Multilink("file"))
582 and add the messageid and inreplyto properties like so::
584 msg = FileClass(db, "msg",
585 author=Link("user"), recipients=Multilink("user"),
586 date=Date(), summary=String(),
587 files=Multilink("file"),
588 messageid=String(), inreplyto=String())
590 Also, configuration is being cleaned up. This means that your dbinit.py will
591 also need to be changed in the open function. If you haven't changed your
592 dbinit.py, the above copy will be enough. If you have, you'll need to change
593 the line (round line 50)::
595 db = Database(instance_config.DATABASE, name)
597 to::
599 db = Database(instance_config, name)
602 0.4.0 Configuration
603 --------------------
604 ``TRACKER_NAME`` and ``EMAIL_SIGNATURE_POSITION`` have been added to the
605 instance_config.py. The simplest solution is to copy the default values
606 from template in the core source.
608 The mail gateway now checks ``ANONYMOUS_REGISTER`` to see if unknown users
609 are to be automatically registered with the tracker. If it is set to "deny"
610 then unknown users will not have access. If it is set to "allow" they will be
611 automatically registered with the tracker.
614 0.4.0 CGI script roundup.cgi
615 ----------------------------
616 The CGI script has been updated with some features and a bugfix, so you should
617 copy it from the roundup cgi-bin source directory again. Make sure you update
618 the ROUNDUP_INSTANCE_HOMES after the copy.
621 0.4.0 Nosy reactor
622 ------------------
623 The nosy reactor has also changed - copy the nosyreactor.py file from the core
624 source::
626 <roundup source>/roundup/templates/<template>/detectors/nosyreactor.py
628 to your instance home "detectors" directory.
631 0.4.0 HTML templating
632 ---------------------
633 The field() function was incorrectly implemented - links and multilinks now
634 display as text fields when rendered using field(). To display a menu (drop-
635 down or select box) you need to use the menu() function.
639 Migrating from 0.2.x to 0.3.x
640 =============================
642 0.3.x Cookie Authentication changes
643 -----------------------------------
644 0.3.0 introduces cookie authentication - you will need to copy the
645 interfaces.py file from the roundup source to your instance home to enable
646 authentication. If you used the classic schema, the interfaces file is in::
648 <roundup source>/roundup/templates/classic/interfaces.py
650 If you used the extended schema, the file is in::
652 <roundup source>/roundup/templates/extended/interfaces.py
654 If you have modified your interfaces.Client class, you will need to take
655 note of the login/logout functionality provided in roundup.cgi_client.Client
656 (classic schema) or roundup.cgi_client.ExtendedClient (extended schema) and
657 modify your instance code apropriately.
660 0.3.x Password encoding
661 -----------------------
662 This release also introduces encoding of passwords in the database. If you
663 have not edited the dbinit.py file in your instance home directory, you may
664 simply copy the new dbinit.py file from the core code. If you used the
665 classic schema, the interfaces file is in::
667 <roundup source>/roundup/templates/classic/dbinit.py
669 If you used the extended schema, the file is in::
671 <roundup source>/roundup/templates/extended/dbinit.py
674 If you have modified your dbinit.py file, you may use encoded passwords:
676 1. Edit the dbinit.py file in your instance home directory
677 a. At the first code line of the open() function::
679 from roundup.hyperdb import String, Date, Link, Multilink
681 alter to include Password, as so::
683 from roundup.hyperdb import String, Password, Date, Link, Multilink
685 b. Where the password property is defined (around line 66)::
687 user = Class(db, "user",
688 username=String(), password=String(),
689 address=String(), realname=String(),
690 phone=String(), organisation=String())
691 user.setkey("username")
693 alter the "password=String()" to "password=Password()"::
695 user = Class(db, "user",
696 username=String(), password=Password(),
697 address=String(), realname=String(),
698 phone=String(), organisation=String())
699 user.setkey("username")
701 2. Any existing passwords in the database will remain cleartext until they
702 are edited. It is recommended that at a minimum the admin password be
703 changed immediately::
705 roundup-admin -i <instance home> set user1 password=<new password>
708 0.3.x Configuration
709 -------------------
710 FILTER_POSITION, ANONYMOUS_ACCESS, ANONYMOUS_REGISTER have been added to
711 the instance_config.py. Simplest solution is to copy the default values from
712 template in the core source.
714 MESSAGES_TO_AUTHOR has been added to the IssueClass in dbinit.py. Set to 'yes'
715 to send nosy messages to the author. Default behaviour is to not send nosy
716 messages to the author. You will need to add MESSAGES_TO_AUTHOR to your
717 dbinit.py in your instance home.
720 0.3.x CGI script roundup.cgi
721 ----------------------------
722 There have been some structural changes to the roundup.cgi script - you will
723 need to install it again from the cgi-bin directory of the source
724 distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the
725 copy.
728 .. _`customisation documentation`: customizing.html
729 .. _`security documentation`: security.html