1 ======================================
2 Upgrading to newer versions of Roundup
3 ======================================
5 Please read each section carefully and edit your tracker home files
6 accordingly.
8 .. contents::
10 Migrating from 0.5 to 0.6
11 =========================
13 - Introduced EMAIL_FROM_TAG config variable.
16 Migrating from 0.4.x to 0.5.0
17 =============================
19 This has been a fairly major revision of Roundup:
21 1. Brand new, much more powerful, flexible, tasty and nutritious templating.
22 Unfortunately, this means all your current templates are useless. Hopefully
23 the new documentation and examples will be enough to help you make the
24 transition. Please don't hesitate to ask on roundup-users for help (or
25 complete conversions if you're completely stuck)!
26 2. The database backed got a lot more flexible, allowing Metakit and SQL
27 databases! The only decent SQL database implemented at present is sqlite,
28 but others shouldn't be a whole lot more work.
29 3. A brand new, highly flexible and much more robust security system including
30 a system of Permissions, Roles and Role assignments to users. You may now
31 define your own Permissions that may be checked in CGI transactions.
32 4. Journalling has been made less storage-hungry, so has been turned on
33 by default *except* for author, recipient and nosy link/unlink events. You
34 are advised to turn it off in your trackers too.
35 5. We've changed the terminology from "instance" to "tracker", to ease the
36 learning curve/impact for new users.
37 6. Because of the above changes, the tracker configuration has seen some
38 major changes. See below for the details.
40 Please, **back up your database** before you start the migration process. This
41 is as simple as copying the "db" directory and all its contents from your
42 tracker to somewhere safe.
45 0.5.0 Configuration
46 -------------------
48 First up, rename your ``instance_config.py`` file to just ``config.py``.
50 Then edit your tracker's ``__init__.py`` module. It'll currently look
51 like this::
53 from instance_config import *
54 try:
55 from dbinit import *
56 except ImportError:
57 pass # in installdir (probably :)
58 from interfaces import *
60 and it needs to be::
62 import config
63 from dbinit import open, init
64 from interfaces import Client, MailGW
66 Due to the new templating having a top-level ``page`` that defines links for
67 searching, indexes, adding items etc, the following variables are no longer
68 used:
70 - HEADER_INDEX_LINKS
71 - HEADER_ADD_LINKS
72 - HEADER_SEARCH_LINKS
73 - SEARCH_FILTERS
74 - DEFAULT_INDEX
75 - UNASSIGNED_INDEX
76 - USER_INDEX
77 - ISSUE_FILTER
79 The new security implementation will require additions to the dbinit module,
80 but also removes the need for the following tracker config variables:
82 - ANONYMOUS_ACCESS
83 - ANONYMOUS_REGISTER
85 but requires two new variables which define the Roles assigned to users who
86 register through the web and e-mail interfaces:
88 - NEW_WEB_USER_ROLES
89 - NEW_EMAIL_USER_ROLES
91 in both cases, 'User' is a good initial setting. To emulate
92 ``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the
93 "Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web
94 Registration" and/or the "Email Registration" Permission from the "Anonymous"
95 Role. See the section on customising security in the `customisation
96 documentation`_ for more information.
98 Finally, the following config variables have been renamed to make more sense:
100 - INSTANCE_HOME -> TRACKER_HOME
101 - INSTANCE_NAME -> TRACKER_NAME
102 - ISSUE_TRACKER_WEB -> TRACKER_WEB
103 - ISSUE_TRACKER_EMAIL -> TRACKER_EMAIL
106 0.5.0 Schema Specification
107 --------------------------
109 0.5.0 Database backend changes
110 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
112 Your select_db module in your tracker has changed a fair bit. Where it used
113 to contain::
115 # WARNING: DO NOT EDIT THIS FILE!!!
116 from roundup.backends.back_anydbm import Database
118 it must now contain::
120 # WARNING: DO NOT EDIT THIS FILE!!!
121 from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass
123 Yes, I realise the irony of the "DO NOT EDIT THIS FILE" statement :)
124 Note the addition of the Class, FileClass, IssueClass imports. These are very
125 important, as they're going to make the next change work too. You now need to
126 modify the top of the dbinit module in your tracker from::
128 import instance_config
129 from roundup import roundupdb
130 from select_db import Database
132 from roundup.roundupdb import Class, FileClass
134 class Database(roundupdb.Database, select_db.Database):
135 ''' Creates a hybrid database from:
136 . the selected database back-end from select_db
137 . the roundup extensions from roundupdb
138 '''
139 pass
141 class IssueClass(roundupdb.IssueClass):
142 ''' issues need the email information
143 '''
144 pass
146 to::
148 import config
149 from select_db import Database, Class, FileClass, IssueClass
151 Yes, remove the Database and IssueClass definitions and those other imports.
152 They're not needed any more!
154 Look for places in dbinit.py where ``instance_config`` is used too, and
155 rename them ``config``.
158 0.5.0 Journalling changes
159 ~~~~~~~~~~~~~~~~~~~~~~~~~
161 Journalling has been optimised for storage. Journalling of links has been
162 turned back on by default. If your tracker has a large user base, you may wish
163 to turn off journalling of nosy list, message author and message recipient
164 link and unlink events. You do this by adding ``do_journal='no'`` to the Class
165 initialisation in your dbinit. For example, your *msg* class initialisation
166 probably looks like this::
168 msg = FileClass(db, "msg",
169 author=Link("user"), recipients=Multilink("user"),
170 date=Date(), summary=String(),
171 files=Multilink("file"),
172 messageid=String(), inreplyto=String())
174 to turn off journalling of author and recipient link events, add
175 ``do_journal='no'`` to the ``author=Link("user")`` part of the statement,
176 like so::
178 msg = FileClass(db, "msg",
179 author=Link("user", do_journal='no'),
180 recipients=Multilink("user", do_journal='no'),
181 date=Date(), summary=String(),
182 files=Multilink("file"),
183 messageid=String(), inreplyto=String())
185 Nosy list link event journalling is actually turned off by default now. If you
186 want to turn it on, change to your issue class' nosy list, change its
187 definition from::
189 issue = IssueClass(db, "issue",
190 assignedto=Link("user"), topic=Multilink("keyword"),
191 priority=Link("priority"), status=Link("status"))
193 to::
195 issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'),
196 assignedto=Link("user"), topic=Multilink("keyword"),
197 priority=Link("priority"), status=Link("status"))
199 noting that your definition of the nosy Multilink will override the normal one.
202 0.5.0 User schema changes
203 ~~~~~~~~~~~~~~~~~~~~~~~~~
205 Users have two more properties, "queries" and "roles". You'll have something
206 like this in your dbinit module now::
208 user = Class(db, "user",
209 username=String(), password=Password(),
210 address=String(), realname=String(),
211 phone=String(), organisation=String(),
212 alternate_addresses=String())
213 user.setkey("username")
215 and you'll need to add the new properties and the new "query" class to it
216 like so::
218 query = Class(db, "query",
219 klass=String(), name=String(),
220 url=String())
221 query.setkey("name")
223 # Note: roles is a comma-separated string of Role names
224 user = Class(db, "user",
225 username=String(), password=Password(),
226 address=String(), realname=String(),
227 phone=String(), organisation=String(),
228 alternate_addresses=String(),
229 queries=Multilink('query'), roles=String())
230 user.setkey("username")
232 The "queries" property is used to store off the user's favourite database
233 queries. The "roles" property is explained below in `0.5.0 Security
234 Settings`_.
237 0.5.0 Security Settings
238 ~~~~~~~~~~~~~~~~~~~~~~~
240 See the `security documentation`_ for an explanation of how the new security
241 system works. In a nutshell though, the security is handled as a four step
242 process:
244 1. Permissions are defined as having a name and optionally a hyperdb class
245 they're specific to,
246 2. Roles are defined that have one or more Permissions,
247 3. Users are assigned Roles in their "roles" property, and finally
248 4. Roundup checks that users have appropriate Permissions at appropriate times
249 (like editing issues).
251 Your tracker dbinit module's *open* function now has to define any
252 Permissions that are specific to your tracker, and also the assignment
253 of Permissions to Roles. At the moment, your open function
254 ends with::
256 import detectors
257 detectors.init(db)
259 return db
261 and what we need to do is insert some commands that will set up the security
262 parameters. Right above the ``import detectors`` line, you'll want to insert
263 these lines::
265 #
266 # SECURITY SETTINGS
267 #
268 # new permissions for this schema
269 for cl in 'issue', 'file', 'msg', 'user':
270 db.security.addPermission(name="Edit", klass=cl,
271 description="User is allowed to edit "+cl)
272 db.security.addPermission(name="View", klass=cl,
273 description="User is allowed to access "+cl)
275 # Assign the access and edit permissions for issue, file and message
276 # to regular users now
277 for cl in 'issue', 'file', 'msg':
278 p = db.security.getPermission('View', cl)
279 db.security.addPermissionToRole('User', p)
280 p = db.security.getPermission('Edit', cl)
281 db.security.addPermissionToRole('User', p)
282 # and give the regular users access to the web and email interface
283 p = db.security.getPermission('Web Access')
284 db.security.addPermissionToRole('User', p)
285 p = db.security.getPermission('Email Access')
286 db.security.addPermissionToRole('User', p)
288 # May users view other user information? Comment these lines out
289 # if you don't want them to
290 p = db.security.getPermission('View', 'user')
291 db.security.addPermissionToRole('User', p)
293 # Assign the appropriate permissions to the anonymous user's Anonymous
294 # Role. Choices here are:
295 # - Allow anonymous users to register through the web
296 p = db.security.getPermission('Web Registration')
297 db.security.addPermissionToRole('Anonymous', p)
298 # - Allow anonymous (new) users to register through the email gateway
299 p = db.security.getPermission('Email Registration')
300 db.security.addPermissionToRole('Anonymous', p)
301 # - Allow anonymous users access to the "issue" class of data
302 # Note: this also grants access to related information like files,
303 # messages, statuses etc that are linked to issues
304 #p = db.security.getPermission('View', 'issue')
305 #db.security.addPermissionToRole('Anonymous', p)
306 # - Allow anonymous users access to edit the "issue" class of data
307 # Note: this also grants access to create related information like
308 # files and messages etc that are linked to issues
309 #p = db.security.getPermission('Edit', 'issue')
310 #db.security.addPermissionToRole('Anonymous', p)
312 # oh, g'wan, let anonymous access the web interface too
313 p = db.security.getPermission('Web Access')
314 db.security.addPermissionToRole('Anonymous', p)
316 Note in the comments there the places where you might change the permissions
317 to restrict users or grant users more access. If you've created additional
318 classes that users should be able to edit and view, then you should add them
319 to the "new permissions for this schema" section at the start of the security
320 block. Then add them to the "Assign the access and edit permissions" section
321 too, so people actually have the new Permission you've created.
323 One final change is needed that finishes off the security system's
324 initialisation. We need to add a call to ``db.post_init()`` at the end of the
325 dbinit open() function. Add it like this::
327 import detectors
328 detectors.init(db)
330 # schema is set up - run any post-initialisation
331 db.post_init()
332 return db
334 You may verify the setup of Permissions and Roles using the new
335 "``roundup-admin security``" command.
338 0.5.0 User changes
339 ~~~~~~~~~~~~~~~~~~
341 To support all those schema changes, you'll need to massage your user database
342 a little too, to:
344 1. make sure there's an "anonymous" user - this user is mandatory now and is
345 the one that unknown users are logged in as.
346 2. make sure all users have at least one Role.
348 If you don't have the "anonymous" user, create it now with the command::
350 roundup-admin create user username=anonymous roles=Anonymous
352 making sure the capitalisation is the same as above. Once you've done that,
353 you'll need to set the roles property on all users to a reasonable default.
354 The admin user should get "Admin", the anonymous user "Anonymous"
355 and all other users "User". The ``fixroles.py`` script in the tools directory
356 will do this. Run it like so (where python is your python 2+ binary)::
358 python tools/fixroles.py -i <tracker home> fixroles
362 0.5.0 CGI interface changes
363 ---------------------------
365 The CGI interface code was completely reorganised and largely rewritten. The
366 end result is that this section of your tracker interfaces module will need
367 changing from::
369 from roundup import cgi_client, mailgw
370 from roundup.i18n import _
372 class Client(cgi_client.Client):
373 ''' derives basic CGI implementation from the standard module,
374 with any specific extensions
375 '''
376 pass
378 to::
380 from roundup import mailgw
381 from roundup.cgi import client
383 class Client(client.Client):
384 ''' derives basic CGI implementation from the standard module,
385 with any specific extensions
386 '''
387 pass
389 You will also need to install the new version of roundup.cgi from the source
390 cgi-bin directory if you're using it.
393 0.5.0 HTML templating
394 ---------------------
396 You'll want to make a backup of your current tracker html directory. You
397 should then copy the html directory from the Roundup source "classic" template
398 and modify it according to your local schema changes.
400 If you need help with the new templating system, please ask questions on the
401 roundup-users mailing list (available through the roundup project page on
402 sourceforge, http://roundup.sf.net/)
405 0.5.0 Detectors
406 ---------------
408 The nosy reactor has been updated to handle the tracker not having an
409 "assignedto" property on issues. You may want to copy it into your tracker's
410 detectors directory. Chances are you've already fixed it though :)
413 Migrating from 0.4.1 to 0.4.2
414 =============================
416 0.4.2 Configuration
417 -------------------
418 The USER_INDEX definition introduced in 0.4.1 was too restrictive in its
419 allowing replacement of 'assignedto' with the user's userid. Users must change
420 the None value of 'assignedto' to 'CURRENT USER' (the string, in quotes) for
421 the replacement behaviour to occur now.
423 The new configuration variables are:
425 - EMAIL_KEEP_QUOTED_TEXT
426 - EMAIL_LEAVE_BODY_UNCHANGED
427 - ADD_RECIPIENTS_TO_NOSY
429 See the sample configuration files in::
431 <roundup source>/roundup/templates/classic/instance_config.py
433 and::
435 <roundup source>/roundup/templates/extended/instance_config.py
437 and the `customisation documentation`_ for information on how they're used.
440 0.4.2 Changes to detectors
441 --------------------------
442 You will need to copy the detectors from the distribution into your instance
443 home "detectors" directory. If you used the classic schema, the detectors
444 are in::
446 <roundup source>/roundup/templates/classic/detectors/
448 If you used the extended schema, the detectors are in::
450 <roundup source>/roundup/templates/extended/detectors/
452 The change means that schema-specific code has been removed from the
453 mail gateway and cgi interface and made into auditors:
455 - nosyreactor.py has now got an updatenosy auditor which updates the nosy
456 list with author, recipient and assignedto information.
457 - statusauditor.py makes the unread or resolved -> chatting changes and
458 presets the status of an issue to unread.
460 There's also a bug or two fixed in the nosyreactor code.
462 0.4.2 HTML templating changes
463 -----------------------------
464 The link() htmltemplate function now has a "showid" option for links and
465 multilinks. When true, it only displays the linked item id as the anchor
466 text. The link value is displayed as a tooltip using the title anchor
467 attribute. To use in eg. the superseder field, have something like this::
469 <td>
470 <display call="field('superseder', showid=1)">
471 <display call="classhelp('issue', 'id,title', label='list', width=500)">
472 <property name="superseder">
473 <br>View: <display call="link('superseder', showid=1)">
474 </property>
475 </td>
477 The stylesheets have been cleaned up too. You may want to use the newer
478 versions in::
480 <roundup source>/roundup/templates/<template>/html/default.css
484 Migrating from 0.4.0 to 0.4.1
485 =============================
487 0.4.1 Files storage
488 -------------------
490 Messages and files from newly created issues will be put into subdierectories
491 in thousands e.g. msg123 will be put into files/msg/0/msg123, file2003
492 will go into files/file/2/file2003. Previous messages are still found, but
493 could be put into this structure.
495 0.4.1 Configuration
496 -------------------
498 To allow more fine-grained access control, the variable used to check
499 permission to auto-register users in the mail gateway is now called
500 ANONYMOUS_REGISTER_MAIL rather than overloading ANONYMOUS_REGISTER. If the
501 variable doesn't exist, then ANONYMOUS_REGISTER is tested as before.
503 Configuring the links in the web header is now easier too. The following
504 variables have been added to the classic instance_config.py::
506 HEADER_INDEX_LINKS - defines the "index" links to be made available
507 HEADER_ADD_LINKS - defines the "add" links
508 DEFAULT_INDEX - specifies the index view for DEFAULT
509 UNASSIGNED_INDEX - specifies the index view for UNASSIGNED
510 USER_INDEX - specifies the index view for USER
512 See the <roundup source>/roundup/templates/classic/instance_config.py for more
513 information - including how the variables are to be set up. Most users will
514 just be able to copy the variables from the source to their instance home. If
515 you've modified the header by changing the source of the interfaces.py file in
516 the instance home, you'll need to remove that customisation and move it into
517 the appropriate variables in instance_config.py.
519 The extended schema has similar variables added too - see the source for more
520 info.
522 0.4.1 Alternate E-Mail Addresses
523 --------------------------------
525 If you add the property "alternate_addresses" to your user class, your users
526 will be able to register alternate email addresses that they may use to
527 communicate with roundup as. All email from roundup will continue to be sent
528 to their primary address.
530 If you have not edited the dbinit.py file in your instance home directory,
531 you may simply copy the new dbinit.py file from the core code. If you used
532 the classic schema, the interfaces file is in::
534 <roundup source>/roundup/templates/classic/dbinit.py
536 If you used the extended schema, the file is in::
538 <roundup source>/roundup/templates/extended/dbinit.py
540 If you have modified your dbinit.py file, you need to edit the dbinit.py
541 file in your instance home directory. Find the lines which define the user
542 class::
544 user = Class(db, "msg",
545 username=String(), password=Password(),
546 address=String(), realname=String(),
547 phone=String(), organisation=String(),
548 alternate_addresses=String())
550 You will also want to add the property to the user's details page. The
551 template for this is the "user.item" file in your instance home "html"
552 directory. Similar to above, you may copy the file from the roundup source if
553 you haven't modified it. Otherwise, add the following to the template::
555 <display call="multiline('alternate_addresses')">
557 with appropriate labelling etc. See the standard template for an idea.
561 Migrating from 0.3.x to 0.4.0
562 =============================
564 0.4.0 Message-ID and In-Reply-To addition
565 -----------------------------------------
566 0.4.0 adds the tracking of messages by message-id and allows threading
567 using in-reply-to. Most e-mail clients support threading using this
568 feature, and we hope to add support for it to the web gateway. If you
569 have not edited the dbinit.py file in your instance home directory, you may
570 simply copy the new dbinit.py file from the core code. If you used the
571 classic schema, the interfaces file is in::
573 <roundup source>/roundup/templates/classic/dbinit.py
575 If you used the extended schema, the file is in::
577 <roundup source>/roundup/templates/extended/dbinit.py
579 If you have modified your dbinit.py file, you need to edit the dbinit.py
580 file in your instance home directory. Find the lines which define the msg
581 class::
583 msg = FileClass(db, "msg",
584 author=Link("user"), recipients=Multilink("user"),
585 date=Date(), summary=String(),
586 files=Multilink("file"))
588 and add the messageid and inreplyto properties like so::
590 msg = FileClass(db, "msg",
591 author=Link("user"), recipients=Multilink("user"),
592 date=Date(), summary=String(),
593 files=Multilink("file"),
594 messageid=String(), inreplyto=String())
596 Also, configuration is being cleaned up. This means that your dbinit.py will
597 also need to be changed in the open function. If you haven't changed your
598 dbinit.py, the above copy will be enough. If you have, you'll need to change
599 the line (round line 50)::
601 db = Database(instance_config.DATABASE, name)
603 to::
605 db = Database(instance_config, name)
608 0.4.0 Configuration
609 --------------------
610 ``TRACKER_NAME`` and ``EMAIL_SIGNATURE_POSITION`` have been added to the
611 instance_config.py. The simplest solution is to copy the default values
612 from template in the core source.
614 The mail gateway now checks ``ANONYMOUS_REGISTER`` to see if unknown users
615 are to be automatically registered with the tracker. If it is set to "deny"
616 then unknown users will not have access. If it is set to "allow" they will be
617 automatically registered with the tracker.
620 0.4.0 CGI script roundup.cgi
621 ----------------------------
622 The CGI script has been updated with some features and a bugfix, so you should
623 copy it from the roundup cgi-bin source directory again. Make sure you update
624 the ROUNDUP_INSTANCE_HOMES after the copy.
627 0.4.0 Nosy reactor
628 ------------------
629 The nosy reactor has also changed - copy the nosyreactor.py file from the core
630 source::
632 <roundup source>/roundup/templates/<template>/detectors/nosyreactor.py
634 to your instance home "detectors" directory.
637 0.4.0 HTML templating
638 ---------------------
639 The field() function was incorrectly implemented - links and multilinks now
640 display as text fields when rendered using field(). To display a menu (drop-
641 down or select box) you need to use the menu() function.
645 Migrating from 0.2.x to 0.3.x
646 =============================
648 0.3.x Cookie Authentication changes
649 -----------------------------------
650 0.3.0 introduces cookie authentication - you will need to copy the
651 interfaces.py file from the roundup source to your instance home to enable
652 authentication. If you used the classic schema, the interfaces file is in::
654 <roundup source>/roundup/templates/classic/interfaces.py
656 If you used the extended schema, the file is in::
658 <roundup source>/roundup/templates/extended/interfaces.py
660 If you have modified your interfaces.Client class, you will need to take
661 note of the login/logout functionality provided in roundup.cgi_client.Client
662 (classic schema) or roundup.cgi_client.ExtendedClient (extended schema) and
663 modify your instance code apropriately.
666 0.3.x Password encoding
667 -----------------------
668 This release also introduces encoding of passwords in the database. If you
669 have not edited the dbinit.py file in your instance home directory, you may
670 simply copy the new dbinit.py file from the core code. If you used the
671 classic schema, the interfaces file is in::
673 <roundup source>/roundup/templates/classic/dbinit.py
675 If you used the extended schema, the file is in::
677 <roundup source>/roundup/templates/extended/dbinit.py
680 If you have modified your dbinit.py file, you may use encoded passwords:
682 1. Edit the dbinit.py file in your instance home directory
683 a. At the first code line of the open() function::
685 from roundup.hyperdb import String, Date, Link, Multilink
687 alter to include Password, as so::
689 from roundup.hyperdb import String, Password, Date, Link, Multilink
691 b. Where the password property is defined (around line 66)::
693 user = Class(db, "user",
694 username=String(), password=String(),
695 address=String(), realname=String(),
696 phone=String(), organisation=String())
697 user.setkey("username")
699 alter the "password=String()" to "password=Password()"::
701 user = Class(db, "user",
702 username=String(), password=Password(),
703 address=String(), realname=String(),
704 phone=String(), organisation=String())
705 user.setkey("username")
707 2. Any existing passwords in the database will remain cleartext until they
708 are edited. It is recommended that at a minimum the admin password be
709 changed immediately::
711 roundup-admin -i <instance home> set user1 password=<new password>
714 0.3.x Configuration
715 -------------------
716 FILTER_POSITION, ANONYMOUS_ACCESS, ANONYMOUS_REGISTER have been added to
717 the instance_config.py. Simplest solution is to copy the default values from
718 template in the core source.
720 MESSAGES_TO_AUTHOR has been added to the IssueClass in dbinit.py. Set to 'yes'
721 to send nosy messages to the author. Default behaviour is to not send nosy
722 messages to the author. You will need to add MESSAGES_TO_AUTHOR to your
723 dbinit.py in your instance home.
726 0.3.x CGI script roundup.cgi
727 ----------------------------
728 There have been some structural changes to the roundup.cgi script - you will
729 need to install it again from the cgi-bin directory of the source
730 distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the
731 copy.
734 .. _`customisation documentation`: customizing.html
735 .. _`security documentation`: security.html