ce6ae6dfaa91843df66e4ec1df283eeefd269eb8
1 ======================================
2 Upgrading to newer versions of Roundup
3 ======================================
5 Please read each section carefully and edit your tracker home files
6 accordingly.
8 .. contents::
10 Migrating from 0.4.x to 0.5.0
11 =============================
13 This has been a fairly major revision of Roundup:
15 1. Brand new, much more powerful, flexible, tasty and nutritious templating.
16 Unfortunately, this means all your current templates are useless. Please
17 don't hesitate to ask on roundup-users for help (or complete conversions
18 if you're completely stuck)!
19 2. The database backed got a lot more flexible, allowing Metakit and SQL
20 databases! The only SQL database implemented at present is gadfly, but
21 others shouldn't be a whole lot more work.
22 3. A brand new, highly flexible and much more robust security system including
23 a system of Permissions, Roles and Role assignments to users. You may now
24 define your own Permissions that may be checked in CGI transactions.
25 4. Journalling has been made less storage-hungry, so has been turned on
26 by default *except* for author, recipient and nosy link/unlink events. You
27 are advised to turn it off in your trackers too.
28 5. We've changed the terminology from "instance" to "tracker", to ease the
29 learning curve/impact for new users.
30 6. Because of the above changes, the tracker configuration has seen some
31 major changes. See below for the details.
33 Please, *back up your database* before you start the migration process. This
34 is as simple as copying the "db" directory and all its contents from your
35 tracker to somewhere safe.
38 0.5.0 Configuration
39 -------------------
41 First up, rename your ``instance_config.py`` file to just ``config.py``.
43 Then edit your tracker's ``__init__.py`` module. It'll currently look
44 like this::
46 from instance_config import *
47 try:
48 from dbinit import *
49 except ImportError:
50 pass # in installdir (probably :)
51 from interfaces import *
53 and it needs to be::
55 import config
56 from dbinit import open, init
57 from interfaces import Client, MailGW
59 Due to the new templating having a top-level ``page`` that defines links for
60 searching, indexes, adding items etc, the following variables are no longer
61 used:
63 - HEADER_INDEX_LINKS
64 - HEADER_ADD_LINKS
65 - HEADER_SEARCH_LINKS
66 - SEARCH_FILTERS
67 - DEFAULT_INDEX
68 - UNASSIGNED_INDEX
69 - USER_INDEX
70 - ISSUE_FILTER
72 The new security implementation will require additions to the dbinit module,
73 but also removes the need for the following tracker config variables:
75 - ANONYMOUS_ACCESS
76 - ANONYMOUS_REGISTER
78 but requires two new variables which define the Roles assigned to users who
79 register through the web and e-mail interfaces:
81 - NEW_WEB_USER_ROLES
82 - NEW_EMAIL_USER_ROLES
84 in both cases, 'User' is a good initial setting. To emulate
85 ``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the
86 "Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web
87 Registration" and/or the "Email Registration" Permission from the "Anonymous"
88 Role. See the section on customising security in the `customisation
89 documentation`_ for more information.
92 0.5.0 Schema Specification
93 --------------------------
95 0.5.0 Database backend changes
96 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98 Your select_db module in your tracker has changed a fair bit. Where it used
99 to contain::
101 # WARNING: DO NOT EDIT THIS FILE!!!
102 from roundup.backends.back_anydbm import Database
104 it must now contain::
106 # WARNING: DO NOT EDIT THIS FILE!!!
107 from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass
109 Yes, I realise the irony of the "DO NOT EDIT THIS FILE" statement :)
110 Note the addition of the Class, FileClass, IssueClass imports. These are very
111 important, as they're going to make the next change work too. You now need to
112 modify the top of the dbinit module in your tracker from::
114 import instance_config
115 from roundup import roundupdb
116 from select_db import Database
118 from roundup.roundupdb import Class, FileClass
120 class Database(roundupdb.Database, select_db.Database):
121 ''' Creates a hybrid database from:
122 . the selected database back-end from select_db
123 . the roundup extensions from roundupdb
124 '''
125 pass
127 class IssueClass(roundupdb.IssueClass):
128 ''' issues need the email information
129 '''
130 pass
132 to::
134 import config
135 from select_db import Database, Class, FileClass, IssueClass
137 Yes, remove the Database and IssueClass definitions and those other imports.
138 They're not needed any more!
140 Look for places in dbinit.py where ``instance_config`` is used too, and
141 rename them ``config``.
144 0.5.0 Journalling changes
145 ~~~~~~~~~~~~~~~~~~~~~~~~~
147 Journalling has been optimised for storage. Journalling of links has been
148 turned back on by default. If your tracker has a large user base, you may wish
149 to turn off journalling of nosy list, message author and message recipient
150 link and unlink events. You do this by adding ``do_journal='no'`` to the Class
151 initialisation in your dbinit. For example, your *msg* class initialisation
152 probably looks like this::
154 msg = FileClass(db, "msg",
155 author=Link("user"), recipients=Multilink("user"),
156 date=Date(), summary=String(),
157 files=Multilink("file"),
158 messageid=String(), inreplyto=String())
160 to turn off journalling of author and recipient link events, add
161 ``do_journal='no'`` to the ``author=Link("user")`` part of the statement,
162 like so::
164 msg = FileClass(db, "msg",
165 author=Link("user", do_journal='no'),
166 recipients=Multilink("user", do_journal='no'),
167 date=Date(), summary=String(),
168 files=Multilink("file"),
169 messageid=String(), inreplyto=String())
171 Nosy list link event journalling is actually turned off by default now. If you
172 want to turn it onn, change to your issue class' nosy list, change its
173 definition from::
175 issue = IssueClass(db, "issue",
176 assignedto=Link("user"), topic=Multilink("keyword"),
177 priority=Link("priority"), status=Link("status"))
179 to::
181 issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'),
182 assignedto=Link("user"), topic=Multilink("keyword"),
183 priority=Link("priority"), status=Link("status"))
185 noting that your definition of the nosy Multilink will override the normal one.
188 0.5.0 User schema changes
189 ~~~~~~~~~~~~~~~~~~~~~~~~~
191 Users have two more properties, "queries" and "roles". You'll have something
192 like this in your dbinit module now::
194 user = Class(db, "user",
195 username=String(), password=Password(),
196 address=String(), realname=String(),
197 phone=String(), organisation=String(),
198 alternate_addresses=String())
199 user.setkey("username")
201 and you'll need to add the new properties and the new "query" class to it
202 like so::
204 query = Class(db, "query",
205 klass=String(), name=String(),
206 url=String())
207 query.setkey("name")
209 # Note: roles is a comma-separated string of Role names
210 user = Class(db, "user",
211 username=String(), password=Password(),
212 address=String(), realname=String(),
213 phone=String(), organisation=String(),
214 alternate_addresses=String(),
215 queries=Multilink('query'), roles=String())
216 user.setkey("username")
218 The "queries" property is used to store off the user's favourite database
219 queries. The "roles" property is explained below in `0.5.0 Security
220 Settings`_.
223 0.5.0 Security Settings
224 ~~~~~~~~~~~~~~~~~~~~~~~
226 See the `security documentation`_ for an explanation of how the new security
227 system works. In a nutshell though, the security is handled as a four step
228 process:
230 1. Permissions are defined as having a name and optionally a hyperdb class
231 they're specific to,
232 2. Roles are defined that have one or more Permissions,
233 3. Users are assigned Roles in their "roles" property, and finally
234 4. Roundup checks that users have appropriate Permissions at appropriate times
235 (like editing issues).
237 Your tracker dbinit module's *open* function now has to define any
238 Permissions that are specific to your tracker, and also the assignment
239 of Permissions to Roles. At the moment, your open function
240 ends with::
242 import detectors
243 detectors.init(db)
245 return db
247 and what we need to do is insert some commands that will set up the security
248 parameters. Right above the ``import detectors`` line, you'll want to insert
249 these lines::
251 #
252 # SECURITY SETTINGS
253 #
254 # new permissions for this schema
255 for cl in 'issue', 'file', 'msg', 'user':
256 db.security.addPermission(name="Edit", klass=cl,
257 description="User is allowed to edit "+cl)
258 db.security.addPermission(name="View", klass=cl,
259 description="User is allowed to access "+cl)
261 # Assign the access and edit permissions for issue, file and message
262 # to regular users now
263 for cl in 'issue', 'file', 'msg':
264 p = db.security.getPermission('View', cl)
265 db.security.addPermissionToRole('User', p)
266 p = db.security.getPermission('Edit', cl)
267 db.security.addPermissionToRole('User', p)
268 # and give the regular users access to the web and email interface
269 p = db.security.getPermission('Web Access')
270 db.security.addPermissionToRole('User', p)
271 p = db.security.getPermission('Email Access')
272 db.security.addPermissionToRole('User', p)
274 # May users view other user information? Comment these lines out
275 # if you don't want them to
276 p = db.security.getPermission('View', 'user')
277 db.security.addPermissionToRole('User', p)
279 # Assign the appropriate permissions to the anonymous user's Anonymous
280 # Role. Choices here are:
281 # - Allow anonymous users to register through the web
282 p = db.security.getPermission('Web Registration')
283 db.security.addPermissionToRole('Anonymous', p)
284 # - Allow anonymous (new) users to register through the email gateway
285 p = db.security.getPermission('Email Registration')
286 db.security.addPermissionToRole('Anonymous', p)
287 # - Allow anonymous users access to the "issue" class of data
288 # Note: this also grants access to related information like files,
289 # messages, statuses etc that are linked to issues
290 #p = db.security.getPermission('View', 'issue')
291 #db.security.addPermissionToRole('Anonymous', p)
292 # - Allow anonymous users access to edit the "issue" class of data
293 # Note: this also grants access to create related information like
294 # files and messages etc that are linked to issues
295 #p = db.security.getPermission('Edit', 'issue')
296 #db.security.addPermissionToRole('Anonymous', p)
298 # oh, g'wan, let anonymous access the web interface too
299 p = db.security.getPermission('Web Access')
300 db.security.addPermissionToRole('Anonymous', p)
302 Note in the comments there the places where you might change the permissions
303 to restrict users or grant users more access. If you've created additional
304 classes that users should be able to edit and view, then you should add them
305 to the "new permissions for this schema" section at the start of the security
306 block. Then add them to the "Assign the access and edit permissions" section
307 too, so people actually have the new Permission you've created.
309 One final change is needed that finishes off the security system's
310 initialisation. We need to add a call to ``db.post_init()`` at the end of the
311 dbinit open() function. Add it like this::
313 import detectors
314 detectors.init(db)
316 # schema is set up - run any post-initialisation
317 db.post_init()
318 return db
320 You may verify the setup of Permissions and Roles using the new
321 "``roundup-admin security``" command.
324 0.5.0 User changes
325 ~~~~~~~~~~~~~~~~~~
327 To support all those schema changes, you'll need to massage your user database
328 a little too, to:
330 1. make sure there's an "anonymous" user - this user is mandatory now and is
331 the one that unknown users are logged in as.
332 2. make sure all users have at least one Role.
334 If you don't have the "anonymous" user, create it now with the command::
336 roundup-admin create user username=anonymous roles=Anonymous
338 making sure the capitalisation is the same as above. Once you've done that,
339 you'll need to set the roles property on all users to a reasonable default.
340 The admin user should get "Admin", the anonymous user "Anonymous"
341 and all other users "User". The ``fixroles.py`` script in the tools directory
342 will do this. Run it like so (where python is your python 2+ binary)::
344 python tools/fixroles.py -i <tracker home>
348 0.5.0 CGI interface changes
349 ---------------------------
351 The CGI interface code was completely reorganised and largely rewritten. The
352 end result is that this section of your tracker interfaces module will need
353 changing from::
355 from roundup import mailgw
356 from roundup.cgi import client
358 class Client(client.Client):
359 ''' derives basic CGI implementation from the standard module,
360 with any specific extensions
361 '''
362 pass
364 to::
366 from roundup import cgi_client, mailgw
367 from roundup.i18n import _
369 class Client(cgi_client.Client):
370 ''' derives basic CGI implementation from the standard module,
371 with any specific extensions
372 '''
373 pass
375 You will also need to install the new version of roundup.cgi from the source
376 cgi-bin directory if you're using it.
379 0.5.0 HTML templating
380 ---------------------
382 You'll want to make a backup of your current tracker html directory. You
383 should then copy the html directory from the Roundup source template that you
384 used to create your tracker, and modify it according to your local schema
385 changes.
387 If you need help with the new templating system, please ask questions on the
388 roundup-users mailing list (available through the roundup project page on
389 sourceforge, http://roundup.sf.net/)
392 0.5.0 Detectors
393 ---------------
395 The nosy reactor has been updated to handle the tracker not having an
396 "assignedto" property on issues. You may want to copy it into your tracker's
397 detectors directory. Chances are you've already fixed it though :)
400 Migrating from 0.4.1 to 0.4.2
401 =============================
403 0.4.2 Configuration
404 -------------------
405 The USER_INDEX definition introduced in 0.4.1 was too restrictive in its
406 allowing replacement of 'assignedto' with the user's userid. Users must change
407 the None value of 'assignedto' to 'CURRENT USER' (the string, in quotes) for
408 the replacement behaviour to occur now.
410 The new configuration variables are:
412 - EMAIL_KEEP_QUOTED_TEXT
413 - EMAIL_LEAVE_BODY_UNCHANGED
414 - ADD_RECIPIENTS_TO_NOSY
416 See the sample configuration files in::
418 <roundup source>/roundup/templates/classic/instance_config.py
420 and::
422 <roundup source>/roundup/templates/extended/instance_config.py
424 and the `customisation documentation`_ for information on how they're used.
427 0.4.2 Changes to detectors
428 --------------------------
429 You will need to copy the detectors from the distribution into your instance
430 home "detectors" directory. If you used the classic schema, the detectors
431 are in::
433 <roundup source>/roundup/templates/classic/detectors/
435 If you used the extended schema, the detectors are in::
437 <roundup source>/roundup/templates/extended/detectors/
439 The change means that schema-specific code has been removed from the
440 mail gateway and cgi interface and made into auditors:
442 - nosyreactor.py has now got an updatenosy auditor which updates the nosy
443 list with author, recipient and assignedto information.
444 - statusauditor.py makes the unread or resolved -> chatting changes and
445 presets the status of an issue to unread.
447 There's also a bug or two fixed in the nosyreactor code.
449 0.4.2 HTML templating changes
450 -----------------------------
451 The link() htmltemplate function now has a "showid" option for links and
452 multilinks. When true, it only displays the linked item id as the anchor
453 text. The link value is displayed as a tooltip using the title anchor
454 attribute. To use in eg. the superseder field, have something like this::
456 <td>
457 <display call="field('superseder', showid=1)">
458 <display call="classhelp('issue', 'id,title', label='list', width=500)">
459 <property name="superseder">
460 <br>View: <display call="link('superseder', showid=1)">
461 </property>
462 </td>
464 The stylesheets have been cleaned up too. You may want to use the newer
465 versions in::
467 <roundup source>/roundup/templates/<template>/html/default.css
471 Migrating from 0.4.0 to 0.4.1
472 =============================
474 0.4.1 Files storage
475 -------------------
477 Messages and files from newly created issues will be put into subdierectories
478 in thousands e.g. msg123 will be put into files/msg/0/msg123, file2003
479 will go into files/file/2/file2003. Previous messages are still found, but
480 could be put into this structure.
482 0.4.1 Configuration
483 -------------------
485 To allow more fine-grained access control, the variable used to check
486 permission to auto-register users in the mail gateway is now called
487 ANONYMOUS_REGISTER_MAIL rather than overloading ANONYMOUS_REGISTER. If the
488 variable doesn't exist, then ANONYMOUS_REGISTER is tested as before.
490 Configuring the links in the web header is now easier too. The following
491 variables have been added to the classic instance_config.py::
493 HEADER_INDEX_LINKS - defines the "index" links to be made available
494 HEADER_ADD_LINKS - defines the "add" links
495 DEFAULT_INDEX - specifies the index view for DEFAULT
496 UNASSIGNED_INDEX - specifies the index view for UNASSIGNED
497 USER_INDEX - specifies the index view for USER
499 See the <roundup source>/roundup/templates/classic/instance_config.py for more
500 information - including how the variables are to be set up. Most users will
501 just be able to copy the variables from the source to their instance home. If
502 you've modified the header by changing the source of the interfaces.py file in
503 the instance home, you'll need to remove that customisation and move it into
504 the appropriate variables in instance_config.py.
506 The extended schema has similar variables added too - see the source for more
507 info.
509 0.4.1 Alternate E-Mail Addresses
510 --------------------------------
512 If you add the property "alternate_addresses" to your user class, your users
513 will be able to register alternate email addresses that they may use to
514 communicate with roundup as. All email from roundup will continue to be sent
515 to their primary address.
517 If you have not edited the dbinit.py file in your instance home directory,
518 you may simply copy the new dbinit.py file from the core code. If you used
519 the classic schema, the interfaces file is in::
521 <roundup source>/roundup/templates/classic/dbinit.py
523 If you used the extended schema, the file is in::
525 <roundup source>/roundup/templates/extended/dbinit.py
527 If you have modified your dbinit.py file, you need to edit the dbinit.py
528 file in your instance home directory. Find the lines which define the user
529 class::
531 user = Class(db, "msg",
532 username=String(), password=Password(),
533 address=String(), realname=String(),
534 phone=String(), organisation=String(),
535 alternate_addresses=String())
537 You will also want to add the property to the user's details page. The
538 template for this is the "user.item" file in your instance home "html"
539 directory. Similar to above, you may copy the file from the roundup source if
540 you haven't modified it. Otherwise, add the following to the template::
542 <display call="multiline('alternate_addresses')">
544 with appropriate labelling etc. See the standard template for an idea.
548 Migrating from 0.3.x to 0.4.0
549 =============================
551 0.4.0 Message-ID and In-Reply-To addition
552 -----------------------------------------
553 0.4.0 adds the tracking of messages by message-id and allows threading
554 using in-reply-to. Most e-mail clients support threading using this
555 feature, and we hope to add support for it to the web gateway. If you
556 have not edited the dbinit.py file in your instance home directory, you may
557 simply copy the new dbinit.py file from the core code. If you used the
558 classic schema, the interfaces file is in::
560 <roundup source>/roundup/templates/classic/dbinit.py
562 If you used the extended schema, the file is in::
564 <roundup source>/roundup/templates/extended/dbinit.py
566 If you have modified your dbinit.py file, you need to edit the dbinit.py
567 file in your instance home directory. Find the lines which define the msg
568 class::
570 msg = FileClass(db, "msg",
571 author=Link("user"), recipients=Multilink("user"),
572 date=Date(), summary=String(),
573 files=Multilink("file"))
575 and add the messageid and inreplyto properties like so::
577 msg = FileClass(db, "msg",
578 author=Link("user"), recipients=Multilink("user"),
579 date=Date(), summary=String(),
580 files=Multilink("file"),
581 messageid=String(), inreplyto=String())
583 Also, configuration is being cleaned up. This means that your dbinit.py will
584 also need to be changed in the open function. If you haven't changed your
585 dbinit.py, the above copy will be enough. If you have, you'll need to change
586 the line (round line 50)::
588 db = Database(instance_config.DATABASE, name)
590 to::
592 db = Database(instance_config, name)
595 0.4.0 Configuration
596 --------------------
597 ``INSTANCE_NAME`` and ``EMAIL_SIGNATURE_POSITION`` have been added to the
598 instance_config.py. The simplest solution is to copy the default values
599 from template in the core source.
601 The mail gateway now checks ``ANONYMOUS_REGISTER`` to see if unknown users
602 are to be automatically registered with the tracker. If it is set to "deny"
603 then unknown users will not have access. If it is set to "allow" they will be
604 automatically registered with the tracker.
607 0.4.0 CGI script roundup.cgi
608 ----------------------------
609 The CGI script has been updated with some features and a bugfix, so you should
610 copy it from the roundup cgi-bin source directory again. Make sure you update
611 the ROUNDUP_INSTANCE_HOMES after the copy.
614 0.4.0 Nosy reactor
615 ------------------
616 The nosy reactor has also changed - copy the nosyreactor.py file from the core
617 source::
619 <roundup source>/roundup/templates/<template>/detectors/nosyreactor.py
621 to your instance home "detectors" directory.
624 0.4.0 HTML templating
625 ---------------------
626 The field() function was incorrectly implemented - links and multilinks now
627 display as text fields when rendered using field(). To display a menu (drop-
628 down or select box) you need to use the menu() function.
632 Migrating from 0.2.x to 0.3.x
633 =============================
635 0.3.x Cookie Authentication changes
636 -----------------------------------
637 0.3.0 introduces cookie authentication - you will need to copy the
638 interfaces.py file from the roundup source to your instance home to enable
639 authentication. If you used the classic schema, the interfaces file is in::
641 <roundup source>/roundup/templates/classic/interfaces.py
643 If you used the extended schema, the file is in::
645 <roundup source>/roundup/templates/extended/interfaces.py
647 If you have modified your interfaces.Client class, you will need to take
648 note of the login/logout functionality provided in roundup.cgi_client.Client
649 (classic schema) or roundup.cgi_client.ExtendedClient (extended schema) and
650 modify your instance code apropriately.
653 0.3.x Password encoding
654 -----------------------
655 This release also introduces encoding of passwords in the database. If you
656 have not edited the dbinit.py file in your instance home directory, you may
657 simply copy the new dbinit.py file from the core code. If you used the
658 classic schema, the interfaces file is in::
660 <roundup source>/roundup/templates/classic/dbinit.py
662 If you used the extended schema, the file is in::
664 <roundup source>/roundup/templates/extended/dbinit.py
667 If you have modified your dbinit.py file, you may use encoded passwords:
669 1. Edit the dbinit.py file in your instance home directory
670 a. At the first code line of the open() function::
672 from roundup.hyperdb import String, Date, Link, Multilink
674 alter to include Password, as so::
676 from roundup.hyperdb import String, Password, Date, Link, Multilink
678 b. Where the password property is defined (around line 66)::
680 user = Class(db, "user",
681 username=String(), password=String(),
682 address=String(), realname=String(),
683 phone=String(), organisation=String())
684 user.setkey("username")
686 alter the "password=String()" to "password=Password()"::
688 user = Class(db, "user",
689 username=String(), password=Password(),
690 address=String(), realname=String(),
691 phone=String(), organisation=String())
692 user.setkey("username")
694 2. Any existing passwords in the database will remain cleartext until they
695 are edited. It is recommended that at a minimum the admin password be
696 changed immediately::
698 roundup-admin -i <instance home> set user1 password=<new password>
701 0.3.x Configuration
702 -------------------
703 FILTER_POSITION, ANONYMOUS_ACCESS, ANONYMOUS_REGISTER have been added to
704 the instance_config.py. Simplest solution is to copy the default values from
705 template in the core source.
707 MESSAGES_TO_AUTHOR has been added to the IssueClass in dbinit.py. Set to 'yes'
708 to send nosy messages to the author. Default behaviour is to not send nosy
709 messages to the author. You will need to add MESSAGES_TO_AUTHOR to your
710 dbinit.py in your instance home.
713 0.3.x CGI script roundup.cgi
714 ----------------------------
715 There have been some structural changes to the roundup.cgi script - you will
716 need to install it again from the cgi-bin directory of the source
717 distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the
718 copy.
721 .. _`customisation documentation`: customizing.html
722 .. _`security documentation`: security.html