1 ======================================
2 Upgrading to newer versions of Roundup
3 ======================================
5 Please read each section carefully and edit your instance home files
6 accordingly.
8 .. contents::
10 Migrating from 0.4.x to 0.5.0
11 =============================
13 This has been a fairly major revision of Roundup:
15 1. Brand new, much more powerful, flexible, tasty and nutritious templating.
16 Unfortunately, this means all your current templates are useless. Please
17 don't hesitate to ask on roundup-users for help (or complete conversions
18 if you're completely stuck)!
19 2. The database backed got a lot more flexible, allowing Metakit and SQL
20 databases! The only SQL database implemented at present is gadfly, but
21 others shouldn't be a whole lot more work.
22 3. A brand new, highly flexible and much more robust security system including
23 a system of Permissions, Roles and Role assignments to users. You may now
24 define your own Permissions that may be checked in CGI transactions.
25 4. Journalling has been made less storage-hungry, so has been turned on
26 by default *except* for author, recipient and nosy link/unlink events. You
27 are advised to turn it off in your instances too.
28 5. Because of the above changes, the instance configuration has seen some
29 major changes. See below for the details.
31 Please, *back up your database* before you start the migration process. This
32 is as simple as copying the "db" directory and all its contents from your
33 instance to somewhere safe.
36 0.5.0 Configuration
37 -------------------
39 Due to the new templating having a top-level ``page`` that defines links for
40 searching, indexes, adding items etc, the following variables are no longer
41 used:
43 - HEADER_INDEX_LINKS
44 - HEADER_ADD_LINKS
45 - HEADER_SEARCH_LINKS
46 - SEARCH_FILTERS
47 - DEFAULT_INDEX
48 - UNASSIGNED_INDEX
49 - USER_INDEX
50 - ISSUE_FILTER
52 The new security implementation will require additions to the dbinit module,
53 but also removes the need for the following instance config variables:
55 - ANONYMOUS_ACCESS
56 - ANONYMOUS_REGISTER
58 but requires two new variables which define the Roles assigned to users who
59 register through the web and e-mail interfaces:
61 - NEW_WEB_USER_ROLES
62 - NEW_EMAIL_USER_ROLES
64 in both cases, 'User' is a good initial setting. To emulate
65 ``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the
66 "Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web
67 Registration" and/or the "Email Registration" Permission from the "Anonymous"
68 Role. See the section on customising security in the `customisation
69 documentation`_ for more information.
72 0.5.0 Schema Specification
73 --------------------------
75 0.5.0 Database backend changes
76 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
78 Your select_db module in your instance has changed a fair bit. Where it used
79 to contain::
81 # WARNING: DO NOT EDIT THIS FILE!!!
82 from roundup.backends.back_anydbm import Database
84 it must now contain::
86 # WARNING: DO NOT EDIT THIS FILE!!!
87 from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass
89 Note the addition of the Class, FileClass, IssueClass imports. These are very
90 important, as they're going to make the next change work too. You now need to
91 modify the top of the dbinit module in your instance from::
93 import instance_config
94 from roundup import roundupdb
95 from select_db import Database
97 from roundup.roundupdb import Class, FileClass
99 class Database(roundupdb.Database, select_db.Database):
100 ''' Creates a hybrid database from:
101 . the selected database back-end from select_db
102 . the roundup extensions from roundupdb
103 '''
104 pass
106 class IssueClass(roundupdb.IssueClass):
107 ''' issues need the email information
108 '''
109 pass
113 to just::
115 import instance_config
116 from select_db import Database, Class, FileClass, IssueClass
118 Yes, remove the Database and IssueClass definitions and those other imports.
119 They're not needed any more!
122 0.5.0 Journalling changes
123 ~~~~~~~~~~~~~~~~~~~~~~~~~
125 Journalling has been optimised for storage. Journalling of links has been
126 turned back on by default. If your tracker has a large user base, you may wish
127 to turn off journalling of nosy list, message author and message recipient
128 link and unlink events. You do this by adding ``do_journal='no'`` to the Class
129 initialisation in your dbinit. For example, your *msg* class initialisation
130 probably looks like this::
132 msg = FileClass(db, "msg",
133 author=Link("user"), recipients=Multilink("user"),
134 date=Date(), summary=String(),
135 files=Multilink("file"),
136 messageid=String(), inreplyto=String())
138 to turn off journalling of author and recipient link events, add
139 ``do_journal='no'`` to the ``author=Link("user")`` part of the statement,
140 like so::
142 msg = FileClass(db, "msg",
143 author=Link("user", do_journal='no'),
144 recipients=Multilink("user", do_journal='no'),
145 date=Date(), summary=String(),
146 files=Multilink("file"),
147 messageid=String(), inreplyto=String())
149 Nosy list link event journalling is actually turned off by default now. If you
150 want to turn it onn, change to your issue class' nosy list, change its
151 definition from::
153 issue = IssueClass(db, "issue",
154 assignedto=Link("user"), topic=Multilink("keyword"),
155 priority=Link("priority"), status=Link("status"))
157 to::
159 issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'),
160 assignedto=Link("user"), topic=Multilink("keyword"),
161 priority=Link("priority"), status=Link("status"))
163 noting that your definition of the nosy Multilink will override the normal one.
166 0.5.0 User schema changes
167 ~~~~~~~~~~~~~~~~~~~~~~~~~
169 Users have two more properties, "queries" and "roles". You'll have something
170 like this in your dbinit module now::
172 user = Class(db, "user",
173 username=String(), password=Password(),
174 address=String(), realname=String(),
175 phone=String(), organisation=String(),
176 alternate_addresses=String())
177 user.setkey("username")
179 and you'll need to add the new properties and the new "query" class to it
180 like so::
182 query = Class(db, "query",
183 klass=String(), name=String(),
184 url=String())
185 query.setkey("name")
187 # Note: roles is a comma-separated string of Role names
188 user = Class(db, "user",
189 username=String(), password=Password(),
190 address=String(), realname=String(),
191 phone=String(), organisation=String(),
192 alternate_addresses=String(),
193 queries=Multilink('query'), roles=String())
194 user.setkey("username")
196 The "queries" property is used to store off the user's favourite database
197 queries. The "roles" property is explained below in `0.5.0 Security
198 Settings`_.
201 0.5.0 Security Settings
202 ~~~~~~~~~~~~~~~~~~~~~~~
204 See the `security documentation`_ for an explanation of how the new security
205 system works. In a nutshell though, the security is handled as a four step
206 process:
208 1. Permissions are defined as having a name and optionally a hyperdb class
209 they're specific to,
210 2. Roles are defined that have one or more Permissions,
211 3. Users are assigned Roles in their "roles" property, and finally
212 4. Roundup checks that users have appropriate Permissions at appropriate times
213 (like editing issues).
215 Your instance dbinit module's *open* function now has to define any
216 Permissions that are specific to your instance, and also the assignment
217 of Permissions to Roles. At the moment, your open function
218 ends with::
220 import detectors
221 detectors.init(db)
223 return db
225 and what we need to do is insert some commands that will set up the security
226 parameters. Right above the ``import detectors`` line, you'll want to insert
227 these lines::
229 #
230 # SECURITY SETTINGS
231 #
232 # new permissions for this schema
233 for cl in 'issue', 'file', 'msg', 'user':
234 db.security.addPermission(name="Edit", klass=cl,
235 description="User is allowed to edit "+cl)
236 db.security.addPermission(name="View", klass=cl,
237 description="User is allowed to access "+cl)
239 # Assign the access and edit permissions for issue, file and message
240 # to regular users now
241 for cl in 'issue', 'file', 'msg':
242 p = db.security.getPermission('View', cl)
243 db.security.addPermissionToRole('User', p)
244 p = db.security.getPermission('Edit', cl)
245 db.security.addPermissionToRole('User', p)
246 # and give the regular users access to the web and email interface
247 p = db.security.getPermission('Web Access')
248 db.security.addPermissionToRole('User', p)
249 p = db.security.getPermission('Email Access')
250 db.security.addPermissionToRole('User', p)
252 # May users view other user information? Comment these lines out
253 # if you don't want them to
254 p = db.security.getPermission('View', 'user')
255 db.security.addPermissionToRole('User', p)
257 # Assign the appropriate permissions to the anonymous user's Anonymous
258 # Role. Choices here are:
259 # - Allow anonymous users to register through the web
260 p = db.security.getPermission('Web Registration')
261 db.security.addPermissionToRole('Anonymous', p)
262 # - Allow anonymous (new) users to register through the email gateway
263 p = db.security.getPermission('Email Registration')
264 db.security.addPermissionToRole('Anonymous', p)
265 # - Allow anonymous users access to the "issue" class of data
266 # Note: this also grants access to related information like files,
267 # messages, statuses etc that are linked to issues
268 #p = db.security.getPermission('View', 'issue')
269 #db.security.addPermissionToRole('Anonymous', p)
270 # - Allow anonymous users access to edit the "issue" class of data
271 # Note: this also grants access to create related information like
272 # files and messages etc that are linked to issues
273 #p = db.security.getPermission('Edit', 'issue')
274 #db.security.addPermissionToRole('Anonymous', p)
276 # oh, g'wan, let anonymous access the web interface too
277 p = db.security.getPermission('Web Access')
278 db.security.addPermissionToRole('Anonymous', p)
280 Note in the comments there the places where you might change the permissions
281 to restrict users or grant users more access. If you've created additional
282 classes that users should be able to edit and view, then you should add them
283 to the "new permissions for this schema" section at the start of the security
284 block. Then add them to the "Assign the access and edit permissions" section
285 too, so people actually have the new Permission you've created.
287 One final change is needed that finishes off the security system's
288 initialisation. We need to add a call to ``db.post_init()`` at the end of the
289 dbinit open() function. Add it like this::
291 import detectors
292 detectors.init(db)
294 # schema is set up - run any post-initialisation
295 db.post_init()
296 return db
298 You may verify the setup of Permissions and Roles using the new
299 "``roundup-admin security``" command.
302 0.5.0 User changes
303 ~~~~~~~~~~~~~~~~~~
305 To support all those schema changes, you'll need to massage your user database
306 a little too, to:
308 1. make sure there's an "anonymous" user - this user is mandatory now and is
309 the one that unknown users are logged in as.
310 2. make sure all users have at least one Role.
312 If you don't have the "anonymous" user, create it now with the command::
314 roundup-admin create user username=anonymous roles=Anonymous
316 making sure the capitalisation is the same as above. Once you've done that,
317 you'll need to set the roles property on all users to a reasonable default.
318 The admin user should get "Admin", the anonymous user "Anonymous"
319 and all other users "User". The ``fixroles.py`` script in the tools directory
320 will do this. Run it like so (where python is your python 2+ binary)::
322 python tools/fixroles.py -i <instance home>
326 0.5.0 CGI interface changes
327 ---------------------------
329 The CGI interface code was completely reorganised and largely rewritten. The
330 end result is that this section of your instance interfaces module will need
331 changing from::
333 from roundup import mailgw
334 from roundup.cgi import client
336 class Client(client.Client):
337 ''' derives basic CGI implementation from the standard module,
338 with any specific extensions
339 '''
340 pass
342 to::
344 from roundup import cgi_client, mailgw
345 from roundup.i18n import _
347 class Client(cgi_client.Client):
348 ''' derives basic CGI implementation from the standard module,
349 with any specific extensions
350 '''
351 pass
353 0.5.0 HTML templating
354 ---------------------
356 You'll want to make a backup of your current instance html directory. You
357 should then copy the html directory from the Roundup source template that you
358 used to create your instance, and modify it according to your local schema
359 changes.
361 If you need help with the new templating system, please ask questions on the
362 roundup-users mailing list (available through the roundup project page on
363 sourceforge, http://roundup.sf.net/)
366 0.5.0 Detectors
367 ---------------
369 The nosy reactor has been updated to handle the instance not having an
370 "assignedto" property on issues. You may want to copy it into your instance's
371 detectors directory. Chances are you've already fixed it though :)
374 Migrating from 0.4.1 to 0.4.2
375 =============================
377 0.4.2 Configuration
378 -------------------
379 The USER_INDEX definition introduced in 0.4.1 was too restrictive in its
380 allowing replacement of 'assignedto' with the user's userid. Users must change
381 the None value of 'assignedto' to 'CURRENT USER' (the string, in quotes) for
382 the replacement behaviour to occur now.
384 The new configuration variables are:
386 - EMAIL_KEEP_QUOTED_TEXT
387 - EMAIL_LEAVE_BODY_UNCHANGED
388 - ADD_RECIPIENTS_TO_NOSY
390 See the sample configuration files in::
392 <roundup source>/roundup/templates/classic/instance_config.py
394 and::
396 <roundup source>/roundup/templates/extended/instance_config.py
398 and the `customisation documentation`_ for information on how they're used.
401 0.4.2 Changes to detectors
402 --------------------------
403 You will need to copy the detectors from the distribution into your instance
404 home "detectors" directory. If you used the classic schema, the detectors
405 are in::
407 <roundup source>/roundup/templates/classic/detectors/
409 If you used the extended schema, the detectors are in::
411 <roundup source>/roundup/templates/extended/detectors/
413 The change means that schema-specific code has been removed from the
414 mail gateway and cgi interface and made into auditors:
416 - nosyreactor.py has now got an updatenosy auditor which updates the nosy
417 list with author, recipient and assignedto information.
418 - statusauditor.py makes the unread or resolved -> chatting changes and
419 presets the status of an issue to unread.
421 There's also a bug or two fixed in the nosyreactor code.
423 0.4.2 HTML templating changes
424 -----------------------------
425 The link() htmltemplate function now has a "showid" option for links and
426 multilinks. When true, it only displays the linked node id as the anchor
427 text. The link value is displayed as a tooltip using the title anchor
428 attribute. To use in eg. the superseder field, have something like this::
430 <td>
431 <display call="field('superseder', showid=1)">
432 <display call="classhelp('issue', 'id,title', label='list', width=500)">
433 <property name="superseder">
434 <br>View: <display call="link('superseder', showid=1)">
435 </property>
436 </td>
438 The stylesheets have been cleaned up too. You may want to use the newer
439 versions in::
441 <roundup source>/roundup/templates/<template>/html/default.css
445 Migrating from 0.4.0 to 0.4.1
446 =============================
448 0.4.1 Files storage
449 -------------------
451 Messages and files from newly created issues will be put into subdierectories
452 in thousands e.g. msg123 will be put into files/msg/0/msg123, file2003
453 will go into files/file/2/file2003. Previous messages are still found, but
454 could be put into this structure.
456 0.4.1 Configuration
457 -------------------
459 To allow more fine-grained access control, the variable used to check
460 permission to auto-register users in the mail gateway is now called
461 ANONYMOUS_REGISTER_MAIL rather than overloading ANONYMOUS_REGISTER. If the
462 variable doesn't exist, then ANONYMOUS_REGISTER is tested as before.
464 Configuring the links in the web header is now easier too. The following
465 variables have been added to the classic instance_config.py::
467 HEADER_INDEX_LINKS - defines the "index" links to be made available
468 HEADER_ADD_LINKS - defines the "add" links
469 DEFAULT_INDEX - specifies the index view for DEFAULT
470 UNASSIGNED_INDEX - specifies the index view for UNASSIGNED
471 USER_INDEX - specifies the index view for USER
473 See the <roundup source>/roundup/templates/classic/instance_config.py for more
474 information - including how the variables are to be set up. Most users will
475 just be able to copy the variables from the source to their instance home. If
476 you've modified the header by changing the source of the interfaces.py file in
477 the instance home, you'll need to remove that customisation and move it into
478 the appropriate variables in instance_config.py.
480 The extended schema has similar variables added too - see the source for more
481 info.
483 0.4.1 Alternate E-Mail Addresses
484 --------------------------------
486 If you add the property "alternate_addresses" to your user class, your users
487 will be able to register alternate email addresses that they may use to
488 communicate with roundup as. All email from roundup will continue to be sent
489 to their primary address.
491 If you have not edited the dbinit.py file in your instance home directory,
492 you may simply copy the new dbinit.py file from the core code. If you used
493 the classic schema, the interfaces file is in::
495 <roundup source>/roundup/templates/classic/dbinit.py
497 If you used the extended schema, the file is in::
499 <roundup source>/roundup/templates/extended/dbinit.py
501 If you have modified your dbinit.py file, you need to edit the dbinit.py
502 file in your instance home directory. Find the lines which define the user
503 class::
505 user = Class(db, "msg",
506 username=String(), password=Password(),
507 address=String(), realname=String(),
508 phone=String(), organisation=String(),
509 alternate_addresses=String())
511 You will also want to add the property to the user's details page. The
512 template for this is the "user.item" file in your instance home "html"
513 directory. Similar to above, you may copy the file from the roundup source if
514 you haven't modified it. Otherwise, add the following to the template::
516 <display call="multiline('alternate_addresses')">
518 with appropriate labelling etc. See the standard template for an idea.
522 Migrating from 0.3.x to 0.4.0
523 =============================
525 0.4.0 Message-ID and In-Reply-To addition
526 -----------------------------------------
527 0.4.0 adds the tracking of messages by message-id and allows threading
528 using in-reply-to. Most e-mail clients support threading using this
529 feature, and we hope to add support for it to the web gateway. If you
530 have not edited the dbinit.py file in your instance home directory, you may
531 simply copy the new dbinit.py file from the core code. If you used the
532 classic schema, the interfaces file is in::
534 <roundup source>/roundup/templates/classic/dbinit.py
536 If you used the extended schema, the file is in::
538 <roundup source>/roundup/templates/extended/dbinit.py
540 If you have modified your dbinit.py file, you need to edit the dbinit.py
541 file in your instance home directory. Find the lines which define the msg
542 class::
544 msg = FileClass(db, "msg",
545 author=Link("user"), recipients=Multilink("user"),
546 date=Date(), summary=String(),
547 files=Multilink("file"))
549 and add the messageid and inreplyto properties like so::
551 msg = FileClass(db, "msg",
552 author=Link("user"), recipients=Multilink("user"),
553 date=Date(), summary=String(),
554 files=Multilink("file"),
555 messageid=String(), inreplyto=String())
557 Also, configuration is being cleaned up. This means that your dbinit.py will
558 also need to be changed in the open function. If you haven't changed your
559 dbinit.py, the above copy will be enough. If you have, you'll need to change
560 the line (round line 50)::
562 db = Database(instance_config.DATABASE, name)
564 to::
566 db = Database(instance_config, name)
569 0.4.0 Configuration
570 --------------------
571 ``INSTANCE_NAME`` and ``EMAIL_SIGNATURE_POSITION`` have been added to the
572 instance_config.py. The simplest solution is to copy the default values
573 from template in the core source.
575 The mail gateway now checks ``ANONYMOUS_REGISTER`` to see if unknown users
576 are to be automatically registered with the tracker. If it is set to "deny"
577 then unknown users will not have access. If it is set to "allow" they will be
578 automatically registered with the tracker.
581 0.4.0 CGI script roundup.cgi
582 ----------------------------
583 The CGI script has been updated with some features and a bugfix, so you should
584 copy it from the roundup cgi-bin source directory again. Make sure you update
585 the ROUNDUP_INSTANCE_HOMES after the copy.
588 0.4.0 Nosy reactor
589 ------------------
590 The nosy reactor has also changed - copy the nosyreactor.py file from the core
591 source::
593 <roundup source>/roundup/templates/<template>/detectors/nosyreactor.py
595 to your instance home "detectors" directory.
598 0.4.0 HTML templating
599 ---------------------
600 The field() function was incorrectly implemented - links and multilinks now
601 display as text fields when rendered using field(). To display a menu (drop-
602 down or select box) you need to use the menu() function.
606 Migrating from 0.2.x to 0.3.x
607 =============================
609 0.3.x Cookie Authentication changes
610 -----------------------------------
611 0.3.0 introduces cookie authentication - you will need to copy the
612 interfaces.py file from the roundup source to your instance home to enable
613 authentication. If you used the classic schema, the interfaces file is in::
615 <roundup source>/roundup/templates/classic/interfaces.py
617 If you used the extended schema, the file is in::
619 <roundup source>/roundup/templates/extended/interfaces.py
621 If you have modified your interfaces.Client class, you will need to take
622 note of the login/logout functionality provided in roundup.cgi_client.Client
623 (classic schema) or roundup.cgi_client.ExtendedClient (extended schema) and
624 modify your instance code apropriately.
627 0.3.x Password encoding
628 -----------------------
629 This release also introduces encoding of passwords in the database. If you
630 have not edited the dbinit.py file in your instance home directory, you may
631 simply copy the new dbinit.py file from the core code. If you used the
632 classic schema, the interfaces file is in::
634 <roundup source>/roundup/templates/classic/dbinit.py
636 If you used the extended schema, the file is in::
638 <roundup source>/roundup/templates/extended/dbinit.py
641 If you have modified your dbinit.py file, you may use encoded passwords:
643 1. Edit the dbinit.py file in your instance home directory
644 a. At the first code line of the open() function::
646 from roundup.hyperdb import String, Date, Link, Multilink
648 alter to include Password, as so::
650 from roundup.hyperdb import String, Password, Date, Link, Multilink
652 b. Where the password property is defined (around line 66)::
654 user = Class(db, "user",
655 username=String(), password=String(),
656 address=String(), realname=String(),
657 phone=String(), organisation=String())
658 user.setkey("username")
660 alter the "password=String()" to "password=Password()"::
662 user = Class(db, "user",
663 username=String(), password=Password(),
664 address=String(), realname=String(),
665 phone=String(), organisation=String())
666 user.setkey("username")
668 2. Any existing passwords in the database will remain cleartext until they
669 are edited. It is recommended that at a minimum the admin password be
670 changed immediately::
672 roundup-admin -i <instance home> set user1 password=<new password>
675 0.3.x Configuration
676 -------------------
677 FILTER_POSITION, ANONYMOUS_ACCESS, ANONYMOUS_REGISTER have been added to
678 the instance_config.py. Simplest solution is to copy the default values from
679 template in the core source.
681 MESSAGES_TO_AUTHOR has been added to the IssueClass in dbinit.py. Set to 'yes'
682 to send nosy messages to the author. Default behaviour is to not send nosy
683 messages to the author. You will need to add MESSAGES_TO_AUTHOR to your
684 dbinit.py in your instance home.
687 0.3.x CGI script roundup.cgi
688 ----------------------------
689 There have been some structural changes to the roundup.cgi script - you will
690 need to install it again from the cgi-bin directory of the source
691 distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the
692 copy.
695 .. _`customisation documentation`: customizing.html
696 .. _`security documentation`: security.html