From: stefan Date: Fri, 6 Feb 2009 04:44:58 +0000 (+0000) Subject: svn repository setup X-Git-Url: https://git.tokkee.org/?a=commitdiff_plain;h=ac6d696f4e811655f68b3a5e869da9c4c62251c5;p=roundup.git svn repository setup git-svn-id: http://svn.roundup-tracker.org/svnroot/roundup/roundup/trunk@4074 57a73879-2fb5-44c3-a270-3262357dd7e2 --- diff --git a/BUILD.txt b/BUILD.txt index aeaf23c..665a5b3 100644 --- a/BUILD.txt +++ b/BUILD.txt @@ -2,58 +2,50 @@ Building Releases ================= Roundup is currently a source-only release - it has no binary components. I -want it to stay that way, too. +want it to stay that way, too. This document describes how to build a +source release. Users of Roundup should read the doc/installation.txt file +to find out how to install this software. -This means that we only need to ever build source releases. This is done by -running: +Building and distributing a release of Roundup is done by running: -1. Make sure the unit tests run! "./run_tests" -2. Edit roundup/__init__.py and doc/announcement.txt to reflect the new +1. Make sure the unit tests run! "./run_tests.py" +2. Tag the CVS for the release, eg. "cvs tag -R release-0-6-3" +3. Edit roundup/__init__.py and doc/announcement.txt to reflect the new version and appropriate announcements. Add truncated announcement to setup.py description field. -3. python setup.py clean --all -4. Edit setup.py to ensure that all information therein (version, contact +4. Clean out all *.orig, *.rej, .#* files from the source. +5. python setup.py clean --all +6. Edit setup.py to ensure that all information therein (version, contact information etc) is correct. -5. python setup.py sdist --manifest-only -6. Check the MANIFEST to make sure that any new files are included. If +7. python setup.py sdist --manifest-only +8. Check the MANIFEST to make sure that any new files are included. If they are not, edit MANIFEST.in to include them. "Documentation" for MANIFEST.in may be found in disutils.filelist._parse_template_line. -7. python setup.py sdist +9. python setup.py sdist (if you find sdist a little verbose, add "--quiet" to the end of the command) -8. unpack the new dist file in /tmp then a) run_test.py and b) demo.py +10. Unpack the new dist file in /tmp then a) run_test.py and b) demo.py with all available Python versions. -9. generate gpg signature with "gpg -a --detach-sign" and upload to - Sourceforge. -10. PyPI registration -11. tag the CVS for the release, eg. "cvs tag -R release-0-6-3" +11. Generate gpg signature with "gpg -a --detach-sign" +12. python setup.py bdist_rpm +13. python setup.py bdist_wininst +14. Send doc/announcement.txt to python-announce@python.org +15. Notify any other news services as appropriate... + + http://freshmeat.net/projects/roundup/ + So, those commands in a nice, cut'n'pasteable form:: + find . -name '*.orig' -exec rm {} \; + find . -name '*.rej' -exec rm {} \; + find . -name '.#*' -exec rm {} \; python setup.py clean --all python setup.py sdist --manifest-only python setup.py sdist --quiet - python2.3 setup.py register - - -Distributing Releases -===================== - -Once a release is built, follow these steps: - -1. FTP the tar.gz from the dist directory to to the "incoming" directory on - "upload.sourceforge.net". -2. Make a quick release at: - http://sourceforge.net/project/admin/qrs.php?package_id=&group_id=31577 -3. Add a news item at: - https://sourceforge.net/news/submit.php?group_id=31577 - using the top of doc/announcement.txt -4. Send doc/announcement.txt to python-announce@python.org -5. Notify any other news services as appropriate... - - -Author -====== + python setup.py bdist_rpm + python setup.py bdist_wininst + python setup.py register + python2.5 setup.py sdist upload --sign -richard@users.sourceforge.net diff --git a/CHANGES.txt b/CHANGES.txt index 262529c..4d7db6f 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,7 +1,806 @@ This file contains the changes to the Roundup system over time. The entries are given with the most recent entry first. -2004-??-?? 0.7.0 +2008-09-01 1.4.6 +Fixed: +- Fix bug introduced in 1.4.5 in RDBMS full-text indexing +- Make URL matching code less matchy +- Try to clarify mail_domain config setting + + +2008-08-19 1.4.5 +Feature: +- Add use of username/password stored in ~/.netrc in mailgw (sf patch + #1912105) + +Fixed: +- 'Make a Copy' failed with more than one person in nosy list (sf #1906147) +- xml-rpc security checks and tests across all backends (sf #1907211) +- Send a Precedence header in email so (well-written) autoresponders don't +- Fix mailgw total failure bounce message generation (thanks Bradley Dean) +- Fix for postgres 8.3 compatibility (and bug) (sf patch #2030479 and bug + #1959261) +- Fix for translations (sf patch #2032526) +- Fire reactors after file storage is all done (sf patch #2001243) +- Allow negative ids other than -1 for item generation (sf patch #1982481) +- Better German translation for retiring users (sf #1998701) +- More improvements to German translation (sf #1919446) +- Add filter() to XML-RPC interface (sf patch #1966456) +- Fix IndexError when there are no messages to an issue (sf patch #1894249) +- Prevent broken pipe errors in csv export (sf patch #1911449) +- New session API and cleanup thanks anatoly t. +- Make WSGI handler threadsafe (sf #1968027) +- Improved URL matching RE (sf #2038858) +- Allow binary file content submission via XML-RPC (sf #1995623) +- Don't run old code on newer database (sf #1979556) +- Fix HTML injection into page title +- Fix indexer handling of indexed Link properties (sf #1936876) + + +2008-03-01 1.4.4 +Fixed: +- Security fixes (thanks Roland Meister) + + +2008-02-27 1.4.3 +Fixed: +- MySQL backend bug introduced in 1.4.2 (TEXT columns need a size when + being indexed) + + +2008-02-08 1.4.2 +Feature: +- New config option in mail section: ignore_alternatives allows to + ignore alternatives besides the text/plain part used for the content + of a message in multipart/alternative attachments. +- Admin copy of error email from mailgw includes traceback (thanks Ulrik + Mikaelsson) +- Messages created through the web are now given an in-reply-to header + when email out to nosy (thanks Martin v. Löwis) +- Nosy messages now include more information about issues (all link + properties with a "name" attribute) (thanks Martin v. Löwis) + +Fixed: +- Searching date range by supplying just a date as the filter spec +- Handle no time.tzset under Windows (sf #1825643) +- Fix race condition in file storage transaction commit (sf #1883580) +- Make user utils JS work with firstname/lastname again (sf #1868323) +- Fix ZRoundup to work with Zope 2.8.5 (sf #1806125) +- Fix race condition for key properties in rdbms backends (sf #1876683) +- Handle Reject in mailgw final set/create (sf #1826425) + + +2007-11-09 1.4.1 +Fixed: +- Removed some metakit references + + +2007-11-04 1.4.0 +Feature: +- Roundup has a new xmlrpc frontend that gives access to a tracker using + XMLRPC. +- Dates can now be in the year-range 1-9999 +- The metakit backend has been removed +- Add simple anti-spam recipe to docs +- Allow customisation of regular expressions used in email parsing, thanks + Bruno Damour +- Italian translation by Marco Ghidinelli +- Multilinks take any iterable +- config option: specify port and local hostname for SMTP connections +- Tracker index templating (i.e. when roundup_server is serving multiple + trackers) (sf bug 1058020) +- config option: Limit nosy attachments based on size (Philipp Gortan) +- roundup_server supports SSL via pyopenssl +- templatable 404 not found messages (sf bug 1403287) +- Unauthorized email includes a link to the registration page for + the tracker +- config options: control whether author info/email is included in email + sent by roundup +- support for receiving OpenPGP MIME messages (signed or encrypted) + +Fixed: +- Handling of unset Link search in RDBMS backend +- Journal export of anydbm didn't correctly export previously empty values +- Fix handling of defaults for date fields +- Fix
name in user editing to allow multilink popups to work +- Fix form handling of editing existing hyperdb items from a new item page. +- Added new rdbms-indexes for full-text index which will speed up + reindexing. +- Turning off indexing for content properties of FileClass instance + (e.g., "file" and "msg") now works for SQL backends. +- Enabled over-riding of content-type in web interface (thanks + John Mitchell) +- Validate user timezones to filter bad entries (sf bug 1738470) +- Classic template allows searching for issues with no topic set + (sf bug 1610787) +- xapian_indexer uses current API for stemming (Rick Benavidez) + (sf bug 1771414) +- Ensure email addresses are unique (sf bug 1611787) +- roundup_admin tracks uncommitted changes in interactive mode + for all backends (sf bug 1297014) +- add template search path for easy_install (Marek Kubica) +- don't spam the roundup admin on client shutdowns (Ulrik Mikaelsson) +- respect umask on filestorage backends (Ulrik Mikaelsson) (sf bug 1744328) +- cope with spam robots posting multiple instances of the same form +- include the author of property-only changes in generated messages +- fuller email validation in templates (sf feature 1216291) +- cope with bad cookies from other apps on same domain (sf bug 1691708) +- updated Spanish translation from Ramiro Morales +- clean up query display of "Private to you items" (sf bug 1481394) +- use local timezone for mail date header (sf bug 1658173) +- allow CSV export of queries on selected issues (sf bug 1783492) +- remove blobfiles on destroy (sf bug 1654132) +- handle postgres exceptions during session cleanup (sf bug 1703116) +- update Xapian indexer to use current API +- handle export and import of old trackers that have data attached to + journal "create" events +- fix a couple more old instances of "type" instead of "ENGINE" for mysql + backend +- make LinkHTMLProperty handle non-existing keys (sf patch 1815895) + + +2007-02-15 1.3.3 +Fixed: +- If-Modified-Since handling was broken +- Updated documentation for customising hard-coded searches in page.html +- Updated Windows installation docs (thanks Bo Berglund) +- Handle rounding of seconds generating invalid date values +- Handle 8-bit untranslateable messages from database properties +- Fix scripts/roundup-reminder date calculation (sf bug 1649979) +- Improved due_date and timelog customisation docs (sf bug 1625124) + + +2006-12-19 1.3.2 +Fixed: +- relax rules for required fields in form_parser.py (sf bug 1599740) +- documentation cleanup from Luke Ross (sf patch 1594860) +- updated Spanish translation from Ramiro Morales (sf patch 1594718) +- handle 8-bit untranslateable messages in tracker templates +- handling of required for boolean False and numeric 0 (sf bug 1608200) +- removed bogus args attr of ConfigurationError (sf bug 1608056) +- implemented start_response in roundup.cgi (sf bug 1604304) +- clarified windows service documentation (sf patch 1597713) +- HTMLClass fixed to work with new item permissions check (sf bug 1602983) +- support POP over SSL (sf patch 1597703) +- clean up input field generation and quoting of values (sf bug 1615616) +- allow use of roundup-server pidfile without forking (sf bug 1614753) +- allow translation of status/priority menu options (sf bug 1613976) + + +2006-11-11 1.3.1 +Fixed: +- setup.py had broken reference to roundup.cgi (sf bug 1593573) +- full-text search wasn't coping with multiple multilinks to the same class +- unicode / sqlite 3 problem (sf bug 1589292) + + +2006-11-09 1.3.0 +Feature: +- WSGI support via roundup.cgi.wsgi_handler + +Fixed: +- sqlite module detection was broken for python 2.5 compiled without sqlite + support +- fixed support for pysqlite2 (version 2.1.0 is the minimum version + supported) +- roundup-server called setuid when run by non-root user +- fix sort/group direction checkbox in issue.index.html (sf bug 1593025) +- fix error detection for non-EN locales of postgres (sf bug 1592249) +- fix email change note rendering of multiline properties (sf patch 1575223) +- fix sidebar search links (sf patch 1574467) +- nicer "permission required" messages (sf patch 1558183) +- fix unstable ordering of detectors (sf bug 1585378) + + +2006-10-07 1.2.1 +Fixed: +- E-mail subject line prefix delimiter configuration was being ignored. +- Password confirm field in user editing. + + +2006-10-04 1.2.0 +Feature: +- supports Python 2.5, including the sqlite3 module +- full timezone support (sf patch 1465296) +- handle connection loss when responding to web requests +- match incoming mail In-Reply-To against existing messages when no issue + id is specified in the Subject +- added StringHTMLProperty wrapped() method to wrap long lines in issue + display +- include the popcal in Date field editing and search fields by default +- @required in forms may now specify properties of linked items (sf patch + 1507093) +- update for latest version of pysqlite (sf bug 1487098; patch 1534227) +- update for latest version of psycopg2 (sf patch 1429391) +- new "exporttables" command in roundup-admin (sf bug 1533791) +- roundup-admin "export" may specify classes to exclude (sf bug 1533791) +- sorting and grouping by multiple properties is now supported by the + backends *and* the classic template. +- sorting, grouping, and searching by transitive properties (e.g., + messages.author.supervisor) is now supported in all backends +- added filter_sql to SQL backends which takes an arbitrary SQL statement + and returns a list of item ids + + +Fixed: +- Verbose option for import and export (sf bug 1505645) +- -c option for roundup-mailgw won't accept parameter (sf bug 1505649) +- '?' in rfc2822-encoded header isn't quoted (sf bug 1505663) +- fix error message in form parser +- updated ZRoundup for Zope 2.9 (sf patch 1511734) +- fix timelog example in customisation doc to mention permissions +- nicer listing of Superseder links (sf non-patch 1497767) +- include roundup-server.ini.example (sf bug 1493859) +- dumb bug in cgi templating utils (sf bug 1490176) +- handle unicode in query names (sf bug 1495702) +- fix error during mailgw bouncing message (sf bug 1413501) +- hyperdb handling of empty raw values for Multilink and Password (sf bug + 1507814) +- don't int() ids (sf bug 1512939) +- fix importing into anydbm backend (sf bug 1512939) +- fix help message for roundup-admin install (sf bug 1494990) +- removed traceback with OTK is used multiple times (sf bug 1240539) +- metakit backend was indexing FileClass content even when asked not to +- anydbm backend will finally sort numerically by ID +- problem with string sorting in anydbm backend fixed: If a string was + fully numeric it was sorted as a number +- Multilink-sorting now sorts by orderprop not by ID and works for all + backends +- Bug with name-collisions in sorted classes when sorting by Link + properties in metakit backend fixed +- Postgres backend allows transaction collisions to be ignored when + committing cleanup in the sessions database +- translate titles of "show all" and "unassigned" issue lists + in classic template (sf patch 1424576) +- "as" is a keyword in Python 2.6 +- "from __future__" statments need to be first line of file in Python 2.6 +- better conflict retry in postgresql backend (sf bug 1552809) +- fix time log example (sf bug 1554630) + + +2006-04-27 1.1.2 +Feature: +- server-ctl script uses server configuration file (sf bug 1443805) +- mail user interface translated (sf patch 1462491) + +Fixed: +- progress display in roundup-admin reindex +- bug in menu() permission filter (sf bug 1444440) +- indexing may be turned off for FileClass "content" now + ("content" and "type" properties are now automatically included in the + FileClass schema where previously the "content" property was faked and + "type" was optional) +- verbose output during import is optional now (sf bug 1475624) +- escape *all* uses of "schema" in mysql backend (sf bug 1472120) +- responses to user rego email (sf bug 1470254) +- dangling connections in session handling (sf bug 1463359) +- reduced frequency of session timestamp update +- classhelp popup pagination forgot about "type" (sf bug 1465836) +- umask is now configurable (with the same 0002 default) +- sorting of entries in classhelp popup (sf bug 1449000) +- allow single digit seconds in date spec (sf bug 1447141) +- prevent generation of new single-digit seconds dates (sf bug 1429390) +- implement close() on all indexers (sf bug 1242477) + + +2006-03-03 1.1.1 +Fixed: +- failure with browsers not sending "Accept-Language" header + (sf bugs 1429646 and 1435335) +- translate class name in "required property not supplied" error message + (sf bug 1429669) +- error in link property lookups with numeric-alike key values (sf bug 1424550) +- ignore UTF-8 BOM in .po files +- add permission filter to menu() implementations (sf bug 1431188) +- lithuanian translation updated by Nerijus Baliunas (sf patch 1411175) +- incompatibility with python2.3 in the mailer module (sf bug 1432602) +- typo in SMTP TLS option name: "MAIL_TLS_CERFILE" (sf bug 1435452) +- email obfuscation code in html templating is more robust +- blank-title subject line handling (sf bug 1442121) +- "All users may only view and edit issues, files and messages they + create" example in docs (sf bug 1439086) +- saving of queries (sf bug 1436169) +- "Adding a new constrained field to the classic schema" example in docs + (sf bug 1433118) +- security check in mailgw (sf bug 1442145) +- "clear this message" (sf bug 1429367) +- escape all uses of "schema" in mysql backend (sf bug 1397569) +- date spec wasn't allowing week intervals + + +2006-02-10 1.1.0 +Feature: +- trackers may configure custom stop-words for the full-text indexer +- login may now be for a single session (and this is the default) +- trackers may hide exceptions from web users (they will be mailed to the + tracker admin) (hiding is the default) +- include "clear this message" link in the "ok" message bar + +Fixed: +- fixes in scripts/import_sf.py +- fix some unicode bugs in roundup-admin import +- Xapian indexer wasn't actually being used and its reindexing of existing + data was busted to boot +- roundup-admin import wasn't indexing message content +- allow dispname to be passed to renderWith (sf bug 1424587) +- rename dispname to @dispname to avoid name clashes in the future +- fixed schema migration problem when Class keys were removed + + +2006-02-03 1.0.1 +Feature: +- scripts/import_sf.py will import a tracker from Sourceforge.NET +- added hasRole() to HTMLUser + +Fixed: +- SQL generation for sort/group by separate Link properties (sf bug + 1417565) +- fix timezone offsetting in email Date: header +- fix security check for hasPermission('Permission', None) + + +2006-01-27 1.0 +Feature: +- Lithuanian translation by Aiste Kesminaite +- Web User Interface language selection by form variable @language, + browser cookie or HTTP header Accept-Language (sf patch 1360321) +- initial values for configuration options may be passed on + 'roundup-admin install' command line (based on sf patch 1237110) +- favicon.ico image may be changed with server config option (sf patch 1355661) +- Password objects initialized from plaintext remember plaintext value + (sf rfe 1379447) +- Roundup installation document includes configuration example + for Exim Internet Mailer (sf bug 1393860) +- enable registration confirmation by web only (sf bug 1381675) +- allow preselection of values in templating menu()s (sf patch 1396085) +- display the query name in the header (sf feature 1298535 / patch 1349387) +- classhelp works with Link properties now (sf bug 1410290) +- added setorderprop() and setlabelprop() to Class (sf features 1379534, + 1379490) +- CSV encoding support (sf bug 1240848) +- fields rendered with StructuredText are hyperlinked by default +- additional attributes for input element may be passed to the 'field' + method of a property wrapper +- added "copy_url" method to generate a URL for copying an item + +Fixed: +- MySQL now creates String columns using the TEXT column type +- password.crypt won't work with md5 passwords (sf bug 1372253) +- use quoted printable encoding for nosy attachments that have MIME + type 'text/plain' but contain 8-bit characters (sf bug 1381559) +- login name and email address fields in the classic template + are highlighted as required fields (sf bug 1392364) +- french translation updated by Patrick Decat (sf patch 1397059) +- HTTP authorization takes precedence over session cookie (sf bug 1396134) +- enforce correct encoding of PostgreSQL backend (sf bug 1374235) +- grouping/sorting on link to same class fixed (sf bug 1404930) +- all backends implement the retired check in getnodeids (sf bug 1290560) +- fix detection of "missing" existing values in CGI form parser (sf bug + 1414149) +- ZRoundup works again (sf bug 1263842) +- default user template does not display password fields and submit button + when editing is not allowed +- fix StructuredText import in cgi.templating +- have "System Messages" be marked as such again (sf bug 1281907) +- enable editing of public queries (sf bug 966144) + + +2005-10-07 0.9.0b1 +Feature: +- added "imapServer.py" script (sf patch 934567) +- added date selection popup windows (thanks Marcus Priesch) +- added Xapian indexer; replaces standard indexers if Xapian is available +- mailgw subject parsing has configurable levels of strictness +- nosy messages may be sent individually to all recipients +- remember where we came from when logging in (sf patch 1312889) + + +2006-??-?? 0.8.6 +Fixed: +- french translation updated by Patrick Decat (sf patch 1397059) +- tighten up Date parsing to not allow 'M/D/YY' (or 'D/M/YY) (sf bug + 1290550) +- handle "schema" being reserved word in MySQL 5+ (sf bug 1397569) +- fixed documentation of filter() in the case of multiple values in a + String search (sf bug 1373396) +- fix comma-separated ID filter spec in web requests (sf bug 1396278) +- fix Date: header generation to be LOCALE-agnostic (sf bug 1352624) +- fix admin doc description of roundup-server config file +- fix redirect after instant registration (sf bug 1381676) +- fix permission checks in cgi interface (sf bug 1289557) +- fix permission check on RetireAction (sf bug 1407342) +- timezone now applied to date for pretty-format (sf bug 1406861) +- fix mangling of "_" in mail Subject class name (sf bug 1413852) +- catch bad classname in URL (related to sf bug 1240541) +- clean up digested_file_types (sf bug 1268303) +- fix permission checks in mailgw (sf bug 1263655) +- fix encoding of subject in generated error message (sf bug 1414465) + + +2005-10-07 0.8.5 +Feature: +- Argentinian Spanish translation by Ramiro Morales + +Fixed: +- Display of Multilinks where linked Class labelprop values are None +- Fix references to the old * Registration Permissions +- Fix missing merge of fix to sf bug 1177057 +- Fix RDBMS indexer indexing UTF-8 words that encode to > 30 chars +- Handle invalidly-specified charsets in incoming email + + +2005-07-18 0.8.4 +Fixed: +- extra CRs in CSV export files on Windows platform (sf bug 1195742) +- activity RDBMS columns were being reported in changes +- fix name collision in roundup.cgi script (sf bug 1203795) +- fix handling of invalid interval input +- search locale files relative ro roundup installation path (sf bug 1219689) +- use translation for boolean property rendering (sf bug 1225152) +- enabled disabling of REMOTE_USER for when it's not a valid username (sf + bug 1190187) +- fix invocation of hasPermission from templating code (sf bug 1224172) +- have 'roundup-admin security' display property restrictions (sf bug + 1222135) +- fixed templating menu() sort_on handling (sf bug 1221936) +- allow specification of pagesize, sorting and filtering in "classhelp" + popups (sf bug 1211800) +- handle dropped properies in rdbms/metakit journal export (sf bug 1203569) +- handle missing Subject lines better (sf bug 1198729) +- sort/group by missing values correctly (sf bugs 1198623, 1176897) +- discard, don't bounce messages to the mailgw when the messages's sender + is invalid (ie. when we try to bounce, we get a 550 "unknown user + account" response from the SMTP server) (sf bug 1190906) +- removed debugging code from cgi/actions.py +- refactored hyperdb.rawToHyperdb, allowing a number of improvements + (thanks Ralf Schlatterbeck) +- don't try to set a timeout for IMAPS (thanks Paul Jimenez) +- present Reject exception messages to web users (sf bug 1237685) + + +2005-05-02 0.8.3 +Feature: +- chinese translation by limodou + +Fixed: +- fix reference to The Zope Book in Roundup FAQ +- disabled file logging in Roundup test suite (sf bug 1155649) +- return original string if message issue xref isn't valid +- fix nosyreaction.py to stop it setting the nosy list unnecessarily + (see doc/upgrading.txt for how to fix in your trackers) +- after logout, always display tracker home page +- web forms don't create new items if no item properties are set from UI +- item creation failed if multilink fields had invalid entries (sf bug + 1177602) +- fix bdist_rpm (sf bug 1164328) +- fix checking of "Email Access" for Anonymous email registration (sf bug + 1177057) +- disable "Email Access" for Anonymous by default to stop spam regsitering + users on public trackers +- send errors in the web interface to a logfile by default. Use the + "debug" multiprocess mode (roundup-server) or the DEBUG_TO_CLIENT var + (roundup.cgi) to have the errors appear in your browser +- fix setgid typo (sf bug 1171346) +- fix faulty find_template filename facility (sf bug 1163629) +- fix roundup-admin "export" so it creates the target dir if needed +- "fix" roundup-admin "import" to not use "universal newline support" since + the csv module appears to have its own ideas about such things (sf bug + 1163890) +- fix installation docs referring to old-style configuration variables +- fix roundup-admin "find" for searching Multilinks (sf bug 1189465) + + +2005-03-03 0.8.2 +Feature: +- roundup-server automatically redirects from trackers list + to the tracker page if there is only one tracker + +Fixed: +- added content to ZRoundup refresh.txt file (sf bug 1147622) +- fix invalid reference to csv.colon_separated +- correct URL to What's New in setup.py meta-data +- change AUTOCOMMIT=OFF to AUTOCOMMIT=0 for MySQL (sf bug 1143707) +- compile message objects in 'setup.py build' +- use backend datatype for journal timestamps in RDBMSes +- fixes to the "Using an external password validation source" + customisation example (sf bugs 1153640 and 1155108) + + +2005-02-17 0.8.1 +Fixed: +- replaced MutlilinkIterator with multilinkGenerator (thanks Bob Ippolito) +- fixed broken csv import in roundup.admin module +- fixed braino in HTMLClass.filter() (sf bug 1124213) +- change ZTUtils Iterator to always iter() its sequence argument + + +2005-01-16 0.8.0 +Fixed: +- fix roundup-server log and PID file paths to be absolute +- fix initialisation of roundup-server in daemon mode so initialisation + errors are visible +- fix: 'Logout' link was enabled on issue index page only +- have Permissions only test the check function if itemid is suppled +- modify cgi templating system to check item-level permissions in listings +- enable batching in message and file listings +- more documentation of security mechanisms (incl. sf patches 1117932, + 1117860) +- better unit tests for security mechanisms +- code cleanup (sf patch 1115329 and additional) +- issue search page allows setting of no sorting / grouping (sf bug + 1119475) +- better edit conflict handling (sf bug 1118790) +- consistent text searching behaviour (AND everywhere) (sf bug 1101036) +- fix handling of invalid date input (sf bug 1102165) +- retain Boolean selections in edit error handling (sf bug 1101492) +- fix initialisation of logging module from config file (sf bug 1108577) +- removed rlog module (py 2.3 is minimum version now) +- fixed class "help" listing paging (sf bug 1106329) +- nicer error looking up values of None (response to sf bug 1108697) +- fallback for (list) popups if javascript disabled (sf patch 1101626) + + +2005-01-13 0.8.0b2 +Fixed: +- note about how to run roundup demo in Windows (sf bug 1082090) +- fix API for templating utils extensions - remove "utils" arg (sf bug 1081981) +- back_sqlite.py is missing "import time" (sf bug 1081959) +- fix (list) popup (sf bug 1083570) +- fix some security assertions (sf bug 1085481) +- 'roundup-server -S' always writes [trackers] section heading (sf bug 1088878) +- fix port number as int in mysql connection info (sf bug 1082530) +- fix setup.py to work with (thanks Roch'e Compaan) +- TAL expressions like 'request/show/whatever' return True + if the request does not contain explicit @columns list +- NumberHTMLProperty should return '' not "None" if not set (thanks + William) +- ensure multilink ordering in RDBMS backends (thanks Marcus Priesch, sf + bug 950963) +- always honor indexme property on Strings (sf patch 1063711) +- make hyperdb value parsing errors readable in mailgw errors +- make anydbm journal export handle removed properties +- allow use of XML templates again + + +2004-10-15 0.7.8 +Fixed: +- Clean out sessions / otks tables when migrating + + +2004-10-11 0.7.7 +Fixed: +- ZRoundup's search interface works now (sf bug 994957) +- fixed history display when "ascending" +- removed references to py2.3+ boolean values (sf bug 995682) +- fix static file path normalisation in security check (thanks David Linke) +- less specific messages for login failures (thanks Chris Withers) +- Reject raised against email messages should result in email rejection, not + discarding of the message +- mailgw can override the MAIL_DEFAULT_CLASS +- handle Py2.3+ datetime objects as Date specs (sf bug 971300) +- use row locking in MySQL newid() (sf bug 1034211) +- add sanity check for sort and group on same property (sf bug 1033477) +- extend OTK and session table value cols to TEXT (sf bug 1031271) +- fix lookup of REMOTE_USER (sf bug 1002923) +- new Interval props weren't created properly in rdbms +- date.Interval() now accepts an Interval as a spec (sf bug 1041266) +- handle deleted properties in RDBMS history +- apply timezone in correct direction in user input (sf bug 1013097) +- more efficient find() in RDBMS (sf bug 1012781) + + +2004-07-21 0.7.6 +Fixed: +- rdbms backend full text search failure after import (sf bug 980314) +- rdbms backends not filtering correctly on link=None +- fix anydbm journal import (sf bug 983166) +- handle postgresql bug in SQL generation (sf bug 984591) +- fix dates-from-Dates (sf bug 984604) +- fix messageid generated when msgid is None for send_message (sf bug 987933) +- make user permissions check more sane (fix search page for anonymous) +- fixed RDBMS filter() for no matches from full-text search (sf bug 990778) +- fixed DateHTMLProperty for invalid date entry (sf bug 986538) +- fixed external password source example (sf bug 986601) +- document the STATIC_FILES config var +- implement the HTTP HEAD command (sf bug 992544) +- fix journal export of files to remove content from CSV files +- API clarification. Previously, the anydbm/bsddb/metakit filter() methods + had required exact matches to Multilink argument lists. The RDBMS + backends treated Multilink matches like all other data types - matching + any of the Multilink argument list is good enough. The latter behaviour + is implemented across the board now. +- fix metakit handling of filter on Link==None + + +2004-06-24 0.7.5 +Fixed: +- force lookup of journal props in anydbm filtering +- fixed lookup of "missing" Link values for new props in anydbm backend +- allow list of values for id, Number and Boolean filtering in anydbm + backend +- fixed some more mysql 0.6->0.7 upgrade bugs (sf bug 950410) +- fixed Boolean values in postgresql (sf bugs 972546 and 972600) +- fixed -g arg to roundup-server (sf bug 973946) +- better roundup-server usage string (sf bug 973352) +- include "context" always, as documented (sf bug 965447) +- fixed REMOTE_USER (external HTTP Basic auth) (sf bug 977309) +- fixed roundup-admin "find" to use better value parsing +- fixed RDBMS Class.find() to handle None value in multiple find +- export now stores file "content" in separate files in export directory + + +2004-06-10 0.7.4 +Fixed: +- re-acquire the OTK manager when we re-open the database +- mailgw handler can close the database on us +- fixed grouping by a NULL Link value +- fixed anydbm import/export (sf bugs 965216, 964457, 964450) +- fix python 2.3.3 strftime deprecation warning (sf patch 968398) +- fix some column datatypes in postgresql and mysql (sf bugs 962611, + 959177 and 964231) +- fixed RDBMS journal packing (sf bug 959177) +- fixed filtering by floats in anydbm (sf bug 963584) + + +2004-05-28 0.7.3 +Fixed: +- add "checked" to truth values for Boolean input +- fixed import in metakit backend +- fix SearchAction use of Class.filter(), and clarify API docs for same +- ensure static files may only be served out of the tracker's "static + files" directory + + +2004-05-17 0.7.2 +Fixed: +- anydbm sorting with None values (sf bug 952853) +- roundup-server -g option not recognised (sf bug 952310) +- HTML templating isset() inverted (sf bug 951779) +- otks manager missing (sf bug 952931) +- mention DEFAULT_TIMEZONE requirement in upgrading doc (sf bug 952932) +- fix DateHTMLProperty so local() can override user timezone (sf bug + 953678) +- fix anydbm sort/group direction handling, and make RDBMS sort/group use + Link'ed "order" properties (sf bug 953148) +- fix Interval editing (sf bug 954891) + + +2004-05-10 0.7.1 +Fixed: +- several temp files made it into the source distribution (sf bug 949243) +- typo in roundup/instance.py +- missing CRLF var in rfc822.py (sf patch 949471) +- fix user creation page +- have roundup server pass though the cause of a "403 Forbidden" response +- fix schema mutation in sqlite backends (thanks Tamer Fahmy) +- make popup Javascript IE 5.0 friendly (thanks Marlon van den Berg) +- fix RDBMS import (thanks Tamer Fahmy) + + +2004-05-06 0.7.0 +Fixed: +- sqlite migration drops some journal information (thanks David Linke) +- user editing Role entry help text always appears +- disable forking server when os.fork() not available (sf bug 938586) +- removed Boolean from source to make py <2.3 happy (sf bug 938790) +- fix nested scope bug in rdbms multilink sorting +- re-seed the random number generator for each request +- postgresql backend altered to not use popen (thanks Georges Martin) +- fixed journal marshalling in RDBMS backends (sf bug 943627) +- fixed handling of key values starting with numbers (sf bug 941363) +- fixed journal "param" column size in RDBMS backends +- fixed static file serving +- fixed rego from email address (sf bug 947414) +- fixed sqlite journal ordering issue +- fixed mysql date range filtering + + +2004-04-18 0.7.0b3 Feature: - added a favicon - added url_quote and html_quote methods to the utils object @@ -13,6 +812,7 @@ Feature: - added search_checkboxes as an option for the search form - added IMAP support to mail gateway (sf rfe 934000) - check MANIFEST against the files actually unpacked +- roundupdb nosymessage() takes an optional bcc list Fixed: - mysql and postgresql schema mutation now handle added Multilinks @@ -31,6 +831,7 @@ Fixed: - sqlite backend had stopped using the global lock - better check for anonymous viewing of user items (sf bug 933510) - stop Interval from displaying an empty string (sf bug 934022) +- fixed storage of some datatypes in some RDBMS backends 2004-03-27 0.7.0b2 @@ -107,9 +908,9 @@ Fixed: - anonymous user can no longer edit or view itself (sf bug 828901). - corrected typo in installation.html (sf bug 822967). - clarified listTemplates docstring. -- print a nicer error message when the address is already in use +- print a nicer error message when the address is already in use (sf bug 798659). -- remove empty lines before sending strings off to the csv parser +- remove empty lines before sending strings off to the csv parser (sf bug 821364). - centralised conversion of user-input data to hyperdb values (sf bug 802405, sf bug 817217, sf rfe 816994) @@ -133,14 +934,21 @@ Cleanup: - tidied up forms in default stylesheet - force textareas to use monospace fonts, lessening surprise on the user - moved out parts of client.py to new modules: - * actions.py - the xxxAction and xxxPermission functions refactored into + * actions.py - the xxxAction and xxxPermission functions refactored into Action classes * exceptions.py - all exceptions * form_parser.py - parsePropsFromForm & extractFormList in a FormParser class -2004-??-?? 0.6.9 +2004-05-17 0.6.10 +Fixed: +- mysql backend wasn't locking tracker +- ensure static files may only be served out of the tracker's "static + files" directory + + +2004-04-18 0.6.9 Fixed: - paging in classhelp popup was broken - socket timeout error logging can fail @@ -284,7 +1092,7 @@ Fixed: - audit some user properties for valid values (roles, address) (sf bugs 742968 and 739653) - fix HTML file detection (hence history xref linking) (sf bug 741478) -- session database caches it's type, rather than calling whichdb each time +- session database caches it's type, rather than calling whichdb each time around. - changed rdbms_common to fix sql backends for new Boolean types under Py2.3 @@ -319,7 +1127,7 @@ Feature: cc addresses, different from address and different nosy list property) (thanks John Rouillard) - applied patch for nicer history display (sf feature 638280) -- cleaning old unused sessions only once per hour, not on every cgi +- cleaning old unused sessions only once per hour, not on every cgi request. It is greatly improves web interface performance, especially on trackers under high load - added mysql backend (see doc/mysql.txt for details) @@ -377,7 +1185,7 @@ Feature: Fixed: - applied unicode patch. All data is stored in utf-8. Incoming messages - converted from any encoding to utf-8, outgoing messages are encoded + converted from any encoding to utf-8, outgoing messages are encoded according to rfc2822 (sf bug 568873) - fixed layout issues with forms in sidebar - fixed timelog example so it handles new issues (sf bug 678908) @@ -460,7 +1268,7 @@ Fixed: - handle :add: better in cgi form parsing (sf bug 663235) - handle all-whitespace multilink values in forms (sf bug 663855) - fixed searching on date / interval fields (sf bug 658157) -- fixed form elements names in search form to allow grouping and sorting +- fixed form elements names in search form to allow grouping and sorting on "creation" field - display of saved queries is now performed correctly @@ -650,7 +1458,7 @@ Feature: - daemonify roundup-server (fork, logfile, pidfile) - modify cgitb to display PageTemplate errors better - rename to "instance" to "tracker" -- have roundup.cgi pick up tracker config from the environment +- have roundup.cgi pick up tracker config from the environment - revamped look and feel in web interface - cleaned up stylesheet usage - several bug fixes and documentation fixes @@ -684,7 +1492,7 @@ Feature: done in the default templates. - the regeneration of the indexes (if necessary) is done once the schema is set up in the dbinit. - - new "reindex" command in roundup-admin used to force regeneration of the + - new "reindex" command in roundup-admin used to force regeneration of the index - added email display function - mangles email addrs so they're not so easily scraped from the web @@ -762,7 +1570,7 @@ Fixed: wants to ignore - fixed the example addresses in the templates to use correct example domains - cleaned out the template stylesheets, removing a bunch of junk that really - wasn't necessary (font specs, styles never used) and added a style for + wasn't necessary (font specs, styles never used) and added a style for message content - build htmlbase if tests are run using CVS checkout - #565979 ] code error in hyperdb.Class.find @@ -775,7 +1583,7 @@ Fixed: - #565992 ] if ISSUE_TRACKER_WEB doesn't have the trailing '/', add it - use the rfc822 module to ensure that every (oddball) email address and real-name is properly quoted -- #558867 ] ZRoundup redirect /instance requests to /instance/ +- #558867 ] ZRoundup redirect /instance requests to /instance/ - #569415 ] {version} - #569178 ] type error was fixed as part of the general cleanup of reactors @@ -839,13 +1647,13 @@ Fixed: 2002-01-24 - 0.4.0 Feature: - much nicer history display (actualy real handling of property types etc) -- journal entries for link and mutlilink properties can be switched on or +- journal entries for link and mutlilink properties can be switched on or off - properties in change note are now sorted - you can now use the roundup-admin tool pack the database Fixed: -- the mail gateway now responds with an error message when invalid values +- the mail gateway now responds with an error message when invalid values for arguments are specified for link or mutlilink properties - modified unit test to check nosy and assignedto when specified as arguments - handle attachments with no name (eg tnef) @@ -952,7 +1760,7 @@ Fixed: - added tests for mailgw -2001-11-23 - 0.3.0 +2001-11-23 - 0.3.0 Feature: - #467129 ] Lossage when username=e-mail-address - #473123 ] Change message generation for author @@ -1260,7 +2068,7 @@ Features: - Added the "classic" template - a direct implementation of the Roundup spec. Well, as close as we're going to get, anyway. - Added an issue priority of support to "extended" -- Added command-line arg handling to roundup-server so it's more useful +- Added command-line arg handling to roundup-server so it's more useful out-of-the-box. - Added distutils-style installation of "lib" files. - Added some unit tests. diff --git a/ChangeLog b/ChangeLog index 84c6a8e..1f08189 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2006-11-22 Stefan Seefeld + + * roundup/cgi/form_parser.py: Allow required fields to be ommitted + if user doesn't have edit permission and the value is already set. + 2001-08-03 11:54 richard * BUILD.txt, CHANGES.txt, README.txt, setup.py, @@ -872,7 +877,10 @@ * cgitb.py, config.py, date.py, hyperdb.py, roundup-mailgw.py, roundup.py, roundup_cgi.py, roundupdb.py, server.py, template.py: - Added CVS keywords $Id: ChangeLog,v 1.7 2001-08-03 02:12:07 anthonybaxter Exp $ and $Log: not supported by cvs2svn $ to all python files. + Added CVS keywords $Id: ChangeLog,v 1.8 2006-11-23 00:44:48 stefan Exp $ and $Log: not supported by cvs2svn $ + Added CVS keywords $Id: ChangeLog,v 1.8 2006-11-23 00:44:48 stefan Exp $ and Revision 1.7 2001/08/03 02:12:07 anthonybaxter + Added CVS keywords $Id: ChangeLog,v 1.8 2006-11-23 00:44:48 stefan Exp $ and regenerated on Fri Aug 3 12:12:00 EST 2001 + Added CVS keywords $Id: ChangeLog,v 1.8 2006-11-23 00:44:48 stefan Exp $ and to all python files. 2001-07-19 15:46 anthonybaxter diff --git a/I18N_PROGRESS.txt b/I18N_PROGRESS.txt deleted file mode 100644 index 2ee110f..0000000 --- a/I18N_PROGRESS.txt +++ /dev/null @@ -1,93 +0,0 @@ -This list has been generated using the MANIFEST file. We should be able to -write a simple script to compare the two and make sure that all MANIFEST -files appear in here. - -To generate a messages.pot file, use this command: - - python tools/pygettext.py roundup roundup-* cgi-bin/roundup.cgi - -"messages.pot" then contains a positive list of files using _(), which can -be extracted by; - - grep "#: " messages.pot | tr ":#0123456789 " "\012" | sort | uniq - -Of course, this does not check whether a file is fully translated, only -whether there is at least one use of "_()". - - -THESE FILES DO NOT USE _() -========================== -roundup/hyperdb.py -roundup/i18n.py -roundup/init.py -roundup/install_util.py -roundup/instance.py -roundup/mailgw.py -roundup/password.py -roundup/roundupdb.py -roundup/templatebuilder.py -roundup/backends/__init__.py -roundup/backends/back_anydbm.py -roundup/backends/back_bsddb.py -roundup/backends/back_bsddb3.py -roundup/templates/__init__.py -roundup/templates/classic/__init__.py -roundup/templates/classic/dbinit.py -roundup/templates/classic/htmlbase.py -roundup/templates/classic/config.py -roundup/templates/classic/interfaces.py -roundup/templates/classic/detectors/__init__.py -roundup/templates/classic/detectors/nosyreaction.py - - -THESE FILES DO USE _() -====================== -roundup-admin -roundup-mailgw -roundup-server -cgi-bin/roundup.cgi -roundup/__init__.py -roundup/admin.py -roundup/date.py -roundup/cgi/cgitb.py - - -WE DON'T CARE ABOUT THESE FILES -=============================== -BUILD.txt -CHANGES.txt -INSTALL.txt -README.txt -setup.py -doc/implementation.txt -doc/index.html -doc/overview.html -doc/spec.html -doc/images/edit.png -doc/images/hyperdb.png -doc/images/logo-acl-medium.png -doc/images/logo-codesourcery-medium.png -doc/images/logo-software-carpentry-standard.png -doc/images/roundup-1.png -doc/images/roundup.png -roundup/templates/classic/html/file.index -roundup/templates/classic/html/file.newitem -roundup/templates/classic/html/issue.filter -roundup/templates/classic/html/issue.index -roundup/templates/classic/html/issue.item -roundup/templates/classic/html/msg.index -roundup/templates/classic/html/msg.item -roundup/templates/classic/html/style.css -roundup/templates/classic/html/user.index -roundup/templates/classic/html/user.item -test/README.txt -test/__init__.py -test/test_dates.py -test/test_db.py -test/test_init.py -test/test_mailsplit.py -test/test_multipart.py -test/test_schema.py -test/test_templating.py -test/unittest.py - diff --git a/MANIFEST.in b/MANIFEST.in index 4ae1a14..960a570 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -2,13 +2,12 @@ recursive-include roundup *.* recursive-include frontends *.* recursive-include scripts *.* *-* recursive-include tools *.* -recursive-include cgi-bin *.cgi recursive-include test *.py *.txt -recursive-include doc *.html *.png *.txt *.css *.1 +recursive-include doc *.html *.png *.txt *.css *.1 *.example recursive-include detectors *.py recursive-include templates *.* home* page* -global-exclude .cvsignore *.pyc *.pyo -include run_tests.py *.txt demo.py MANIFEST.in +global-exclude .cvsignore *.pyc *.pyo .DS_Store +include run_tests.py *.txt demo.py MANIFEST.in MANIFEST exclude BUILD.txt I18N_PROGRESS.txt TODO.txt exclude doc/security.txt doc/templating.txt - +include locale/*.po locale/*.mo locale/roundup.pot diff --git a/README.txt b/README.txt index 6457786..f0b6689 100644 --- a/README.txt +++ b/README.txt @@ -18,6 +18,10 @@ To start anew (a fresh demo instance):: python demo.py nuke +Run demo.py from the *source* directory; don't try to run demo.py from +the *installed* directory, it will *break*. + + Installation ============ For installation instructions, please see installation.txt in the "doc" @@ -27,11 +31,14 @@ directory. Upgrading ========= For upgrading instructions, please see upgrading.txt in the "doc" directory. - + Usage and Other Information =========================== See the index.txt file in the "doc" directory. +The *.txt files in the "doc" directory are written in reStructedText. If +you have rst2html installed (part of the docutils suite) you can convert +these to HTML by running "make html" in the "doc" directory. License diff --git a/TODO.txt b/TODO.txt deleted file mode 100644 index 29af698..0000000 --- a/TODO.txt +++ /dev/null @@ -1,8 +0,0 @@ -This file contains items that need doing before the next release: - -. make Intervals store timestamps, not strings - - -Optionally: -- have rdbms backends look up the journal for actor if it's not set -- migrate to numeric ID values (fixes bug 817217) diff --git a/cgi-bin/roundup.cgi b/cgi-bin/roundup.cgi index bc01294..9674437 100755 --- a/cgi-bin/roundup.cgi +++ b/cgi-bin/roundup.cgi @@ -16,12 +16,12 @@ # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. # -# $Id: roundup.cgi,v 1.37 2003-11-03 23:37:06 richard Exp $ +# $Id: roundup.cgi,v 1.42 2005-05-18 05:39:21 richard Exp $ # python version check from roundup import version_check from roundup.i18n import _ -import sys +import sys, time # ## Configuration @@ -42,8 +42,9 @@ import sys # ROUNDUP_LOG is the name of the logfile; if it's empty or does not exist, # logging is turned off (unless you changed the default below). -# ROUNDUP_DEBUG is a debug level, currently only 0 (OFF) and 1 (ON) are -# used in the code. Higher numbers means more debugging output. +# DEBUG_TO_CLIENT specifies whether debugging goes to the HTTP server (via +# stderr) or to the web client (via cgitb). +DEBUG_TO_CLIENT = False # This indicates where the Roundup tracker lives TRACKER_HOMES = { @@ -158,16 +159,19 @@ def main(out, err): else: tracker_home = TRACKER_HOMES[tracker] tracker = roundup.instance.open(tracker_home) - from roundup.cgi.client import Unauthorised, NotFound - client = tracker.Client(tracker, request, os.environ) + import roundup.cgi.client + if hasattr(tracker, 'Client'): + client = tracker.Client(tracker, request, os.environ) + else: + client = roundup.cgi.client.Client(tracker, request, os.environ) try: client.main() - except Unauthorised: + except roundup.cgi.client.Unauthorised: request.send_response(403) request.send_header('Content-Type', 'text/html') request.end_headers() out.write('Unauthorised') - except NotFound: + except roundup.cgi.client.NotFound: request.send_response(404) request.send_header('Content-Type', 'text/html') request.end_headers() @@ -208,7 +212,16 @@ except SystemExit: except: sys.stdout, sys.stderr = out, err out.write('Content-Type: text/html\n\n') - cgitb.handler() + if DEBUG_TO_CLIENT: + cgitb.handler() + else: + out.write(cgitb.breaker()) + ts = time.ctime() + out.write('''

%s: An error occurred. Please check + the server log for more infomation.

'''%ts) + print >> sys.stderr, 'EXCEPTION AT', ts + traceback.print_exc(0, sys.stderr) + sys.stdout.flush() sys.stdout, sys.stderr = out, err LOG.close() diff --git a/demo.py b/demo.py index 7d7f947..b911541 100644 --- a/demo.py +++ b/demo.py @@ -1,36 +1,51 @@ #! /usr/bin/env python # # Copyright (c) 2003 Richard Jones (richard@mechanicalcat.net) -# -# $Id: demo.py,v 1.10 2004-03-31 23:07:51 richard Exp $ +# +# $Id: demo.py,v 1.26 2007-08-28 22:37:45 jpend Exp $ -import sys, os, string, re, urlparse -import shutil, socket, errno, BaseHTTPServer +import errno +import os +import socket +import sys +import urlparse from glob import glob -def install_demo(home, backend): - # create the instance - if os.path.exists(home): - shutil.rmtree(home) +from roundup import configuration +from roundup.scripts import roundup_server + +def install_demo(home, backend, template): + """Install a demo tracker + + Parameters: + home: + tracker home directory path + backend: + database backend name + template: + full path to the tracker template directory + + """ from roundup import init, instance, password, backends + # set up the config for this tracker + config = configuration.CoreConfig() + config['TRACKER_HOME'] = home + config['MAIL_DOMAIN'] = 'localhost' + config['DATABASE'] = 'db' + config['WEB_DEBUG'] = True + if backend in ('mysql', 'postgresql'): + config['RDBMS_HOST'] = 'localhost' + config['RDBMS_USER'] = 'rounduptest' + config['RDBMS_PASSWORD'] = 'rounduptest' + config['RDBMS_NAME'] = 'rounduptest' + # see if we have further db nuking to perform - module = getattr(backends, backend) - if backend == 'mysql': - class config: - MYSQL_DBHOST = 'localhost' - MYSQL_DBUSER = 'rounduptest' - MYSQL_DBPASSWORD = 'rounduptest' - MYSQL_DBNAME = 'rounduptest' - DATABASE = 'home' + module = backends.get_backend(backend) + if module.db_exists(config): module.db_nuke(config) - elif backend == 'postgresql': - class config: - POSTGRESQL_DATABASE = {'database': 'rounduptest'} - DATABASE = 'home' - module.db_nuke(config, 1) - init.install(home, os.path.join('templates', 'classic')) + init.install(home, template) # don't have email flying around os.remove(os.path.join(home, 'detectors', 'nosyreaction.py')) try: @@ -41,7 +56,7 @@ def install_demo(home, backend): init.write_select_db(home, backend) # figure basic params for server - hostname = socket.gethostname() + hostname = 'localhost' # pick a fairly odd, random port port = 8917 while 1: @@ -59,71 +74,63 @@ def install_demo(home, backend): s.close() print 'already in use.' port += 100 - url = 'http://%s:%s/demo/'%(hostname, port) + config['TRACKER_WEB'] = 'http://%s:%s/demo/'%(hostname, port) # write the config - f = open(os.path.join(home, 'config.py'), 'r') - s = f.read().replace('http://tracker.example/cgi-bin/roundup.cgi/bugs/', - url) - f.close() - # DB connection stuff for mysql and postgresql - s = s + """ -MYSQL_DBHOST = 'localhost' -MYSQL_DBUSER = 'rounduptest' -MYSQL_DBPASSWORD = 'rounduptest' -MYSQL_DBNAME = 'rounduptest' -MYSQL_DATABASE = (MYSQL_DBHOST, MYSQL_DBUSER, MYSQL_DBPASSWORD, MYSQL_DBNAME) -POSTGRESQL_DATABASE = {'database': 'rounduptest'} -""" - f = open(os.path.join(home, 'config.py'), 'w') - f.write(s) - f.close() - - # initialise the database - init.initialise(home, 'admin') + config['INSTANT_REGISTRATION'] = 1 + config.save(os.path.join(home, config.INI_FILE)) - # add the "demo" user + # open the tracker and initialise tracker = instance.open(home) + tracker.init(password.Password('admin')) + + # add the "demo" user db = tracker.open('admin') db.user.create(username='demo', password=password.Password('demo'), realname='Demo User', roles='User') db.commit() db.close() -def run_demo(): - ''' Run a demo server for users to play with for instant gratification. - - Sets up the web service on localhost. Disables nosy lists. - ''' - home = os.path.abspath('demo') - backend = 'anydbm' - if not os.path.exists(home) or sys.argv[-1] == 'nuke': - if len(sys.argv) > 2: - backend = sys.argv[1] - install_demo(home, backend) - - f = open(os.path.join(home, 'config.py'), 'r') - url = re.search(r'^TRACKER_WEB\s*=\s*[\'"](http.+/)[\'"]$', f.read(), - re.M|re.I).group(1) - f.close() +def run_demo(home): + """Run the demo tracker installed in ``home``""" + cfg = configuration.CoreConfig(home) + url = cfg["TRACKER_WEB"] hostname, port = urlparse.urlparse(url)[1].split(':') port = int(port) - - # ok, so start up the server - from roundup.scripts import roundup_server - roundup_server.RoundupRequestHandler.TRACKER_HOMES = {'demo': home} - success_message = '''Server running - connect to: %s 1. Log in as "demo"/"demo" or "admin"/"admin". 2. Hit Control-C to stop the server. -3. Re-start the server by running "python demo.py" again. -4. Re-initialise the server by running "python demo.py nuke".''' % url +3. Re-start the server by running "roundup-demo" again. +4. Re-initialise the server by running "roundup-demo nuke". + +Demo tracker is set up to be accessed by localhost browser. If you +run demo on a server host, please stop the demo, open file +"demo/config.ini" with your editor, change the host name in the "web" +option in section "[tracker]", save the file, then re-run the demo +program. + +''' % url - sys.argv = sys.argv[:1] - roundup_server.run(port, success_message) + # disable command line processing in roundup_server + sys.argv = sys.argv[:1] + ['-p', str(port), 'demo=' + home] + roundup_server.run(success_message=success_message) + +def demo_main(): + """Run a demo server for users to play with for instant gratification. + + Sets up the web service on localhost. Disables nosy lists. + """ + home = os.path.abspath('demo') + if not os.path.exists(home) or (sys.argv[-1] == 'nuke'): + if len(sys.argv) > 2: + backend = sys.argv[-2] + else: + backend = 'anydbm' + install_demo(home, backend, os.path.join('templates', 'classic')) + run_demo(home) if __name__ == '__main__': - run_demo() + demo_main() -# vim: set filetype=python ts=4 sw=4 et si +# vim: set filetype=python sts=4 sw=4 et si : diff --git a/detectors/emailauditor.py b/detectors/emailauditor.py index 2f1c290..e4af8bc 100644 --- a/detectors/emailauditor.py +++ b/detectors/emailauditor.py @@ -28,7 +28,7 @@ def eml_to_mht(db, cl, nodeid, newvalues): name extension. So... we do that. :)''' - if newalues.get('type', '').lower() == "message/rfc822": + if newvalues.get('type', '').lower() == "message/rfc822": if not newvalues.has_key('name'): newvalues['name'] = 'email.mht' return diff --git a/doc/.cvsignore b/doc/.cvsignore index 0f9a7cf..f16764b 100644 --- a/doc/.cvsignore +++ b/doc/.cvsignore @@ -17,3 +17,5 @@ mysql.html postgresql.html tracker_templates.html whatsnew-0.7.html +whatsnew-0.8.html +*.ht diff --git a/doc/FAQ.txt b/doc/FAQ.txt index 62a7dbe..dbdbeb5 100644 --- a/doc/FAQ.txt +++ b/doc/FAQ.txt @@ -2,7 +2,7 @@ Roundup FAQ =========== -:Version: $Revision: 1.16 $ +:Version: $Revision: 1.23 $ .. contents:: @@ -13,7 +13,7 @@ Installation Living without a mailserver ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Remove the nosy reactor, means delete the tracker file +Remove the nosy reactor - delete the tracker file ``detectors/nosyreactor.py`` from your tracker home. @@ -23,24 +23,27 @@ The cgi-bin is very slow! Yep, it sure is. It has to start up Python and load all of the support libraries for *every* request. -The solution is to use the built in server. +The solution is to use the built in server (or possibly the mod_python +or WSGI support). To make Roundup more seamless with your website, you may place the built -in server behind apache and link it into your web tree +in server behind apache and link it into your web tree (see below). How do I put Roundup behind Apache ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -We have a project (foo) running on ``foohost.com:8888``. -We want ``http://foohost.com/FooIssues`` to use the roundup server, so we -set that up on port 8888 on ``foohost.com`` with the ``config.py`` line:: +We have a project (foo) running on ``tracker.example:8080``. +We want ``http://tracker.example/issues`` to use the roundup server, so we +set that up on port 8080 on ``tracker.example`` with the ``config.ini`` line:: - TRACKER_WEB = 'http://foohost.com/FooIssues/' + [tracker] + ... + web = 'http://tracker.example/issues/' We have a "foo_issues" tracker and we run the server with:: - roundup-server -p 8888 foo_issues=/home/roundup/trackers/foo_issues + roundup-server -p 8080 issues=/home/roundup/trackers/issues Then, on the Apache machine (eg. redhat 7.3 with apache 1.3), in ``/etc/httpd/conf/httpd.conf`` uncomment:: @@ -53,27 +56,50 @@ and:: Then add:: + # roundup stuff (added manually) + + # proxy through one tracker + ProxyPass /issues/ http://tracker.example:8080/issues/ + # proxy through all tracker(*) + #ProxyPass /roundup/ http://tracker.example:8080/ + + +Then restart Apache. Now Apache will proxy the request on to the +roundup-server. + +Note that if you're proxying multiple trackers, you'll need to use the +second ProxyPass rule described above. It will mean that your TRACKER_WEB +will change to:: + + TRACKER_WEB = 'http://tracker.example/roundup/issues/' + +Once you're done, you can firewall off port 8080 from the rest of the world. + +Note that in some situations (eg. virtual hosting) you might need to use a +more complex rewrite rule instead of the simpler ProxyPass above. The +following should be useful as a starting template:: + # roundup stuff (added manually) RewriteEngine on # General Roundup - RewriteRule ^/Roundup$ Roundup/ [R] - RewriteRule ^/Roundup/(.*)$ http://foohost.com:8888/$1 [P,L] + RewriteRule ^/roundup$ roundup/ [R] + RewriteRule ^/roundup/(.*)$ http://tracker.example:8080/$1 [P,L] # Handle Foo Issues - RewriteRule ^/FooIssues$ FooIssues/ [R] - RewriteRule ^/FooIssues/(.*)$ http://foohost.com:8888/foo_issues/$1 [P,L] + RewriteRule ^/issues$ issues/ [R] + RewriteRule ^/issues/(.*)$ http://tracker.example:8080/issues/$1 [P,L] -Then restart Apache. Now Apache will proxy the request on to the -roundup-server. -You need to add the last 3 RewriteRule lines for each tracker that you have. +How do I run Roundup through SSL (HTTPS)? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can now firewall off port 8888 from the rest of the world. +You should proxy through apache and use its SSL service. See the previous +question on how to proxy through apache. Roundup runs very slowly on my XP machine when accessed from the Internet @@ -100,9 +126,9 @@ This is based upon the template markup language in Zope called, oddly enough "Zope Page Templates". There's documentation in the Roundup customisation_ documentation. For more information have a look at: - http://www.zope.org/Documentation/Books/ZopeBook/current/contents + http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/ -specifically chapter 5 "Using Zope Page Templates" and chapter 9 "Advanced +specifically chapter 10 "Using Zope Page Templates" and chapter 14 "Advanced Page Templates". @@ -156,9 +182,27 @@ If you're using IE then install Mozilla and try again ;^) I keep getting logged out ~~~~~~~~~~~~~~~~~~~~~~~~~ -Make sure that the TRACKER_WEB setting in your tracker's config.py is set -to the URL of the tracker. +Make sure that the ``tracker`` -> ``web`` setting in your tracker's +config.ini is set to the URL of the tracker. + + +How is sorting performed, and why does it seem to fail sometimes? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When we sort items in the hyperdb, we use one of a number of methods, +depending on the properties being sorted on: + +1. If it's a String, Number, Date or Interval property, we just sort the + scalar value of the property. Strings are sorted case-sensitively. +2. If it's a Link property, we sort by either the linked item's "order" + property (if it has one) or the linked item's "id". +3. Mulitlinks sort similar to #2, but we start with the first + Multilink list item, and if they're the same, we sort by the second item, + and so on. +Note that if an "order" property is defined on a Class that is used for +sorting, all items of that Class *must* have a value against the "order" +property, or sorting will result in random ordering. ----------------- diff --git a/doc/Makefile b/doc/Makefile index e17fbd9..a5ce16e 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -1,17 +1,28 @@ -PYTHON = /usr/bin/python2 STXTOHTML = rst2html +STXTOHT = rst2ht.py +WEBDIR = ../../htdocs/htdocs/doc-1.0 SOURCE = announcement.txt customizing.txt developers.txt FAQ.txt features.txt \ glossary.txt implementation.txt index.txt design.txt mysql.txt \ installation.txt upgrading.txt user_guide.txt admin_guide.txt \ - postgresql.txt tracker_templates.txt whatsnew-0.7.txt + postgresql.txt tracker_templates.txt xmlrpc.txt COMPILED := $(SOURCE:.txt=.html) +WEBHT := $(SOURCE:.txt=.ht) -all: ${COMPILED} +all: html ht +html: ${COMPILED} +ht: ${WEBHT} + +website: ${WEBHT} + cp *.ht ${WEBDIR} + cp -r images ${WEBDIR} %.html: %.txt ${STXTOHTML} --report=warning -d $< $@ +%.ht: %.txt + ${STXTOHT} --report=warning -d $< $@ + clean: rm -f ${COMPILED} diff --git a/doc/admin_guide.txt b/doc/admin_guide.txt index fe02582..ebf4c5e 100644 --- a/doc/admin_guide.txt +++ b/doc/admin_guide.txt @@ -2,7 +2,7 @@ Administration Guide ==================== -:Version: $Revision: 1.5 $ +:Version: $Revision: 1.28 $ .. contents:: @@ -36,8 +36,98 @@ There's two "installations" that we talk about when using Roundup: "inst" (and "init") commands, you're creating a new Roundup tracker. This installs configuration files, HTML templates, detector code and a new database. You have complete control over where this stuff goes through - both choosing your "tracker home" and the DATABASE variable in - config.py. + both choosing your "tracker home" and the ``main`` -> ``database`` variable + in the tracker's config.ini. + + +Configuring Roundup's Logging of Messages For Sysadmins +======================================================= + +You may configure where Roundup logs messages in your tracker's config.ini +file. Roundup will use the standard Python (2.3+) logging implementation +when available. If not, then a very basic logging implementation will be used +(see BasicLogging in the roundup.rlog module for details). + +Configuration for standard "logging" module: + - tracker configuration file specifies the location of a logging + configration file as ``logging`` -> ``config`` + - ``roundup-server`` specifies the location of a logging configuration + file on the command line +Configuration for "BasicLogging" implementation: + - tracker configuration file specifies the location of a log file + ``logging`` -> ``filename`` + - tracker configuration file specifies the level to log to as + ``logging`` -> ``level`` + - ``roundup-server`` specifies the location of a log file on the command + line + - ``roundup-server`` specifies the level to log to on the command line + +(``roundup-mailgw`` always logs to the tracker's log file) + +In both cases, if no logfile is specified then logging will simply be sent +to sys.stderr with only logging of ERROR messages. + + +Configuring roundup-server +========================== + +The basic configuration file layout is as follows (take from the +``roundup-server.ini.example`` file in the "doc" directory):: + + [main] + port = 8080 + ;hostname = + ;user = + ;group = + ;log_ip = yes + ;pidfile = + ;logfile = + ;template = + ;ssl = no + ;pem = + + [trackers] + ; Add one of these per tracker being served + name = /path/to/tracker/name + +Values ";commented out" are optional. The meaning of the various options +are as follows: + +**port** + Defines the local TCP port to listen for clients on. +**hostname** + Defines the local hostname to listen for clients on. Only required if + "localhost" is not sufficient. +**user** and **group** + Defines the Unix user and group to run the server as. Only work if the + server is started as root. +**log_ip** + If ``yes`` then we log IP addresses against accesses. If ``no`` then we + log the hostname of the client. The latter can be much slower. +**pidfile** + If specified, the server will fork at startup and write its new PID to + the file. +**logfile** + Any unhandled exception messages or other output from Roundup will be + written to this file. It must be specified if **pidfile** is specified. + If per-tracker logging is specified, then very little will be written to + this file. +**template** + Specifies a template used for displaying the tracker index when + multiple trackers are being used. The variable "trackers" is available + to the template and is a dict of all configured trackers. +**ssl** + Enables the use of SSL to secure the connection to the roundup-server. + If you enable this, ensure that your tracker's config.ini specifies + an *https* URL. +**pem** + If specified, the SSL PEM file containing the private key and certificate. + If not specified, roundup will generate a temporary, self-signed certificate + for use. +**trackers** section + Each line denotes a mapping from a URL component to a tracker home. + Make sure the name part doesn't include any url-unsafe characters like + spaces. Stick to alphanumeric characters and you'll be ok. Users and Security @@ -64,7 +154,7 @@ Roundup identifies users in a number of ways: In both cases, Roundup's behaviour when dealing with unknown users is controlled by Permissions defined in the "SECURITY SETTINGS" section of the -tracker's ``dbinit.py`` module: +tracker's ``schema.py`` module: Web Registration If granted to the Anonymous Role, then anonymous users will be able to @@ -76,22 +166,32 @@ Email Registration More information about how to customise your tracker's security settings may be found in the `customisation documentation`_. + Tasks ===== Maintenance of Roundup can involve one of the following: -1. `tracker backup`_ +1. `tracker backup`_ 2. `software upgrade`_ 3. `migrating backends`_ 4. `moving a tracker`_ +5. `migrating from other software`_ +6. `adding a user from the command-line`_ Tracker Backup -------------- -Stop the web and email frontends and to copy the contents of the tracker home -directory to some other place using standard backup tools. +The roundup-admin import and export commands are **not** recommended for +performing backup. + +Optionally stop the web and email frontends and to copy the contents of the +tracker home directory to some other place using standard backup tools. +This means using +*pg_dump* to take a snapshot of your Postgres backend database, for example. +A simple copy of the tracker home (and files storage area if you've configured +it to be elsewhere) will then complete the backup. Software Upgrade @@ -100,34 +200,53 @@ Software Upgrade Always make a backup of your tracker before upgrading software. Steps you may take: -1. ensure that the unit tests run on your system -2. copy your tracker home to a new directory -3. follow the steps in the upgrading documentation for the new version of - the software -4. test each of the admin tool, web interface and mail gateway using the new - version of the software -5. stop the production web and email frontends -6. perform the upgrade steps on the existing tracker directory -7. upgrade the software -8. restart your tracker +1. Ensure that the unit tests run on your system:: + + python run_tests.py + +2. If you're using an RDBMS backend, make a backup of its contents now. +3. Make a backup of the tracker home itself. +4. Stop the tracker web and email frontends. +5. Install the new version of the software:: + + python setup.py install + +6. Follow the steps in the `upgrading documentation`_ for the new version of + the software in the copied. + + Usually you will be asked to run `roundup_admin migrate` on your tracker + before you allow users to start accessing the tracker. + + It's safe to run this even if it's not required, so just get into the + habit. +7. Restart your tracker web and email frontends. + +If something bad happens, you may reinstate your backup of the tracker and +reinstall the older version of the sofware using the same install command:: + + python setup.py install Migrating Backends ------------------ -1. stop the existing tracker web and email frontends (preventing changes) -2. use the roundup-admin tool "export" command to export the contents of - your tracker to disk -3. copy the tracker home to a new directory -4. change the backend used in the tracker home ``select_db.py`` file -5. delete the "db" directory from the new directory -6. use the roundup-admin "import" command to import the previous export with - the new tracker home -7. test each of the admin tool, web interface and mail gateway using the new - backend -8. move the old tracker home out of the way (rename to "tracker.old") and - move the new tracker home into its place -9. restart web and email frontends +1. Stop the existing tracker web and email frontends (preventing changes). +2. Use the roundup-admin tool "export" command to export the contents of + your tracker to disk. +3. Copy the tracker home to a new directory. +4. Delete the "db" directory from the new directory. +5. Enter the new backend name in the tracker home ``db/backend_name`` file. +6. Use the roundup-admin "import" command to import the previous export with + the new tracker home. If non-interactively:: + + roundup-admin -i import + + If interactively, enter 'commit' before exitting. +7. Test each of the admin tool, web interface and mail gateway using the new + backend. +8. Move the old tracker home out of the way (rename to "tracker.old") and + move the new tracker home into its place. +9. Restart web and email frontends. Moving a Tracker @@ -154,6 +273,54 @@ moved using the above steps) then you'll need to: 6. start the tracker web and email frontends on the new machine. +Migrating From Other Software +----------------------------- + +You have a couple of choices. You can either use a CSV import into Roundup, +or you can write a simple Python script which uses the Roundup API +directly. The latter is almost always simpler -- see the "scripts" +directory in the Roundup source for some example uses of the API. + +"roundup-admin import" will import data into your tracker from a +directory containing files with the following format: + +- one colon-separated-values file per Class with columns for each property, + named .csv +- one colon-separated-values file per Class with journal information, + named -journals.csv (this is required, even if it's empty) +- if the Class is a FileClass, you may have the "content" property + stored in separate files from the csv files. This goes in a directory + structure:: + + -files// + + where ```` is the item's ```` combination. + The ```` value is ``int( / 1000)``. + + +Adding A User From The Command-Line +----------------------------------- + +The ``roundup-admin`` program can create any data you wish to in the +database. To create a new user, use:: + + roundup-admin create user + +To figure out what good values might be for some of the fields (eg. Roles) +you can just display another user:: + + roundup-admin list user + +(or if you know their username, and it happens to be "richard"):: + + roundup-admin find username=richard + +then using the user id you get from one of the above commands, you may +display the user's details:: + + roundup-admin display + + Running the Servers =================== @@ -168,18 +335,28 @@ to reflect your specific installation. Windows ------- -On Windows, the roundup-server program runs as a Windows Service, and -therefore may be controlled through the Services control panel. +On Windows, the roundup-server program runs as a Windows Service, and +therefore may be controlled through the Services control panel. The +roundup-server program may also control the service directly: -TODO: info how to bring up the services panel. +**install the service** + ``roundup-server -C /path/to/my/roundup-server.ini -c install`` +**start the service** + ``roundup-server -c start`` +**stop the service** + ``roundup-server -c stop`` -TODO: how to start the server in "service" mode. +To bring up the services panel: -On windows, use:: +Windows 2000 and later + Start/Control Panel/Administrative Tools/Services +Windows NT4 + Start/Control Panel/Services - roundup-server -c stop - -to stop the running server. +You will need a server configuration file (as described in +`Configuring roundup-server`_) for specifying tracker homes +and other roundup-server configuration. Specify the name of +this file using the ``-C`` switch when installing the service. Running the Mail Gateway Script ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -199,4 +376,4 @@ Back to `Table of Contents`_ .. _`Table of Contents`: index.html .. _`customisation documentation`: customizing.html - +.. _`upgrading documentation`: upgrading.html diff --git a/doc/announcement.txt b/doc/announcement.txt index 0b4f052..148ccdd 100644 --- a/doc/announcement.txt +++ b/doc/announcement.txt @@ -1,52 +1,23 @@ -This is the second beta release of Roundup version 0.7. It fixes some bugs -in the previous beta release: - -- Boolean, Date and Link HTML templating was broken -- fix reporting of test inclusion in postgresql test -- EditAction was confused about who "self" was -- edit collision detection was broken for index-page edits -- sqlite backend wasn't migrating multilink tables correctly -- use SimpleCookie instead of Cookie (is an alias for the evil SmartCookie) -- handle older sessions in session dbm -- make presetunread more resilient to status Class changes -- HTMLDatabase classes() was broken +I'm proud to release version 1.4.6 of Roundup. -If you're upgrading from an older version of Roundup you *must* follow -the "Software Upgrade" guidelines given in the maintenance documentation. - -No, really, this is a BETA and if you don't follow the upgrading steps, -particularly the bit about BACKING UP YOUR DATA, I'm NOT GOING TO BE HELD -RESPONSIBLE. This release is NOT FOR GENERAL USE. - -I would *greatly* appreciate people giving this release a whirl with a -copy of their existing setup. It's only through real-world testing of -beta releases that we can ensure that older trackers will be OK. +1.4.6 is a bugfix release: -Version 0.7 introduces far too many features to list here. I've put -together a What's New page: +- Fix bug introduced in 1.4.5 in RDBMS full-text indexing +- Make URL matching code less matchy - http://roundup.sourceforge.net/doc-0.7/whatsnew-0.7.html - -Some highlights: - -- added postgresql backend -- RDBMS backends have no external locking requirements -- new "actor" automatic property (user who caused the last "activity") -- RDBMS backends have data typed columns and indexes on several columns -- we support confirming registration by replying to the email -- all HTML templating methods now automatically check for permissions, - greatly simplifying templates +If you're upgrading from an older version of Roundup you *must* follow +the "Software Upgrade" guidelines given in the maintenance documentation. -Roundup requires python 2.1.3 or later for correct operation. +Roundup requires python 2.3 or later for correct operation. To give Roundup a try, just download (see below), unpack and run:: - python demo.py + roundup-demo +Release info and download page: + http://cheeseshop.python.org/pypi/roundup Source and documentation is available at the website: http://roundup.sourceforge.net/ -Release Info (via download page): - http://sourceforge.net/projects/roundup Mailing lists - the place to ask questions: http://sourceforge.net/mail/?group_id=31577 @@ -71,11 +42,11 @@ Roundup manages a number of issues (with flexible properties such as The system will facilitate communication among the participants by managing discussions and notifying interested parties when issues are edited. One of the major design goals for Roundup that it be simple to get going. Roundup -is therefore usable "out of the box" with any python 2.1+ installation. It +is therefore usable "out of the box" with any python 2.3+ installation. It doesn't even need to be "installed" to be operational, though a disutils-based install script is provided. It comes with two issue tracker templates (a classic bug/feature tracker and -a minimal skeleton) and seven database back-ends (anydbm, bsddb, bsddb3, -sqlite, metakit, mysql and postgresql). +a minimal skeleton) and four database back-ends (anydbm, sqlite, mysql +and postgresql). diff --git a/doc/customizing.txt b/doc/customizing.txt index abc945c..5273e80 100644 --- a/doc/customizing.txt +++ b/doc/customizing.txt @@ -2,7 +2,7 @@ Customising Roundup =================== -:Version: $Revision: 1.134 $ +:Version: $Revision: 1.223 $ .. This document borrows from the ZopeBook section on ZPT. The original is at: http://www.zope.org/Documentation/Books/ZopeBook/current/ZPT.stx @@ -18,7 +18,7 @@ Before you get too far, it's probably worth having a quick read of the Roundup Customisation of Roundup can take one of six forms: -1. `tracker configuration`_ file changes +1. `tracker configuration`_ changes 2. database, or `tracker schema`_ changes 3. "definition" class `database content`_ changes 4. behavioural changes, through detectors_ @@ -39,290 +39,448 @@ Trackers have the following structure: =================== ======================================================== Tracker File Description =================== ======================================================== -config.py Holds the basic `tracker configuration`_ -dbinit.py Holds the `tracker schema`_ -interfaces.py Defines the Web and E-Mail interfaces for the tracker -select_db.py Selects the database back-end for the tracker +config.ini Holds the basic `tracker configuration`_ +schema.py Holds the `tracker schema`_ +initial_data.py Holds any data to be entered into the database when the + tracker is initialised. db/ Holds the tracker's database db/files/ Holds the tracker's upload files and messages +db/backend_name Names the database back-end for the tracker detectors/ Auditors and reactors for this tracker +extensions/ Additional web actions and templating utilities. html/ Web interface templates, images and style sheets +lib/ optional common imports for detectors and extensions =================== ======================================================== + Tracker Configuration ===================== -The ``config.py`` located in your tracker home contains the basic +The ``config.ini`` located in your tracker home contains the basic configuration for the web and e-mail components of roundup's interfaces. -As the name suggests, this file is a Python module. This means that any -valid python expression may be used in the file. Mostly though, you'll -be setting the configuration variables to string values. Python string -values must be quoted with either single or double quotes:: - - 'this is a string' - "this is also a string - use it when the value has 'single quotes'" - this is not a string - it's not quoted - -Python strings may use formatting that's almost identical to C string -formatting. The ``%`` operator is used to perform the formatting, like -so:: - - 'roundup-admin@%s'%MAIL_DOMAIN - -this will create a string ``'roundup-admin@tracker.domain.example'`` if -MAIL_DOMAIN is set to ``'tracker.domain.example'``. - -You'll also note some values are set to:: - - os.path.join(TRACKER_HOME, 'db') - -or similar. This creates a new string which holds the path to the -``'db'`` directory in the TRACKER_HOME directory. This is just a -convenience so if the TRACKER_HOME changes you don't have to edit -multiple valoues. -The configuration variables available are: - -**TRACKER_HOME** - ``os.path.split(__file__)[0]`` - The tracker home directory. The above default code will automatically - determine the tracker home for you, so you can just leave it alone. +Changes to the data captured by your tracker is controlled by the `tracker +schema`_. Some configuration is also performed using permissions - see the +`security / access controls`_ section. For example, to allow users to +automatically register through the email interface, you must grant the +"Anonymous" Role the "Email Access" Permission. + +The following is taken from the `Python Library Reference`__ (May 20, 2004) +section "ConfigParser -- Configuration file parser": + + The configuration file consists of sections, led by a "[section]" header + and followed by "name = value" entries, with line continuations on a + newline with leading whitespace. Note that leading whitespace is removed + from values. The optional values can contain format strings which + refer to other values in the same section. Lines beginning with "#" or ";" + are ignored and may be used to provide comments. + + For example:: + + [My Section] + foodir = %(dir)s/whatever + dir = frob + + would resolve the "%(dir)s" to the value of "dir" ("frob" in this case) + resulting in "foodir" being "frob/whatever". + +__ http://docs.python.org/lib/module-ConfigParser.html + +Section **main** + database -- ``db`` + Database directory path. The path may be either absolute or relative + to the directory containig this config file. + + templates -- ``html`` + Path to the HTML templates directory. The path may be either absolute + or relative to the directory containig this config file. + + static_files -- default *blank* + Path to directory holding additional static files available via Web + UI. This directory may contain sitewide images, CSS stylesheets etc. + and is searched for these files prior to the TEMPLATES directory + specified above. If this option is not set, all static files are + taken from the TEMPLATES directory The path may be either absolute or + relative to the directory containig this config file. + + admin_email -- ``roundup-admin`` + Email address that roundup will complain to if it runs into trouble. If + the email address doesn't contain an ``@`` part, the MAIL_DOMAIN defined + below is used. + + dispatcher_email -- ``roundup-admin`` + The 'dispatcher' is a role that can get notified of new items to the + database. It is used by the ERROR_MESSAGES_TO config setting. If the + email address doesn't contain an ``@`` part, the MAIL_DOMAIN defined + below is used. + + email_from_tag -- default *blank* + Additional text to include in the "name" part of the From: address used + in nosy messages. If the sending user is "Foo Bar", the From: line + is usually: ``"Foo Bar" `` + the EMAIL_FROM_TAG goes inside the "Foo Bar" quotes like so: + ``"Foo Bar EMAIL_FROM_TAG" `` + + new_web_user_roles -- ``User`` + Roles that a user gets when they register with Web User Interface. + This is a comma-separated list of role names (e.g. ``Admin,User``). + + new_email_user_roles -- ``User`` + Roles that a user gets when they register with Email Gateway. + This is a comma-separated string of role names (e.g. ``Admin,User``). + + error_messages_to -- ``user`` + Send error message emails to the ``dispatcher``, ``user``, or ``both``? + The dispatcher is configured using the DISPATCHER_EMAIL setting. + Allowed values: ``dispatcher``, ``user``, or ``both`` + + html_version -- ``html4`` + HTML version to generate. The templates are ``html4`` by default. + If you wish to make them xhtml, then you'll need to change this + var to ``xhtml`` too so all auto-generated HTML is compliant. + Allowed values: ``html4``, ``xhtml`` + + timezone -- ``0`` + Numeric timezone offset used when users do not choose their own + in their settings. + + instant_registration -- ``yes`` + Register new users instantly, or require confirmation via + email? + Allowed values: ``yes``, ``no`` + + email_registration_confirmation -- ``yes`` + Offer registration confirmation by email or only through the web? + Allowed values: ``yes``, ``no`` + + indexer_stopwords -- default *blank* + Additional stop-words for the full-text indexer specific to + your tracker. See the indexer source for the default list of + stop-words (e.g. ``A,AND,ARE,AS,AT,BE,BUT,BY, ...``). + + umask -- ``02`` + Defines the file creation mode mask. + +Section **tracker** + name -- ``Roundup issue tracker`` + A descriptive name for your roundup instance. + + web -- ``http://host.example/demo/`` + The web address that the tracker is viewable at. + This will be included in information sent to users of the tracker. + The URL MUST include the cgi-bin part or anything else + that is required to get to the home page of the tracker. + You MUST include a trailing '/' in the URL. + + email -- ``issue_tracker`` + Email address that mail to roundup should go to. + + language -- default *blank* + Default locale name for this tracker. If this option is not set, the + language is determined by the environment variable LANGUAGE, LC_ALL, + LC_MESSAGES, or LANG, in that order of preference. + +Section **web** + http_auth -- ``yes`` + Whether to use HTTP Basic Authentication, if present. + Roundup will use either the REMOTE_USER or HTTP_AUTHORIZATION + variables supplied by your web server (in that order). + Set this option to 'no' if you do not wish to use HTTP Basic + Authentication in your web interface. + + use_browser_language -- ``yes`` + Whether to use HTTP Accept-Language, if present. + Browsers send a language-region preference list. + It's usually set in the client's browser or in their + Operating System. + Set this option to 'no' if you want to ignore it. + + debug -- ``no`` + Setting this option makes Roundup display error tracebacks + in the user's browser rather than emailing them to the + tracker admin."), + +Section **rdbms** + Settings in this section are used by Postgresql and MySQL backends only + + name -- ``roundup`` + Name of the database to use. + + host -- ``localhost`` + Database server host. + + port -- default *blank* + TCP port number of the database server. Postgresql usually resides on + port 5432 (if any), for MySQL default port number is 3306. Leave this + option empty to use backend default. + + user -- ``roundup`` + Database user name that Roundup should use. + + password -- ``roundup`` + Database user password. + + read_default_file -- ``~/.my.cnf`` + Name of the MySQL defaults file. Only used in MySQL connections. + + read_default_group -- ``roundup`` + Name of the group to use in the MySQL defaults file. Only used in + MySQL connections. + +Section **logging** + config -- default *blank* + Path to configuration file for standard Python logging module. If this + option is set, logging configuration is loaded from specified file; + options 'filename' and 'level' in this section are ignored. The path may + be either absolute or relative to the directory containig this config file. + + filename -- default *blank* + Log file name for minimal logging facility built into Roundup. If no file + name specified, log messages are written on stderr. If above 'config' + option is set, this option has no effect. The path may be either absolute + or relative to the directory containig this config file. + + level -- ``ERROR`` + Minimal severity level of messages written to log file. If above 'config' + option is set, this option has no effect. + Allowed values: ``DEBUG``, ``INFO``, ``WARNING``, ``ERROR`` + +Section **mail** + Outgoing email options. Used for nosy messages, password reset and + registration approval requests. + + domain -- ``localhost`` + Domain name used for email addresses. + + host -- default *blank* + SMTP mail host that roundup will use to send mail + + username -- default *blank* + SMTP login name. Set this if your mail host requires authenticated access. + If username is not empty, password (below) MUST be set! + + password -- default *blank* + SMTP login password. + Set this if your mail host requires authenticated access. + + port -- default *25* + SMTP port on mail host. + Set this if your mail host runs on a different port. + + local_hostname -- default *blank* + The fully qualified domain name (FQDN) to use during SMTP sessions. If left + blank, the underlying SMTP library will attempt to detect your FQDN. If your + mail host requires something specific, specify the FQDN to use. + + tls -- ``no`` + If your SMTP mail host provides or requires TLS (Transport Layer Security) + then you may set this option to 'yes'. + Allowed values: ``yes``, ``no`` + + tls_keyfile -- default *blank* + If TLS is used, you may set this option to the name of a PEM formatted + file that contains your private key. The path may be either absolute or + relative to the directory containig this config file. + + tls_certfile -- default *blank* + If TLS is used, you may set this option to the name of a PEM formatted + certificate chain file. The path may be either absolute or relative + to the directory containig this config file. + + charset -- utf-8 + Character set to encode email headers with. We use utf-8 by default, as + it's the most flexible. Some mail readers (eg. Eudora) can't cope with + that, so you might need to specify a more limited character set + (eg. iso-8859-1). + + debug -- default *blank* + Setting this option makes Roundup to write all outgoing email messages + to this file *instead* of sending them. This option has the same effect + as environment variable SENDMAILDEBUG. Environment variable takes + precedence. The path may be either absolute or relative to the directory + containig this config file. + + add_authorinfo -- ``yes`` + Add a line with author information at top of all messages send by + roundup. + + add_authoremail -- ``yes`` + Add the mail address of the author to the author information at the + top of all messages. If this is false but add_authorinfo is true, + only the name of the actor is added which protects the mail address + of the actor from being exposed at mail archives, etc. + +Section **mailgw** + Roundup Mail Gateway options + + keep_quoted_text -- ``yes`` + Keep email citations when accepting messages. Setting this to ``no`` strips + out "quoted" text from the message. Signatures are also stripped. + Allowed values: ``yes``, ``no`` + + leave_body_unchanged -- ``no`` + Preserve the email body as is - that is, keep the citations *and* + signatures. + Allowed values: ``yes``, ``no`` + + default_class -- ``issue`` + Default class to use in the mailgw if one isn't supplied in email subjects. + To disable, leave the value blank. + + language -- default *blank* + Default locale name for the tracker mail gateway. If this option is + not set, mail gateway will use the language of the tracker instance. + + subject_prefix_parsing -- ``strict`` + Controls the parsing of the [prefix] on subject lines in incoming emails. + ``strict`` will return an error to the sender if the [prefix] is not + recognised. ``loose`` will attempt to parse the [prefix] but just + pass it through as part of the issue title if not recognised. ``none`` + will always pass any [prefix] through as part of the issue title. + + subject_suffix_parsing -- ``strict`` + Controls the parsing of the [suffix] on subject lines in incoming emails. + ``strict`` will return an error to the sender if the [suffix] is not + recognised. ``loose`` will attempt to parse the [suffix] but just + pass it through as part of the issue title if not recognised. ``none`` + will always pass any [suffix] through as part of the issue title. + + subject_suffix_delimiters -- ``[]`` + Defines the brackets used for delimiting the commands suffix in a subject + line. + + subject_content_match -- ``always`` + Controls matching of the incoming email subject line against issue titles + in the case where there is no designator [prefix]. ``never`` turns off + matching. ``creation + interval`` or ``activity + interval`` will match + an issue for the interval after the issue's creation or last activity. + The interval is a standard Roundup interval. + + refwd_re -- ``(\s*\W?\s*(fw|fwd|re|aw|sv|ang)\W)+`` + Regular expression matching a single reply or forward prefix + prepended by the mailer. This is explicitly stripped from the + subject during parsing. Value is Python Regular Expression + (UTF8-encoded). + + origmsg_re -- `` ^[>|\s]*-----\s?Original Message\s?-----$`` + Regular expression matching start of an original message if quoted + the in body. Value is Python Regular Expression (UTF8-encoded). + + sign_re -- ``^[>|\s]*-- ?$`` + Regular expression matching the start of a signature in the message + body. Value is Python Regular Expression (UTF8-encoded). + + eol_re -- ``[\r\n]+`` + Regular expression matching end of line. Value is Python Regular + Expression (UTF8-encoded). + + blankline_re -- ``[\r\n]+\s*[\r\n]+`` + Regular expression matching a blank line. Value is Python Regular + Expression (UTF8-encoded). + +Section **pgp** + OpenPGP mail processing options + + enable -- ``no`` + Enable PGP processing. Requires pyme. + + roles -- default *blank* + If specified, a comma-separated list of roles to perform PGP + processing on. If not specified, it happens for all users. + + homedir -- default *blank* + Location of PGP directory. Defaults to $HOME/.gnupg if not + specified. + +Section **nosy** + Nosy messages sending + + messages_to_author -- ``no`` + Send nosy messages to the author of the message. + Allowed values: ``yes``, ``no``, ``new`` + + signature_position -- ``bottom`` + Where to place the email signature. + Allowed values: ``top``, ``bottom``, ``none`` + + add_author -- ``new`` + Does the author of a message get placed on the nosy list automatically? + If ``new`` is used, then the author will only be added when a message + creates a new issue. If ``yes``, then the author will be added on + followups too. If ``no``, they're never added to the nosy. + Allowed values: ``yes``, ``no``, ``new`` + + add_recipients -- ``new`` + Do the recipients (``To:``, ``Cc:``) of a message get placed on the nosy + list? If ``new`` is used, then the recipients will only be added when a + message creates a new issue. If ``yes``, then the recipients will be added + on followups too. If ``no``, they're never added to the nosy. + Allowed values: ``yes``, ``no``, ``new`` -**MAILHOST** - ``'localhost'`` - The SMTP mail host that roundup will use to send e-mail. + email_sending -- ``single`` + Controls the email sending from the nosy reactor. If ``multiple`` then + a separate email is sent to each recipient. If ``single`` then a single + email is sent with each recipient as a CC address. -**MAILUSER** - ``()`` - If your SMTP mail host requires a username and password for access, then - specify them here. eg. ``MAILUSER = ('username', 'password')`` + max_attachment_size -- ``2147483647`` + Attachments larger than the given number of bytes won't be attached + to nosy mails. They will be replaced by a link to the tracker's + download page for the file. -**MAILHOST_TLS** - ``'no'`` - If your SMTP mail host provides or requires TLS (Transport Layer - Security) then set ``MAILHOST_TLS = 'yes'`` -**MAILHOST_TLS_KEYFILE** - ``''`` - If you're using TLS, you may also set MAILHOST_TLS_KEYFILE to the name of - a PEM formatted file that contains your private key. - -**MAILHOST_TLS_CERTFILE** - ``''`` - If you're using TLS and have specified a MAILHOST_TLS_KEYFILE, you may - also set MAILHOST_TLS_CERTFILE to the name of a PEM formatted certificate - chain file. - -**MAIL_DOMAIN** - ``'tracker.domain.example'`` - The domain name used for email addresses. - -**DATABASE** - ``os.path.join(TRACKER_HOME, 'db')`` - This is the directory that the database is going to be stored in. By default - it is in the tracker home. - -**TEMPLATES** - ``os.path.join(TRACKER_HOME, 'html')`` - This is the directory that the HTML templates reside in. By default they are - in the tracker home. +You may generate a new default config file using the ``roundup-admin +genconfig`` command. -**TRACKER_NAME** - ``'Roundup issue tracker'`` - A descriptive name for your roundup tracker. This is sent out in e-mails and - appears in the heading of CGI pages. +Configuration variables may be referred to in lower or upper case. In code, +variables not in the "main" section are referred to using their section and +name, so "domain" in the section "mail" becomes MAIL_DOMAIN. The +configuration variables available are: -**TRACKER_EMAIL** - ``'issue_tracker@%s'%MAIL_DOMAIN`` - The email address that e-mail sent to roundup should go to. Think of it as the - tracker's personal e-mail address. +Extending the configuration file +-------------------------------- -**TRACKER_WEB** - ``'http://tracker.example/cgi-bin/roundup.cgi/bugs/'`` - The web address that the tracker is viewable at. This will be included in - information sent to users of the tracker. The URL **must** include the - cgi-bin part or anything else that is required to get to the home page of - the tracker. You **must** include a trailing '/' in the URL. +You can't add new variables to the config.ini file in the tracker home but +you can add two new config.ini files: -**ADMIN_EMAIL** - ``'roundup-admin@%s'%MAIL_DOMAIN`` - The email address that roundup will complain to if it runs into trouble. +- a config.ini in the ``extensions`` directory will be loaded and attached + to the config variable as "ext". +- a config.ini in the ``detectors`` directory will be loaded and attached + to the config variable as "detectors". -**EMAIL_FROM_TAG** - ``''`` - Additional text to include in the "name" part of the ``From:`` address used - in nosy messages. If the sending user is "Foo Bar", the ``From:`` line is - usually:: +For example, the following in ``detectors/config.ini``:: - "Foo Bar" + [main] + qa_recipients = email@example.com - The EMAIL_FROM_TAG goes inside the "Foo Bar" quotes like so:: +is accessible as:: - "Foo Bar EMAIL_FROM_TAG" + db.config.detectors['QA_RECIPIENTS'] -**ERROR_MESSAGES_TO** - ``'user'``, ``'dispatcher'`` or ``'both'`` - Sends error messages to the dispatcher, user, or both. It will use the - ``DISPATCHER_EMAIL`` address if set, otherwise it'll use the - ``ADMIN_EMAIL`` address. +Note that the name grouping applied to the main configuration file is +applied to the extension config files, so if you instead have:: -**DISPATCHER_EMAIL** - ``''`` - The email address that Roundup will issue all error messages to. This is - also useful if you want to switch your 'new message' notification to - a central user. + [qa] + recipients = email@example.com -**MESSAGES_TO_AUTHOR** - ``'new'``, ``'yes'`` or``'no'`` - Send nosy messages to the author of the message? - If 'new' is used, then the author will only be sent the message when the - message creates a new issue. If 'yes' then the author will always be sent - a copy of the message they wrote. +then the above ``db.config.detectors['QA_RECIPIENTS']`` will still work. -**ADD_AUTHOR_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'`` - Does the author of a message get placed on the nosy list automatically? - If ``'new'`` is used, then the author will only be added when a message - creates a new issue. If ``'yes'``, then the author will be added on followups - too. If ``'no'``, they're never added to the nosy. - -**ADD_RECIPIENTS_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'`` - Do the recipients (To:, Cc:) of a message get placed on the nosy list? - If ``'new'`` is used, then the recipients will only be added when a message - creates a new issue. If ``'yes'``, then the recipients will be added on - followups too. If ``'no'``, they're never added to the nosy. - -**EMAIL_SIGNATURE_POSITION** - ``'top'``, ``'bottom'`` or ``'none'`` - Where to place the email signature in messages that Roundup generates. - -**EMAIL_KEEP_QUOTED_TEXT** - ``'yes'`` or ``'no'`` - Keep email citations. Citations are the part of e-mail which the sender has - quoted in their reply to previous e-mail with ``>`` or ``|`` characters at - the start of the line. - -**EMAIL_LEAVE_BODY_UNCHANGED** - ``'no'`` - Preserve the email body as is. Enabiling this will cause the entire message - body to be stored, including all citations, signatures and Outlook-quoted - sections (ie. "Original Message" blocks). It should be either ``'yes'`` - or ``'no'``. - -**MAIL_DEFAULT_CLASS** - ``'issue'`` or ``''`` - Default class to use in the mailgw if one isn't supplied in email - subjects. To disable, comment out the variable below or leave it blank. - -**HTML_VERSION** - ``'html4'`` or ``'xhtml'`` - HTML version to generate. The templates are html4 by default. If you - wish to make them xhtml, then you'll need to change this var to 'xhtml' - too so all auto-generated HTML is compliant. - -**EMAIL_CHARSET** - ``utf-8`` (or ``iso-8859-1`` for Eudora users) - Character set to encode email headers with. We use utf-8 by default, as - it's the most flexible. Some mail readers (eg. Eudora) can't cope with - that, so you might need to specify a more limited character set (eg. - 'iso-8859-1'. - -**DEFAULT_TIMEZONE** - ``0`` - Numeric hour timezone offest to be used when displaying local times. - The default timezone is used when users do not choose their own in - their settings. - -The default config.py is given below - as you -can see, the MAIL_DOMAIN must be edited before any interaction with the -tracker is attempted.:: - - # roundup home is this package's directory - TRACKER_HOME=os.path.split(__file__)[0] - - # The SMTP mail host that roundup will use to send mail - MAILHOST = 'localhost' - - # The domain name used for email addresses. - MAIL_DOMAIN = 'your.tracker.email.domain.example' - - # This is the directory that the database is going to be stored in - DATABASE = os.path.join(TRACKER_HOME, 'db') - - # This is the directory that the HTML templates reside in - TEMPLATES = os.path.join(TRACKER_HOME, 'html') - - # A descriptive name for your roundup tracker - TRACKER_NAME = 'Roundup issue tracker' - - # The email address that mail to roundup should go to - TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN - - # The web address that the tracker is viewable at. This will be - # included in information sent to users of the tracker. The URL MUST - # include the cgi-bin part or anything else that is required to get - # to the home page of the tracker. You MUST include a trailing '/' - # in the URL. - TRACKER_WEB = 'http://tracker.example/cgi-bin/roundup.cgi/bugs/' - - # The email address that roundup will complain to if it runs into - # trouble - ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN - - # Additional text to include in the "name" part of the From: address - # used in nosy messages. If the sending user is "Foo Bar", the From: - # line is usually: "Foo Bar" - # the EMAIL_FROM_TAG goes inside the "Foo Bar" quotes like so: - # "Foo Bar EMAIL_FROM_TAG" - EMAIL_FROM_TAG = "" - - # Send nosy messages to the author of the message - MESSAGES_TO_AUTHOR = 'no' # either 'yes' or 'no' - - # Does the author of a message get placed on the nosy list - # automatically? If 'new' is used, then the author will only be - # added when a message creates a new issue. If 'yes', then the - # author will be added on followups too. If 'no', they're never - # added to the nosy. - ADD_AUTHOR_TO_NOSY = 'new' # one of 'yes', 'no', 'new' - - # Do the recipients (To:, Cc:) of a message get placed on the nosy - # list? If 'new' is used, then the recipients will only be added - # when a message creates a new issue. If 'yes', then the recipients - # will be added on followups too. If 'no', they're never added to - # the nosy. - ADD_RECIPIENTS_TO_NOSY = 'new' # either 'yes', 'no', 'new' - - # Where to place the email signature - EMAIL_SIGNATURE_POSITION = 'bottom' # one of 'top', 'bottom', 'none' - - # Keep email citations - EMAIL_KEEP_QUOTED_TEXT = 'no' # either 'yes' or 'no' - - # Preserve the email body as is - EMAIL_LEAVE_BODY_UNCHANGED = 'no' # either 'yes' or 'no' - - # Default class to use in the mailgw if one isn't supplied in email - # subjects. To disable, comment out the variable below or leave it - # blank. Examples: - MAIL_DEFAULT_CLASS = 'issue' # use "issue" class by default - #MAIL_DEFAULT_CLASS = '' # disable (or just comment the var out) - - # HTML version to generate. The templates are html4 by default. If you - # wish to make them xhtml, then you'll need to change this var to 'xhtml' - # too so all auto-generated HTML is compliant. - HTML_VERSION = 'html4' # either 'html4' or 'xhtml' - - # Character set to encode email headers with. We use utf-8 by default, as - # it's the most flexible. Some mail readers (eg. Eudora) can't cope with - # that, so you might need to specify a more limited character set (eg. - # 'iso-8859-1'. - EMAIL_CHARSET = 'utf-8' - #EMAIL_CHARSET = 'iso-8859-1' # use this instead for Eudora users - - # You may specify a different default timezone, for use when users do not - # choose their own in their settings. - DEFAULT_TIMEZONE = 0 # specify as numeric hour offest - - # - # SECURITY DEFINITIONS - # - # define the Roles that a user gets when they register with the - # tracker these are a comma-separated string of role names (e.g. - # 'Admin,User') - NEW_WEB_USER_ROLES = 'User' - NEW_EMAIL_USER_ROLES = 'User' Tracker Schema ============== -Note: if you modify the schema, you'll most likely need to edit the - `web interface`_ HTML template files and `detectors`_ to reflect - your changes. +.. note:: + if you modify the schema, you'll most likely need to edit the + `web interface`_ HTML template files and `detectors`_ to reflect + your changes. A tracker schema defines what data is stored in the tracker's database. -Schemas are defined using Python code in the ``dbinit.py`` module of your +Schemas are defined using Python code in the ``schema.py`` module of your tracker. -The ``dbinit.py`` module +The ``schema.py`` module ------------------------ -The ``dbinit.py`` module contains two functions: +The ``schema.py`` module contains two functions: **open** This function defines what your tracker looks like on the inside, the @@ -340,8 +498,10 @@ The ``dbinit.py`` module contains two functions: The "classic" schema -------------------- -The "classic" schema looks like this (see below for the meaning -of ``'setkey'``):: +The "classic" schema looks like this (see section `setkey(property)`_ +below for the meaning of ``'setkey'`` -- you may also want to look into +the sections `setlabelprop(property)`_ and `setorderprop(property)`_ for +specifying (default) labelling and ordering of classes.):: pri = Class(db, "priority", name=String(), order=String()) pri.setkey("name") @@ -354,16 +514,17 @@ of ``'setkey'``):: user = Class(db, "user", username=String(), organisation=String(), password=String(), address=String(), realname=String(), - phone=String()) + phone=String(), alternate_addresses=String(), + queries=Multilink('query'), roles=String(), timezone=String()) user.setkey("username") msg = FileClass(db, "msg", author=Link("user"), summary=String(), date=Date(), recipients=Multilink("user"), - files=Multilink("file")) + files=Multilink("file"), messageid=String(), inreplyto=String()) - file = FileClass(db, "file", name=String(), type=String()) + file = FileClass(db, "file", name=String()) - issue = IssueClass(db, "issue", topic=Multilink("keyword"), + issue = IssueClass(db, "issue", keyword=Multilink("keyword"), status=Link("status"), assignedto=Link("user"), priority=Link("priority")) issue.setkey('title') @@ -375,8 +536,10 @@ What you can't do to the schema You must never: **Remove the users class** - This class is the only *required* class in Roundup. Similarly, its - username, password and address properties must never be removed. + This class is the only *required* class in Roundup. + +**Remove the "username", "address", "password" or "realname" user properties** + Various parts of Roundup require these properties. Don't remove them. **Change the type of a property** Property types must *never* be changed - the database simply doesn't take @@ -470,6 +633,17 @@ A Class is comprised of one or more properties of the following types: * A Multilink property refers to possibly many items in a specified class. The value is a list of integers. +All Classes automatically have a number of properties by default: + +*creator* + Link to the user that created the item. +*creation* + Date the item was created. +*actor* + Link to the user that last modified the item. +*activity* + Date the item was last modified. + FileClass ~~~~~~~~~ @@ -479,7 +653,8 @@ the rest of the database. This reduces the number of large entries in the database, which generally makes databases more efficient, and also allows us to use command-line tools to operate on the files. They are stored in the files sub-directory of the ``'db'`` directory in your -tracker. +tracker. FileClasses also have a "type" attribute to store the MIME +type of the file. IssueClass @@ -523,12 +698,37 @@ or:: Note, the same thing can be done in the web and e-mail interfaces. -If a class does not have an "order" property, the key is also used to -sort instances of the class when it is rendered in the user interface. -(If a class has no "order" property, sorting is by the labelproperty of -the class. This is computed, in order of precedence, as the key, the -"name", the "title", or the first property alphabetically.) +setlabelprop(property) +~~~~~~~~~~~~~~~~~~~~~~ + +Select a property of the class to be the label property. The label +property is used whereever an item should be uniquely identified, e.g., +when displaying a link to an item. If setlabelprop is not specified for +a class, the following values are tried for the label: + + * the key of the class (see the `setkey(property)`_ section above) + * the "name" property + * the "title" property + * the first property from the sorted property name list + +So in most cases you can get away without specifying setlabelprop +explicitly. + +setorderprop(property) +~~~~~~~~~~~~~~~~~~~~~~ + +Select a property of the class to be the order property. The order +property is used whenever using a default sort order for the class, +e.g., when grouping or sorting class A by a link to class B in the user +interface, the order property of class B is used for sorting. If +setorderprop is not specified for a class, the following values are tried +for the order property: + * the property named "order" + * the label property (see `setlabelprop(property)`_ above) + +So in most cases you can get away without specifying setorderprop +explicitly. create(information) ~~~~~~~~~~~~~~~~~~~ @@ -537,10 +737,30 @@ Create an item in the database. This is generally used to create items in the "definitional" classes like "priority" and "status". +A note about ordering +~~~~~~~~~~~~~~~~~~~~~ + +When we sort items in the hyperdb, we use one of a number of methods, +depending on the properties being sorted on: + +1. If it's a String, Number, Date or Interval property, we just sort the + scalar value of the property. Strings are sorted case-sensitively. +2. If it's a Link property, we sort by either the linked item's "order" + property (if it has one) or the linked item's "id". +3. Mulitlinks sort similar to #2, but we start with the first Multilink + list item, and if they're the same, we sort by the second item, and + so on. + +Note that if an "order" property is defined on a Class that is used for +sorting, all items of that Class *must* have a value against the "order" +property, or sorting will result in random ordering. + + Examples of adding to your schema --------------------------------- -TODO +The Roundup wiki has examples of how schemas can be customised to add +new functionality. Detectors - adding behaviour to your tracker @@ -549,7 +769,7 @@ Detectors - adding behaviour to your tracker Detectors are initialised every time you open your tracker database, so you're free to add and remove them any time, even after the database is -initialised via the "roundup-admin initialise" command. +initialised via the ``roundup-admin initialise`` command. The detectors in your tracker fire *before* (**auditors**) and *after* (**reactors**) changes to the contents of your database. They are Python @@ -691,15 +911,16 @@ to generate email messages from Roundup. In addition, the ``IssueClass`` methods ``nosymessage()`` and ``send_message()`` are used to generate nosy messages, and may generate messages which only consist of a change note (ie. the message id parameter -is not required). +is not required - this is referred to as a "System Message" because it +comes from "the system" and not a user). Database Content ================ -Note: if you modify the content of definitional classes, you'll most - likely need to edit the tracker `detectors`_ to reflect your - changes. +.. note:: + If you modify the content of definitional classes, you'll most + likely need to edit the tracker `detectors`_ to reflect your changes. Customisation of the special "definitional" classes (eg. status, priority, resolution, ...) may be done either before or after the @@ -707,8 +928,8 @@ tracker is initialised. The actual method of doing so is completely different in each case though, so be careful to use the right one. **Changing content before tracker initialisation** - Edit the dbinit module in your tracker to alter the items created in - using the ``create()`` methods. + Edit the initial_data.py module in your tracker to alter the items + created using the ``create( ... )`` methods. **Changing content after tracker initialisation** As the "admin" user, click on the "class list" link in the web @@ -728,62 +949,144 @@ Security / Access Controls A set of Permissions is built into the security module by default: +- Create (everything) - Edit (everything) - View (everything) -Every Class you define in your tracker's schema also gets an Edit and View -Permission of its own. - -The default interfaces define: - -- Web Registration -- Web Access -- Web Roles -- Email Registration -- Email Access +These are assigned to the "Admin" Role by default, and allow a user to do +anything. Every Class you define in your `tracker schema`_ also gets an +Create, Edit and View Permission of its own. The web and email interfaces +also define: + +*Email Access* + If defined, the user may use the email interface. Used by default to deny + Anonymous users access to the email interface. When granted to the + Anonymous user, they will be automatically registered by the email + interface (see also the ``new_email_user_roles`` configuration option). +*Web Access* + If defined, the user may use the web interface. All users are able to see + the login form, regardless of this setting (thus enabling logging in). +*Web Roles* + Controls user access to editing the "roles" property of the "user" class. + TODO: deprecate in favour of a property-based control. These are hooked into the default Roles: -- Admin (Edit everything, View everything, Web Roles) -- User (Web Access, Email Access) -- Anonymous (Web Registration, Email Registration) +- Admin (Create, Edit, View and everything; Web Roles) +- User (Web Access; Email Access) +- Anonymous (Web Access) And finally, the "admin" user gets the "Admin" Role, and the "anonymous" -user gets "Anonymous" assigned when the database is initialised on -installation. The two default schemas then define: +user gets "Anonymous" assigned when the tracker is installed. + +For the "User" Role, the "classic" tracker defines: -- Edit issue, View issue (both) -- Edit file, View file (both) -- Edit msg, View msg (both) -- Edit support, View support (extended only) +- Create, Edit and View issue, file, msg, query, keyword +- View priority, status +- View user +- Edit their own user record -and assign those Permissions to the "User" Role. Put together, these -settings appear in the ``open()`` function of the tracker ``dbinit.py`` -(the following is taken from the "minimal" template's ``dbinit.py``):: +And the "Anonymous" Role is defined as: + +- Web interface access +- Create user (for registration) +- View issue, file, msg, query, keyword, priority, status + +Put together, these settings appear in the tracker's ``schema.py`` file:: # - # SECURITY SETTINGS + # TRACKER SECURITY SETTINGS # - # and give the regular users access to the web and email interface - p = db.security.getPermission('Web Access') - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Email Access') - db.security.addPermissionToRole('User', p) + # See the configuration and customisation document for information + # about security setup. + + # + # REGULAR USERS + # + # Give the regular users access to the web and email interface + db.security.addPermissionToRole('User', 'Web Access') + db.security.addPermissionToRole('User', 'Email Access') + + # Assign the access and edit Permissions for issue, file and message + # to regular users now + for cl in 'issue', 'file', 'msg', 'query', 'keyword': + db.security.addPermissionToRole('User', 'View', cl) + db.security.addPermissionToRole('User', 'Edit', cl) + db.security.addPermissionToRole('User', 'Create', cl) + for cl in 'priority', 'status': + db.security.addPermissionToRole('User', 'View', cl) # May users view other user information? Comment these lines out # if you don't want them to - p = db.security.getPermission('View', 'user') + db.security.addPermissionToRole('User', 'View', 'user') + + # Users should be able to edit their own details -- this permission + # is limited to only the situation where the Viewed or Edited item + # is their own. + def own_record(db, userid, itemid): + '''Determine whether the userid matches the item being accessed.''' + return userid == itemid + p = db.security.addPermission(name='View', klass='user', check=own_record, + description="User is allowed to view their own user details") + db.security.addPermissionToRole('User', p) + p = db.security.addPermission(name='Edit', klass='user', check=own_record, + description="User is allowed to edit their own user details") db.security.addPermissionToRole('User', p) - # Assign the appropriate permissions to the anonymous user's - # Anonymous role. Choices here are: - # - Allow anonymous users to register through the web - p = db.security.getPermission('Web Registration') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous (new) users to register through the email - # gateway - p = db.security.getPermission('Email Registration') - db.security.addPermissionToRole('Anonymous', p) + # + # ANONYMOUS USER PERMISSIONS + # + # Let anonymous users access the web interface. Note that almost all + # trackers will need this Permission. The only situation where it's not + # required is in a tracker that uses an HTTP Basic Authenticated front-end. + db.security.addPermissionToRole('Anonymous', 'Web Access') + + # Let anonymous users access the email interface (note that this implies + # that they will be registered automatically, hence they will need the + # "Create" user Permission below) + # This is disabled by default to stop spam from auto-registering users on + # public trackers. + #db.security.addPermissionToRole('Anonymous', 'Email Access') + + # Assign the appropriate permissions to the anonymous user's Anonymous + # Role. Choices here are: + # - Allow anonymous users to register + db.security.addPermissionToRole('Anonymous', 'Create', 'user') + + # Allow anonymous users access to view issues (and the related, linked + # information) + for cl in 'issue', 'file', 'msg', 'keyword', 'priority', 'status': + db.security.addPermissionToRole('Anonymous', 'View', cl) + + # [OPTIONAL] + # Allow anonymous users access to create or edit "issue" items (and the + # related file and message items) + #for cl in 'issue', 'file', 'msg': + # db.security.addPermissionToRole('Anonymous', 'Create', cl) + # db.security.addPermissionToRole('Anonymous', 'Edit', cl) + + +Automatic Permission Checks +--------------------------- + +Permissions are automatically checked when information is rendered +through the web. This includes: + +1. View checks for properties when being rendered via the ``plain()`` or + similar methods. If the check fails, the text "[hidden]" will be + displayed. +2. Edit checks for properties when the edit field is being rendered via + the ``field()`` or similar methods. If the check fails, the property + will be rendered via the ``plain()`` method (see point 1. for subsequent + checking performed) +3. View checks are performed in index pages for each item being displayed + such that if the user does not have permission, the row is not rendered. +4. View checks are performed at the top of item pages for the Item being + displayed. If the user does not have permission, the text "You are not + allowed to view this page." will be displayed. +5. View checks are performed at the top of index pages for the Class being + displayed. If the user does not have permission, the text "You are not + allowed to view this page." will be displayed. New User Roles @@ -794,6 +1097,9 @@ New users are assigned the Roles defined in the config file as: - NEW_WEB_USER_ROLES - NEW_EMAIL_USER_ROLES +The `users may only edit their issues`_ example shows customisation of +these parameters. + Changing Access Controls ------------------------ @@ -811,7 +1117,7 @@ Adding a new Permission When adding a new Permission, you will need to: -1. add it to your tracker's dbinit so it is created, using +1. add it to your tracker's ``schema.py`` so it is created, using ``security.addPermission``, for example:: self.security.addPermission(name="View", klass='frozzle', @@ -824,26 +1130,44 @@ When adding a new Permission, you will need to: 4. add it to the appropriate xxxPermission methods on in your tracker interfaces module +The ``addPermission`` method takes a couple of optional parameters: + +**properties** + A sequence of property names that are the only properties to apply the + new Permission to (eg. ``... klass='user', properties=('name', + 'email') ...``) +**check** + A function to be execute which returns boolean determining whether the + Permission is allowed. The function has the signature ``check(db, userid, + itemid)`` where ``db`` is a handle on the open database, ``userid`` is + the user attempting access and ``itemid`` is the specific item being + accessed. Example Scenarios ~~~~~~~~~~~~~~~~~ -**automatic registration of users in the e-mail gateway** - By giving the "anonymous" user the "Email Registration" Role, any - unidentified user will automatically be registered with the tracker - (with no password, so they won't be able to log in through the web - until an admin sets their password). Note: this is the default - behaviour in the tracker templates that ship with Roundup. +See the `examples`_ section for longer examples of customisation. **anonymous access through the e-mail gateway** - Give the "anonymous" user the "Email Access" and ("Edit", "issue") - Roles but do not not give them the "Email Registration" Role. This - means that when an unknown user sends email into the tracker, they're - automatically logged in as "anonymous". Since they don't have the - "Email Registration" Role, they won't be automatically registered, but - since "anonymous" has permission to use the gateway, they'll still be - able to submit issues. Note that the Sender information - their email - address - will not be available - they're *anonymous*. + Give the "anonymous" user the "Email Access", ("Edit", "issue") and + ("Create", "msg") Permissions but do not not give them the ("Create", + "user") Permission. This means that when an unknown user sends email + into the tracker, they're automatically logged in as "anonymous". + Since they don't have the ("Create", "user") Permission, they won't + be automatically registered, but since "anonymous" has permission to + use the gateway, they'll still be able to submit issues. Note that + the Sender information - their email address - will not be available + - they're *anonymous*. + +**automatic registration of users in the e-mail gateway** + By giving the "anonymous" user the ("Create", "user") Permission, any + unidentified user will automatically be registered with the tracker + (with no password, so they won't be able to log in through + the web until an admin sets their password). By default new Roundup + trackers don't allow this as it opens them up to spam. It may be enabled + by uncommenting the appropriate addPermissionToRole in your tracker's + ``schema.py`` file. The new user is given the Roles list defined in the + "new_email_user_roles" config variable. **only developers may be assigned issues** Create a new Permission called "Fixer" for the "issue" class. Create a @@ -892,10 +1216,11 @@ is used by ``roundup.cgi``, ``roundup-server`` and ``ZRoundup`` (``ZRoundup`` is broken, until further notice). In all cases, we determine which tracker is being accessed (the first part of the URL path inside the scope of the CGI handler) and pass control on to the -tracker ``interfaces.Client`` class - which uses the ``Client`` class -from ``roundup.cgi.client`` - which handles the rest of the access -through its ``main()`` method. This means that you can do pretty much -anything you want as a web interface to your tracker. +``roundup.cgi.client.Client`` class - which handles the rest of the +access through its ``main()`` method. This means that you can do pretty +much anything you want as a web interface to your tracker. + + Repercussions of changing the tracker schema --------------------------------------------- @@ -908,6 +1233,7 @@ the web interface knows about it: 2. The "page" template may require links to be changed, as might the "home" page's content arguments. + How requests are processed -------------------------- @@ -934,6 +1260,7 @@ In some situations, exceptions occur: this exception percolates up to the CGI interface that called the client + Determining web context ----------------------- @@ -943,20 +1270,23 @@ identifier is examined. Typical URL paths look like: 1. ``/tracker/issue`` 2. ``/tracker/issue1`` -3. ``/tracker/@file/style.css`` +3. ``/tracker/@@file/style.css`` 4. ``/cgi-bin/roundup.cgi/tracker/file1`` 5. ``/cgi-bin/roundup.cgi/tracker/file1/kitten.png`` where the "tracker identifier" is "tracker" in the above cases. That means -we're looking at "issue", "issue1", "@file/style.css", "file1" and +we're looking at "issue", "issue1", "@@file/style.css", "file1" and "file1/kitten.png" in the cases above. The path is generally only one entry long - longer paths are handled differently. -a. if there is no path, then we are in the "home" context. -b. if the path starts with "@file" (as in example 3, - "/tracker/@file/style.css"), then the additional path entry, +a. if there is no path, then we are in the "home" context. See `the "home" + context`_ below for more information about how it may be used. +b. if the path starts with "@@file" (as in example 3, + "/tracker/@@file/style.css"), then the additional path entry, "style.css" specifies the filename of a static file we're to serve up - from the tracker "html" directory. Raises a SendStaticFile exception. + from the tracker TEMPLATES (or STATIC_FILES, if configured) directory. + This is usually the tracker's "html" directory. Raises a SendStaticFile + exception. c. if there is something in the path (as in example 1, "issue"), it identifies the tracker class we're to display. d. if the path is an item designator (as in examples 2 and 4, "issue1" @@ -978,11 +1308,30 @@ defaults to: - full item designator supplied: "item" +The "home" Context +------------------ + +The "home" context is special because it allows you to add templated +pages to your tracker that don't rely on a class or item (ie. an issues +list or specific issue). + +Let's say you wish to add frames to control the layout of your tracker's +interface. You'd probably have: + +- A top-level frameset page. This page probably wouldn't be templated, so + it could be served as a static file (see `serving static content`_) +- A sidebar frame that is templated. Let's call this page + "home.navigation.html" in your tracker's "html" directory. To load that + page up, you use the URL: + + /home?@template=navigation + + Serving static content ---------------------- See the previous section `determining web context`_ where it describes -``@file`` paths. +``@@file`` paths. Performing actions in web requests @@ -1033,40 +1382,33 @@ of: Each of the actions is implemented by a corresponding ``*XxxAction*`` (where "Xxx" is the name of the action) class in the ``roundup.cgi.actions`` module. -These classes are registered with ``roundup.cgi.client.Client`` which also -happens to be available in your tracker instance as ``interfaces.Client``. So -if you need to define new actions, you may add them there (see `defining new +These classes are registered with ``roundup.cgi.client.Client``. If you need +to define new actions, you may add them there (see `defining new web actions`_). Each action class also has a ``*permission*`` method which determines whether the action is permissible given the current user. The base permission checks -are: +for each action are: **login** - Determine whether the user has permission to log in. Base behaviour is - to check the user has "Web Access". + Determine whether the user has the "Web Access" Permission. **logout** No permission checks are made. **register** - Determine whether the user has permission to register. Base behaviour - is to check the user has the "Web Registration" Permission. + Determine whether the user has the ("Create", "user") Permission. **edit** - Determine whether the user has permission to edit this item. Base - behaviour is to check whether the user can edit this class. If we're + Determine whether the user has permission to edit this item. If we're editing the "user" class, users are allowed to edit their own details - unless they try to edit the "roles" property, which requires the special Permission "Web Roles". **new** - Determine whether the user has permission to create (or edit) this - item. Base behaviour is to check the user can edit this class. No + Determine whether the user has permission to create this item. No additional property checks are made. Additionally, new user items may - be created if the user has the "Web Registration" Permission. + be created if the user has the ("Create", "user") Permission. **editCSV** - Determine whether the user has permission to edit this class. Base - behaviour is to check whether the user may edit this class. + Determine whether the user has permission to edit this class. **search** - Determine whether the user has permission to search this class. Base - behaviour is to check whether the user may view this class. + Determine whether the user has permission to view this class. Special form variables @@ -1083,10 +1425,32 @@ variables and their values. You can: - Remove items from a multilink property of the current item. - Specify that some properties are required for the edit operation to be successful. +- Set up user interface locale. + +These operations will only take place if the form action (the +``@action`` variable) is "edit" or "new". In the following, values are variable, "@" may be either ":" or "@", and other text "required" is fixed. +Two special form variables are used to specify user language preferences: + +``@language`` + value may be locale name or ``none``. If this variable is set to + locale name, web interface language is changed to given value + (provided that appropriate translation is available), the value + is stored in the browser cookie and will be used for all following + requests. If value is ``none`` the cookie is removed and the + language is changed to the tracker default, set up in the tracker + configuration or OS environment. + +``@charset`` + value may be character set name or ``none``. Character set name + is stored in the browser cookie and sets output encoding for all + HTML pages generated by Roundup. If value is ``none`` the cookie + is removed and HTML output is reset to Roundup internal encoding + (UTF-8). + Most properties are specified as form variables: ```` @@ -1145,11 +1509,11 @@ None of the above (ie. just a simple form value) For a Link('klass') property, the form value is a single key for 'klass', where the key field is - specified in dbinit.py. + specified in schema.py. For a Multilink('klass') property, the form value is a comma-separated list of keys for 'klass', where the - key field is specified in dbinit.py. + key field is specified in schema.py. Note that for simple-form-variables specifiying Link and Multilink properties, the linked-to class must @@ -1201,13 +1565,12 @@ actual content, otherwise we remove them from all_props before returning. - Default templates ----------------- The default templates are html4 compliant. If you wish to change them to be -xhtml compliant, you'll need to change the ``HTML_VERSION`` configuration -variable in ``config.py`` to ``'xhtml'`` instead of ``'html4'``. +xhtml compliant, you'll need to change the ``html_version`` configuration +variable in ``config.ini`` to ``'xhtml'`` instead of ``'html4'``. Most customisation of the web view can be done by modifying the templates in the tracker ``'html'`` directory. There are several types @@ -1246,7 +1609,7 @@ of files in there. The *minimal* template includes: The *classic* template has a number of additional templates. -Note: Remember that you can create any template extension you want to, +Remember that you can create any template extension you want to, so if you just want to play around with the templating for new issues, you can copy the current "issue.item" template to "issue.test", and then access the test template using the "@template" URL argument:: @@ -1264,8 +1627,8 @@ Basic Templating Actions ~~~~~~~~~~~~~~~~~~~~~~~~ Roundup's templates consist of special attributes on the HTML tags. -These attributes form the Template Attribute Language, or TAL. The basic -TAL commands are: +These attributes form the `Template Attribute Language`_, or TAL. +The basic TAL commands are: **tal:define="variable expression; variable expression; ..."** Define a new variable that is local to this tag and its contents. For @@ -1306,7 +1669,10 @@ TAL commands are: The example would iterate over the sequence of users returned by - "user/list" and define the local variable "u" for each entry. + "user/list" and define the local variable "u" for each entry. Using + the repeat command creates a new variable called "repeat" which you + may access to gather information about the iteration. See the section + below on `the repeat variable`_. **tal:replace="expression"** Replace this tag with the result of the expression. For example:: @@ -1358,12 +1724,17 @@ making arbitrary blocks of HTML conditional or repeatable (very handy for repeating multiple table rows, which would othewise require an illegal tag placement to effect the repeat). +.. _TAL: +.. _Template Attribute Language: + http://dev.zope.org/Wikis/DevSite/Projects/ZPT/TAL%20Specification%201.4 + Templating Expressions ~~~~~~~~~~~~~~~~~~~~~~ -The expressions you may use in the attribute values may be one of the -following forms: +Templating Expressions are covered by `Template Attribute Language +Expression Syntax`_, or TALES. The expressions you may use in the +attribute values may be one of the following forms: **Path Expressions** - eg. ``item/status/checklist`` These are object attribute / item accesses. Roughly speaking, the @@ -1420,6 +1791,10 @@ Modifiers: This simply inverts the logical true/false value of another expression. +.. _TALES: +.. _Template Attribute Language Expression Syntax: + http://dev.zope.org/Wikis/DevSite/Projects/ZPT/TALES%20Specification%201.3 + Template Macros ~~~~~~~~~~~~~~~ @@ -1430,8 +1805,8 @@ you'll use is the "icing" macro defined in the "page" template. Macros are generated and used inside your templates using special attributes similar to the `basic templating actions`_. In this case, -though, the attributes belong to the Macro Expansion Template Attribute -Language, or METAL. The macro commands are: +though, the attributes belong to the `Macro Expansion Template +Attribute Language`_, or METAL. The macro commands are: **metal:define-macro="macro name"** Define that the tag and its contents are now a macro that may be @@ -1477,16 +1852,18 @@ Language, or METAL. The macro commands are: where the tag that fills the slot completely replaces the one defined as the slot in the macro. -Note that you may not mix METAL and TAL commands on the same tag, but +Note that you may not mix `METAL`_ and `TAL`_ commands on the same tag, but TAL commands may be used freely inside METAL-using tags (so your *fill-slots* tags may have all manner of TAL inside them). +.. _METAL: +.. _Macro Expansion Template Attribute Language: + http://dev.zope.org/Wikis/DevSite/Projects/ZPT/METAL%20Specification%201.0 Information available to templates ---------------------------------- -Note: this is implemented by -``roundup.cgi.templating.RoundupPageTemplate`` +This is implemented by ``roundup.cgi.templating.RoundupPageTemplate`` The following variables are available to templates. @@ -1498,12 +1875,17 @@ The following variables are available to templates. - the current index information (``filterspec``, ``filter`` args, ``properties``, etc) parsed out of the form. - methods for easy filterspec link generation - - *user*, the current user item as an HTMLItem instance - - *form* + - "form" The current CGI form information as a mapping of form argument name - to value + to value (specifically a cgi.FieldStorage) + - "env" the CGI environment variables + - "base" the base URL for this instance + - "user" a HTMLItem instance for the current user + - "language" as determined by the browser or config + - "classname" the current classname (possibly None) + - "template" the current template (suffix, also possibly None) **config** - This variable holds all the values defined in the tracker config.py + This variable holds all the values defined in the tracker config.ini file (eg. TRACKER_NAME, etc.) **db** The current database, used to access arbitrary database items. @@ -1536,6 +1918,23 @@ The following variables are available to templates. Hello, World! +**true**, **false** + Boolean constants that may be used in `templating expressions`_ + instead of ``python:1`` and ``python:0``. +**i18n** + Internationalization service, providing two string translation methods: + + **gettext** (*message*) + Return the localized translation of message + **ngettext** (*singular*, *plural*, *number*) + Like ``gettext()``, but consider plural forms. If a translation + is found, apply the plural formula to *number*, and return the + resulting message (some languages have more than two plural forms). + If no translation is found, return singular if *number* is 1; + return plural otherwise. + + This function requires python2.3; in earlier python versions + may not work as expected. The context variable ~~~~~~~~~~~~~~~~~~~~ @@ -1563,7 +1962,7 @@ item. The only real difference between cases 2 and 3 above are: Hyperdb class wrapper ::::::::::::::::::::: -Note: this is implemented by the ``roundup.cgi.templating.HTMLClass`` +This is implemented by the ``roundup.cgi.templating.HTMLClass`` class. This wrapper object provides access to a hyperb class. It is used @@ -1586,10 +1985,64 @@ properties return a `hyperdb property wrapper`_ for all of this class's list lists all of the active (not retired) items in the class. csv return the items of this class as a chunk of CSV text. propnames lists the names of the properties of this class. -filter lists of items from this class, filtered and sorted by the - current *request* filterspec/filter/sort/group args +filter lists of items from this class, filtered and sorted. Two + options are avaible for sorting: + + 1. by the current *request* filterspec/filter/sort/group args + 2. by the "filterspec", "sort" and "group" keyword args. + "filterspec" is ``{propname: value(s)}``. "sort" and + "group" are an optionally empty list ``[(dir, prop)]`` + where dir is '+', '-' or None + and prop is a prop name or None. + + The propname in filterspec and prop in a sort/group spec + may be transitive, i.e., it may contain properties of + the form link.link.link.name. + + eg. All issues with a priority of "1" with messages added in + the last week, sorted by activity date: + ``issue.filter(filterspec={"priority": "1", + 'messages.creation' : '.-1w;'}, sort=[('activity', '+')])`` + +filter_sql **Only in SQL backends** + + Lists the items that match the SQL provided. The SQL is a + complete "select" statement. + + The SQL select must include the item id as the first column. + + This function **does not** filter out retired items, add + on a where clause "__retired__ <> 1" if you don't want + retired nodes. + classhelp display a link to a javascript popup containing this class' "help" template. + + This generates a link to a popup window which displays the + properties indicated by "properties" of the class named by + "classname". The "properties" should be a comma-separated list + (eg. 'id,name,description'). Properties defaults to all the + properties of a class (excluding id, creator, created and + activity). + + You may optionally override the "label" displayed, the "width", + the "height", the number of items per page ("pagesize") and + the field on which the list is sorted ("sort"). + + With the "filter" arg it is possible to specify a filter for + which items are supposed to be displayed. It has to be of + the format "=;=;...". + + The popup window will be resizable and scrollable. + + If the "property" arg is given, it's passed through to the + javascript help_window function. This allows updating of a + property in the calling HTML page. + + If the "form" arg is given, it's passed through to the + javascript help_window function - it's the name of the form + the "property" belongs to. + submit generate a submit button (and action hidden element) renderWith render this class with the given template. history returns 'New node - no history' :) @@ -1609,7 +2062,7 @@ will access the "list" property, rather than the list method. Hyperdb item wrapper :::::::::::::::::::: -Note: this is implemented by the ``roundup.cgi.templating.HTMLItem`` +This is implemented by the ``roundup.cgi.templating.HTMLItem`` class. This wrapper object provides access to a hyperb item. @@ -1630,12 +2083,30 @@ history render the journal of the current item as HTML renderQueryForm specific to the "query" class - render the search form for the query hasPermission specific to the "user" class - determine whether the - user has a Permission + user has a Permission. The signature is:: + + hasPermission(self, permission, [classname=], + [property=], [itemid=]) + + where the classname defaults to the current context. +hasRole specific to the "user" class - determine whether the + user has a Role. The signature is:: + + hasRole(self, rolename) + is_edit_ok is the user allowed to Edit the current item? is_view_ok is the user allowed to View the current item? is_retired is the item retired? -download_url generates a url-quoted link for download of FileClass +download_url generate a url-quoted link for download of FileClass item contents (ie. file/) +copy_url generate a url-quoted link for creating a copy + of this item. By default, the copy will acquire + all properties of the current item except for + ``messages`` and ``files``. This can be overridden + by passing ``exclude`` argument which contains a list + (or any iterable) of property names that shall not be + copied. Database-driven properties like ``id`` or + ``activity`` cannot be copied. =============== ======================================================== Note that if you have a property of the same name as one of the above @@ -1650,7 +2121,7 @@ will access the "journal" property, rather than the journal method. Hyperdb property wrapper :::::::::::::::::::::::: -Note: this is implemented by subclasses of the +This is implemented by subclasses of the ``roundup.cgi.templating.HTMLProperty`` class (``HTMLStringProperty``, ``HTMLNumberProperty``, and so on). @@ -1704,15 +2175,30 @@ plain render a "plain" representation of the property. This method "structure python:msg.content.plain(hyperlink=1)" - Note also that the text is automatically HTML-escaped before - the hyperlinking transformation. + The text is automatically HTML-escaped before the hyperlinking + transformation done in the plain() method. + hyperlinked The same as msg.content.plain(hyperlink=1), but nicer:: "structure msg/content/hyperlinked" field render an appropriate form edit field for the property - for most types this is a text entry box, but for Booleans it's a - tri-state yes/no/neither selection. + tri-state yes/no/neither selection. This method may take some + arguments: + + size + Sets the width in characters of the edit field + + format (Date properties only) + Sets the format of the date in the field - uses the same + format string argument as supplied to the ``pretty`` method + below. + + popcal (Date properties only) + Include the Javascript-based popup calendar for date + selection. Defaults to on. + stext only on String properties - render the value of the property as StructuredText (requires the StructureText module to be installed separately) @@ -1742,14 +2228,62 @@ pretty Date properties - render the date as "dd Mon YYYY" (eg. "19 Will format as "2004-03-19" instead. Interval properties - render the interval in a pretty - format (eg. "yesterday") + format (eg. "yesterday"). The format arguments are those used + in the standard ``strftime`` call (see the `Python Library + Reference: time module`__) +popcal Generate a link to a popup calendar which may be used to + edit the date field, for example:: + + + + you still need to include the ``field`` for the property, so + typically you'd have:: + + + + menu only on Link and Multilink properties - render a form select - list for this property + list for this property. Takes a number of optional arguments + + size + is used to limit the length of the list labels + height + is used to set the - - + + Adding category to the default view @@ -2765,7 +3447,7 @@ user hasn't asked for it to be hidden. The next part is to set the content of the cell to be the category part of "i" - the current issue. Finally we have to edit ``html/page.html`` again. This time, we need to -tell it that when the user clicks on "Unasigned Issues" or "All Issues", +tell it that when the user clicks on "Unassigned Issues" or "All Issues", the category column should be included in the resulting list. If you scroll down the page file, you can see the links with lots of options. The option that we are interested in is the ``:columns=`` one which @@ -2778,7 +3460,7 @@ Adding a time log to your issues We want to log the dates and amount of time spent working on issues, and be able to give a summary of the total time spent on a particular issue. -1. Add a new class to your tracker ``dbinit.py``:: +1. Add a new class to your tracker ``schema.py``:: # storage for time logging timelog = Class(db, "timelog", period=Interval()) @@ -2786,11 +3468,21 @@ be able to give a summary of the total time spent on a particular issue. Note that we automatically get the date of the time log entry creation through the standard property "creation". + You will need to grant "Creation" permission to the users who are + allowed to add timelog entries. You may do this with:: + + db.security.addPermissionToRole('User', 'Create', 'timelog') + db.security.addPermissionToRole('User', 'View', 'timelog') + + If users are also able to *edit* timelog entries, then also include:: + + db.security.addPermissionToRole('User', 'Edit', 'timelog') + 2. Link to the new class from your issue class (again, in - ``dbinit.py``):: + ``schema.py``):: issue = IssueClass(db, "issue", - assignedto=Link("user"), topic=Multilink("keyword"), + assignedto=Link("user"), keyword=Multilink("keyword"), priority=Link("priority"), status=Link("status"), times=Multilink("timelog")) @@ -2798,17 +3490,17 @@ be able to give a summary of the total time spent on a particular issue. 3. We'll need to let people add in times to the issue, so in the web interface we'll have a new entry field. This is a special field - because unlike the other fields in the issue.item template, it + because unlike the other fields in the ``issue.item`` template, it affects a different item (a timelog item) and not the template's - item, an issue. We have a special syntax for form fields that affect + item (an issue). We have a special syntax for form fields that affect items other than the template default item (see the cgi documentation on `special form variables`_). In particular, we add a - field to capture a new timelog item's perdiod:: + field to capture a new timelog item's period:: Time Log -
(enter as '3y 1m 4d 2:40:02' or parts thereof) + (enter as '3y 1m 4d 2:40:02' or parts thereof) @@ -2821,31 +3513,43 @@ be able to give a summary of the total time spent on a particular issue. On submission, the "-1" timelog item will be created and assigned a real item id. The "times" property of the issue will have the new id added to it. + + The full entry will now look like this:: + + + Time Log + + (enter as '3y 1m 4d 2:40:02' or parts thereof) + + + + -4. We want to display a total of the time log times that have been +4. We want to display a total of the timelog times that have been accumulated for an issue. To do this, we'll need to actually write some Python code, since it's beyond the scope of PageTemplates to - perform such calculations. We do this by adding a method to the - TemplatingUtils class in our tracker ``interfaces.py`` module:: + perform such calculations. We do this by adding a module ``timespent.py`` + to the ``extensions`` directory in our tracker. The contents of this + file is as follows:: + + from roundup import date - class TemplatingUtils: - ''' Methods implemented on this class will be available to HTML - templates through the 'utils' variable. + def totalTimeSpent(times): + ''' Call me with a list of timelog items (which have an + Interval "period" property) ''' - def totalTimeSpent(self, times): - ''' Call me with a list of timelog items (which have an - Interval "period" property) - ''' - total = Interval('0d') - for time in times: - total += time.period._value - return total + total = date.Interval('0d') + for time in times: + total += time.period._value + return total + + def init(instance): + instance.registerUtil('totalTimeSpent', totalTimeSpent) - Replace the ``pass`` line if one appears in your TemplatingUtils - class. As indicated in the docstrings, we will be able to access the - ``totalTimeSpent`` method via the ``utils`` variable in our templates. + We will now be able to access the ``totalTimeSpent`` function via the + ``utils`` variable in our templates, as shown in the next step. -5. Display the time log for an issue:: +5. Display the timelog for an issue:: + + + + + The "times" property of the message will have the new id added to it. + +C. Add the timelog listing from step 5. to the ``msg.item.html`` template + so that the timelog entry appears on the message view page. Note that + the call to totalTimeSpent is not used here since there will only be one + single timelog entry for each message. + + I placed it after the Date entry like this:: + + + + + +
Time Log @@ -2866,11 +3570,69 @@ be able to give a summary of the total time spent on a particular issue. displayed in the template as text like "+ 1y 2:40" (1 year, 2 hours and 40 minutes). -8. If you're using a persistent web server - roundup-server or - mod_python for example - then you'll need to restart that to pick up +6. If you're using a persistent web server - ``roundup-server`` or + ``mod_python`` for example - then you'll need to restart that to pick up the code changes. When that's done, you'll be able to use the new time logging interface. +An extension of this modification attaches the timelog entries to any +change message entered at the time of the timelog entry: + +A. Add a link to the timelog to the msg class in ``schema.py``: + + msg = FileClass(db, "msg", + author=Link("user", do_journal='no'), + recipients=Multilink("user", do_journal='no'), + date=Date(), + summary=String(), + files=Multilink("file"), + messageid=String(), + inreplyto=String(), + times=Multilink("timelog")) + +B. Add a new hidden field that links that new timelog item (new + because it's marked as having id "-1") to the new message. + The link is placed in ``issue.item.html`` in the same section that + handles the timelog entry. + + It looks like this after this addition:: + +
Time Log + (enter as '3y 1m 4d 2:40:02' or parts thereof) + + +
Date:
+ + + + + + + + + +
Time Log
DatePeriodLogged By
+ + + Tracking different types of issues ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2878,51 +3640,54 @@ Tracking different types of issues Sometimes you will want to track different types of issues - developer, customer support, systems, sales leads, etc. A single Roundup tracker is able to support multiple types of issues. This example demonstrates adding -a customer support issue class to a tracker. +a system support issue class to a tracker. 1. Figure out what information you're going to want to capture. OK, so this is obvious, but sometimes it's better to actually sit down for a while and think about the schema you're going to implement. -2. Add the new issue class to your tracker's ``dbinit.py`` - in this - example, we're adding a "system support" class. Just after the "issue" - class definition in the "open" function, add:: +2. Add the new issue class to your tracker's ``schema.py``. Just after the + "issue" class definition, add:: + # list our systems + system = Class(db, "system", name=String(), order=Number()) + system.setkey("name") + + # store issues related to those systems support = IssueClass(db, "support", - assignedto=Link("user"), topic=Multilink("keyword"), + assignedto=Link("user"), keyword=Multilink("keyword"), status=Link("status"), deadline=Date(), affects=Multilink("system")) -3. Copy the existing "issue.*" (item, search and index) templates in the - tracker's "html" to "support.*". Edit them so they use the properties - defined in the "support" class. Be sure to check for hidden form +3. Copy the existing ``issue.*`` (item, search and index) templates in the + tracker's ``html`` to ``support.*``. Edit them so they use the properties + defined in the ``support`` class. Be sure to check for hidden form variables like "required" to make sure they have the correct set of required properties. -4. Edit the modules in the "detectors", adding lines to their "init" - functions where appropriate. Look for "audit" and "react" registrations - on the "issue" class, and duplicate them for "support". +4. Edit the modules in the ``detectors``, adding lines to their ``init`` + functions where appropriate. Look for ``audit`` and ``react`` registrations + on the ``issue`` class, and duplicate them for ``support``. 5. Create a new sidebar box for the new support class. Duplicate the - existing issues one, changing the "issue" class name to "support". + existing issues one, changing the ``issue`` class name to ``support``. -6. Re-start your tracker and start using the new "support" class. +6. Re-start your tracker and start using the new ``support`` class. Optionally, you might want to restrict the users able to access this new class to just the users with a new "SysAdmin" Role. To do this, we add some security declarations:: - p = db.security.getPermission('View', 'support') - db.security.addPermissionToRole('SysAdmin', p) - p = db.security.getPermission('Edit', 'support') - db.security.addPermissionToRole('SysAdmin', p) + db.security.addPermissionToRole('SysAdmin', 'View', 'support') + db.security.addPermissionToRole('SysAdmin', 'Create', 'support') + db.security.addPermissionToRole('SysAdmin', 'Edit', 'support') You would then (as an "admin" user) edit the details of the appropriate users, and add "SysAdmin" to their Roles list. Alternatively, you might want to change the Edit/View permissions granted -for the "issue" class so that it's only available to users with the "System" +for the ``issue`` class so that it's only available to users with the "System" or "Developer" Role, and then the new class you're adding is available to all with the "User" Role. @@ -2933,6 +3698,12 @@ Using External User Databases Using an external password validation source ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. note:: You will need to either have an "admin" user in your external + password source *or* have one of your regular users have + the Admin Role assigned. If you need to assign the Role *after* + making the changes below, you may use the ``roundup-admin`` + program to edit a user's details. + We have a centrally-managed password changing system for our users. This results in a UN*X passwd-style file that we use for verification of users. Entries in the file consist of ``name:password`` where the @@ -2945,13 +3716,18 @@ would be:: Each user of Roundup must still have their information stored in the Roundup database - we just use the passwd file to check their password. To do this, we need to override the standard ``verifyPassword`` method defined in -``roundup.cgi.actions.LoginAction`` and register the new class with our -``Client`` class in the tracker home ``interfaces.py`` module:: +``roundup.cgi.actions.LoginAction`` and register the new class. The +following is added as ``externalpassword.py`` in the tracker ``extensions`` +directory:: + import os, crypt from roundup.cgi.actions import LoginAction class ExternalPasswordLoginAction(LoginAction): def verifyPassword(self, userid, password): + '''Look through the file, line by line, looking for a + name that matches. + ''' # get the user's username username = self.db.user.get(userid, 'username') @@ -2968,15 +3744,10 @@ need to override the standard ``verifyPassword`` method defined in # user doesn't exist in the file return 0 - class Client(client.Client): - actions = client.Client.actions + ( - ('login', ExternalPasswordLoginAction) - ) - -What this does is look through the file, line by line, looking for a -name that matches. + def init(instance): + instance.registerAction('login', ExternalPasswordLoginAction) -We also remove the redundant password fields from the ``user.item`` +You should also remove the redundant password fields from the ``user.item`` template. @@ -2996,7 +3767,7 @@ which the users are removed when they no longer have access to a system. To make use of the passwd file, we therefore synchronise between the two user stores. We also use the passwd file to validate the user logins, as described in the previous example, `using an external password -validation source`_. We keep the users lists in sync using a fairly +validation source`_. We keep the user lists in sync using a fairly simple script that runs once a day, or several times an hour if more immediate access is needed. In short, it: @@ -3011,7 +3782,7 @@ immediate access is needed. In short, it: The retiring and updating are simple operations, requiring only a call to ``retire()`` or ``set()``. The creation operation requires more -information though - the user's email address and their roundup Roles. +information though - the user's email address and their Roundup Roles. We're going to assume that the user's email address is the same as their login name, so we just append the domain name to that. The Roles are determined using the passwd group identifier - mapping their UN*X group @@ -3030,8 +3801,8 @@ tracker we're to work on:: Next we read in the *passwd* file from the tracker home:: - # read in the users - file = os.path.join(tracker_home, 'users.passwd') + # read in the users from the "passwd.txt" file + file = os.path.join(tracker_home, 'passwd.txt') users = [x.strip().split(':') for x in open(file).readlines()] Handle special users (those to ignore in the file, and those who don't @@ -3127,11 +3898,11 @@ workflow). See the example `Using a UN*X passwd file as the user database`_ for more information about doing this. To authenticate off the LDAP store (rather than using the passwords in the -roundup user database) you'd use the same python-ldap module inside an +Roundup user database) you'd use the same python-ldap module inside an extension to the cgi interface. You'd do this by overriding the method called -"verifyPassword" on the LoginAction class in your tracker's interfaces.py -module (see `using an external password validation source`_). The method is -implemented by default as:: +``verifyPassword`` on the ``LoginAction`` class in your tracker's +``extensions`` directory (see `using an external password validation +source`_). The method is implemented by default as:: def verifyPassword(self, userid, password): ''' Verify the password that the user has supplied @@ -3156,6 +3927,37 @@ So you could reimplement this as something like:: Changes to Tracker Behaviour ---------------------------- +Preventing SPAM +~~~~~~~~~~~~~~~ + +The following detector code may be installed in your tracker's +``detectors`` directory. It will block any messages being created that +have HTML attachments (a very common vector for spam and phishing) +and any messages that have more than 2 HTTP URLs in them. Just copy +the following into ``detectors/anti_spam.py`` in your tracker:: + + from roundup.exceptions import Reject + + def reject_html(db, cl, nodeid, newvalues): + if newvalues['type'] == 'text/html': + raise Reject, 'not allowed' + + def reject_manylinks(db, cl, nodeid, newvalues): + content = newvalues['content'] + if content.count('http://') > 2: + raise Reject, 'not allowed' + + def init(db): + db.file.audit('create', reject_html) + db.msg.audit('create', reject_manylinks) + +You may also wish to block image attachments if your tracker does not +need that ability:: + + if newvalues['type'].startswith('image/'): + raise Reject, 'not allowed' + + Stop "nosy" messages going to people on vacation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -3256,8 +4058,8 @@ vacation". Not very useful, and relatively easy to stop. Adding in state transition control ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Sometimes tracker admins want to control the states that users may move -issues to. You can do this by following these steps: +Sometimes tracker admins want to control the states to which users may +move issues. You can do this by following these steps: 1. make "status" a required variable. This is achieved by adding the following to the top of the form in the ``issue.item.html`` @@ -3265,7 +4067,7 @@ issues to. You can do this by following these steps: - this will force users to select a status. + This will force users to select a status. 2. add a Multilink property to the status class:: @@ -3275,7 +4077,7 @@ issues to. You can do this by following these steps: and then edit the statuses already created, either: a. through the web using the class list -> status class editor, or - b. using the roundup-admin "set" command. + b. using the ``roundup-admin`` "set" command. 3. add an auditor module ``checktransition.py`` in your tracker's ``detectors`` directory, for example:: @@ -3332,47 +4134,50 @@ We needed the ability to mark certain issues as "blockers" - that is, they can't be resolved until another issue (the blocker) they rely on is resolved. To achieve this: -1. Create a new property on the issue Class, - ``blockers=Multilink("issue")``. Edit your tracker's dbinit.py file. - Where the "issue" class is defined, something like:: +1. Create a new property on the ``issue`` class: + ``blockers=Multilink("issue")``. To do this, edit the definition of + this class in your tracker's ``schema.py`` file. Change this:: issue = IssueClass(db, "issue", - assignedto=Link("user"), topic=Multilink("keyword"), + assignedto=Link("user"), keyword=Multilink("keyword"), priority=Link("priority"), status=Link("status")) - add the blockers entry like so:: + to this, adding the blockers entry:: issue = IssueClass(db, "issue", blockers=Multilink("issue"), - assignedto=Link("user"), topic=Multilink("keyword"), + assignedto=Link("user"), keyword=Multilink("keyword"), priority=Link("priority"), status=Link("status")) -2. Add the new "blockers" property to the issue.item edit page, using - something like:: +2. Add the new ``blockers`` property to the ``issue.item.html`` edit + page, using something like:: You'll need to fiddle with your item page layout to find an appropriate place to put it - I'll leave that fun part up to you. Just make sure it appears in the first table, possibly somewhere near the "superseders" field. -3. Create a new detector module (attached) which enforces the rules: +3. Create a new detector module (see below) which enforces the rules: - issues may not be resolved if they have blockers - when a blocker is resolved, it's removed from issues it blocks The contents of the detector should be something like this:: + def blockresolution(db, cl, nodeid, newvalues): ''' If the issue has blockers, don't allow it to be resolved. ''' @@ -3403,18 +4208,21 @@ resolved. To achieve this: if newvalues['status'] == resolved_id: raise ValueError, "This issue can't be resolved until %s resolved."%s - def resolveblockers(db, cl, nodeid, newvalues): + + def resolveblockers(db, cl, nodeid, oldvalues): ''' When we resolve an issue that's a blocker, remove it from the blockers list of the issue(s) it blocks. ''' - if not newvalues.has_key('status'): + newstatus = cl.get(nodeid,'status') + + # no change? + if oldvalues.get('status', None) == newstatus: return - # get the resolved state ID resolved_id = db.status.lookup('resolved') # interesting? - if newvalues['status'] != resolved_id: + if newstatus != resolved_id: return # yes - find all the blocked issues, if any, and remove me from @@ -3426,7 +4234,6 @@ resolved. To achieve this: blockers.remove(nodeid) cl.set(issueid, blockers=blockers) - def init(db): # might, in an obscure situation, happen in a create db.issue.audit('create', blockresolution) @@ -3444,12 +4251,36 @@ resolved. To achieve this: example, the existing "Show All" link in the "page" template (in the tracker's "html" directory) looks like this:: - Show All
+ Show All
modify it to add the "blockers" info to the URL (note, both the - ":filter" *and* "blockers" values must be specified):: - - Show All
+ "@filter" *and* "blockers" values must be specified):: + + Show All
+ + The above examples are line-wrapped on the trailing & and should + be unwrapped. That's it. You should now be able to set blockers on your issues. Note that if you want to know whether an issue has any other issues dependent @@ -3457,33 +4288,32 @@ on it (i.e. it's in their blockers list) you can look at the journal history at the bottom of the issue page - look for a "link" event to another issue's "blockers" property. -Add users to the nosy list based on the topic -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Add users to the nosy list based on the keyword +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -We need the ability to automatically add users to the nosy list based -on the occurence of a topic. Every user should be allowed to edit his -own list of topics for which he wants to be added to the nosy list. +Let's say we need the ability to automatically add users to the nosy +list based +on the occurance of a keyword. Every user should be allowed to edit their +own list of keywords for which they want to be added to the nosy list. -Below will be showed that such a change can be performed with only -minimal understanding of the roundup system, but with clever use -of Copy and Paste. +Below, we'll show that this change can be done with minimal +understanding of the Roundup system, using only copy and paste. This requires three changes to the tracker: a change in the database to -allow per-user recording of the lists of topics for which he wants to -be put on the nosy list, a change in the user view allowing to edit -this list of topics, and addition of an auditor which updates the nosy -list when a topic is set. - -Adding the nosy topic list -:::::::::::::::::::::::::: - -The change in the database to make is that for any user there should be -a list of topics for which he wants to be put on the nosy list. Adding -a ``Multilink`` of ``keyword`` seem to fullfill this (note that within -the code topics are called ``keywords``.) As such, all what has to be -done is to add a new field to the definition of ``user`` within the -file ``dbinit.py``. We will call this new field ``nosy_keywords``, and -the updated definition of user will be:: +allow per-user recording of the lists of keywords for which he wants to +be put on the nosy list, a change in the user view allowing them to edit +this list of keywords, and addition of an auditor which updates the nosy +list when a keyword is set. + +Adding the nosy keyword list +:::::::::::::::::::::::::::: + +The change to make in the database, is that for any user there should be a list +of keywords for which he wants to be put on the nosy list. Adding a +``Multilink`` of ``keyword`` seems to fullfill this. As such, all that has to +be done is to add a new field to the definition of ``user`` within the file +``schema.py``. We will call this new field ``nosy_keywords``, and the updated +definition of user will be:: user = Class(db, "user", username=String(), password=Password(), @@ -3494,22 +4324,22 @@ the updated definition of user will be:: timezone=String(), nosy_keywords=Multilink('keyword')) -Changing the user view to allow changing the nosy topic list -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: +Changing the user view to allow changing the nosy keyword list +:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -We want any user to be able to change the list of topics for which +We want any user to be able to change the list of keywords for which he will by default be added to the nosy list. We choose to add this to the user view, as is generated by the file ``html/user.item.html``. -We easily can -see that the topic field in the issue view has very similar editting -requirements as our nosy topics, both being a list of topics. As -such, we search for Topics in ``issue.item.html``, and extract the +We can easily +see that the keyword field in the issue view has very similar editing +requirements as our nosy keywords, both being lists of keywords. As +such, we look for Keywords in ``issue.item.html``, and extract the associated parts from there. We add this to ``user.item.html`` at the bottom of the list of viewed items (i.e. just below the 'Alternate E-mail addresses' in the classic template):: - + `` that displays the -actual rows of data:: +the ``issue.index.html`` template, add this to the ```` that +displays the rows of data:: and then in your stylesheet (``style.css``) specify the colouring for the -different priorities, like:: +different priorities, as follows:: tr.priority-critical td { background-color: red; @@ -3802,7 +4648,8 @@ Editing multiple items in an index view To edit the status of all items in the item index view, edit the ``issue.item.html``: -1. add a form around the listing table, so at the top it reads:: +1. add a form around the listing table (separate from the existing + index-page form), so at the top it reads::
Waiting On - +
View: +
Nosy TopicsNosy Keywords @@ -3520,39 +4350,39 @@ E-mail addresses' in the classic template):: Addition of an auditor to update the nosy list :::::::::::::::::::::::::::::::::::::::::::::: -The more difficult part is the addition of the logic to actually -at the users to the nosy list when it is required. -The choice is made to perform this action when the topics on an -item are set, including when an item is created. +The more difficult part is the logic to add +the users to the nosy list when required. +We choose to perform this action whenever the keywords on an +item are set (this includes the creation of items). Here we choose to start out with a copy of the ``detectors/nosyreaction.py`` detector, which we copy to the file ``detectors/nosy_keyword_reaction.py``. This looks like a good start as it also adds users to the nosy list. A look through the code reveals that the -``nosyreaction`` function actually is sending the e-mail, which -we do not need. As such, we can change the init function to:: +``nosyreaction`` function actually sends the e-mail. +We don't need this. Therefore, we can change the ``init`` function to:: def init(db): db.issue.audit('create', update_kw_nosy) db.issue.audit('set', update_kw_nosy) -After that we rename the ``updatenosy`` function to ``update_kw_nosy``. -The first two blocks of code in that function relate to settings +After that, we rename the ``updatenosy`` function to ``update_kw_nosy``. +The first two blocks of code in that function relate to setting ``current`` to a combination of the old and new nosy lists. This functionality is left in the new auditor. The following block of -code, which in ``updatenosy`` handled adding the assignedto user(s) -to the nosy list, should be replaced by a block of code to add the +code, which handled adding the assignedto user(s) to the nosy list in +``updatenosy``, should be replaced by a block of code to add the interested users to the nosy list. We choose here to loop over all -new topics, than loop over all users, -and assign the user to the nosy list when the topic in the user's -nosy_keywords. The next part in ``updatenosy``, adding the author -and/or recipients of a message to the nosy list, obviously is not -relevant here and thus is deleted from the new auditor. The last -part, copying the new nosy list to newvalues, does not have to be changed. -This brings the following function:: +new keywords, than looping over all users, +and assign the user to the nosy list when the keyword occurs in the user's +``nosy_keywords``. The next part in ``updatenosy`` -- adding the author +and/or recipients of a message to the nosy list -- is obviously not +relevant here and is thus deleted from the new auditor. The last +part, copying the new nosy list to ``newvalues``, can stay as is. +This results in the following function:: def update_kw_nosy(db, cl, nodeid, newvalues): - '''Update the nosy list for changes to the topics + '''Update the nosy list for changes to the keywords ''' # nodeid will be None if this is a new node current = {} @@ -3577,17 +4407,17 @@ This brings the following function:: if not current.has_key(value): current[value] = 1 - # add users with topic in nosy_keywords to the nosy list - if newvalues.has_key('topic') and newvalues['topic'] is not None: - topic_ids = newvalues['topic'] - for topic in topic_ids: + # add users with keyword in nosy_keywords to the nosy list + if newvalues.has_key('keyword') and newvalues['keyword'] is not None: + keyword_ids = newvalues['keyword'] + for keyword in keyword_ids: # loop over all users, - # and assign user to nosy when topic in nosy_keywords + # and assign user to nosy when keyword in nosy_keywords for user_id in db.user.list(): nosy_kw = db.user.get(user_id, "nosy_keywords") found = 0 for kw in nosy_kw: - if kw == topic: + if kw == keyword: found = 1 if found: current[user_id] = 1 @@ -3595,9 +4425,9 @@ This brings the following function:: # that's it, save off the new nosy list newvalues['nosy'] = current.keys() -and these two function are the only ones needed in the file. +These two function are the only ones needed in the file. -TODO: update this example to use the find() Class method. +TODO: update this example to use the ``find()`` Class method. Caveats ::::::: @@ -3606,22 +4436,22 @@ A few problems with the design here can be noted: Multiple additions When a user, after automatic selection, is manually removed - from the nosy list, he again is added to the nosy list when the - topic list of the issue is updated. A better design might be - to only check which topics are new compared to the old list - of topics, and only add users when they have indicated - interest on a new topic. + from the nosy list, he is added to the nosy list again when the + keyword list of the issue is updated. A better design might be + to only check which keywords are new compared to the old list + of keywords, and only add users when they have indicated + interest on a new keyword. - The code could also be changed to only trigger on the create() event, - rather than also on the set() event, thus only setting the nosy list - when the issue is created. + The code could also be changed to only trigger on the ``create()`` + event, rather than also on the ``set()`` event, thus only setting + the nosy list when the issue is created. Scalability - In the auditor there is a loop over all users. For a site with - only few users this will pose no serious problem, however, with + In the auditor, there is a loop over all users. For a site with + only few users this will pose no serious problem; however, with many users this will be a serious performance bottleneck. - A way out will be to link from the topics to the users which - selected these topics a nosy topics. This will eliminate the + A way out would be to link from the keywords to the users who + selected these keywords as nosy keywords. This will eliminate the loop over all users. Changes to Security and Permissions @@ -3630,7 +4460,7 @@ Changes to Security and Permissions Restricting the list of users that are assignable to a task ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. In your tracker's "dbinit.py", create a new Role, say "Developer":: +1. In your tracker's ``schema.py``, create a new Role, say "Developer":: db.security.addRole(name='Developer', description='A developer') @@ -3644,7 +4474,7 @@ Restricting the list of users that are assignable to a task db.security.addPermissionToRole('Developer', p) -4. In the issue item edit page ("html/issue.item.html" in your tracker +4. In the issue item edit page (``html/issue.item.html`` in your tracker directory), use the new Permission in restricting the "assignedto" list:: @@ -3661,11 +4491,11 @@ Restricting the list of users that are assignable to a task For extra security, you may wish to setup an auditor to enforce the -Permission requirement (install this as "assignedtoFixer.py" in your -tracker "detectors" directory):: +Permission requirement (install this as ``assignedtoFixer.py`` in your +tracker ``detectors`` directory):: def assignedtoMustBeFixer(db, cl, nodeid, newvalues): - ''' Ensure the assignedto value in newvalues is a used with the + ''' Ensure the assignedto value in newvalues is used with the Fixer Permission ''' if not newvalues.has_key('assignedto'): @@ -3684,74 +4514,90 @@ tracker "detectors" directory):: So now, if an edit action attempts to set "assignedto" to a user that doesn't have the "Fixer" Permission, the error will be raised. + Users may only edit their issues ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Users registering themselves are granted Provisional access - meaning they +In this case, users registering themselves are granted Provisional +access, meaning they have access to edit the issues they submit, but not others. We create a new Role called "Provisional User" which is granted to newly-registered users, and has limited access. One of the Permissions they have is the new "Edit -Own" on issues (regular users have "Edit".) We back up the permissions with -an auditor. +Own" on issues (regular users have "Edit".) First up, we create the new Role and Permission structure in -``dbinit.py``:: +``schema.py``:: + # # New users not approved by the admin + # db.security.addRole(name='Provisional User', description='New user registered via web or email') - p = db.security.addPermission(name='Edit Own', klass='issue', - description='Can only edit own issues') - db.security.addPermissionToRole('Provisional User', p) - # Assign the access and edit Permissions for issue to new users now - p = db.security.getPermission('View', 'issue') + # These users need to be able to view and create issues but only edit + # and view their own + db.security.addPermissionToRole('Provisional User', 'Create', 'issue') + def own_issue(db, userid, itemid): + '''Determine whether the userid matches the creator of the issue.''' + return userid == db.issue.get(itemid, 'creator') + p = db.security.addPermission(name='Edit', klass='issue', + check=own_issue, description='Can only edit own issues') db.security.addPermissionToRole('Provisional User', p) - p = db.security.getPermission('Edit', 'issue') + p = db.security.addPermission(name='View', klass='issue', + check=own_issue, description='Can only view own issues') db.security.addPermissionToRole('Provisional User', p) + # Assign the Permissions for issue-related classes + for cl in 'file', 'msg', 'query', 'keyword': + db.security.addPermissionToRole('Provisional User', 'View', cl) + db.security.addPermissionToRole('Provisional User', 'Edit', cl) + db.security.addPermissionToRole('Provisional User', 'Create', cl) + for cl in 'priority', 'status': + db.security.addPermissionToRole('Provisional User', 'View', cl) + # and give the new users access to the web and email interface - p = db.security.getPermission('Web Access') + db.security.addPermissionToRole('Provisional User', 'Web Access') + db.security.addPermissionToRole('Provisional User', 'Email Access') + + # make sure they can view & edit their own user record + def own_record(db, userid, itemid): + '''Determine whether the userid matches the item being accessed.''' + return userid == itemid + p = db.security.addPermission(name='View', klass='user', check=own_record, + description="User is allowed to view their own user details") db.security.addPermissionToRole('Provisional User', p) - p = db.security.getPermission('Email Access') + p = db.security.addPermission(name='Edit', klass='user', check=own_record, + description="User is allowed to edit their own user details") db.security.addPermissionToRole('Provisional User', p) - -Then in the ``config.py`` we change the Role assigned to newly-registered +Then, in ``config.ini``, we change the Role assigned to newly-registered users, replacing the existing ``'User'`` values:: - NEW_WEB_USER_ROLES = 'Provisional User' - NEW_EMAIL_USER_ROLES = 'Provisional User' - -Finally we add a new *auditor* to the ``detectors`` directory called -``provisional_user_auditor.py``:: + [main] + ... + new_web_user_roles = 'Provisional User' + new_email_user_roles = 'Provisional User' - def audit_provisionaluser(db, cl, nodeid, newvalues): - ''' New users are only allowed to modify their own issues. - ''' - if (db.getuid() != cl.get(nodeid, 'creator') - and db.security.hasPermission('Edit Own', db.getuid(), cl.classname)): - raise ValueError, ('You are only allowed to edit your own %s' - % cl.classname) - def init(db): - # fire before changes are made - db.issue.audit('set', audit_provisionaluser) - db.issue.audit('retire', audit_provisionaluser) - db.issue.audit('restore', audit_provisionaluser) +All users may only view and edit issues, files and messages they create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Note that some older trackers might also want to change the ``page.html`` -template as follows:: +Replace the standard "classic" tracker View and Edit Permission assignments +for the "issue", "file" and "msg" classes with the following:: -

- + tal:condition="python:request.user.hasPermission('View', 'user')"> - Administration
- - Class List
+ def checker(klass): + def check(db, userid, itemid, klass=klass): + return db.getclass(klass).get(itemid, 'creator') == userid + return check + for cl in 'issue', 'file', 'msg': + p = db.security.addPermission(name='View', klass=cl, + check=checker(cl)) + db.security.addPermissionToRole('User', p) + p = db.security.addPermission(name='Edit', klass=cl, + check=checker(cl)) + db.security.addPermissionToRole('User', p) + db.security.addPermissionToRole('User', 'Create', cl) -(note that the "-" indicates a removed line, and the "+" indicates an added -line). Changes to the Web User Interface @@ -3760,7 +4606,7 @@ Changes to the Web User Interface Adding action links to the index page ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Add a column to the item.index.html template. +Add a column to the ``item.index.html`` template. Resolving the issue:: @@ -3772,19 +4618,19 @@ Resolving the issue:: take -... and so on +... and so on. Colouring the rows in the issue index according to priority ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A simple ``tal:attributes`` statement will do the bulk of the work here. In -the ``issue.index.html`` template, add to the ``

@@ -3827,7 +4674,7 @@ To edit the status of all items in the item index view, edit the this will result in an edit field for the status property. -3. after the ``tal:block`` which lists the actual index items (marked by +3. after the ``tal:block`` which lists the index items (marked by ``tal:repeat="i batch"``) add a new table row:: @@ -3839,7 +4686,7 @@ To edit the status of all items in the item index view, edit the which gives us a submit button, indicates that we are performing an edit - on any changed statuses and the final block will make sure that the + on any changed statuses. The final ``tal:block`` will make sure that the current index view parameters (filtering, columns, etc) will be used in rendering the next page (the results of the editing). @@ -3847,7 +4694,7 @@ To edit the status of all items in the item index view, edit the Displaying only message summaries in the issue display ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Alter the issue.item template section for messages to:: +Alter the ``issue.item`` template section for messages to::
@@ -3871,7 +4718,7 @@ Enabling display of either message summaries or the entire messages This is pretty simple - all we need to do is copy the code from the example `displaying only message summaries in the issue display`_ into our template alongside the summary display, and then introduce a switch -that shows either one or the other. We'll use a new form variable, +that shows either the one or the other. We'll use a new form variable, ``@whole_messages`` to achieve this::
Messages
@@ -3912,6 +4759,7 @@ that shows either one or the other. We'll use a new form variable,
+ Setting up a "wizard" (or "druid") for controlled adding of issues ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -3947,8 +4795,8 @@ Setting up a "wizard" (or "druid") for controlled adding of issues . - Note that later in the form, I test the value of "cat" include form - elements that are appropriate. For example:: + Note that later in the form, I use the value of "cat" to decide which + form elements should be displayed. For example:: @@ -3966,9 +4814,9 @@ Setting up a "wizard" (or "druid") for controlled adding of issues 3. Determine what actions need to be taken between the pages - these are usually to validate user choices and determine what page is next. Now encode - those actions in a new ``Action`` class and insert hooks to those actions in - the "actions" attribute on on the ``interfaces.Client`` class, like so (see - `defining new web actions`_):: + those actions in a new ``Action`` class (see `defining new web actions`_):: + + from roundup.cgi.actions import Action class Page1SubmitAction(Action): def handle(self): @@ -3982,14 +4830,30 @@ Setting up a "wizard" (or "druid") for controlled adding of issues # everything's ok, move on to the next page self.template = 'add_page2' - actions = client.Client.actions + ( - ('page1_submit', Page1SubmitAction), - ) + def init(instance): + instance.registerAction('page1_submit', Page1SubmitAction) 4. Use the usual "new" action as the ``@action`` on the final page, and you're done (the standard context/submit method can do this for you). +Debugging Trackers +================== + +There are three switches in tracker configs that turn on debugging in +Roundup: + +1. web :: debug +2. mail :: debug +3. logging :: level + +See the config.ini file or the `tracker configuration`_ section above for +more information. + +Additionally, the ``roundup-server.py`` script has its own debugging mode +in which it reloads edited templates immediately when they are changed, +rather than requiring a web server restart. + ------------------- @@ -3997,4 +4861,5 @@ Back to `Table of Contents`_ .. _`Table of Contents`: index.html .. _`design documentation`: design.html +.. _`admin guide`: admin_guide.html diff --git a/doc/default.css b/doc/default.css index bc29452..bd5b5ac 100644 --- a/doc/default.css +++ b/doc/default.css @@ -1,15 +1,15 @@ /* :Author: David Goodger :Contact: goodger@users.sourceforge.net -:date: $Date: 2004-03-26 06:11:32 $ -:version: $Revision: 1.11 $ +:date: $Date: 2004-06-09 00:25:32 $ +:version: $Revision: 1.13 $ :copyright: This stylesheet has been placed in the public domain. Default cascading style sheet for the HTML output of Docutils. */ a.target { - color: black } + color: blue } a.toc-backref { text-decoration: none ; @@ -25,12 +25,23 @@ div.abstract p.topic-title { font-weight: bold ; text-align: center } -div.attention, div.caution, div.danger, div.error, div.hint, -div.important, div.note, div.tip, div.warning { +div.attention, div.caution, div.danger, div.error, +div.important, div.tip, div.warning { margin: 2em ; border: medium outset ; padding: 1em } +div.hint, div.note { + font-size: 80%; + float: right; + width: 15em; + margin: 0.5em; + margin-left: 1em ; + border: solid #aaa; + background: #eee; + padding: 1em; +} + div.attention p.admonition-title, div.caution p.admonition-title, div.danger p.admonition-title, div.error p.admonition-title, div.warning p.admonition-title { @@ -75,8 +86,15 @@ div.system-message p.system-message-title { div.topic { margin: 2em } +h1 { + margin-top: 2em; + text-decoration: underline; +} + h1.title { - text-align: center } + text-align: center; + margin-top: .5em; +} h2.subtitle { text-align: center } diff --git a/doc/design.txt b/doc/design.txt index 8856d37..4cee32e 100644 --- a/doc/design.txt +++ b/doc/design.txt @@ -2,9 +2,7 @@ Roundup - An Issue-Tracking System for Knowledge Workers ======================================================== -:Authors: Ka-Ping Yee (original__), Richard Jones (implementation) - -__ spec.html +:Authors: Ka-Ping Yee (original), Richard Jones (implementation) .. contents:: @@ -14,10 +12,12 @@ Introduction This document presents a description of the components of the Roundup system and specifies their interfaces and behaviour in sufficient detail to guide an implementation. For the philosophy and rationale behind the -Roundup design, see the first-round Software Carpentry submission for -Roundup. This document fleshes out that design as well as specifying +Roundup design, see the first-round Software Carpentry `submission for +Roundup`__. This document fleshes out that design as well as specifying interfaces so that the components can be developed separately. +__ spec.html + The Layer Cake ----------------- @@ -50,13 +50,15 @@ Hyperdatabase ------------- The lowest-level component to be implemented is the hyperdatabase. The -hyperdatabase is intended to be a flexible data store that can hold -configurable data in records which we call items. +hyperdatabase is a flexible data store that can hold configurable data +in records which we call items. The hyperdatabase is implemented on top of the storage layer, an -external module for storing its data. The storage layer could be a -third-party RDBMS; for a "batteries-included" distribution, implementing -the hyperdatabase on the standard bsddb module is suggested. +external module for storing its data. The "batteries-includes" distribution +implements the hyperdatabase on the standard anydbm module. The storage +layer could be a third-party RDBMS; for a low-maintenance solution, +implementing the hyperdatabase on the SQLite RDBMS is suggested. + Dates and Date Arithmetic ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -450,27 +452,50 @@ Here is the interface provided by the hyperdatabase:: id is returned; otherwise a KeyError is raised. """ - def find(self, propname, itemid): + def find(self, **propspec): """Get the ids of items in this class which link to the given items. - 'propspec' consists of keyword args propname={itemid:1,} - 'propname' must be the name of a property in this class, or - a KeyError is raised. That property must be a Link or - Multilink property, or a TypeError is raised. + 'propspec' consists of keyword args propname=itemid or + propname={:1, : 1, ...} + 'propname' must be the name of a property in this class, + or a KeyError is raised. That property must + be a Link or Multilink property, or a TypeError + is raised. Any item in this class whose 'propname' property links to - any of the itemids will be returned. Used by the full text - indexing, which knows that "foo" occurs in msg1, msg3 and - file7, so we have hits on these issues: + any of the itemids will be returned. Examples:: + db.issue.find(messages='1') db.issue.find(messages={'1':1,'3':1}, files={'7':1}) """ def filter(self, search_matches, filterspec, sort, group): - """ Return a list of the ids of the active items in this - class that match the 'filter' spec, sorted by the group spec - and then the sort spec. + """Return a list of the ids of the active nodes in this class that + match the 'filter' spec, sorted by the group spec and then the + sort spec. + + "filterspec" is {propname: value(s)} + + "sort" and "group" are [(dir, prop), ...] where dir is '+', '-' + or None and prop is a prop name or None. Note that for + backward-compatibility reasons a single (dir, prop) tuple is + also allowed. + + "search_matches" is {nodeid: marker} + + The filter must match all properties specificed. If the property + value to match is a list: + + 1. String properties must match all elements in the list, and + 2. Other properties must match any of the elements in the list. + + The propname in filterspec and prop in a sort/group spec may be + transitive, i.e., it may contain properties of the form + link.link.link.name, e.g. you can search for all issues where + a message was added by a certain user in the last week with a + filterspec of + {'messages.author' : '42', 'messages.creation' : '.-1w;'} """ def list(self): @@ -574,7 +599,7 @@ practice:: 4 >>> db.issue.create(title="abuse", status=1) 5 - >>> hyperdb.Class(db, "user", username=hyperdb.Key(), + >>> hyperdb.Class(db, "user", username=hyperdb.String(), ... password=hyperdb.String()) >>> db.issue.addprop(fixer=hyperdb.Link("user")) @@ -726,12 +751,6 @@ hyperdatabase, except for the following changes and additional methods:: properties or "actor" cause a KeyError. """ - # New methods: - - def audit(self, event, detector): - def react(self, event, detector): - """Register a detector (see below for more details).""" - class IssueClass(Class): # Overridden methods: @@ -800,7 +819,7 @@ software bug tracker. The database is set up like this:: Class(db, "keyword", name=hyperdb.String()) Class(db, "issue", fixer=hyperdb.Multilink("user"), - topic=hyperdb.Multilink("keyword"), + keyword=hyperdb.Multilink("keyword"), priority=hyperdb.Link("priority"), status=hyperdb.Link("status")) @@ -852,20 +871,22 @@ The ``audit()`` and ``react()`` methods register detectors on a given class of items:: class Class: - def audit(self, event, detector): + def audit(self, event, detector, priority=100): """Register an auditor on this class. 'event' should be one of "create", "set", "retire", or "restore". 'detector' should be a function accepting four - arguments. + arguments. Detectors are called in priority order, execution + order is undefined for detectors with the same priority. """ - def react(self, event, detector): + def react(self, event, detector, priority=100): """Register a reactor on this class. 'event' should be one of "create", "set", "retire", or "restore". 'detector' should be a function accepting four - arguments. + arguments. Detectors are called in priority order, execution + order is undefined for detectors with the same priority. """ Auditors are called with the arguments:: @@ -1229,10 +1250,10 @@ An index view specifier looks like this (whitespace has been added for clarity):: /issue?status=unread,in-progress,resolved& - topic=security,ui& - :group=priority& + keyword=security,ui& + :group=priority,-status& :sort=-activity& - :filters=status,topic& + :filters=status,keyword& :columns=title,status,fixer @@ -1253,12 +1274,12 @@ of issues with values matching any specified Multilink properties. The example specifies an index of "issue" items. Only issues with a "status" of either "unread" or "in-progres" or "resolved" are displayed, -and only issues with "topic" values including both "security" and "ui" -are displayed. The issues are grouped by priority, arranged in -ascending order; and within groups, sorted by activity, arranged in -descending order. The filter section shows filters for the "status" and -"topic" properties, and the table includes columns for the "title", -"status", and "fixer" properties. +and only issues with "keyword" values including both "security" and "ui" +are displayed. The items are grouped by priority arranged in ascending +order and in descending order by status; and within groups, sorted by +activity, arranged in descending order. The filter section shows +filters for the "status" and "keyword" properties, and the table includes +columns for the "title", "status", and "fixer" properties. Associated with each issue class is a default layout specifier. The layout specifier in the above example is the default layout to be @@ -1382,11 +1403,12 @@ this path, and allow the multiple assignment of Roles to Users, and multiple Permissions to Roles. These definitions are not persistent - they're defined when the application initialises. -There will be two levels of Permission. The Class level permissions +There will be three levels of Permission. The Class level permissions define logical permissions associated with all items of a particular class (or all classes). The Item level permissions define logical permissions associated with specific items by way of their user-linked -properties. +properties. The Property level permissions define logical permissions +associated with a specific property of an item. Access Control Interface Specification @@ -1399,11 +1421,20 @@ The security module defines:: - name - description - klass (optional) + - properties (optional) + - check function (optional) The klass may be unset, indicating that this permission is not locked to a particular hyperdb class. There may be multiple Permissions for the same name for different classes. + + If property names are set, permission is restricted to those + properties only. + + If check function is set, permission is granted only when + the function returns value interpreted as boolean true. + The function is called with arguments db, userid, itemid. ''' class Role: @@ -1419,36 +1450,41 @@ The security module defines:: the base roles (for admin user). ''' - def getPermission(self, permission, classname=None): - ''' Find the Permission matching the name and for the class, - if the classname is specified. + def getPermission(self, permission, classname=None, properties=None, + check=None): + ''' Find the Permission exactly matching the name, class, + properties list and check function. Raise ValueError if there is no exact match. ''' - def hasPermission(self, permission, userid, classname=None): + def hasPermission(self, permission, userid, classname=None, + property=None, itemid=None): ''' Look through all the Roles, and hence Permissions, and - see if "permission" is there for the specified - classname. - ''' + see if "permission" exists given the constraints of + classname, property and itemid. - def hasItemPermission(self, classname, itemid, **propspec): - ''' Check the named properties of the given item to see if - the userid appears in them. If it does, then the user is - granted this permission check. + If classname is specified (and only classname) then the + search will match if there is *any* Permission for that + classname, even if the Permission has additional + constraints. - 'propspec' consists of a set of properties and values - that must be present on the given item for access to be - granted. + If property is specified, the Permission matched must have + either no properties listed or the property must appear in + the list. - If a property is a Link, the value must match the - property value. If a property is a Multilink, the value - must appear in the Multilink list. + If itemid is specified, the Permission matched must have + either no check function defined or the check function, + when invoked, must return a True value. + + Note that this functionality is actually implemented by the + Permission.test() method. ''' def addPermission(self, **propspec): ''' Create a new Permission with the properties defined in - 'propspec' + 'propspec'. See the Permission class for the possible + keyword args. ''' def addRole(self, **propspec): diff --git a/doc/developers.txt b/doc/developers.txt index d49dba3..a37347c 100644 --- a/doc/developers.txt +++ b/doc/developers.txt @@ -2,11 +2,12 @@ Developing Roundup ================== -:Version: $Revision: 1.6 $ +:Version: $Revision: 1.16 $ -Note: the intended audience of this document is the developers of the core -Roundup code. If you just wish to alter some behaviour of your Roundup -installation, see `customising roundup`_. +.. note:: + The intended audience of this document is the developers of the core + Roundup code. If you just wish to alter some behaviour of your Roundup + installation, see `customising roundup`_. .. contents:: @@ -26,8 +27,8 @@ All development is coordinated through two resources: Small Changes ------------- -Most small changes can be submitted through the Feature tracker, with patches -attached that give context diffs of the affected source. +Most small changes can be submitted through the `feature tracker`_, with +patches attached that give context diffs of the affected source. CVS Access @@ -43,7 +44,7 @@ CVS stuff: 1. to make a branch (eg. branching for code freeze/release):: - cvs co -d maint-0-5 -r release-0-5-0-pr1 + cvs co -d maint-0-5 -r release-0-5-0-pr1 roundup cd maint-0-5 cvs tag -b maint-0-5 @@ -87,7 +88,7 @@ relaxed sometimes). In short: - 80 column width code - 4-space indentations -- All modules must have a CVS Id line near the top, and a CVS Log at the end +- All modules must have a CVS Id line near the top Other project rules: @@ -107,6 +108,327 @@ consistently check in code which is either broken or takes the codebase in directions that have not been agreed to. +Debugging Aids +-------------- + +Try turning on logging of DEBUG level messages. This may be done a number +of ways, depending on what it is you're testing: + +1. If you're testing the database unit tests, then set the environment + variable ``LOGGING_LEVEL=DEBUG``. This may be done like so: + + LOGGING_LEVEL=DEBUG python run_tests.py + + This variable replaces the older HYPERDBDEBUG environment var. + +2. If you're testing a particular tracker, then set the logging level in + your tracker's ``config.ini``. + + +Internationalization Notes +-------------------------- + +How stuff works: + +1. Strings that may require translation (messages in human language) + are marked in the source code. This step is discussed in + `Marking Strings for Translation`_ section. + +2. These strings are all extracted into Message Template File + ``locale/roundup.pot`` (_`POT` file). See `Extracting Translatable + Messages`_ below. + +3. Language teams use POT file to make Message Files for national + languages (_`PO` files). All PO files for Roundup are kept in + the ``locale`` directory. Names of these files are target + locale names, usually just 2-letter language codes. `Translating + Messages`_ section of this chapter gives useful hints for + message translators. + +4. Translated Message Files are compiled into binary form (_`MO` files) + and stored in ``locale`` directory (but not kept in the `Roundup + CVS`_ repository, as they may be easily made from PO files). + See `Compiling Message Catalogs`_ section. + +5. Roundup installer creates runtime locale structure on the file + system, putting MO files in their appropriate places. + +6. Runtime internationalization (_`I18N`) services use these MO files + to translate program messages into language selected by current + Roundup user. Roundup command line interface uses locale name + set in OS environment variable ``LANGUAGE``, ``LC_ALL``, + ``LC_MESSAGES``, or ``LANG`` (in that order). Roundup Web User + Interface uses language selected by currently authenticated user. + +Additional details may be found in `GNU gettext`_ and Python `gettext +module`_ documentation. + +`Roundup source distribution`_ includes POT and PO files for message +translators, and also pre-built MO files to facilitate installations +from source. Roundup binary distribution includes MO files only. + +.. _GNU gettext: + +GNU gettext package +^^^^^^^^^^^^^^^^^^^ + +This chapter is full of references to GNU `gettext package`_. +GNU gettext is a "must have" for nearly all steps of internationalizing +any program, and it's manual is definetely a recommended reading +for people involved in `I18N`_. + +There are GNU gettext ports to all major OS platforms. +Windows binaries are available from `GNU mirror sites`_. + +Roundup does not use GNU gettext at runtime, but it's tools +are used for `extracting translatable messages`_, `compiling +message catalogs`_ and, optionally, for `translating messages`_. + +Note that ``gettext`` package in some OS distributions means just +runtime tools and libraries. In such cases gettext development tools +are usually distributed in separate package named ``gettext-devel``. + +Marking Strings for Translation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Strings that need translation must be marked in the source code. +Following subsections explain how this is done in different cases. + +If translatable string is used as a format string, it is recommended +to always use *named* format specifiers:: + + _('Index of %(classname)s') % locals() + +This helps translators to better understand the context of the +message and, with Python formatting, remove format specifier altogether +(which is sometimes useful, especially in singular cases of `Plural Forms`_). + +When there is more than one format specifier in the translatable +format string, named format specifiers **must** be used almost always, +because translation may require different order of items. + +It is better to *not* mark for translation strings that are not +locale-dependent, as this makes it more difficult to keep track +of translation completeness. For example, string ```` +(in ``index()`` method of the request handler in ``roundup_server`` +script) has no human readable parts at all, and needs no translations. +Such strings are left untranslated in PO files, and are reported +as such by PO status checkers (e.g. ``msgfmt --statistics``). + +Command Line Interfaces +~~~~~~~~~~~~~~~~~~~~~~~ + +Scripts and routines run from the command line use "static" language +defined by environment variables recognized by ``gettext`` module +from Python library (``LANGUAGE``, ``LC_ALL``, ``LC_MESSAGES``, and +``LANG``). Primarilly, these are ``roundup-admin`` script and +``admin.py`` module, but also help texts and startup error messages +in other scripts and their supporting modules. + +For these interfaces, Python ``gettext`` engine must be initialized +to use Roundup message catalogs. This is normally done by including +the following line in the module imports:: + + from i18n import _, ngettext + +Simple translations are automatically marked by calls to builtin +message translation function ``_()``:: + + print _("This message is translated") + +Translations for messages whose grammatical depends on a number +must be done by ``ngettext()`` function:: + + print ngettext("Nuked %i file", "Nuked %i files", number_of_files_nuked) + +Deferred Translations +~~~~~~~~~~~~~~~~~~~~~ + +Sometimes translatable strings appear in the source code in untranslated +form [#note_admin.py]_ and must be translated elsewhere. +Example:: + + for meal in ("spam", "egg", "beacon"): + print _(meal) + +In such cases, strings must be marked for translation without actual +call to the translating function. To mark these strings, we use Python +feature of automatic concatenation of adjacent strings and different +types of string quotes:: + + strings_to_translate = ( + ''"This string will be translated", + ""'me too', + ''r"\raw string", + ''""" + multiline string""" + ) + +.. [#note_admin.py] In current Roundup sources, this feature is + extensively used in the ``admin`` module using method docstrings + as help messages. + +Web User Interface +~~~~~~~~~~~~~~~~~~ + +For Web User Interface, translation services are provided by Client +object. Action classes have methods ``_()`` and ``gettext()``, +delegating translation to the Client instance. In HTML templates, +translator object is available as context variable ``i18n``. + +HTML templates have special markup for translatable strings. +The syntax for this markup is defined on `ZPTInternationalizationSupport`_ +page. Roundup translation service currently ignores values for +``i18n:domain``, ``i18n:source`` and ``i18n:target``. + +Template markup examples: + +* simplest case:: + +
+ Say + no + more! +
+ + this will result in msgid ``"Say no more!"``, with all leading and + trailing whitespace stripped, and inner blanks replaced with single + space character. + +* using variable slots:: + +
+ And now...
+ No.
+ THE LARCH +
+ + Msgid will be: ``"And now...
No.${slideNo}
THE LARCH"``. + Template rendering will use context variable ``number`` (you may use + any expression) to put instead of ``${slideNo}`` in translation. + +* attribute translation:: + +