1 ==================
2 Developing Roundup
3 ==================
5 .. note::
6 The intended audience of this document is the developers of the core
7 Roundup code. If you just wish to alter some behaviour of your Roundup
8 installation, see `customising roundup`_.
10 .. contents::
12 Getting Started
13 ---------------
15 Anyone wishing to help in the development of Roundup must read `Roundup's
16 Design Document`_ and the `implementation notes`_.
18 All development is coordinated through two resources:
20 - roundup-dev mailing list at
21 http://lists.sourceforge.net/mailman/listinfo/roundup-devel
22 - Sourceforge's issue trackers at
23 https://sourceforge.net/tracker/?group_id=31577
25 Small Changes
26 -------------
28 Most small changes can be submitted through the `feature tracker`_, with
29 patches attached that give context diffs of the affected source.
32 CVS Access
33 ----------
35 To get CVS access, contact richard@users.sourceforge.net.
37 CVS stuff:
39 1. to tag a release (eg. the pre-release of 0.5.0)::
41 cvs tag release-0-5-0-pr1
43 1. to make a branch (eg. branching for code freeze/release)::
45 cvs co -d maint-0-5 -r release-0-5-0-pr1 roundup
46 cd maint-0-5
47 cvs tag -b maint-0-5
49 2. to check out a branch (eg. the maintenance branch for 0.5.x)::
51 cvs co -d maint-0-5 -r maint-0-5
53 3. to merge changes from the maintenance branch to the trunk, in the
54 directory containing the HEAD checkout::
56 cvs up -j maint-0-5
58 though this is highly discouraged, as it generally creates a whole swag
59 of conflicts :(
61 Standard tag names:
63 *release-maj-min-patch[-sub]*
64 Release of the major.minor.patch release, possibly a beta or pre-release,
65 in which case *sub* will be one of "b*N*" or "pr*N*".
66 *maint-maj-min*
67 Maintenance branch for the major.minor release. Patch releases are tagged in
68 this branch.
70 Typically, release happen like this:
72 1. work progresses in the HEAD branch until milestones are met,
73 2. a series of beta releases are tagged in the HEAD until the code is
74 stable enough to freeze,
75 3. the pre-release is tagged in the HEAD, with the resultant code branched
76 to the maintenance branch for that release,
77 4. bugs in the release are patched in the maintenance branch, and the final
78 and patch releases are tagged there, and
79 5. further major work happens in the HEAD.
81 Project Rules
82 -------------
84 Mostly the project follows Guido's Style (though naming tends to be a little
85 relaxed sometimes). In short:
87 - 80 column width code
88 - 4-space indentations
89 - All modules must have a CVS Id line near the top
91 Other project rules:
93 - New functionality must be documented, even briefly (so at least we know
94 where there's missing documentation) and changes to tracker configuration
95 must be logged in the upgrading document.
96 - subscribe to roundup-checkins to receive checkin notifications from the
97 other developers with CVS access
98 - discuss any changes with the other developers on roundup-dev. If nothing
99 else, this makes sure there's no rude shocks
100 - write unit tests for changes you make (where possible), and ensure that
101 all unit tests run before committing changes
102 - run pychecker over changed code
104 The administrators of the project reserve the right to boot developers who
105 consistently check in code which is either broken or takes the codebase in
106 directions that have not been agreed to.
109 Debugging Aids
110 --------------
112 Try turning on logging of DEBUG level messages. This may be done a number
113 of ways, depending on what it is you're testing:
115 1. If you're testing the database unit tests, then set the environment
116 variable ``LOGGING_LEVEL=DEBUG``. This may be done like so:
118 LOGGING_LEVEL=DEBUG python run_tests.py
120 This variable replaces the older HYPERDBDEBUG environment var.
122 2. If you're testing a particular tracker, then set the logging level in
123 your tracker's ``config.ini``.
126 Internationalization Notes
127 --------------------------
129 How stuff works:
131 1. Strings that may require translation (messages in human language)
132 are marked in the source code. This step is discussed in
133 `Marking Strings for Translation`_ section.
135 2. These strings are all extracted into Message Template File
136 ``locale/roundup.pot`` (_`POT` file). See `Extracting Translatable
137 Messages`_ below.
139 3. Language teams use POT file to make Message Files for national
140 languages (_`PO` files). All PO files for Roundup are kept in
141 the ``locale`` directory. Names of these files are target
142 locale names, usually just 2-letter language codes. `Translating
143 Messages`_ section of this chapter gives useful hints for
144 message translators.
146 4. Translated Message Files are compiled into binary form (_`MO` files)
147 and stored in ``locale`` directory (but not kept in the `Roundup
148 CVS`_ repository, as they may be easily made from PO files).
149 See `Compiling Message Catalogs`_ section.
151 5. Roundup installer creates runtime locale structure on the file
152 system, putting MO files in their appropriate places.
154 6. Runtime internationalization (_`I18N`) services use these MO files
155 to translate program messages into language selected by current
156 Roundup user. Roundup command line interface uses locale name
157 set in OS environment variable ``LANGUAGE``, ``LC_ALL``,
158 ``LC_MESSAGES``, or ``LANG`` (in that order). Roundup Web User
159 Interface uses language selected by currently authenticated user.
161 Additional details may be found in `GNU gettext`_ and Python `gettext
162 module`_ documentation.
164 `Roundup source distribution`_ includes POT and PO files for message
165 translators, and also pre-built MO files to facilitate installations
166 from source. Roundup binary distribution includes MO files only.
168 .. _GNU gettext:
170 GNU gettext package
171 ^^^^^^^^^^^^^^^^^^^
173 This chapter is full of references to GNU `gettext package`_.
174 GNU gettext is a "must have" for nearly all steps of internationalizing
175 any program, and it's manual is definetely a recommended reading
176 for people involved in `I18N`_.
178 There are GNU gettext ports to all major OS platforms.
179 Windows binaries are available from `GNU mirror sites`_.
181 Roundup does not use GNU gettext at runtime, but it's tools
182 are used for `extracting translatable messages`_, `compiling
183 message catalogs`_ and, optionally, for `translating messages`_.
185 Note that ``gettext`` package in some OS distributions means just
186 runtime tools and libraries. In such cases gettext development tools
187 are usually distributed in separate package named ``gettext-devel``.
189 Marking Strings for Translation
190 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
192 Strings that need translation must be marked in the source code.
193 Following subsections explain how this is done in different cases.
195 If translatable string is used as a format string, it is recommended
196 to always use *named* format specifiers::
198 _('Index of %(classname)s') % locals()
200 This helps translators to better understand the context of the
201 message and, with Python formatting, remove format specifier altogether
202 (which is sometimes useful, especially in singular cases of `Plural Forms`_).
204 When there is more than one format specifier in the translatable
205 format string, named format specifiers **must** be used almost always,
206 because translation may require different order of items.
208 It is better to *not* mark for translation strings that are not
209 locale-dependent, as this makes it more difficult to keep track
210 of translation completeness. For example, string ``</ol></body></html>``
211 (in ``index()`` method of the request handler in ``roundup_server``
212 script) has no human readable parts at all, and needs no translations.
213 Such strings are left untranslated in PO files, and are reported
214 as such by PO status checkers (e.g. ``msgfmt --statistics``).
216 Command Line Interfaces
217 ~~~~~~~~~~~~~~~~~~~~~~~
219 Scripts and routines run from the command line use "static" language
220 defined by environment variables recognized by ``gettext`` module
221 from Python library (``LANGUAGE``, ``LC_ALL``, ``LC_MESSAGES``, and
222 ``LANG``). Primarilly, these are ``roundup-admin`` script and
223 ``admin.py`` module, but also help texts and startup error messages
224 in other scripts and their supporting modules.
226 For these interfaces, Python ``gettext`` engine must be initialized
227 to use Roundup message catalogs. This is normally done by including
228 the following line in the module imports::
230 from i18n import _, ngettext
232 Simple translations are automatically marked by calls to builtin
233 message translation function ``_()``::
235 print _("This message is translated")
237 Translations for messages whose grammatical depends on a number
238 must be done by ``ngettext()`` function::
240 print ngettext("Nuked %i file", "Nuked %i files", number_of_files_nuked)
242 Deferred Translations
243 ~~~~~~~~~~~~~~~~~~~~~
245 Sometimes translatable strings appear in the source code in untranslated
246 form [#note_admin.py]_ and must be translated elsewhere.
247 Example::
249 for meal in ("spam", "egg", "beacon"):
250 print _(meal)
252 In such cases, strings must be marked for translation without actual
253 call to the translating function. To mark these strings, we use Python
254 feature of automatic concatenation of adjacent strings and different
255 types of string quotes::
257 strings_to_translate = (
258 ''"This string will be translated",
259 ""'me too',
260 ''r"\raw string",
261 ''"""
262 multiline string"""
263 )
265 .. [#note_admin.py] In current Roundup sources, this feature is
266 extensively used in the ``admin`` module using method docstrings
267 as help messages.
269 Web User Interface
270 ~~~~~~~~~~~~~~~~~~
272 For Web User Interface, translation services are provided by Client
273 object. Action classes have methods ``_()`` and ``gettext()``,
274 delegating translation to the Client instance. In HTML templates,
275 translator object is available as context variable ``i18n``.
277 HTML templates have special markup for translatable strings.
278 The syntax for this markup is defined on `ZPTInternationalizationSupport`_
279 page. Roundup translation service currently ignores values for
280 ``i18n:domain``, ``i18n:source`` and ``i18n:target``.
282 Template markup examples:
284 * simplest case::
286 <div i18n:translate="">
287 Say
288 no
289 more!
290 </div>
292 this will result in msgid ``"Say no more!"``, with all leading and
293 trailing whitespace stripped, and inner blanks replaced with single
294 space character.
296 * using variable slots::
298 <div i18n:translate="">
299 And now...<br/>
300 No.<span tal:replace="number" i18n:name="slideNo" /><br/>
301 THE LARCH
302 </div>
304 Msgid will be: ``"And now...<br /> No.${slideNo}<br /> THE LARCH"``.
305 Template rendering will use context variable ``number`` (you may use
306 any expression) to put instead of ``${slideNo}`` in translation.
308 * attribute translation::
310 <button name="btn_wink" value=" Wink " i18n:attributes="value" />
312 will translate the caption (and return value) for the "wink" button.
314 * explicit msgids. Sometimes it may be useful to specify msgid
315 for the element translation explicitely, like this::
317 <span i18n:translate="know what i mean?">this text is ignored</span>
319 When rendered, element contents will be replaced by translation
320 of the string specified in ``i18n:translate`` attribute.
322 * ``i18n`` in `TALES`_. You may translate strings in `TALES`_ python
323 expressions::
325 <span tal:replace="python: i18n.gettext('Oh, wicked.')" />
327 * plural forms. There is no markup for plural forms in `TAL`_ i18n.
328 You must use python expression for that::
330 <span tal:replace="python: i18n.ngettext(
331 'Oh but it\'s only %i shilling.',
332 'Oh but it\'s only %i shillings.',
333 fine) % fine"
334 />
336 Extracting Translatable Messages
337 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
339 The most common tool for message extraction is ``xgettext`` utility
340 from `GNU gettext package`_. Unfortunately, this utility has no means
341 of `Deferred Translations`_ in Python sources. There is ``xpot`` tool
342 from Francois Pinard free `PO utilities`_ that allows to mark strings
343 for deferred translations, but it does not handle `plural forms`_.
345 Roundup overcomes these limitations by using both of these utilities.
346 This means that you need both `GNU gettext`_ tools and `PO utilities`_
347 to build the Message Template File yourself.
349 Latest Message Template File is kept in `Roundup CVS`_ and distributed
350 with `Roundup Source`_. If you wish to rebuild the template yourself,
351 make sure that you have both ``xpot`` and ``xgettext`` installed and
352 just run ``gmake`` (or ``make``, if you are on a `GNU`_ system like
353 `linux`_ or `cygwin`_) in the ``locale`` directory.
355 For on-site i18n, Roundup provides command-line utility::
357 roundup-gettext <tracker_home>
359 extracting translatable messages from tracker's html templates.
360 This utility creates message template file ``messages.pot`` in
361 ``locale`` subdirectory of the tracker home directory. Translated
362 messages may be put in *locale*.po files (where *locale* is selected
363 locale name) in the same directory, e.g.: ``locale/ru.po``.
364 These message catalogs are searched prior to system-wide translations
365 kept in the ``share`` directory.
367 Translating Messages
368 ^^^^^^^^^^^^^^^^^^^^
370 Gettext Message File (`PO`_ file) is a plain text file, that can be created
371 by simple copying ``roundup.pot`` to new .po file, like this::
373 $ cp roundup.pot ru.po
375 The name of PO file is target locale name, usually just 2-letter language
376 code (``ru`` for Russian in the above example). Alternatively, PO file
377 may be initialized by ``msginit`` utility from `GNU gettext`_ tools::
379 $ msginit -i roundup.pot
381 ``msginit`` will check your current locale, and initialize the header
382 entry, setting language name, rules for `plural forms`_ and, if available,
383 translator's name and email address. The name for PO file is also chosen
384 based on current locale.
386 Next, you will need to edit this file, filling all ``msgstr`` lines with
387 translations of the above ``msgid`` entries. PO file is a plain text
388 file that can be edited with any text editor. However, there are several
389 tools that may help you with this process:
391 - `poEdit`_ by Vaclav Slavik. Very nice cross-platform GUI editor.
393 - `KBabel`_. Being part of `KDE`_, it works in X windows only.
394 At the first glance looks pretty hairy, with all bells and whistles.
395 Haven't had much experience with it, though.
397 - ``po-mode`` for `emacs`_. One of `GNU gettext`_ tools. Very handy,
398 definitely recommended if you are comfortable with emacs. Cannot
399 handle `plural forms`_ per se, but allows to edit them in simple
400 text mode.
402 - `po filetype plugin`_ for `vim`_. Does not do as much as ``po-mode``,
403 but helps in finding untranslated and fuzzy strings, and checking
404 code references. Please contact `alexander smishlajev`_ if you
405 prefer this, as i have patched this plugin a bit. I have also
406 informed the original plugin author about these changes, but got
407 no reply so far.
409 Compiling Message Catalogs
410 ^^^^^^^^^^^^^^^^^^^^^^^^^^
412 Message catalogs (`PO`_ files) must be compiled into binary form
413 (`MO`_ files) before they can be used in the application. This
414 compilation is handled by ``msgfmt`` utility from `GNU gettext`_
415 tools. ``GNUmakefile`` in the ``locale`` directory automatically
416 compiles all existing message catalogs after updating them from
417 Roundup source files. If you wish to rebuild an individual `MO`_
418 file without making everything else, you may, for example::
420 $ msgfmt --statistics -o ru.mo ru.po
422 This way, message translators can check their `PO`_ files without
423 extracting strings from source. (Note: String extraction requires
424 additional utility that is not part of `GNU gettext`_. See `Extracting
425 Translatable Messages`_.)
427 At run time, Roundup automatically compiles message catalogs whenever
428 `PO`_ file is changed.
430 .. _`Customising Roundup`: customizing.html
431 .. _`Roundup's Design Document`: spec.html
432 .. _`implementation notes`: implementation.html
435 .. _External hyperlink targets:
437 .. _alexander smishlajev:
438 .. _als: http://sourceforge.net/users/a1s/
439 .. _cygwin: http://www.cygwin.com/
440 .. _emacs: http://www.gnu.org/software/emacs/
441 .. _gettext package: http://www.gnu.org/software/gettext/
442 .. _gettext module: http://docs.python.org/lib/module-gettext.html
443 .. _GNU: http://www.gnu.org/
444 .. _GNU mirror sites: http://www.gnu.org/prep/ftp.html
445 .. _KBabel: http://i18n.kde.org/tools/kbabel/
446 .. _KDE: http://www.kde.org/
447 .. _linux: http://www.linux.org/
448 .. _Plural Forms:
449 http://www.gnu.org/software/gettext/manual/html_node/gettext_150.html
450 .. _po filetype plugin:
451 http://vim.sourceforge.net/scripts/script.php?script_id=695
452 .. _PO utilities: http://po-utils.progiciels-bpi.ca/
453 .. _poEdit: http://poedit.sourceforge.net/
454 .. _Roundup CVS: http://sourceforge.net/cvs/?group_id=31577
455 .. _Roundup Source:
456 .. _Roundup source distribution:
457 .. _Roundup binary distribution:
458 http://sourceforge.net/project/showfiles.php?group_id=31577
459 .. _TAL:
460 .. _Template Attribute Language:
461 http://dev.zope.org/Wikis/DevSite/Projects/ZPT/TAL%20Specification%201.4
462 .. _TALES:
463 .. _Template Attribute Language Expression Syntax:
464 http://dev.zope.org/Wikis/DevSite/Projects/ZPT/TALES%20Specification%201.3
465 .. _vim: http://www.vim.org/
466 .. _ZPTInternationalizationSupport: http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/ZPTInternationalizationSupport
467 .. _feature tracker: http://sourceforge.net/tracker/?group_id=31577&atid=402791