How to get a new feature into Docutils

Question:

I would very much like to have this new feature in the Docutils core. What exactly do I have to do to make this possible?

• Present your idea at the Docutils-develop mailing list,

  +1	usually triggers a fast response,
  -1	may be forgotten later,

  and/or file a feature request at the docutils tracker

  +1	there is a permanent record,
  -1	it may stay forgotten among all the other feature requests.

• Convince a Docutils developer that this is a valuable addition worth the effort.

• Contribute. The developers will make sure that the contribution fits nicely into Docutils (cf. the review criteria). This might involve discussing (and compromising on) design and implementation details. It might also lead to the conclusion that the addition fits better in the extensions and related projects.

• Be patient, and be persistent. None of us are paid to do this, it's all in our spare time -- which is precious and rare.

How to make code contributions that are easily accepted:

• Have a look at the Python coding conventions and documentation conventions below.

• Prepare test cases. This vastly helps when integrating new code. Test cases are also examples and showcases for new features. See Docutils Testing for a description of the test suite in docutils/test/.

  Ensure the addition works with all supported Python versions (2.4 ... 3.2).

• Look at the Docutils sources to see how similar features are implemented, learn to do it "the Docutils way".

• Prepare the a patch or an addition to the existing documentation. Include links.

• If you are familiar with version control, consider creating a feature branch in a Docutils repository checkout. (Working with branches is much easier with git. You can get a git clone of the repository from http://repo.or.cz/w/docutils.git or with git-svn.)

Python Coding Conventions

Contributed code will not be refused merely because it does not strictly adhere to these conditions; as long as it's internally consistent, clean, and correct, it probably will be accepted. But don't be surprised if the "offending" code gets fiddled over time to conform to these conventions.

The Docutils project shall follow the generic coding conventions as specified in the Style Guide for Python Code and Docstring Conventions PEPs, summarized, clarified, and extended as follows:

• 4 spaces per indentation level. No hard tabs.
• Use only 7-bit ASCII, no 8-bit strings. See Docutils Internationalization.
• No one-liner compound statements (i.e., no if x: return: use two lines & indentation), except for degenerate class or method definitions (i.e., class X: pass is OK.).
• Lines should be no more than 78 characters long.
• Use "StudlyCaps" for class names (except for element classes in docutils.nodes).
• Use "lowercase" or "lowercase_with_underscores" for function, method, and variable names. For short names, maximum two words, joined lowercase may be used (e.g. "tagname"). For long names with three or more words, or where it's hard to parse the split between two words, use lowercase_with_underscores (e.g., "note_explicit_target", "explicit_target"). If in doubt, use underscores.
• Avoid lambda expressions, which are inherently difficult to understand. Named functions are preferable and superior: they're faster (no run-time compilation), and well-chosen names serve to document and aid understanding.
• Avoid functional constructs (filter, map, etc.). Use list comprehensions instead.
• Avoid from __future__ import constructs. They are inappropriate for production code.
• Use 'single quotes' for string literals, and """triple double quotes""" for docstrings.

Documentation Conventions

• Docutils documentation is written using reStructuredText, of course.

• Use 7-bit ASCII if at all possible, and Unicode substitutions when necessary.

• Use the following section title adornment styles:

  ================
   Document Title
  ================
  
  --------------------------------------------
   Document Subtitle, or Major Division Title
  --------------------------------------------
  
  Section
  =======
  
  Subsection
  ----------
  
  Sub-Subsection
  ``````````````
  
  Sub-Sub-Subsection
  ..................
  

• Use two blank lines before each section/subsection/etc. title. One blank line is sufficient between immediately adjacent titles.

• Add a bibliographic field list immediately after the document title/subtitle. See the beginning of this document for an example.

• Add an Emacs "local variables" block in a comment at the end of the document. See the end of this document for an example.

Copyrights and Licensing

The majority of the Docutils project code and documentation has been placed in the public domain. Unless clearly and explicitly indicated otherwise, any patches (modifications to existing files) submitted to the project for inclusion (via Subversion, SourceForge trackers, mailing lists, or private email) are assumed to be in the public domain as well.

Any new files contributed to the project should clearly state their intentions regarding copyright, in one of the following ways:

• Public domain (preferred): include the statement "This module/document has been placed in the public domain."

• Copyright & open source license: include a copyright notice, along with either an embedded license statement, a reference to an accompanying license file, or a license URL.

  The license should be well known, simple and compatible with other open source software licenses. To keep the number of different licenses at a minimum, using the 2-Clause BSD license (also known as "Simplified" or FreeBSD license) is recommended.

The following licenses are described as simple and permissible on Various Licenses and Comments about Them by the GNU Project and listed as popular in OSI Approved Licenses:

Expat License / MIT License

http://www.jclark.com/xml/copying.txt http://www.opensource.org/licenses/MIT http://www.spdx.org/licenses/MIT

The term MIT License is ambiguous and may accurately refer to the Expat license or the X11 license [http://en.wikipedia.org/wiki/MIT_License]

-1	called MIT license by OSI and in the SPDX Open Source License Registry but Expat license by GNU

2-Clause BSD license / FreeBSD license

http://www.freebsd.org/copyright/freebsd-license.html http://www.opensource.org/licenses/BSD-2-Clause http://www.spdx.org/licenses/BSD-2-Clause

If you want a simple, permissive non-copyleft free software license, the FreeBSD license is a reasonable choice. However, please don't call it a “BSD” or “BSD-style” license, because that is likely to cause confusion which could lead to use of the flawed original BSD license.

—Various Licenses and Comments about Them

+1	license used by the closely related Sphinx project
+1	"2-Clause BSD license" is a non-ambiguous name, used by both, OSI and GNU.
+1	clear wording, structured text

Repository

Please see the repository documentation for details on how to access Docutils' Subversion repository. Anyone can access the repository anonymously. Only project developers can make changes. (If you would like to become a project developer, just ask!) Also see Setting Up For Docutils Development below for some useful info.

Unless you really really know what you're doing, please do not use svn import. It's quite easy to mess up the repository with an import.

Version Numbering

Docutils version numbering uses a major.minor.micro scheme (x.y.z; for example, 0.4.1).

Major releases (x.0, e.g. 1.0) will be rare, and will represent major changes in API, functionality, or commitment. For example, as long as the major version of Docutils is 0, it is to be considered experimental code. When Docutils reaches version 1.0, the major APIs will be considered frozen and backward compatibility will become of paramount importance.

Releases that change the minor number (x.y, e.g. 0.5) will be feature releases; new features from the Docutils core will be included.

Releases that change the micro number (x.y.z, e.g. 0.4.1) will be bug-fix releases. No new features will be introduced in these releases; only bug fixes off of maintenance branches will be included.

This policy was adopted in October 2005, and will take effect with Docutils version 0.4. Prior to version 0.4, Docutils didn't have an official version numbering policy, and micro releases contained both bug fixes and new features.

Snapshots

Snapshot tarballs will be generated regularly from

• the Docutils core, representing the current cutting-edge state of development;
• each active maintenance branch, for bug fixes;
• each development branch, representing the unstable seat-of-your-pants bleeding edge.

The sandbox/infrastructure/docutils-update shell script, run as an hourly cron job on the BerliOS server, is responsible for automatically generating the snapshots and updating the web site. See the web site docs.

Setting Up For Docutils Development

When making changes to the code, testing is a must. The code should be run to verify that it produces the expected results, and the entire test suite should be run too. The modified Docutils code has to be accessible to Python for the tests to have any meaning. There are two ways to keep the Docutils code accessible during development:

1. Update your PYTHONPATH environment variable so that Python picks up your local working copy of the code. This is the recommended method.

   We'll assume that the Docutils trunk is checked out under your ~/projects/ directory as follows:

   svn co https://<user>@docutils.svn.sourceforge.net/svnroot/docutils/trunk \
       docutils
   

   For the bash shell, add this to your ~/.profile:

   PYTHONPATH=$HOME/projects/docutils/docutils
   PYTHONPATH=$PYTHONPATH:$HOME/projects/docutils/docutils/extras
   export PYTHONPATH
   

   The first line points to the directory containing the docutils package. The second line adds the directory containing the third-party modules Docutils depends on. The third line exports this environment variable. You may also wish to add the tools directory to your PATH:

   PATH=$PATH:$HOME/projects/docutils/docutils/tools
   export PATH
   

2. Before you run anything, every time you make a change, reinstall Docutils:

   python setup.py install
   

   Caution!

   This method is not recommended for day-to-day development; it's too easy to forget. Confusion inevitably ensues.

   If you install Docutils this way, Python will always pick up the last-installed copy of the code. If you ever forget to reinstall the "docutils" package, Python won't see your latest changes.

A useful addition to the docutils top-level directory in branches and alternate copies of the code is a set-PATHS file containing the following lines:

# source this file
export PYTHONPATH=$PWD:$PWD/extras
export PATH=$PWD/tools:$PATH

Open a shell for this branch, cd to the docutils top-level directory, and "source" this file. For example, using the bash shell:

$ cd some-branch/docutils
$ . set-PATHS

Mailing Lists

Developers are recommended to subscribe to all Docutils mailing lists.

The Wiki

There is a development wiki at http://docutils.python-hosting.com/ as a scratchpad for transient notes. Please use the repository for permament document storage.

Extensions and Related Projects

sandbox/
    project_name/ # For a collaborative project.
        README.txt  # Describe the requirements, purpose/goals, usage,
                    # and list the maintainers.
        docs/
            ...
        component.py    # The component is a single module.
                        # *OR* (but *not* both)
        component/      # The component is a package.
            __init__.py  # Contains the Reader/Writer class.
            other1.py    # Other modules and data files used
            data.txt     # by this component.
            ...
        test/       # Test suite.
            ...
        tools/      # For front ends etc.
            ...
        setup.py    # Use Distutils to install the component
                    # code and tools/ files into the right
                    # places in Docutils.
    userid/       # For *temporary* personal space.