String size

When dealing with Unicode strings, the concept of “size” is tricky, since the number of characters doesn't necessarily correspond to the number of bytes in the UTF-8 representation.

To prevent subtle errors, this library does not provide the PyStr_Size function.

Instead, use PyStr_AsUTF8AndSize.Under Python 3, this as an alias for PyUnicode_AsUTF8AndSize. Under Python 2, it functions like 3's PyUnicode_AsUTF8AndSize, except the string is not encoded (as it is not an Unicode string), the size pointer must not be NULL, and the size may be stored even if an error occurs.

PyModuleDef and PyModule_Create

Defining a module with this library is similar to the Python 3 way.

First, create a PyModuleDef structure:

static struct PyModuleDef moduledef = {
    PyModuleDef_HEAD_INIT,
    .m_name = "spam",
    .m_doc = PyDoc_STR("Python wrapper for the spam submodule."),
    .m_size = -1,
    .m_methods = spam_methods,
};

Then, where a Python 2 module would have

m = Py_InitModule3("spam", spam_methods, "Python wrapper ...");

use instead

m = PyModule_Create(&moduledef);

For m_size, use -1. (For the case of the module supporting multiple subinterpreters, 0 is also accepted, but this is tricky to achieve portably.) Additional members of the PyModuleDef structure are accepted, but unused under Python 2. Do not rely on them. See Python documentation for details.

Module creation entrypoint

Instead of the void init<name> function in Python 2, or a Python3-style PyObject *PyInit_<name> function, use the MODULE_INIT_FUNC macro to define an initialization function, and return the created module from it:

MODULE_INIT_FUNC(name)
{
    ...
    m = PyModule_Create(&moduledef);
    ...
    if (error) {
        return NULL;
    }
    ...
    return m;
}

Under Python 3, the macro expands to the PyInit_<name> function header (including a prototype, to squelch -Wmissing-prototypes warnings). For Python 2, it additionally defines an init<name> function that calls PyInit_<name> and discards the result.

Adding module-level constants

Samba includes code like this

PyModule_AddObject(m, "RDWR", PyInt_FromLong(O_RDWR));
PyModule_AddObject(m, "__version__", PyString_FromString(MOD_VERSION));

Python 2.6 introduced convenience functions for this, which are shorter to write

PyModule_AddIntConstant(m, "RDWR", O_RDWR)
PyModule_AddStringConstant(m, "__version__", MOD_VERSION)

PyModule_AddStringConstant will use the string type native to the current Python version.

ABI flags in library filename

In Python 3, extension libraries have a flag attached to the filename. Instead of spam.so, the library is named spam.cpython-34m.so. Files with the shorter "generic" name are loaded if the file specific to a given version does not exist. In addition to making it the target Python version clear, this allows for keeping modules for several Python versions in one directory.

Samba's build system uses this feature. Libraries with only the .so extension are built for Python 2 only. (This may change if and when Samba will officially support Python 3, or even its stable ABI.)

Building for Python 3

To build for Python 3, define the environment variable PYTHON=/usr/bin/python3 at configure time. This only works when building components that have been ported to Python 3.

Configuring with "--extra-python=/usr/bin/python3" will build a copy of the Python 3 bindings in addition to the ones for Python 2. These will get tested with make test.

To support this, changes to the wscript need to be made. In the configure step, use SAMBA_CHECK_PYTHON instead of calling find_program, check_tool and check_python_version indifidually.

Then, for each

bld.SAMBA_PYTHON('<modulename>',
                 ...,
                 realname='<modulename>.so')

create a corresponding call for the "extrapython" module:

if bld.env['EXTRA_PYTHON']:
    bld.SAMBA_PYTHON('extra-<modulename>',
                     ...,
                     realname='<modulename>.so',
                     extra_python=True)

For non-module libraries that uses Python runtime, i.e. "pyfeature='pyembed'", use the pyembed_libname helper to derive the filename. This name must then be used by all dependencies. For example:

name = bld.pyembed_libname('<modulename>')
bld.SAMBA_LIBRARY(name,
                  pyfeature='pyembed',
                  pc_files='<modulename>.pc',
                  ...)

Then, also build a version for "extrapython":

if bld.env['EXTRA_PYTHON']:
    name = bld.pyembed_libname('<modulename>', extrapython=True)
    bld.SAMBA_LIBRARY(name,
                      pyfeature='extrapyembed',
                      pc_files=None,
                      ...)

The @PYTHON_SO_ABI_FLAG@ conf variable is available for use in templates, such as .pc.in files for pkgconfig. It should be affixed to "pyembed" library names.

Testing Python libraries

The function samba_utils.RUN_PYTHON_TESTS will run given test scripts. When building for two Python versions, it'll test on both.