Chapter 9. Advanced packaging

Table of Contents

9.1. Historical perspective
9.2. Current trends
9.3. Note on build system
9.4. Continuous integration
9.5. Bootstrapping
9.6. Compiler hardening
9.7. Reproducible build
9.8. Substvar
9.9. Library package
9.10. Multiarch
9.11. Split of a Debian binary package
9.12. Package split scenario and examples
9.13. Multiarch library path
9.14. Multiarch header file path
9.15. Multiarch *.pc file path
9.16. Library symbols
9.17. Library package name
9.18. Library transition
9.19. binNMU safe
9.20. Debugging information
9.21. -dbgsym package
9.22. debconf

Let’s describe advanced topics on Debian packaging.

Let me oversimplify historical perspective of Debian packaging practices focused on the non-native packaging.

Debian was started in 1990s when upstream packages were available from public FTP sites such as Sunsite. In those early days, Debian packaging used Debian source format currently known as the Debian source format 1.0:

  • The Debian source package ships a set of files for the Debian source package.

    • package_version.orig.tar.gz : symlink to or copy of the upstream released file.
    • package_version-revision.diff.gz : One big patch for Debian modifications.
    • package_version-revision.dsc : package description.
  • Several workaround approaches such as dpatch, dbs, or cdbs were deplyoed to manage multiple topic patches.

The modern Debian source format 3.0 (quilt) was invented around 2008 (see ProjectsDebSrc3.0):

  • The Debian source package ships a set of files for the Debian source package.

    • package_version.orig.tar.?z : symlink to or copy of the upstream released file.
    • package_version-revision.debian.tar.?z : tarball of debian/ for Debian modifications.

      • The debian/source/format file contains 3.0 (quilt).
      • Optional multiple topic patches are stored in the debian/patches/ directory.
    • package_version-revision.dsc : package description.
  • The standarized approach to manage multiple topic patches using quilt(1) is deployed for the Debian source format 3.0 (quilt).

Most Debian packages adopted the Debian source formats 3.0 (quilt) and 3.0 (native).

Now, the git(1) is popular with upstream and Debian developers. The git and its associated tools are important part of the modern Debian packaging workflow. This modern workflow involving git will be mentioned later in Chapter 10, Packaging with git.

Current Debian packaging practices and their trends are moving target. See:

  • Debian Trends” — Hints for De facto standard of Debian practices

    • Build systems: dh
    • Debian source format: 3.0 (quilt)
    • VCS: git
    • VCS Hosting: salsa
    • Rules-Requires-Root: adopted, fakeroot
    • Copyright format: DEP-5
  • debhelper-compat-upgrade-checklist(7) manpage — Upgrade checklist for debhelper
  • DEP - Debian Enhancement Proposals — Formal proposals to enhance Debian

You can also search entire Debian source code data by yourself, too.

Auto-generated files of the build system may be found in the released upstream tarball. These should be regenerated when Debian package is build. E.g.:

  • dh $@ --with autoreconf should be used in the debian/rules if Autotools (autoconf + automake) are used.

Some modern build system may be able to download required source codes and binary files from arbitrary remote hosts to satisfy build requirements. Don’t use this download feature. The official Debian package is required to be build only with packages listed in Build-Depends: of the debian/control file.

The dh_auto_test(1) command is a debhelper command that tries to automatically run the test suite provided by the upstream developer during the Debian package building process.

The autopkgtest(1) command can be used after the Debian package building process. It tests generated Debian binary packages in the virtual environment using the debian/tests/control RFC822-style metadata file as continuous integration (CI). See:

There are several other CI tools on Debian for you to explore.

Debian cares about supporting new ports or flavours. The new ports or flavours require bootstrapping operation for the cross-build of the initial minimal native-building system. In order to avoid build-dependency loops during bootstrapping, the build-dependency needs to be reduced using the DEB_BUILD_PROFILES environment variable.

See Debian wiki: BuildProfileSpec.

[Tip]Tip

If a core package foo build depends on a package bar with deep build dependency chains but bar is only used in the test target in foo, you can safely mark the bar with <!nocheck> in the Build-depends of foo to avoid build loops.

The compiler hardening support spreading for Debian jessie (8.0) demands that we pay extra attention to the packaging.

You should read the following references in detail.

The debmake command adds template comments to the debian/rules file as needed for DEB_BUILD_MAINT_OPTIONS, DEB_CFLAGS_MAINT_APPEND, and DEB_LDFLAGS_MAINT_APPEND (see Chapter 5, Simple packaging and dpkg-buildflags(1)).

Here are some recommendations to attain a reproducible build result.

The control file source-name_source-version_arch.buildinfo generated by dpkg-genbuildinfo(1) records the build environment. See deb-buildinfo(5)

The debian/control file also defines the package dependency in which the variable substitutions mechanism” (substvar) may be used to free package maintainers from chores of tracking most of the simple package dependency cases. See deb-substvars(5).

The debmake command supports the following substvars:

  • ${misc:Depends} for all binary packages
  • ${misc:Pre-Depends} for all multiarch packages
  • ${shlibs:Depends} for all binary executable and library packages
  • ${python:Depends} for all Python packages
  • ${python3:Depends} for all Python3 packages
  • ${perl:Depends} for all Perl packages
  • ${ruby:Depends} for all Ruby packages

For the shared library, required libraries found simply by objdump -p /path/to/program | grep NEEDED are covered by the shlib substvar.

For Python and other interpreters, required modules found simply looking for lines with import, use, require, etc., are covered by the corresponding substvars.

For other programs which do not deploy their own substvars, the misc substvar covers their dependency.

For POSIX shell programs, there is no easy way to identify the dependency and no substvar covers their dependency.

For libraries and modules required via the dynamic loading mechanism including the GObject introspection mechanism, there is no easy way to identify the dependency and no substvar covers their dependency.

Packaging library software requires you to perform much more work than usual. Here are some reminders for packaging library software:

Before packaging shared library software, see:

For the historic background study, see:

Multiarch support for cross-architecture installation of binary packages (particularly i386 and amd64, but also other combinations) in the dpkg and apt packages introduced in Debian wheezy (7.0, May 2013), demands that we pay extra attention to packaging.

You should read the following references in detail.

The multiarch is enabled by using the <triplet> value such as i386-linux-gnu and x86_64-linux-gnu in the install path of shared libraries as /usr/lib/<triplet>/, etc..

  • The <triplet> value required internally by debhelper scripts is implicitly set in themselves. The maintainer doesn’t need to worry.
  • The <triplet> value used in override_dh_* target scripts must be explicitly set in the debian/rules file by the maintainer. The <triplet> value is stored in the $(DEB_HOST_MULTIARCH) variable in the following debian/rules snippet example:

    DEB_HOST_MULTIARCH = $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
    ...
    override_dh_install:
    	mkdir -p package1/lib/$(DEB_HOST_MULTIARCH)
    	cp -dR tmp/lib/. package1/lib/$(DEB_HOST_MULTIARCH)

See:

For well behaving build systems, the split of a Debian binary package into small ones can be realized as follows.

  • Create binary package entries for all binary packages in the debian/control file.
  • List all file paths (relative to debian/tmp) in the corresponding debian/binarypackage.install files.

Please check examples in this guide:

An intuitive and flexible method to create the initial template debian/control file defining the split of the Debian binary packages is accommodated with the -b option. See Section 15.2, “debmake -b.

Here are some typical multiarch package split scenarios for the following upstream source examples using the debmake command:

  • a library source libfoo-1.0.tar.gz
  • a tool source bar-1.0.tar.gz written in a compiled language
  • a tool source baz-1.0.tar.gz written in an interpreted language
binarypackagetypeArchitecture:Multi-Arch:Package content

libfoo1

lib*

any

same

the shared library, co-installable

libfoo-dev

dev*

any

same

the shared library header files etc., co-installable

libfoo-tools

bin*

any

foreign

the run-time support programs, not co-installable

libfoo-doc

doc*

all

foreign

the shared library documentation files

bar

bin*

any

foreign

the compiled program files, not co-installable

bar-doc

doc*

all

foreign

the documentation files for the program

baz

script

all

foreign

the interpreted program files

Debian policy requires to comply with the Filesystem Hierarchy Standard (FHS), version 3.0”, with the exceptions noted in File System Structure.

The most notable exception is the use of /usr/lib/<triplet>/ instead of /usr/lib<qual>/ (e.g., /lib32/ and /lib64/) to support a multiarch library.


For Autotools based packages under the debhelper package (compat>=9), this path setting is automatically taken care by the dh_auto_configure command.

For other packages with non-supported build systems, you need to manually adjust the install path as follows.

  • If ./configure is used in the override_dh_auto_configure target in debian/rules, make sure to replace it with dh_auto_configure -- while re-targeting the install path from /usr/lib/ to /usr/lib/$(DEB_HOST_MULTIARCH)/.
  • Replace all occurrences of /usr/lib/ with /usr/lib/*/ in debian/foo.install files.

All files installed simultaneously as the multiarch package to the same file path should have exactly the same file content. You must be careful with differences generated by the data byte order and by the compression algorithm.

The shared library files in the default path /usr/lib/ and /usr/lib/<triplet>/ are loaded automatically.

For shared library files in another path, the GCC option -l must be set by the pkg-config command to make them load properly.

GCC includes both /usr/include/ and /usr/include/<triplet>/ by default on the multiarch Debian system.

If the header file is not in those paths, the GCC option -I must be set by the pkg-config command to make "#include <foo.h>" work properly.


The use of the /usr/lib/<triplet>/packagename/ path for the library files allows the upstream maintainer to use the same install script for the multiatch system with /usr/lib/<triplet> and the biarch system with /usr/lib<qual>/. [19]

The use of the file path containing packagename enables having more than 2 development libraries simultaneously installed on a system.

The pkg-config program is used to retrieve information about installed libraries in the system. It stores its configuration parameters in the *.pc file and is used for setting the -I and -l options for GCC.


The symbols support in dpkg introduced in Debian lenny (5.0, May 2009) helps us to manage the backward ABI compatibility of the library package with the same package name. The DEBIAN/symbols file in the binary package provides the minimal version associated with each symbol.

An oversimplified method for the library packaging is as follows.

  • Extract the old DEBIAN/symbols file of the immediate previous binary package with the dpkg-deb -e command.

    • Alternatively, the mc command may be used to extract the DEBIAN/symbols file.
  • Copy it to the debian/binarypackage.symbols file.

    • If this is the first package, use an empty content file instead.
  • Build the binary package.

    • If the dpkg-gensymbols command warns about some new symbols:

      • Extract the updated DEBIAN/symbols file with the dpkg-deb -e command.
      • Trim the Debian revision such as -1 in it.
      • Copy it to the debian/binarypackage.symbols file.
      • Re-build the binary package.
    • If the dpkg-gensymbols command does not warn about new symbols:

      • You are done with the library packaging.

For the details, you should read the following primary references.

  • 8.6.3 The symbols system” of the Debian Policy Manual
  • dh_makeshlibs(1) manapage
  • dpkg-gensymbols(1) manapage
  • dpkg-shlibdeps(1) manapage
  • deb-symbols(5) manapage

You should also check:

[Tip]Tip

For C++ libraries and other cases where the tracking of symbols is problematic, follow 8.6.4 The shlibs system of the Debian Policy Manual, instead. Please make sure to erase the empty debian/binarypackage.symbols file generated by the debmake command. For this case, the DEBIAN/shlibs file is used.

Let’s consider that the upstream source tarball of the libfoo library is updated from libfoo-7.0.tar.gz to libfoo-8.0.tar.gz with a new SONAME major version which affects other packages.

The binary library package must be renamed from libfoo7 to libfoo8 to keep the unstable suite system working for all dependent packages after the upload of the package based on the libfoo-8.0.tar.gz.

[Warning]Warning

If the binary library package isn’t renamed, many dependent packages in the unstable suite become broken just after the library upload even if a binNMU upload is requested. The binNMU may not happen immediately after the upload due to several reasons.

The -dev package must follow one of the following naming rules:

[Tip]Tip

If the data encoding scheme changes (e.g., latin1 to utf-8), the same care as the API change needs to be taken.

See Section 9.9, “Library package”.

When you package a new library package version which affects other packages, you must file a transition bug report against the release.debian.org pseudo package using the reportbug command with the ben file and wait for the approval for its upload from the Release Team.

Release team has the transition tracker. See Transitions.

[Caution]Caution

Please make sure to rename binary packages as in Section 9.17, “Library package name”.

A binNMU” is a binary-only non-maintainer upload performed for library transitions etc. In a binNMU upload, only the Architecture: any packages are rebuilt with a suffixed version number (e.g. version 2.3.4-3 will become 2.3.4-3+b1). The Architecture: all packages are not built.

The dependency defined in the debian/control file among binary packages from the same source package should be safe for the binNMU. This needs attention if there are both Architecture: any and Architecture: all packages involved in it.

  • Architecture: any package: depends on Architecture: any foo package

    • Depends: foo (= ${binary:Version})
  • Architecture: any package: depends on Architecture: all bar package

    • Depends: bar (= ${source:Version})
  • Architecture: all package: depends on Architecture: any baz package

    • Depends: baz (>= ${source:Version}), baz (<< ${source:Version}.0~)

The Debian package is built with the debugging information but packaged into the binary package after stripping the debugging information as required by Chapter 10 - Files” of the Debian Policy Manual.

See

The debugging information is automatically packaged separately as the debug package using the dh_strip command with its default behavior. The name of such a debug package normally has the -dbgsym suffix.

  • The debian/rules file shouldn’t explicitly contain dh_strip.
  • Set the Build-Depends to debhelper-compat (>=13) while removing Build-Depends to debhelper in debian/control.

The debconf package enables us to configure packages during their installation in 2 main ways:

  • non-interactively from the debian-installer pre-seeding.
  • interactively from the menu interface (dialog, gnome, kde, …​)

    • the package installation: invoked by the dpkg command
    • the installed package: invoked by the dpkg-reconfigure command

All user interactions for the package installation must be handled by this debconf system using the following files.

  • debian/binarypackage.config

    • This is the debconf config script used for asking any questions necessary to configure the package.
  • debian/binarypackage.template

    • This is the debconf templates file used for asking any questions necessary to configure the package.

These debconf files are called by package configuration scripts in the binary Debian package

  • DEBIAN/binarypackage.preinst
  • DEBIAN/binarypackage.prerm
  • DEBIAN/binarypackage.postinst
  • DEBIAN/binarypackage.postrm

See dh_installdebconf(1), debconf(7), debconf-devel(7) and 3.9.1 Prompting in maintainer scripts” in the Debian Policy Manual.



[17] This document was written before the introduction of the symbols file.

[18] The strong preference is to use the SONAME versioned -dev package names over the single -dev package name in Chapter 6. Development (-DEV) packages, which does not seem to be shared by the former ftp-master (Steve Langasek). This document was written before the introduction of the multiarch system and the symbols file.

[19] This path is compliant with the FHS. Filesystem Hierarchy Standard: /usr/lib : Libraries for programming and packages states Applications may use a single subdirectory under /usr/lib. If an application uses a subdirectory, all architecture-dependent data exclusively used by the application must be placed within that subdirectory.