How to merge a package

Merging is the process of taking all Ubuntu changes made on top of one Debian version of a package, and re-doing them on top of a new Debian version of the package. See Merges & syncs for a more context information.

See the Ubuntu wiki for a more detailed workflow. This guide is intended to cover the majority of use cases.

For a list of packages that have been changed in Debian, but not merged into Ubuntu, see the Merge-o-Matic tool:

Overview

Merging is done using the git-ubuntu tool. As such, the process in many ways follows that of a git rebase where commits from one point are replayed on top of another point:

        gitGraph
   commit id: "something 1.2"
   branch 1.2ubuntu
   checkout 1.2ubuntu
   commit id: "Ubuntu changes a, b, c on 1.2"
   commit id: "1.2ubuntu1"
   checkout main
   commit id: "something 1.3"
   branch 1.3ubuntu
   checkout 1.3ubuntu
   commit id: "Ubuntu changes a, b, c on 1.3"
   commit id: "1.3ubuntu1"

git-ubuntu process overview

At a more detailed level, there are other sub-tasks to be done, such as:

  • Splitting out large “omnibus” style commits into smaller logical units (one commit per logical unit).

  • Harmonizing debian/changelog commits into two commits: a changelog merge and a reconstruction.

With this process, we keep the Ubuntu version of a package cleanly applied to the end of the latest Debian version and make it easy to drop changes as they become redundant.

Process steps

Manual steps:

Preliminary steps

Decide on a merge candidate

First, check if a newer version is available from Debian. Use the rmadison(1) tool:

$ rmadison <package>
$ rmadison -u debian <package>

Example:

$ rmadison at
 at | 3.1.13-1ubuntu1   | precise | source, amd64, armel, armhf, i386, powerpc
 at | 3.1.14-1ubuntu1   | trusty  | source, amd64, arm64, armhf, i386, powerpc, ppc64el
 at | 3.1.18-2ubuntu1   | xenial  | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
 at | 3.1.20-3.1ubuntu2 | bionic  | source, amd64, arm64, armhf, i386, ppc64el, s390x
 at | 3.1.20-3.1ubuntu2 | cosmic  | source, amd64, arm64, armhf, i386, ppc64el, s390x
 at | 3.1.20-3.1ubuntu2 | disco   | source, amd64, arm64, armhf, i386, ppc64el, s390x
$ rmadison -u debian at
at         | 3.1.13-2+deb7u1 | oldoldstable       | source, amd64, armel, armhf, i386, ia64, kfreebsd-amd64, kfreebsd-i386, mips, mipsel, powerpc, s390, s390x, sparc
at         | 3.1.16-1        | oldstable          | source, amd64, arm64, armel, armhf, i386, mips, mipsel, powerpc, ppc64el, s390x
at         | 3.1.16-1        | oldstable-kfreebsd | source, kfreebsd-amd64, kfreebsd-i386
at         | 3.1.20-3        | stable             | source, amd64, arm64, armel, armhf, i386, mips, mips64el, mipsel, ppc64el, s390x
at         | 3.1.23-1        | testing            | source, amd64, arm64, armel, armhf, i386, mips, mips64el, mipsel, ppc64el, s390x
at         | 3.1.23-1        | unstable           | source, amd64, arm64, armel, armhf, hurd-i386, i386, kfreebsd-amd64, kfreebsd-i386, mips, mips64el, mipsel, ppc64el, s390x
at         | 3.1.23-1        | unstable-debug     | source

You’re be merging from Debian unstable, which in this example is 3.1.23-1.

Check existing bug entries

Check for any low-hanging fruit in the Debian or Ubuntu bug list that can be wrapped into this merge.

  • Ubuntu bug tracker: https://bugs.launchpad.net/ubuntu/+source/<package>

  • Debian bug tracker: https://tracker.debian.org/pkg/<package>

If there are bugs you’d like to fix, make a new SRU-style commit at the end of the merge process and put them together in the same merge proposal. This process is described in the Adding new changes section.

Make a bug report for the merge

Many regular Ubuntu team merges are pre-planned and likely already exist as bugs, or can be found in the team merge schedule.

Merges can also be picked up from Merge-o-Matic, weekly Merge Opportunities Reports (e.g. the Ubuntu Server report), or through awareness being raised for other reasons.

If there is no obvious pre-created bug yet, check if there is an existing merge request bug entry in Launchpad. If you don’t find one, create one to avoid duplicate efforts and to allow coordination.

To do so, go to the package’s Launchpad page:

https://bugs.launchpad.net/ubuntu/+source/<package>

From there, create a new bug report requesting a merge.

Example bug:

URL:

https://bugs.launchpad.net/ubuntu/+source/at/+filebug

Summary:

“Please merge 3.1.23-1 into noble”

Description:

“tracking bug”

Result:

https://bugs.launchpad.net/ubuntu/+source/at/+bug/1802914

Set the bug status to “in-progress” and assign it to yourself.

To let people who only use Merge-o-Matic know, go to the summary page (for example, universe), and if the package is listed there, leave a comment linking to the bug.

This way, those not studying the LP bugs discover more easily that there is already a bug filed for that merge. To do so:

  • Click in the Comment column on the invisible text entry field.

  • Leave a comment like bug #123456 and press Enter.

  • The page updates and links to your bug.

Important

Save the bug report number, because you’ll be using it throughout the merge process.

Get the package repository

Cloning the repository is the start of all further interactions. If you have already cloned the repository, update it to ensure you have the newest content before taking any further action.

Clone the package repository

$ git ubuntu clone <package> [<package>-gu]

Example:

$ git ubuntu clone at at-gu

It’s a good idea to append some git-ubuntu specific label (like -gu) to distinguish it from clones of Debian or upstream Git repositories (which tend to want to clone as the same name).

Update the package repository

Since this is just Git, the best way to update the git-ubuntu-based content (and any other remotes) is to update them all before going into the merge process.

$ git fetch --all

The merge process

Start a Git Ubuntu merge

From within the Git source tree:

$ git ubuntu merge start pkg/ubuntu/devel

This generates the following tags for you:

Tag

Source

old/ubuntu

ubuntu/devel

old/debian

last import tag prior to old/ubuntu without ubuntu suffix in version

new/debian

debian/sid

If git ubuntu merge start fails, Start a merge manually.

Make a merge branch

Use the merge tracking bug and the current Ubuntu devel version it’s going into (in the example of doing a merge below, the current Ubuntu devel was disco, and the merge bug for the case was LP #1802914).

$ git checkout -b merge-lp1802914-disco

If there’s no merge bug, the Debian package version you’re merging into can be used (for example, merge-3.1.23-1-disco).

Empty directories warning

A message like the following one when making the merge branch indicates a problem with empty directories:

$ git checkout -b merge-augeas-mirespace-testing
Switched to a new branch 'merge-augeas-mirespace-testing'

WARNING: empty directories exist but are not tracked by git:

tests/root/etc/postfix
tests/root/etc/xinetd.d/arch

These will silently disappear on commit, causing extraneous
unintended changes. See: LP: #1687057.

These empty directories can cause the rich history to become lost when uploading them to the Archive. When that happens, use a workaround: Empty directories.

Split commits

In this phase, split old-style commits that grouped multiple changes together.

Check if there are commits to split

$ git log --oneline

2af0cb7 (HEAD -> merge-3.1.20-6-disco, tag: reconstruct/3.1.20-3.1ubuntu2, tag: split/3.1.20-3.1ubuntu2) import patches-unapplied version 3.1.20-3.1ubuntu2 to ubuntu/disco-proposed
2a71755 (tag: pkg/import/3.1.20-5) Import patches-unapplied version 3.1.20-5 to debian/sid
9c3cf29 (tag: pkg/import/3.1.20-3.1) Import patches-unapplied version 3.1.20-3.1 to debian/sid
...

Get all commit hashes since old/debian and check the summary for what they changed using:

$ git log --stat old/debian..

Example (from merging the heimdal package):

$ git log --stat old/debian..

commit 9fc91638b0a50392eb9f79d45d68bc5ac6cd6944 (HEAD ->
merge-7.8.git20221117.28daf24+dfsg-1-lunar)
Author: Michal Maloszewski <[email protected]>
Date:   Tue Jan 17 16:16:01 2023 +0100

    Changelog for 7.8.git20221117.28daf24+dfsg-1

 debian/changelog | 1 -
 1 file changed, 1 deletion(-)


commit e217fae2dc54a0a13e4ac5397ec7d3be527fa243
Author: Michal Maloszewski <[email protected]>
Date:   Tue Jan 17 16:13:49 2023 +0100

    update-maintainer

 debian/control | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


commit 3c66d873330dd594d593d21870f4700b5e7fd153
Author: Michal Maloszewski <[email protected]>
Date:   Tue Jan 17 16:13:49 2023 +0100

    reconstruct-changelog

 debian/changelog | 10 ++++++++++
 1 file changed, 10 insertions(+)


commit 58b895f5ff6333b1a0956dd83e478542dc7a10d3
Author: Michal Maloszewski <[email protected]>
Date:   Tue Jan 17 16:13:46 2023 +0100

    merge-changelogs

 debian/changelog | 68
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

This command shows the specific commit, as well as what changed within the commit (i.e., how many files were changed and how many insertions and deletions there were).

Common examples of commits that need to be split:

  • changelog with any other file(s) changed in a single commit

  • debian/changelog with any other file(s) changed in a single commit

  • commit named: Import patches-unapplied version 1.2.3ubuntu4 to ubuntu/cosmic-proposed, where it’s applying from an Ubuntu source rather than a Debian one (in this case ubuntu4)

You should still look over all commits just to make sure.

If there are no commits to split, add the “split” tag and continue to Prepare the logical view.

Identify logical changes

The next step is to separate the changes into logical units. For the at package, this is trivial: put the changelog change in one commit and the control change in the other.

The second example, for nspr, is more instructive. Here we have 5 files changed that need to be split out:

  • All changelog changes go to one commit called changelog.

  • Update maintainer (in debian/control) goes to one commit called update maintainers.

  • All other logically separable commits go into individual commits.

Look in debian/changelog:

nspr (2:4.18-1ubuntu1) bionic; urgency=medium

  * Resynchronize with Debian, remaining changes
    - rules: Enable Thumb2 build on armel, armhf.
    - d/p/fix_test_errcodes_for_runpath.patch: Fix testcases to handle
      zesty linker default changing to --enable-new-dtags for -rpath.

There are two logical changes, which you need to separate. Look at the changes in individual files to see which file changes should be logically grouped together.

Example:

$ git show d7ebe661 -- debian/rules

In this case, the following file changes are separated into logical units:

File(s)

Logical unit

debian/rules

Enable Thumb2 build on armel, armhf

debian/patches/*

Fix test cases to handle zesty linker default
changing to --enable-new-dtags for -rpath

debian/control

Change maintainer

debian/changelog

Changelog

Split into logical commits

Start a rebase at old/debian, and then reset to HEAD^ to bring back the changes as uncommitted changes.

  1. Start a rebase: git rebase -i old/debian

  2. Change the commit(s) you’re going to separate from pick to edit

  3. Do a git reset to get your changes back: git reset HEAD^

Next, add the commits:

Logical unit:

$ git add debian/patches/*
$ git commit

Commit message:

  * d/p/fix_test_errcodes_for_runpath.patch: Fix testcases to handle
    zesty linker default changing to --enable-new-dtags for -rpath.

Logical unit:

$ git add debian/rules
$ git commit

Commit message:

  * d/rules: Enable Thumb2 build on armel, armhf.

Maintainers:

$ git commit -m "update maintainers" debian/control

Changelog:

$ git commit -m changelog debian/changelog

Finally, complete the rebase:

$ git rebase --continue

The result of this rebase should be a sequence of smaller commits, one per debian/changelog entry (with potentially additional commits for previously undocumented changes).

It should represent a granular history (viewable with git log) for the latest Ubuntu version and no content differences to that Ubuntu version. This can be verified with git diff -p old/ubuntu.

Tag split

Do this even if there were no commits to split:

$ git ubuntu tag --split

Purpose of logical tags

If we do this step, we have a distinct boundary between looking at the past (analysis of what is there already) and the actual work we want to perform (bringing the old Ubuntu delta forward).

By having a well-defined point, it is easier to do the future work, and also for a reviewer to start from the same point. The reviewer can easily and almost completely automatically determine if the logical tag is correct.

The logical tag is the cleanest possible representation of a previous Ubuntu delta. By determining this representation, we make it as easy as possible to bring the delta forward.

Prepare the logical view

In this phase, make a clean, “logical” view of the history. This history is cleaned up (but has the same delta) and only contains the actual changes that affect the package’s behavior.

Start with a rebase from old/debian:

$ git rebase -i old/debian

Do some cleaning:

  • Delete imports, etc.

  • Delete any commit that only changes metadata like changelog or maintainer.

  • Possibly rearrange commits if it makes logical sense.

Squash these kinds of commits together:

  • Changes and reversions of those changes because they result in no change.

  • Multiple changes to the same patch file because they should be a logical unit.

To squash a commit, move its line underneath the one you want it to become part of, and then change it from pick to fixup.

Check the result

At the end of the “squash and clean” phase, the only delta you should see from the split tag is:

$ git diff --stat split/6.8-0ubuntu2
 debian/changelog | 31 -------------------------------
 debian/control   |  3 +--
 2 files changed, 1 insertion(+), 33 deletions(-)

Only changelog and control were changed, which is what we want.

Create logical tag

What is the logical tag? It is a representation of the Ubuntu delta present against a specific historical package version in Ubuntu.

$ git ubuntu tag --logical

This may fail with an error like:

ERROR:HEAD is not a defined object in this git repository.

In which case, Create logical tag manually.

Rebase onto new Debian

$ git rebase -i --onto new/debian old/debian

Conflicts

If a conflict occurs, you must resolve them. Do so by modifying the conflicting commit during the rebase.

For example, merging logwatch 7.5.0-1:

$ git rebase -i --onto new/debian old/debian
...
CONFLICT (content): Merge conflict in debian/control
error: could not apply c0efd06... - Drop libsys-cpu-perl and libsys-meminfo-perl from Recommends to
...

Look at the conflict in debian/control:

    <<<<<<< HEAD
    Recommends: libdate-manip-perl, libsys-cpu-perl, libsys-meminfo-perl
    =======
    Recommends: libdate-manip-perl
    Suggests: fortune-mod, libsys-cpu-perl, libsys-meminfo-perl
    >>>>>>> c0efd06... - Drop libsys-cpu-perl and libsys-meminfo-perl from Recommends to

Upstream removed fortune-mod and deleted the entire line because it was no longer needed. Resolve it to:

Recommends: libdate-manip-perl
Suggests: libsys-cpu-perl, libsys-meminfo-perl

Continue with the rebase:

$ git add debian/control
$ git rebase --continue

Corollaries

Mistake corrections are squashed.

Changes that fix mistakes made previously in the same delta are squashed against them. For example:

  • 2.3-4ubuntu1 was the previous merge.

  • 2.3-4ubuntu2 adjusted debian/rules to add the --build-beter configure flag.

  • 2.3-4ubuntu3 fixed the typo in debian/rules to say --build-better instead.

  • When the logical tag is created, there is one commit relating to --build-better, which omits any mention of the typo.

Note

If a mistake exists in the delta itself, it is retained. For example, if 2.3-4ubuntu3 was never uploaded and the typo is still present in 2.3-4ubuntu2, then logical/2.3.-4ubuntu2 should contain a commit adding the configure flag with the typo still present.

Empty commits

If a commit becomes empty, it’s because the change has already been applied upstream:

The previous cherry-pick is now empty, possibly due to conflict resolution.

In such a case, drop the commit:

$ git rebase --abort
$ git rebase -i old/debian

Keep a copy of the redundant commit’s commit message, then delete it in the rebase.

Sync request

If all the commits are empty, or you realized there are no logical changes, you’re facing a sync request, not a merge. Refer to the sync guidelines to continue.

Check patches still apply cleanly

$ quilt push -a --fuzz=0

If quilt fails

Quilt can fail at this point if the file being patched has changed significantly upstream. The most common reason is that the issue the patch addresses has since been fixed upstream.

For example:

$ quilt push -a --fuzz=0
...
Applying patch ssh-ignore-disconnected.patch
patching file scripts/services/sshd
Hunk #1 FAILED at 297.
1 out of 1 hunk FAILED -- rejects in file scripts/services/sshd
Patch ssh-ignore-disconnected.patch does not apply (enforce with -f)

If this patch fails because the changes in ssh-ignore-disconnected.patch are already applied upstream, you must remove this patch.

$ git log --oneline

1aed93f (HEAD -> ubuntu/devel)   * d/p/ssh-ignore-disconnected.patch: [sshd] ignore disconnected from user     USER (LP: 1644057)
7d9d752 - Drop libsys-cpu-perl and libsys-meminfo-perl from Recommends to   Suggests as they are in universe.

Removing 1aed93f removes the patch.

  1. Save the commit message from 1aed93f for later including it in the Drop Changes section of the new changelog entry.

  2. git rebase -i 7d9d752 and delete commit 1aed93f.

Unapply patches before continuing

$ quilt pop -a

Adding new changes

Add any new changes you want to include with the merge. For instance, the new package version may fail to build from source (FTBFS) in Ubuntu due to new versions of specific libraries or runtimes.

Each logical change should be in its own commit to match the work done up to this point on splitting the logical changes.

Moreover, there is no need to add changelog entries for these changes manually. They are generated from the commit messages with the merge finish process described below.

Finish the merge

$ git ubuntu merge finish pkg/ubuntu/devel

If this fails, Finish the merge manually.

Fix the changelog

git-ubuntu attempts to put together a changelog entry, but it will likely have problems. Fix it to make sure it follows the standards. See committing your changes for information about what it should look like.

Add dropped changes

If you dropped any changes (due to upstream fixes), you must note them in the changelog entry:

  * Drop Changes:
    - Foo: change to bar
      [Fixed in 1.2.3-4]

Format any new added changes

If you added any new changes, they should be in their own section in the changelog:

  * New Changes:
    - Bar: change to foo
    - Baz: adjust for Foo changes

Commit the changelog fix

$ git commit debian/changelog -m changelog

No changes to debian/changelog

The range old/ubuntu..logical/<version> should contain no changes to debian/changelog at all. We do not consider this part of the logical delta. So, any commits that contain only changes to debian/changelog should be dropped.

Note

If you diff your final logical tag against the Ubuntu package it analyses, the diff should be empty, except:

  1. All changes to debian/changelog:

    We deliberately exclude these from the logical tag, relying on commit messages instead.

  2. The change that update-maintainer introduced, and (rarely) similar changes like a change to Vcs-Git headers to point to an Ubuntu VCS instead.

    For the purposes of this workflow, these are not considered part of our “logical delta”, and instead are re-added at the end.

Tip

You can use the execsnoop-bpfcc tool from the bpfcc-tools package to find what debhelper scripts were called for a certain package. This is helpful for debugging what scripts were called, and what parameters were passed to them.

For example:

$ sudo execsnoop-bpfcc -n multipath

Now in another shell run:

$ sudo apt install --reinstall multipath-tools

In the original shell, you should see something like:

PCOMM            PID     PPID    RET ARGS
multipath-tools  13939   13931     0 /var/lib/dpkg/info/multipath-tools.prerm upgrade 0.9.4-5ubuntu3
multipath-tools  13951   13931     0 /var/lib/dpkg/info/multipath-tools.postrm upgrade 0.9.4-5ubuntu3
multipath-tools  13959   13956     0 /var/lib/dpkg/info/multipath-tools.postinst configure 0.9.4-5ubuntu3
multipathd       14009   1         0 /sbin/multipathd -d -s

A brief summary of this phase (cheat sheet)

  1. rmadison <package_name>

  2. rmadison -u debian <package_name>

  3. git ubuntu clone <package_name> <package_name>-gu

  4. cd <package_name>-gu

  5. git ubuntu merge start pkg/ubuntu/devel

  6. git checkout -b merge-<version_of_debian_unstable>-<current_ubuntu_devel_name>

  7. git log --stat old/debian..

  8. git ubuntu tag --split (if there’s nothing to split, type that command straight away)

  9. git rebase -i old/debian

  10. Drop metadata changes and reorder/merge/split commits.

  11. git diff split/

  12. git ubuntu tag --logical

  13. git show logical/<version> (check if the new tag exists)

  14. git rebase -i --onto new/debian old/debian

  15. quilt push -a --fuzz=0

  16. quilt pop -a

  17. git ubuntu merge finish pkg/ubuntu/devel

Upload a PPA

Get the orig tarball

Ubuntu doesn’t know about the new tarball yet, so we must create it.

$ git ubuntu export-orig

If the upstream version does not yet exist in Ubuntu, that is, the new package from Debian also includes a new upstream version, you should add the --for-merge option:

$ git ubuntu export-orig --for-merge

If this fails, do it manually.

Build source package

$ dpkg-buildpackage \
    --build=source \
    --no-pre-clean \
    --no-check-builddeps \
    -sa \
    -v3.1.20-3.1ubuntu2

The switches are:

-sa

include orig tarball (required on a merge)

-vXYZ

include changelog since XYZ

As the merge upload represents all changes that happened in Debian since the last merge plus anything added as part of the merge itself, -v should usually point to the last published Ubuntu version. For example:

  1. Ubuntu merged as 1.3-1ubuntu1.

  2. Then Ubuntu had a fix in 1.3-1ubuntu2.

  3. But in the meantime, Debian merged upstream as 1.4-1.

  4. And then Debian added a fix in 1.4-2.

  5. New Ubuntu will be 1.4-2ubuntu1.

  6. -v should be set to 1.3-1ubuntu2.

    1. Thereby the .changes file will include 1.4-1, 1.4-2, and 1.4-2ubuntu1.

    2. That represents all the changes that happened from the perspective of an Ubuntu user upgrading from 1.3-1ubuntu2 to 1.4-2ubuntu1.

Note

If sponsoring a merge or any other upload for someone else, remember the need to sign their upload with your key. See Sponsor a package for more information about that.

Furthermore just like you, the sponsor needs to know about setting -v right and using -sa when needed. If in doubt, coordinating with them is helpful.

Push to your Launchpad repository

Now that the package builds successfully, push it to your Launchpad repository:

git push <your-lp-username>

You get an error message and a suggestion for how to set an upstream branch. For example:

$ git push kstenerud
fatal: The current branch merge-lp1802914-disco has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream kstenerud merge-lp1802914-disco

Run the suggested command to push to your repository.

Push your lp tags

$ git push <your-git-remote> old/ubuntu old/debian new/debian \
           reconstruct/<version> logical/<version> split/<version>

For example:

$ git push git+ssh://[email protected]/~kstenerud/ubuntu/+source/at \
           old/ubuntu old/debian new/debian \
           reconstruct/3.1.20-3.1ubuntu2 \
           logical/3.1.20-3.1ubuntu2 \
           split/3.1.20-3.1ubuntu2

To ssh://git.launchpad.net/~kstenerud/ubuntu/+source/at
 * [new tag]         split/3.1.20-3.1ubuntu2 -> split/3.1.20-3.1ubuntu2
 * [new tag]         logical/3.1.20-3.1ubuntu2 -> logical/3.1.20-3.1ubuntu2
 * [new tag]         new/debian -> new/debian
 * [new tag]         old/debian -> old/debian
 * [new tag]         old/ubuntu -> old/ubuntu
 * [new tag]         reconstruct/3.1.20-3.1ubuntu2 -> reconstruct/3.1.20-3.1ubuntu2

Create a PPA

You need to have a PPA for reviewers to test.

Create a PPA repository

https://launchpad.net/~<your-username>/+activate-ppa

Give it a name that identifies the package name, Ubuntu version, and bug number, such as at-merge-1802914.

Important

Be sure to enable all architectures to check that it builds (click Change details in the top right corner of the newly created PPA page).

The URL of the PPA is formed as follows:

https://launchpad.net/~<your-username>/+archive/ubuntu/<PPA-name>

For example:

https://launchpad.net/~kstenerud/+archive/ubuntu/disco-at-merge-1802914

Upload files

$ dput ppa:kstenerud/disco-at-merge-1802914 ../at_3.1.23-1ubuntu1_source.changes

Wait for packages to be ready

  1. Check the PPA page to see when packages are finished building:

    https://launchpad.net/~kstenerud/+archive/ubuntu/disco-at-merge-1802914

  2. Look at the package contents to make sure they have actually been published (click View package details in the top right corner of the PPA page, or append +packages to its URL):

    https://launchpad.net/~kstenerud/+archive/ubuntu/disco-at-merge-1802914/+packages

Test the new build

Test the following:

  1. Run package tests (if any)

  2. Upgrading from the previous version

  3. Installing the latest where nothing was installed before

  4. Other smoke tests

Test upgrading from the previous version

  1. Create and start a new LXD container to test in:

    $ lxc launch ubuntu-daily:ubuntu/<ubuntu-codename> tester && lxc exec tester bash

  2. Install the currently available version of the package you’ve been working on:

    $ apt update && apt dist-upgrade -y && apt install -y at

  3. Run the test:

    echo "echo xyz > test.txt" | at now + 1 minute && sleep 1m && cat test.txt && rm test.txt

  4. Add your PPA to the virtual system to upgrade the package to the version you want to test:

    $ sudo add-apt-repository -y ppa:kstenerud/at-merge-lp1802914

    If the Ubuntu release for which you’ve built the package is not yet available, modify the source list entry. For example:

    $ vi /etc/apt/sources.list.d/kstenerud-ubuntu-at-merge-lp1802914-cosmic.list
* change cosmic to disco

  5. Upgrade to the new version of the package from your PPA:

    $ apt update && apt dist-upgrade -y

  6. Test the upgraded version:

    $ echo "echo abc > test.txt" | at now + 1 minute && sleep 1m && cat test.txt && rm test.txt

Test installing the latest from scratch

  1. Create and start a new LXD container to test in:

    $ lxc launch ubuntu-daily:ubuntu/<ubuntu-codename> tester && lxc exec tester bash

  2. Add your PPA to the virtual system to upgrade the package to the version you want to test:

    $ sudo add-apt-repository -y ppa:kstenerud/at-merge-lp1802914

  3. Install the new version of the package from your PPA:

    $ apt update && apt dist-upgrade -y && apt install -y at

  4. Run the test:

    echo "echo xyz > test.txt" | at now + 1 minute && sleep 1m && cat test.txt && rm test.txt

Other smoke tests

  • Run various basic commands.

  • Run regression tests.

  • For a package that Build-Depends on itself (openjdk, jruby, kotlin, etc.), build it using the new version.

Submit Merge Proposal (MP)

Note

Git branches with % in their name don’t work. Use something like _.

$ git ubuntu submit --reviewer $REVIEWER --target-branch debian/sid
Your merge proposal is now available at: https://code.launchpad.net/~kstenerud/ubuntu/+source/at/+git/at/+merge/358655
If it looks OK, please move it to the 'Needs Review' state.

Set --reviewer to the team (or user) on Launchpad that should look at your change – by default it is --reviewer ubuntu-sponsors.

  • If you do not have upload rights for this package, use ubuntu-sponsors here. That adds your Merge Proposal to the Ubuntu sponsoring queue, so people with upload rights for that package may eventually review it for you.

  • To notify a specific team, use, e.g. canonical-foundations, canonical-public-cloud, or ubuntu-server.

To avoid having to specify the --reviewer flag, configure the reviewers for git-ubuntu. Include a section like the following either globally in ~/.gitconfig, or in individual repositories in .git/config:

[gitubuntu.submit]
    defaultReviewer = <your-ubuntu-teamname>, \
                      <canonical-more-reviewers>, \
                      <canonical-otherteam>

The equivalent git config command is:

$ git config [--global] gitubuntu.submit.defaultReviewer <launchpad-reviewer>

Note

Using a target branch of debian/sid may seem wrong, but is a workaround for LP: #1976112.

If this fails, do it manually.

Update the merge proposal

  • Link the PPA.

  • Add any other info (as a comment) that can help the reviewer.

    Example:

    PPA: https://launchpad.net/~kstenerud/+archive/ubuntu/disco-at-merge-1802914

Basic test:
$ echo "echo abc >test.txt" | at now + 1 minute && sleep 1m && cat test.txt && rm test.txt

Package tests:
This package contains no tests.

Open the review

Change the MP status from work in progress to needs review.

Follow the migration

Once the merge proposal goes through, you must follow the package to make sure it gets to its destination.

Package tests

The results from the latest package tests is published for each Ubuntu release. For example: autopkgtest.ubuntu.com/packages/o/openssh/questing/amd64.

Proposed migration

The status of all packages is available from the Ubuntu archive or one of its subdirectories. The top level directory is for the current dev release. Previous releases are in subdirectories.

Start a merge manually

Generate the merge branch

Create a branch to do the merge work in:

$ git checkout -b merge-lp1802914-disco

Create tags

Tag

Source

old/ubuntu

ubuntu/<Ubuntu_release>-devel

old/debian

Last import tag prior to old/ubuntu without ubuntu suffix in version

new/debian

debian/sid

As per Debian releases, debian/sid always matches Debian “unstable”.

Find the last import tag using git log:

git log | grep "tag: pkg/import" | grep -v ubuntu | head -1

commit 9c3cf29c05c3fddd7359e71c978ff9a9a76e4404 (tag: pkg/import/3.1.20-3.1)

Based on that, create the following tags:

$ git tag old/ubuntu pkg/ubuntu/disco-devel
$ git tag old/debian 9c3cf29c05c3fddd7359e71c978ff9a9a76e4404
$ git tag new/debian pkg/debian/sid

Start a rebase

$ git rebase -i old/debian

Clear any history, up to and including the last Debian version

Clear any history, up to and including the last Debian version. If the package hasn’t been updated since the Git repository structure changed, it grabs all changes throughout time rather than since the last Debian version. Delete the older lines from the interactive rebase.

In this case, up to, and including the import of 3.1.20-3.1.

Create reconstruct tag

$ git ubuntu tag --reconstruct

Next step: Split commits.

Create logical tag manually

Use the version number of the last ubuntu change. So, if there are:

  • 3.1.20-3.1ubuntu1

  • 3.1.20-3.1ubuntu2

  • 3.1.20-3.1ubuntu2

Run:

$ git tag -a -m "Logical delta of 3.1.20-3.1ubuntu2" logical/3.1.20-3.1ubuntu2

Note

Certain characters aren’t allowed in Git. For example, replace colons (:) with percentage signs (%).

Next step: Rebase onto new Debian.

Finish the merge manually

  1. Merge the changelogs of old Ubuntu and new Debian:

    $ git show new/debian:debian/changelog > /tmp/debnew.txt
$ git show old/ubuntu:debian/changelog > /tmp/ubuntuold.txt
$ merge-changelog /tmp/debnew.txt /tmp/ubuntuold.txt > debian/changelog
$ git commit -m "Merge changelogs" debian/changelog

  2. Create a new changelog entry for the merge:

    $ dch -i

    Which creates, for example:

    at (3.1.23-1ubuntu1) disco; urgency=medium

* Merge with Debian unstable (LP: #1802914). Remaining changes:
    - Suggest an MTA rather than Recommending one.

-- Karl Stenerud <[email protected]>  Mon, 12 Nov 2018 18:11:53 +0100

  3. Commit the changelog:

    $ git commit -m "changelog: Merge of 3.1.23-1" debian/changelog

  4. Update maintainer:

    $ update-maintainer
$ git commit -m "Update maintainer" debian/control

Next step: Fix the changelog

Get the orig tarball manually

Use the pristine-tar(1) tool to regenerate the orig (upstream) tarball:

  1. Create a new branch for the orig tarball:

    $ git checkout -b pkg/importer/debian/pristine-tar

  2. Regenerate the pristine tarball:

    $ pristine-tar checkout at_3.1.23.orig.tar.gz

  3. Switch to the merge branch:

    $ git checkout merge-3.1.23-1-disco

    TODO: Is this ^ branch name correct?

If git checkout also fails

$ git checkout merge-lp1802914-disco
$ cd /tmp
$ pull-debian-source at
$ mv at_3.1.23.orig.tar.gz{,.asc} ~/work/packages/ubuntu/
$ cd -

TODO: This step needs context/explanation.

Next step: Check the source for errors.

Submit merge proposal manually

$ git push kstenerud merge-lp1802914-disco

Then create a MP manually in Launchpad, and save the URL.

Next step: Update the merge proposal.

Known issues

Empty directories

Use the emptydirfixup.py Python script (by Robie Basak, @racb) to locally restore empty directories (see the script for a detailed explanation of why empty directories are a problem).

Example use:

$ git ubuntu clone apache2
$ cd apache2
$ git tag -f base
  <add commits>
$ python3 emptydirfixup.py fix-many base
$ git ubuntu tag --upload

For an entire real case, follow this workflow:

  1. Get the emptydirfixup.py script:

    $ wget -O ../emptydirfixup.py \
  "https://git.launchpad.net/~racb/usd-importer/plain/wip/emptydirfixup.py?h=emptydirfixup"

  2. Clone as usual:

    $ git ubuntu clone "<source_package>" "<source_package>-gu"
$ cd "<source_package>-gu/"

  3. Create the merge branch (here you see the warning message):

    $ git checkout "<last_remote>" -b "<branch_name>"

  4. Tag the base and rebase on the ubuntu/devel branch:

    $ git tag -f base
$ git checkout ubuntu/devel
$ git rebase base

  5. Start the merge:

    $ git ubuntu merge -f start

  6. Merge work as usual…

  7. Workaround for ERROR: Empty directories exist... (see LP: #1939747):

    $ rm .git/hooks/pre-commit

  8. Finish the merge:

    $ git ubuntu merge finish pkg/ubuntu/devel

  9. Create a MP as usual, get reviewed/approved, etc.

  10. Fix the set of commits with empty directories:

    $ python3 ../emptydirfixup.py fix-many base

  11. Build the package:

    $ debuild -S $(git ubuntu push-for-upload)

  12. Upload:

    $ git push pkg "upload/<version>"
$ dput ubuntu "<changes_file>"

  13. Done!