Docker stack updates for Mageia 9

Long time since I wrote on my preferred distribution ! The Mageia dev team has been working very hard to release Mageia 9 … which is not there yet, but which will be really soon now.

I’ve been using Docker for a long time, especially to build software. My recurrent usage is to build a Mageia cauldron container to build a new package for that distribution before submitting it.

But the Docker stack in current Mageia cauldron is pretty old (based on 20.10.x) and miss an important security fix provided by the latest 24.0.5 and the new compose v2 at least.

So I started to work on it so that we could have a new shiny stack, easier to maintain during the life of Mageia 9. However, I had some aspects to handle before being able to claim victory:

• manage the lack of support in 24.0.5 for aufs storage driver which had been used in Mageia in all versions before.
• manage the migration of all existing images using that aufs format to the new overlay2 format, if possible automatically during update (I myself have 525 existing images I didn’t want to loose !)
• deal with the new CLI plugin support for buildx (a new feature) and compose (replacing v1 as a standalone python binary)
• update the rest of low level components to support that (docker-containerd-1.7.3, opencontainers-runc-1.1.9) and adding missing go modules (golang-github-mrunalp-fileutils-0.5.0) so everything works fine.
• Test that my existing docker usages were still working afterwards with the new stack.

For managing the migration, it turned out that the best solution was to export images with 20.x having aufs and re-import them with 24.x having overlay2. And even if you can export multiple images at once, it may not work when you have so many like me, especially when some images have unreadable files in them causing errors. So I had to loop on each existing image to export it to /var/cache/docker/save, using its image id for that, and then storing informtation on their tags in order to be able to re-import them afterwards and recreate all the correct tags. At least for me it works fine and that is now part of the %pre and %post scripts of the docker package when doing un upgrade. At then end it let under /var/cache/docker/oldsave (directory renamed if operation occured) the exported content that you’ll have to remove manually. If something goes wrong, you can always renamed it back to /var/cache/docker/save and relaunch the install of the docker package to re-import your content. Worst case, it’s feasible manually as well (look at the result of rpm -q –scripts docker).

So during an upgrade all images are saved and restored to allow for a smooth migration, however, this requires time *and * free storage under the directory used (/var/cache/docker by default). On a system with 525 images sizing 88 Go compressed, migration through urpmi took 34 hours for backup and 2 hours for restore. So it is advised to run this operation separately from the global update to avoid a long lock on the RPM database and allow to control it.

We used also to have docker-compose v1 (python based) and the upstream project in v2 is now using a CLI plugin for the docker command written in go. The binary is placed under /usr/libexec/docker/cli-plugins where the docker command will find and use it. Seems to work in a great compatible way on the docker-compose.yml files I tested.

Similarly, now you may want to use the Moby buildx plugin to give you more features to build containers, and it’s now available as a new package for Mageia 9 named … docker-buildx and installed as well under /usr/libexec/docker/cli-plugins.

docker-containerd has also been updated to 1.7.3 and now requires a new dependency golang-github-mrunalp-fileutils.

And to have all that building, I also updated go to 1.21 !

None of these packages are yet available, as we’re in version freeze and no update is currently allowed. I’ll see whether the QA team can/want to test it so we have it day one after announce of availability as an update for our Docker users. If not, it may take some additional days before you can benefit from it.

Tags: Docker, FLOSS, Mageia, Open Source