Meet at the Open Source Summit NA 2017

Despite difficult times for HPE, I’m still supported by my management to continue to represent HPE during tier 1 Open Source events such as the Open Source Summit North America, held in Los Angeles from the 11th to the 14th of September. Well, of course, because I was accepted as a speaker again !

I’ll lead a round table to cover with panelists whether “Containers are the future of IaaS ?”

So as usual, if you are around, I’ll happy to discuss with you around Open Source in general, and the projects I’m involved with in particular such as MondoRescue (yes, there will be a new version soon, now remains to see the definition of soon), project-builder.org (same thing), Redfish, containers, early music or more.

Meet at the RMLL 2017

This year the RMLL are coming back, from the 1st to the 7th of July and in my region, in St Etienne ! And this year, I am not only a speaker, but also a co-track leader for the Server track.

With my talks, You’ll be able to learn about the latest news for MondoRescue, project-builder.org, Docker, Redfish and python-redfish, Mageia and much more from my co-speakers (Ethereum, CMS, Cloud Native app, Unikernels, …) and even more from Keynote speakers as well.

Lodging is now available, so book quickly and let meet us there !

This week, gift continues with project-builder

Normally, today I receive gifts, priviledge (really ?) of being older 🙂 But I can also make gifts myself ! So after python-redfish 0.4 yesterday, I’m announcing the release of project-builder 0.14.3 today (and maybe more to come later this week !)

This project seems to have a quicker pace than MondoRescue, maybe due to the fact I’m using it daily, and that it’s now used by python-redfish, MondoRescue, and also FOSSology (WIP more on that later again). I realize I didn’t announced previous versions 0.14.1 last July, and 0.14.2 last January, but lots have been added since.

I’ve worked on this version to improve deb support (no arch pkg) and especially deb repositories (with GPG keys) as well as OpenSUSE GPG signed repositories as well. Previous versions brought support for latest distros (Fedora 25, Ubuntu 16.10 e.g.), robust build using Docker in addition to VMs or chroot VEs, fixes some issues around conf files analysis, improving rpm package name for smooth upgrade between test and stable versions, renaming temporary dirs for consistency, as well as moving the main conf file outside of /etc reserved for local modifications, … Changelogs have all the nasty details.

One of the task I’d like to make now is to move the conf file format to JSON, to benefit from existing libs to manage them, and make them even easier to deal with. That would allow some other improvements afterwards, but too early to talk about.

Oh, and BTW, we decided to also publish python-redfish 0.4.1 (with its packages made with project-builder.org to keep this article’s focus 😉 ) to fix a nasty error we made when pushing to pypi. Enjoy as well !

MondoRescue 3.3.0 on its way

Well, it’s time for gifts no ? So I passed some days last week to go back to various problems reported, in order to solve them.

The most tricky was mindi-busybox not able to compile on recent distros such as Ubuntu 16 or Fedora 24 (glibc changes in them made that incompatibility). I made tries with busybox 1.25 and it turned out that it seems pretty compatible with 1.21 we were using, but working on these latest distros and working easily on the old distros I want to support as well (I went back to RHEL4 with this patch)

I’ve been able to do backup / restore cycles with both CentOS 6 and 7 without issues with the latest version, after having fixed a lot of small more or less blocking problems.

Now I still have stuff I’d like to fix before release:

For Debian:

• mindi aborts on Debian 8 on UEFI system
• Dependencies issue on Debian 8
• ldlinux.c32 not foud while booting
• mondoarchive doesn’t create a bootable UEFI media on Debian 8

In short it’s UEFI support with Debian. I should have worked on that since a long time, but last months were very loaded, both at work and at home. And I’ve not done any progress with it for now.

For SLES:

• Wrong dependencies in mondo SLES 12 rpm package
• Provide SLES 11 SP4 support with UEFI

That will probably wait till 3.3.1

and the most important for me (and some customers), more generic:

• mr-parted2fdisk should use parted when there’s a lot of disks

I’ll start with this one after Christmas.

And also:

• Verify doesn’t work with tape

But I have a MondoRescue contributor developing code for that !! That’s really great !

And also two improvements that would make everybody’s life easier:

• Calculate EXTRA_SPACE value before assigning it
• mondorestore says a partition is occupied

So, yes, some work on my plate for the new year 2017. But I took also some time off this week to recharge, and so I’d be able to do that, my work back next week, and the rest (concerts planned) with lots of energy !!

Let me take that opportunity to wish to all of you a very good new year 2017, that peace may gain more than loose, that FLOSS may gain more than loose, and that we’ll be actors in this.

Meet during LinuxCon EMEA

After having talked on project-builder.org during the latest LinuxCon NA in Toronto, it’s now time to prepare for LinuxCon EMEA in Berlin.

I’l again talk about packaging, this time “Packaging for Linux Distributions with Docker“. And I’ll also replay the now popular (!) “101 Docker training” I’ve also delivered in Toronto, this time trying to deliver it before the one Jérome Patazzoni from Docker is doing on Docker orchestration, in order to allow for a complete overview of the Docker ecosystem.

Of course I’ll enjoy attending many sessions on various topics, and will see if in EMEA as well Docker Orchestration is the new Eldorado this year (In Toronto many talks and booth were dedicated to that theme). At least my colleague Mike Bright will cover it well !

See you around !

Project-Builder 0.14.1 is now out for LinuxConNA !

Well, in fact I froze that version back in July, but didn’t officially announced it till today (vacations were just in the middle). So that version 0.14.1 is now available, it breaks compatibility with previous ones, so you’ll need to update your VMs/VEs with it as well if you want to use it correctly (that’s why it’s still version 0.x !)

But I’ve been able to build a lot using docker containers with that version, and I’m now more and more happy with these results. Especially because it’s easier to fully automate an environment from absolutely nothing except project-builder.org to working packages. I already described it when I released 0.13.1 last year through a short tutorial, but it’s really working reliably now for a large set of distributions:

$ docker images | grep pb-pb
pb   mageia-5-x86_64-pb-pb              fc479023c876        3 weeks ago         413.9 MB
pb   opensuse-13.1-x86_64-pb-pb         2c31c56e7bda        3 weeks ago         354.3 MB
pb   opensuse-13.2-x86_64-pb-pb         c8edeba0440b        3 weeks ago         438.2 MB
pb   opensuse-42.1-x86_64-pb-pb         8e91d2b584ed        3 weeks ago         461.5 MB
pb   debian-7-x86_64-pb-pb              0c133063d4be        3 weeks ago         449.8 MB
pb   debian-8-x86_64-pb-pb              8fb919480351        3 weeks ago         588.3 MB
pb   centos-5-x86_64-pb-pb              c8b6f1a0846b        3 weeks ago         706.6 MB
pb   centos-6-x86_64-pb-pb              a3c1ea318031        3 weeks ago         545.9 MB
pb   centos-7-x86_64-pb-pb              d5790ee97e13        3 weeks ago         392.1 MB
pb   ubuntu-12.04-x86_64-pb-pb          053f112ed939        3 weeks ago         398.2 MB
pb   ubuntu-14.04-x86_64-pb-pb          bc239427fc3a        3 weeks ago         501.7 MB
pb   ubuntu-15.04-x86_64-pb-pb          e6cf8d0fafd0        3 weeks ago         503.8 MB
pb   ubuntu-15.10-x86_64-pb-pb          dba3b580bdc8        3 weeks ago         550.1 MB
pb   ubuntu-16.04-x86_64-pb-pb          02a337752837        3 weeks ago         522.9 MB
pb   fedora-21-x86_64-pb-pb             07223685d24f        3 weeks ago         965.1 MB
pb   fedora-22-x86_64-pb-pb             211b865c1ec6        3 weeks ago         749.5 MB
pb   fedora-23-x86_64-pb-pb             f6012ff7b6b8        3 weeks ago         879.6 MB
pb   fedora-24-x86_64-pb-pb             6d878b489584        3 weeks ago         678.5 MB
pb   gentoo-nover-x86_64-pb-pb          b6ead3da5b85        4 weeks ago         1.392 GB

But I’ve not really worked yet on the ideas expressed when I issued 0.13.2 in February this year.

Instead I’ve done other improvements such as managing signed deb repositories (was asked for a long time), have a better naming for temporary versions (allowing easier updates between temporary versions and also with stables, but creating the incompatibility with 0.13.x), docker tests for lots of distributions (paving the way for automatic testing here from nothing to package build), adding support for pbr, standardizing temporary directory names, …

That will be the version I’ll use to make demos during my presentation next week at LinuxCon North America in Toronto as already announced. Come to see it live and so that we can discuss continuous packaging !

Network install of Debian 8 on a HPE ProLiant Gen9 server

There are articles to do this in a pure Debian environment, such as the excellent one I used as a base from Raphaël Hertzog.

But my deployment server in that case is a CentOS 6 one, so I needed some adaptations to make it work.

My target system is a HPE ProLiant BL 460 Gen9, equiped with bnx2x NICs (Broadcom BCM 57840 10/20 Gb/s cards). And during a network boot install, I have had messages indicating that the firmware was missing (failed to load bnx2x/bnx2x-…)

So in order for my system to network boot, I needed to adapt my initrd and add the missing firmware in it (Debian’s policy prevents the distribution of
non-free firmware, which these are). The Firmware in my case is available as a standalone package at http://ftp.fr.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2x_0.43_all.deb (hint from this article)

So on your non-debian deployment server, you can do the following:

mkdir /tmp/fw
cd /tmp/fw
# This is the network boot environment required for booting Jessie
wget http://ftp.debian.org/debian/dists/jessie/main/installer-amd64/current/images/netboot/netboot.tar.gz
[...]
tar xvfz netboot.tar.gz
[...]
# Keep the kernel in your deployment infra
mv debian-installer/amd64/linux /where/your/tftpboot/k/debian-8
# Extract the initrd to modify it
wget http://ftp.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2x_0.43_all.deb
[...]
mkdir initrd
cd initrd
gzip -cd ../debian-installer/amd64/initrd.gz | cpio -ivdum
[...]
# You need the dpkg commdand to make your life easier (part of CentOS, Fedora or Mageia)
dpkg -x ../firmware-bnx2x_0.43_all.deb  .
ls lib/firmware/
[...]
# Should show you the firmware in the right place
find . -print | cpio -o -H newc | gzip -c9 > /where/your/tftpboot/i/debian-8.img
[...]

Now you can add an entry to your boot loader (grub2 in my case) for the network boot part (more info on Network boot and Debian at https://wiki.debian.org/PXEBootInstall)

menuentry 'Debian 8' {
  insmod gzio
  insmod part_gpt
  insmod ext2
  insmod iso9660
  linux /k/debian-8 ip=dhcp --
  initrd /i/debian-8.img
}

If you have another firmware to deal with, refer to the dedicated Debian page at https://wiki.debian.org/fr/firmware

After that your system should be able to network boot and install your Debian Jessie distribution on it.

The next step is to automate the installation, which consists just in changing the linux line in the grug conf file:

linux /k/debian-8 ip=dhcp auto priority=critical url=http://deploy-server/ks/debian-8 --

where the debian-8 file under the ks directory is just the preseed file you’ll build for your automatic installation.

Meet again next week in Paris and at LinuxCon NA 2016 in Toronto

I’m just back from an HPE Event (the ETSS in Gothenburg, Sweden) and will soon have to start the next ones.

First, I’ll be this week in Paris to talk during a MySQL Meetup to cover automatic deployment, in particular with Redfish, images of reference (and a bit of MondoRescue) and automation (ansible e.g.) for DBA.

Then I’m again lucky this year to be retained as a speaker for the upcoming LinuxCon North America which will be in Toronto, Canada from the 22nd of August to the 27th. I’ll have a session on Continuous Packaging, mandatory for DevOps and a 3 hour Initiation Lab on Docker. Interestingly enough at the same time, Jérôme Petazzoni, Docker, will do one on Swarm !! Maybe we should coordinate and make them one after the other (I’d like to attend his Lab BTW !!)

I have now put all my lab material that I do for HPE on GitHub, to make it easier for attendees to report errors, and for others to use them and learn. I have content for Docker (Done with René Ribaud, Yann Allandit, Michael Bright), Git (Done with Clément Poulain), Redfish (Originally written by François Donzé and done with René Ribaud) and soon for Ironic.

Hope to be able to meet with you here or there !

MondoRescue 3.2.2 is finally available

I don’t know you, but I was waiting for this version for a long time. Way too long ! But I’m lucky, I’m not time based for releases. I’m really much more features based. And the one I really needed to catch for this version was UEFI support for RHEL/CentOS 6 & 7. HPE has teams and customers which were really needing that to work correctly, which is, from my series of tests, now the case. And I had confirmation from my colleagues that it was indeed working for them as expected just recently. So it was then time to publish !

While I was at it, I tried to fix at the same time some other annoying issues that were reported by the community on the mailing list, while leaving some other problems for the next time, as it was already too much delayed. So I know the Debian and Ubuntu users won’t get UEFI support working, as I had not enough time to test it (my test infra is now being used for trainings for 2 weeks), similarly for SLES 11 SP4 with UEFI (which uses elilo, while I now support grub, grub2 and syslinux, but this one doesn’t work correctly with UEFI IMO).

In addition to what was already done and reported here previously, I have hopefully fixed the following additional issues:

• #777: Full UEFI support with grub 0.9 and grub2 configuration generation in addition to syslinux (limited to RHEL6, RHEL7)
• #778 use dosfslabel/mr-label to recreate labelled FAT or VFAT FS
• #757, #780 and #781 to fix md support
• #792 to fix LVM support, by forcing fixed values for LE if percentage calculated for it is 0
• #787 to fix USB support by cleaning the target dir of its syslinux content

In total 24 bugs have been officially closed and additional features improved or fixed as well. Look at the details in trac for the complete list of changes during these last 8 months of pregnancy of the new version 🙂

But, of course, things do not stop here. In addition to fixes which needs to be done as mentioned upper, I’d like to add support for Secure Boot e.g. in the next version to have a full UEFI support, and work again on code refactoring even more.

For now, let’s take time to celebrate and hopefully this will be for you as users the best version possible of MondoRescue available to date !

Time to test what should be the latest beta of MondoRescue 3.2.2

Well, I thought it could still take more time, but I had the possibility to dedicate enough time to fix and test the UEFI support for RHEL7 so I think it’s worth sharing and hopefully get good feedback from you before tagging it as final.

So a new beta of MondoRescue 3.2. is on its way. Please test it, especially if you try to do a backup/restore of a UEFI based system as this should now work flawlessly, including putting back EPS UUIDs. Should also work for other grub 2.x type of distributions such as Debian 8 e.g.

And share your report on our mailing-list !

I may have time to work on some remaining pretty annoying issues (#757 and #780 which look similar, maybe #781) but can’t promise for sure they will be part of this one which has been delayed way too much.

Current state of the changelogs are at http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/mindi/pbcl, http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/mondo/pbcl and http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/MondoRescue/pbcl