Posts Tagged ‘Linux’

MondoRescue 3.3.0 on its way

2016/12/31

Well, it’s time for gifts no ? So I passed some days last week to go back to various problems reported, in order to solve them.

The most tricky was mindi-busybox not able to compile on recent distros such as Ubuntu 16 or Fedora 24 (glibc changes in them made that incompatibility). I made tries with busybox 1.25 and it turned out that it seems pretty compatible with 1.21 we were using, but working on these latest distros and working easily on the old distros I want to support as well (I went back to RHEL4 with this patch)

I’ve been able to do backup / restore cycles with both CentOS 6 and 7 without issues with the latest version, after having fixed a lot of small more or less blocking problems.

Now I still have stuff I’d like to fix before release:

For Debian:

In short it’s UEFI support with Debian. I should have worked on that since a long time, but last months were very loaded, both at work and at home. And I’ve not done any progress with it for now.

For SLES:

That will probably wait till 3.3.1

and the most important for me (and some customers), more generic:

I’ll start with this one after Christmas.

And also:

But I have a MondoRescue contributor developing code for that !! That’s really great !

And also two improvements that would make everybody’s life easier:

So, yes, some work on my plate for the new year 2017. But I took also some time off this week to recharge, and so I’d be able to do that, my work back next week, and the rest (concerts planned) with lots of energy !!

Let me take that opportunity to wish to all of you a very good new year 2017, that peace may gain more than loose, that FLOSS may gain more than loose, and that we’ll be actors in this.

Meet during LinuxCon EMEA

2016/09/23

After having talked on project-builder.org during the latest LinuxCon NA in Toronto, it’s now time to prepare for LinuxCon EMEA in Berlin.

I’l again talk about packaging, this time “Packaging for Linux Distributions with Docker“. And I’ll also replay the now popular (!) “101 Docker training” I’ve also delivered in Toronto, this time trying to deliver it before the one Jérome Patazzoni from Docker is doing on Docker orchestration, in order to allow for a complete overview of the Docker ecosystem.

Of course I’ll enjoy attending many sessions on various topics, and will see if in EMEA as well Docker Orchestration is the new Eldorado this year (In Toronto many talks and booth were dedicated to that theme). At least my colleague Mike Bright will cover it well !

See you around !

Project-Builder 0.14.1 is now out for LinuxConNA !

2016/08/19

Well, in fact I froze that version back in July, but didn’t officially announced it till today (vacations were just in the middle). So that version 0.14.1 is now available, it breaks compatibility with previous ones, so you’ll need to update your VMs/VEs with it as well if you want to use it correctly (that’s why it’s still version 0.x !)

But I’ve been able to build a lot using docker containers with that version, and I’m now more and more happy with these results. Especially because it’s easier to fully automate an environment from absolutely nothing except project-builder.org to working packages. I already described it when I released 0.13.1 last year through a short tutorial, but it’s really working reliably now for a large set of distributions:

$ docker images | grep pb-pb
pb   mageia-5-x86_64-pb-pb              fc479023c876        3 weeks ago         413.9 MB
pb   opensuse-13.1-x86_64-pb-pb         2c31c56e7bda        3 weeks ago         354.3 MB
pb   opensuse-13.2-x86_64-pb-pb         c8edeba0440b        3 weeks ago         438.2 MB
pb   opensuse-42.1-x86_64-pb-pb         8e91d2b584ed        3 weeks ago         461.5 MB
pb   debian-7-x86_64-pb-pb              0c133063d4be        3 weeks ago         449.8 MB
pb   debian-8-x86_64-pb-pb              8fb919480351        3 weeks ago         588.3 MB
pb   centos-5-x86_64-pb-pb              c8b6f1a0846b        3 weeks ago         706.6 MB
pb   centos-6-x86_64-pb-pb              a3c1ea318031        3 weeks ago         545.9 MB
pb   centos-7-x86_64-pb-pb              d5790ee97e13        3 weeks ago         392.1 MB
pb   ubuntu-12.04-x86_64-pb-pb          053f112ed939        3 weeks ago         398.2 MB
pb   ubuntu-14.04-x86_64-pb-pb          bc239427fc3a        3 weeks ago         501.7 MB
pb   ubuntu-15.04-x86_64-pb-pb          e6cf8d0fafd0        3 weeks ago         503.8 MB
pb   ubuntu-15.10-x86_64-pb-pb          dba3b580bdc8        3 weeks ago         550.1 MB
pb   ubuntu-16.04-x86_64-pb-pb          02a337752837        3 weeks ago         522.9 MB
pb   fedora-21-x86_64-pb-pb             07223685d24f        3 weeks ago         965.1 MB
pb   fedora-22-x86_64-pb-pb             211b865c1ec6        3 weeks ago         749.5 MB
pb   fedora-23-x86_64-pb-pb             f6012ff7b6b8        3 weeks ago         879.6 MB
pb   fedora-24-x86_64-pb-pb             6d878b489584        3 weeks ago         678.5 MB
pb   gentoo-nover-x86_64-pb-pb          b6ead3da5b85        4 weeks ago         1.392 GB

But I’ve not really worked yet on the ideas expressed when I issued 0.13.2 in February this year.

Instead I’ve done other improvements such as managing signed deb repositories (was asked for a long time), have a better naming for temporary versions (allowing easier updates between temporary versions and also with stables, but creating the incompatibility with 0.13.x), docker tests for lots of distributions (paving the way for automatic testing here from nothing to package build), adding support for pbr, standardizing temporary directory names, …

That will be the version I’ll use to make demos during my presentation next week at LinuxCon North America in Toronto as already announced. Come to see it live and so that we can discuss continuous packaging !

Network install of Debian 8 on a HPE ProLiant Gen9 server

2016/06/29

There are articles to do this in a pure Debian environment, such as the excellent one I used as a base from Raphaël Hertzog.

But my deployment server in that case is a CentOS 6 one, so I needed some adaptations to make it work.

My target system is a HPE ProLiant BL 460 Gen9, equiped with bnx2x NICs (Broadcom BCM 57840 10/20 Gb/s cards). And during a network boot install, I have had messages indicating that the firmware was missing (failed to load bnx2x/bnx2x-…)

So in order for my system to network boot, I needed to adapt my initrd and add the missing firmware in it (Debian’s policy prevents the distribution of
non-free firmware, which these are). The Firmware in my case is available as a standalone package at http://ftp.fr.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2x_0.43_all.deb (hint from this article)

So on your non-debian deployment server, you can do the following:

mkdir /tmp/fw
cd /tmp/fw
# This is the network boot environment required for booting Jessie
wget http://ftp.debian.org/debian/dists/jessie/main/installer-amd64/current/images/netboot/netboot.tar.gz
[...]
tar xvfz netboot.tar.gz
[...]
# Keep the kernel in your deployment infra
mv debian-installer/amd64/linux /where/your/tftpboot/k/debian-8
# Extract the initrd to modify it
wget http://ftp.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2x_0.43_all.deb
[...]
mkdir initrd
cd initrd
gzip -cd ../debian-installer/amd64/initrd.gz | cpio -ivdum
[...]
# You need the dpkg commdand to make your life easier (part of CentOS, Fedora or Mageia)
dpkg -x ../firmware-bnx2x_0.43_all.deb  .
ls lib/firmware/
[...]
# Should show you the firmware in the right place
find . -print | cpio -o -H newc | gzip -c9 > /where/your/tftpboot/i/debian-8.img
[...]

Now you can add an entry to your boot loader (grub2 in my case) for the network boot part (more info on Network boot and Debian at https://wiki.debian.org/PXEBootInstall)

menuentry 'Debian 8' {
  insmod gzio
  insmod part_gpt
  insmod ext2
  insmod iso9660
  linux /k/debian-8 ip=dhcp --
  initrd /i/debian-8.img
}

If you have another firmware to deal with, refer to the dedicated Debian page at https://wiki.debian.org/fr/firmware

After that your system should be able to network boot and install your Debian Jessie distribution on it.

The next step is to automate the installation, which consists just in changing the linux line in the grug conf file:

linux /k/debian-8 ip=dhcp auto priority=critical url=http://deploy-server/ks/debian-8 --

where the debian-8 file under the ks directory is just the preseed file you’ll build for your automatic installation.

Meet again next week in Paris and at LinuxCon NA 2016 in Toronto

2016/06/19

I’m just back from an HPE Event (the ETSS in Gothenburg, Sweden) and will soon have to start the next ones.

First, I’ll be this week in Paris to talk during a MySQL Meetup to cover automatic deployment, in particular with Redfish, images of reference (and a bit of MondoRescue) and automation (ansible e.g.) for DBA.

Then I’m again lucky this year to be retained as a speaker for the upcoming LinuxCon North America which will be in Toronto, Canada from the 22nd of August to the 27th. I’ll have a session on Continuous Packaging, mandatory for DevOps and a 3 hour Initiation Lab on Docker. Interestingly enough at the same time, Jérôme Petazzoni, Docker, will do one on Swarm !! Maybe we should coordinate and make them one after the other (I’d like to attend his Lab BTW !!)

I have now put all my lab material that I do for HPE on GitHub, to make it easier for attendees to report errors, and for others to use them and learn. I have content for Docker (Done with René Ribaud, Yann Allandit, Michael Bright), Git (Done with Clément Poulain), Redfish (Originally written by François Donzé and done with René Ribaud) and soon for Ironic.

Hope to be able to meet with you here or there !

MondoRescue 3.2.2 is finally available

2016/04/29

I don’t know you, but I was waiting for this version for a long time. Way too long ! But I’m lucky, I’m not time based for releases. I’m really much more features based. And the one I really needed to catch for this version was UEFI support for RHEL/CentOS 6 & 7. HPE has teams and customers which were really needing that to work correctly, which is, from my series of tests, now the case. And I had confirmation from my colleagues that it was indeed working for them as expected just recently. So it was then time to publish !

While I was at it, I tried to fix at the same time some other annoying issues that were reported by the community on the mailing list, while leaving some other problems for the next time, as it was already too much delayed. So I know the Debian and Ubuntu users won’t get UEFI support working, as I had not enough time to test it (my test infra is now being used for trainings for 2 weeks), similarly for SLES 11 SP4 with UEFI (which uses elilo, while I now support grub, grub2 and syslinux, but this one doesn’t work correctly with UEFI IMO).

In addition to what was already done and reported here previously, I have hopefully fixed the following additional issues:

  • #777: Full UEFI support with grub 0.9 and grub2 configuration generation in addition to syslinux (limited to RHEL6, RHEL7)
  • #778 use dosfslabel/mr-label to recreate labelled FAT or VFAT FS
  • #757, #780 and #781 to fix md support
  • #792 to fix LVM support, by forcing fixed values for LE if percentage calculated for it is 0
  • #787 to fix USB support by cleaning the target dir of its syslinux content

In total 24 bugs have been officially closed and additional features improved or fixed as well. Look at the details in trac for the complete list of changes during these last 8 months of pregnancy of the new version 🙂

But, of course, things do not stop here. In addition to fixes which needs to be done as mentioned upper, I’d like to add support for Secure Boot e.g. in the next version to have a full UEFI support, and work again on code refactoring even more.

For now, let’s take time to celebrate and hopefully this will be for you as users the best version possible of MondoRescue available to date !

Time to test what should be the latest beta of MondoRescue 3.2.2

2016/03/25

Well, I thought it could still take more time, but I had the possibility to dedicate enough time to fix and test the UEFI support for RHEL7 so I think it’s worth sharing and hopefully get good feedback from you before tagging it as final.

So a new beta of MondoRescue 3.2. is on its way. Please test it, especially if you try to do a backup/restore of a UEFI based system as this should now work flawlessly, including putting back EPS UUIDs. Should also work for other grub 2.x type of distributions such as Debian 8 e.g.

And share your report on our mailing-list !

I may have time to work on some remaining pretty annoying issues (#757 and #780 which look similar, maybe #781) but can’t promise for sure they will be part of this one which has been delayed way too much.

Current state of the changelogs are at http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/mindi/pbcl, http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/mondo/pbcl and http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/MondoRescue/pbcl

Progresses with MondoRescue 3.2.2

2016/03/22

Of course I’m late. No surprise here 😦 I think the only day I’m not will be for my death 😉

So 3.2.1 was published the 31st of August 2015.
Version 3.2.2 was supposed to be a maintenance version of that one and was thus planned to be published mid October 2015. However, there were requests from various users to provide a version supporting UEFI boot mode on ProLiant servers. That enhancement request alone accounts for most of the delay encountered while developing this version as it turned out to be way more impacting than I thought.

So some bugs encountered on previous versions have been solved:

However with regards to the single enhancement request in that version to add UEFI support, many tickets were made:

The last one, is the most tricky to solve.
MondoRescue uses since day one a single bootloader, coming from the syslinux project, to manage all boot media it generate :
– isolinux for bootable CD/DVD and tapes
– syslinux for bootable USB devices
– pxelinux for bootable network setup.

This has the advantage of homogeneity and simplifies what the mindi tool has to generate in order to implement that support.

However, when using syslinux.efi to provide a bootable media on HPE ProLiant servers setup in UEFI boot mode, we obtain red screen showing a boot time crash.

Despite many attempts first with various options, then with the help of the syslinux community back in October 2015 (Cf: http://www.syslinux.org/archives/2015-October/024469.html and
the related thread) using beta versions of syslinux or the latest ProLiant Firmware, I never succeeded in obtaining a UEFI bootable syslinux based media 😦

I’ve had contacts in December 2015 with the ProLiant Firmware engineering team, but we didn’t had yet a joint work on this issue. My plan would be to first diagnose precisely the origin of that error with this team, and then depending on the results to invite them around the table with the syslinux developers during potentially an upcoming LinuxCon NA (August 2016) to solve it for good.

As all that means we won’t have a final solution with syslinux.efi before end of summer 2016 at the earliest. Therefore, I’ve worked on adding support for other boot
loaders in MondoRescue.

Grub 0.99 support was added in August 2015 before the publication of 3.2.1 and we had success testing a full backup/recovery cycle on RHEL 6, while some other users encountered issues with that. Since November 2015, that should was considered fairly stable, and thus was made the default for RHEL 6 only.

Grub 2.0x support was added very recently (the 22nd of February 2016) with other modifications required to better support correctly mutiple boot loaders on multiple backup media (consolidation of code split into different functions in mindi). This was now tested successfully yesterday and I was able to make a full cycle with mindi on a physical server with RHEL 7. The restore now even present the MondoRescue menu to choose from.

However, while mindi now seems to be working fine for this configuration, this is still not the case for mondoarchive. I’ve started to unify the UEFI support between the 2 tools, and hopefully this week I should see positive results leading to a test version published for the MondoRescue community to try RSN ! And with your retunrs, I should be able to publish that long awaited version rapidly after around start of April probably.

I understand it took much more time than expected initially, and I’ve been myself giving wrong estimates for the delivery of the new version many times. Let’s hope this time I’ll be more accurate.
Hope this explains more the ground reasons why it was so, and the attempt to mitigate the real problem we have with syslinux on UEFI boot mode that may not be solved soon.

Fosdem is here again !

2016/01/26

In addition to next week event participation for HPE in Cannes, I’ll be at Fosdem the coming week-end in Brussels. And I’m so happy to be able to be there again, as it has been since 2012 that i’ve not participated.

This time, it’s not MondoRescue or Project-Builder.org related. But the presentation that was selected is part of the Distributions dev room. The title is “Building Linux distribution packages with Docker”. I’ve been using Docker to help in building packages since a long time now, and Project-Builder.org latest version is now supporting it pretty well to help me package my upstream projects. But I want to cover a different aspect.

Docker has brought an ease of use without comparison with VMs typically to build native upstream distribution packages. Where before it was needed to
launch a complete environment, copy the sources into it, invoke the build tools to create the packages and then copy them back to the host, Docker has made all these steps much easier and straight forward, allowing for more rapid package production and automation.

This presentation will show a detailed use case for building packages for both Mageia and Fedora with their respective bm or koji tools encapsulated in Docker containers. It should help any upstream packager adopt a similar approach to make his packaging task a breathe.

Will be there Saturday the 30th of January (my 20′ talk is at 11:30 in K.4.201) and I’ll stay till Sunday early afternoon. Feel free again to catch me and discuss. When not in my talk, I’ll be in other sessions and also around the Mageia Booth.

And come in mass as this is one of the best FLOSS event WW. Really !

Time to Meet at Hewlett Packard Enterprise Technology & Solutions Summit 2016 !

2016/01/26

New company name, new event name for our presales and channel partners, but same content of quality with the best experts to transfer knowledge on everything HPE !

And this year, we’ll have a track dedicated to Open Source and Linux ! It’s a first (and hopefully not the last time it’ll be given).

So I’ll be in Cannes next week to cover:

  • HP Helion Openstack 2.0: architecture and live demo – Breakout 145 – Tuesday 2nd of February 2016 – 5:40PM-6:30PM
  • HPE’s Open Source Strategy – Breakout 153 – Wednesday 3rd of February 2016 – 3:00PM-3:50PM
  • Linux Distributions roadmap – Breakout 154 – Thursday 4th of February 2016 – 9:30AM-10:20AM
  • HPE ProLiant Gen9 value add on Linux – Hands-on Lab 24 – Thursday 4th of February 2016 – 4:45PM-6:45PM

And you should be able to meet the Geeks on Wednesday afternoon, myself included.

Other subject of interest are the breakouts 86 (Intro to open source infra automation tools), 48 (container and next gen architecture), and 151 (building an open source high perf object storage cluster with Ceph) and of course the Labs (the Redfish and UEfI ones in particular) and SuSE and Red Hat sponsor sessions. Lots to see !

For those of you wondering why project-builder.org and MondoRescue versions are not published more regularly, that gives you again a hint 😉 Not speaking of the next event in March in Grenoble, TES, where I’m part of the Event team this time.

See you there to talk of these subjects or something else as you see fit. I’m around the whole week.