Archive for the ‘FLOSS’ Category

Project-Builder 0.14.1 is now out for LinuxConNA !

2016/08/19

Well, in fact I froze that version back in July, but didn’t officially announced it till today (vacations were just in the middle). So that version 0.14.1 is now available, it breaks compatibility with previous ones, so you’ll need to update your VMs/VEs with it as well if you want to use it correctly (that’s why it’s still version 0.x !)

But I’ve been able to build a lot using docker containers with that version, and I’m now more and more happy with these results. Especially because it’s easier to fully automate an environment from absolutely nothing except project-builder.org to working packages. I already described it when I released 0.13.1 last year through a short tutorial, but it’s really working reliably now for a large set of distributions:

$ docker images | grep pb-pb
pb   mageia-5-x86_64-pb-pb              fc479023c876        3 weeks ago         413.9 MB
pb   opensuse-13.1-x86_64-pb-pb         2c31c56e7bda        3 weeks ago         354.3 MB
pb   opensuse-13.2-x86_64-pb-pb         c8edeba0440b        3 weeks ago         438.2 MB
pb   opensuse-42.1-x86_64-pb-pb         8e91d2b584ed        3 weeks ago         461.5 MB
pb   debian-7-x86_64-pb-pb              0c133063d4be        3 weeks ago         449.8 MB
pb   debian-8-x86_64-pb-pb              8fb919480351        3 weeks ago         588.3 MB
pb   centos-5-x86_64-pb-pb              c8b6f1a0846b        3 weeks ago         706.6 MB
pb   centos-6-x86_64-pb-pb              a3c1ea318031        3 weeks ago         545.9 MB
pb   centos-7-x86_64-pb-pb              d5790ee97e13        3 weeks ago         392.1 MB
pb   ubuntu-12.04-x86_64-pb-pb          053f112ed939        3 weeks ago         398.2 MB
pb   ubuntu-14.04-x86_64-pb-pb          bc239427fc3a        3 weeks ago         501.7 MB
pb   ubuntu-15.04-x86_64-pb-pb          e6cf8d0fafd0        3 weeks ago         503.8 MB
pb   ubuntu-15.10-x86_64-pb-pb          dba3b580bdc8        3 weeks ago         550.1 MB
pb   ubuntu-16.04-x86_64-pb-pb          02a337752837        3 weeks ago         522.9 MB
pb   fedora-21-x86_64-pb-pb             07223685d24f        3 weeks ago         965.1 MB
pb   fedora-22-x86_64-pb-pb             211b865c1ec6        3 weeks ago         749.5 MB
pb   fedora-23-x86_64-pb-pb             f6012ff7b6b8        3 weeks ago         879.6 MB
pb   fedora-24-x86_64-pb-pb             6d878b489584        3 weeks ago         678.5 MB
pb   gentoo-nover-x86_64-pb-pb          b6ead3da5b85        4 weeks ago         1.392 GB

But I’ve not really worked yet on the ideas expressed when I issued 0.13.2 in February this year.

Instead I’ve done other improvements such as managing signed deb repositories (was asked for a long time), have a better naming for temporary versions (allowing easier updates between temporary versions and also with stables, but creating the incompatibility with 0.13.x), docker tests for lots of distributions (paving the way for automatic testing here from nothing to package build), adding support for pbr, standardizing temporary directory names, …

That will be the version I’ll use to make demos during my presentation next week at LinuxCon North America in Toronto as already announced. Come to see it live and so that we can discuss continuous packaging !

OpenStack Summit proposals

2016/08/07

While I’m in vacation around Barcelone, I’ve made before leaving proposals for the upcoming OpenStack Summit which will be … in Barcelona !

If you think they are valuable for the OpenStack community, let them know by voting for them😉 Here are they:

  • The nightmare of Ironic Standalone
  • Empowering Ironic with Redfish support
  • From Bare Metal to CMDB: ITIL as Code

The easiest way to vote is to go to https://www.openstack.org/summit/barcelona-2016/vote-for-speakers/ and search for cornec in the search area on the top left. You’ll just get these 3 ones. But of course there are tons of other interesting presentations proposed and as usual it will be difficult for these one to be selected. And you should really vote for the ones which are the most interesting.

Hopefully even if not selected I’ll be able to come back in this great place and meet again with peers after Paris and Vancouver but of course, it will be much more difficult without being a presenter. Let’s see !

Network install of Debian 8 on a HPE ProLiant Gen9 server

2016/06/29

There are articles to do this in a pure Debian environment, such as the excellent one I used as a base from Raphaël Hertzog.

But my deployment server in that case is a CentOS 6 one, so I needed some adaptations to make it work.

My target system is a HPE ProLiant BL 460 Gen9, equiped with bnx2x NICs (Broadcom BCM 57840 10/20 Gb/s cards). And during a network boot install, I have had messages indicating that the firmware was missing (failed to load bnx2x/bnx2x-…)

So in order for my system to network boot, I needed to adapt my initrd and add the missing firmware in it (Debian’s policy prevents the distribution of
non-free firmware, which these are). The Firmware in my case is available as a standalone package at http://ftp.fr.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2x_0.43_all.deb (hint from this article)

So on your non-debian deployment server, you can do the following:

mkdir /tmp/fw
cd /tmp/fw
# This is the network boot environment required for booting Jessie
wget http://ftp.debian.org/debian/dists/jessie/main/installer-amd64/current/images/netboot/netboot.tar.gz
[...]
tar xvfz netboot.tar.gz
[...]
# Keep the kernel in your deployment infra
mv debian-installer/amd64/linux /where/your/tftpboot/k/debian-8
# Extract the initrd to modify it
wget http://ftp.debian.org/debian/pool/non-free/f/firmware-nonfree/firmware-bnx2x_0.43_all.deb
[...]
mkdir initrd
cd initrd
gzip -cd ../debian-installer/amd64/initrd.gz | cpio -ivdum
[...]
# You need the dpkg commdand to make your life easier (part of CentOS, Fedora or Mageia)
dpkg -x ../firmware-bnx2x_0.43_all.deb  .
ls lib/firmware/
[...]
# Should show you the firmware in the right place
find . -print | cpio -o -H newc | gzip -c9 > /where/your/tftpboot/i/debian-8.img
[...]

Now you can add an entry to your boot loader (grub2 in my case) for the network boot part (more info on Network boot and Debian at https://wiki.debian.org/PXEBootInstall)

menuentry 'Debian 8' {
  insmod gzio
  insmod part_gpt
  insmod ext2
  insmod iso9660
  linux /k/debian-8 ip=dhcp --
  initrd /i/debian-8.img
}

If you have another firmware to deal with, refer to the dedicated Debian page at https://wiki.debian.org/fr/firmware

After that your system should be able to network boot and install your Debian Jessie distribution on it.

The next step is to automate the installation, which consists just in changing the linux line in the grug conf file:

linux /k/debian-8 ip=dhcp auto priority=critical url=http://deploy-server/ks/debian-8 --

where the debian-8 file under the ks directory is just the preseed file you’ll build for your automatic installation.

Meet again next week in Paris and at LinuxCon NA 2016 in Toronto

2016/06/19

I’m just back from an HPE Event (the ETSS in Gothenburg, Sweden) and will soon have to start the next ones.

First, I’ll be this week in Paris to talk during a MySQL Meetup to cover automatic deployment, in particular with Redfish, images of reference (and a bit of MondoRescue) and automation (ansible e.g.) for DBA.

Then I’m again lucky this year to be retained as a speaker for the upcoming LinuxCon North America which will be in Toronto, Canada from the 22nd of August to the 27th. I’ll have a session on Continuous Packaging, mandatory for DevOps and a 3 hour Initiation Lab on Docker. Interestingly enough at the same time, Jérôme Petazzoni, Docker, will do one on Swarm !! Maybe we should coordinate and make them one after the other (I’d like to attend his Lab BTW !!)

I have now put all my lab material that I do for HPE on GitHub, to make it easier for attendees to report errors, and for others to use them and learn. I have content for Docker (Done with René Ribaud, Yann Allandit, Michael Bright), Git (Done with Clément Poulain), Redfish (Originally written by François Donzé and done with René Ribaud) and soon for Ironic.

Hope to be able to meet with you here or there !

Microsoft recrute … des RHCT/RHCE

2016/06/12

Si, si ! Et c’est même écrit ici

De là à penser qu’ils vont basculer dans une adoption large et massive des technologies que nous promouvons, il y a un pas que je ne franchirais pas. Pour avoir discuté récemment avec certains de leurs représentants d’Azure justement, je pense qu’il y a encore beaucoup de chemin avant qu’ils comprennent et ensuite admettent les vertus de ce que nous prônons.

Quant au job, dans ces temps difficiles, c’est toujours bon à savoir, même si pour un job de “Technology Solutions Professionals” il vaut mieux avoir “Une formation complémentaire dans la vente, services de livraison, les affaires ou marketing” ! En clair c’est de l’avant vente pour justifier auprès de clients combien Azure est la bonne solution pour héberger des environnements informatiques basés sur Linux. Et dire qu’ils ont au moins 25% de leurs clients qui les croient et qui l’ont fait ! J’espère que les autres hébergent eux-mêmes, ou font appel à des solutions plus ouvertes, ce n’est pas le choix qui manque.

Il est vrai que je ne devrais pas être la cible d’une telle campagne, ayant un job bien plus intéressant que cela, mais visiblement, ils ratissent large sur LinkedIn, et n’ont sans doute pas compris mon activité🙂

Bon, moi je retourne à mes patchs pour Ironic, Biforst et DiskImageBuilder;-)

python-redfish reached 0.3, first interesting version for others

2016/05/12

Back in 2015, during our internal TES event in Grenoble, a group of people gathered to talk about the new Redfish DMTF standard and how we could add its support into Ironic.

Since that, we have worked first on a new Open Source python library to interact programmatically with Redfish 1.0 based systems.

We tried to promote it during OpenStack Summits but except in Vancouver when I did a short video explaining that, not enough interest from the community. Of course we had very few to show at that time.

One year after, we have now reached version 0.3 of the project. And while that doesn’t sound a lot, in fact the libray is now usable by others to interact with Redfish based systems.

What it provides:

Our next steps are in multiple directions:

Hopefully, as you’re reading this, you’re interested by the topic, and we’d be happy to get your feedback, and more over your patches in order to improve our work, and speed up the development !

And don’t forget the HPE Open Source project available as well on github which provides much more capabilities, due to its support of the iLO and iLOCM extensions to the Redfish standard.

MondoRescue 3.2.2 is finally available

2016/04/29

I don’t know you, but I was waiting for this version for a long time. Way too long ! But I’m lucky, I’m not time based for releases. I’m really much more features based. And the one I really needed to catch for this version was UEFI support for RHEL/CentOS 6 & 7. HPE has teams and customers which were really needing that to work correctly, which is, from my series of tests, now the case. And I had confirmation from my colleagues that it was indeed working for them as expected just recently. So it was then time to publish !

While I was at it, I tried to fix at the same time some other annoying issues that were reported by the community on the mailing list, while leaving some other problems for the next time, as it was already too much delayed. So I know the Debian and Ubuntu users won’t get UEFI support working, as I had not enough time to test it (my test infra is now being used for trainings for 2 weeks), similarly for SLES 11 SP4 with UEFI (which uses elilo, while I now support grub, grub2 and syslinux, but this one doesn’t work correctly with UEFI IMO).

In addition to what was already done and reported here previously, I have hopefully fixed the following additional issues:

  • #777: Full UEFI support with grub 0.9 and grub2 configuration generation in addition to syslinux (limited to RHEL6, RHEL7)
  • #778 use dosfslabel/mr-label to recreate labelled FAT or VFAT FS
  • #757, #780 and #781 to fix md support
  • #792 to fix LVM support, by forcing fixed values for LE if percentage calculated for it is 0
  • #787 to fix USB support by cleaning the target dir of its syslinux content

In total 24 bugs have been officially closed and additional features improved or fixed as well. Look at the details in trac for the complete list of changes during these last 8 months of pregnancy of the new version🙂

But, of course, things do not stop here. In addition to fixes which needs to be done as mentioned upper, I’d like to add support for Secure Boot e.g. in the next version to have a full UEFI support, and work again on code refactoring even more.

For now, let’s take time to celebrate and hopefully this will be for you as users the best version possible of MondoRescue available to date !

Time to test what should be the latest beta of MondoRescue 3.2.2

2016/03/25

Well, I thought it could still take more time, but I had the possibility to dedicate enough time to fix and test the UEFI support for RHEL7 so I think it’s worth sharing and hopefully get good feedback from you before tagging it as final.

So a new beta of MondoRescue 3.2. is on its way. Please test it, especially if you try to do a backup/restore of a UEFI based system as this should now work flawlessly, including putting back EPS UUIDs. Should also work for other grub 2.x type of distributions such as Debian 8 e.g.

And share your report on our mailing-list !

I may have time to work on some remaining pretty annoying issues (#757 and #780 which look similar, maybe #781) but can’t promise for sure they will be part of this one which has been delayed way too much.

Current state of the changelogs are at http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/mindi/pbcl, http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/mondo/pbcl and http://trac.project-builder.org/browser/projects/mondorescue/pbconf/branches/3.2/MondoRescue/pbcl

Progresses with MondoRescue 3.2.2

2016/03/22

Of course I’m late. No surprise here😦 I think the only day I’m not will be for my death😉

So 3.2.1 was published the 31st of August 2015.
Version 3.2.2 was supposed to be a maintenance version of that one and was thus planned to be published mid October 2015. However, there were requests from various users to provide a version supporting UEFI boot mode on ProLiant servers. That enhancement request alone accounts for most of the delay encountered while developing this version as it turned out to be way more impacting than I thought.

So some bugs encountered on previous versions have been solved:

However with regards to the single enhancement request in that version to add UEFI support, many tickets were made:

The last one, is the most tricky to solve.
MondoRescue uses since day one a single bootloader, coming from the syslinux project, to manage all boot media it generate :
– isolinux for bootable CD/DVD and tapes
– syslinux for bootable USB devices
– pxelinux for bootable network setup.

This has the advantage of homogeneity and simplifies what the mindi tool has to generate in order to implement that support.

However, when using syslinux.efi to provide a bootable media on HPE ProLiant servers setup in UEFI boot mode, we obtain red screen showing a boot time crash.

Despite many attempts first with various options, then with the help of the syslinux community back in October 2015 (Cf: http://www.syslinux.org/archives/2015-October/024469.html and
the related thread) using beta versions of syslinux or the latest ProLiant Firmware, I never succeeded in obtaining a UEFI bootable syslinux based media😦

I’ve had contacts in December 2015 with the ProLiant Firmware engineering team, but we didn’t had yet a joint work on this issue. My plan would be to first diagnose precisely the origin of that error with this team, and then depending on the results to invite them around the table with the syslinux developers during potentially an upcoming LinuxCon NA (August 2016) to solve it for good.

As all that means we won’t have a final solution with syslinux.efi before end of summer 2016 at the earliest. Therefore, I’ve worked on adding support for other boot
loaders in MondoRescue.

Grub 0.99 support was added in August 2015 before the publication of 3.2.1 and we had success testing a full backup/recovery cycle on RHEL 6, while some other users encountered issues with that. Since November 2015, that should was considered fairly stable, and thus was made the default for RHEL 6 only.

Grub 2.0x support was added very recently (the 22nd of February 2016) with other modifications required to better support correctly mutiple boot loaders on multiple backup media (consolidation of code split into different functions in mindi). This was now tested successfully yesterday and I was able to make a full cycle with mindi on a physical server with RHEL 7. The restore now even present the MondoRescue menu to choose from.

However, while mindi now seems to be working fine for this configuration, this is still not the case for mondoarchive. I’ve started to unify the UEFI support between the 2 tools, and hopefully this week I should see positive results leading to a test version published for the MondoRescue community to try RSN ! And with your retunrs, I should be able to publish that long awaited version rapidly after around start of April probably.

I understand it took much more time than expected initially, and I’ve been myself giving wrong estimates for the delivery of the new version many times. Let’s hope this time I’ll be more accurate.
Hope this explains more the ground reasons why it was so, and the attempt to mitigate the real problem we have with syslinux on UEFI boot mode that may not be solved soon.

Project-Builder.org reaches 0.13.2

2016/02/24

Before leaving for a 15 days trip to the US, I’ve published the latest version 0.13.2 of project-builder.org.

It will be very useful for my colleagues and friends working on python-redfish to make local packages for testing that module, and then to publish it in upstream distributions such as Mageia and Fedora to start with.

That version improves git support, ssh support with ECDSA (for RHEL 7 distributions which have removed DSA by default), provides Fedora 22 and 23 support as well (with dnf).

Of course, project-builder.org is available for 160 distribution tuples because it uses project-builder.org to make that possible ! And if I can do that for this project, then you can probably do it as well for yours. The goal again is to help upstream projects to provide upstream packages for various distributions to help users, testers to install the software on their system. And so is the tool itself, you can download it easily and test it following the instructions described in the Lab document.

I have the intention to work next on improving the conf files to support regular expressions for the labels (to allow making conf entries such as minosdep fedora-2[0-9] = ….), looking at transforming them to use JSON instead of their own format (JSON wasn’t an option when I started back in 2005) and also looking at the Swagger2 perl module to help build a RESTful API around project-builder.org

As usual, that’s not the lack of idea which is preventing all that to arrive but the lack of time. So don’t hold your breath if you plan to do GET/POST request to build your packages with project-builder.org but it will come. And now this version is available, back to MondoRescue to fix the UEFI support !


Follow

Get every new post delivered to your Inbox.

Join 135 other followers