Posts Tagged ‘project-builder.org’

Meet during the Open Source Summit in Prague

2017/10/16

Getting approval for my participation at that conference was the toughest I ever had in the past, but I’ll be there again after having spoken in all the previous editions.

This time, this presentation will be made by a duo, as I’ve worked since the last LinuxCon EMEA with the FOSSology project to help them adopt a Continuous Packaging approach and Michael Jaeger from Siemens who leads the project will be with me to cover the topic. On my side, I’ll explain how project-builder.org was used, the setup done with Vagrant and containers to generate packages for 7 different Linux distributions (ongoing effort as we speak ;-), automating fully the production of packages as part of the LinuxFoundation build infrastructure for FOSSology. And of course the changes I had to make to the code to better support the project needs !

So I’ll be back in Prague (no problem as I love that city !) next week, to celebrate also the 10 years of project-builder.org, just after my week at DockerCon which is really starting today !

Advertisements

Meet at DockerCon Europe

2017/10/12

This is a late announce, as I wasn’t sure to be able to attend the event, but now everything is on track and I’ll be at DockerCon Europe next week ! One of my proposals has been selected as a 20′ session in the Community Theater and I’m glad as this is the first time I can participate to DockerCon. Project-Builder.org is using Docker since September 2014, so I’ll be able to celebrate the 3 years of Docker usage in pb after the 10 years of pb itself 😉 More seriously my goal is to promote again Continuous Packaging and demonstrate how beneficial it has been to introduce Docker in the chain, and how my upstream projects, as well as FOSSology are benefiting from it. Come to see that !

Of course, HPE is a platinum sponsor of the event so if you’re one of our customers, I’d also be happy to talk with you around your Docker projects as HPE is sponsoring my travel there ! (Thanks to the management to support again this despite hard times for travel requests and the HPE Next plan). Maybe I’ll still have a job when I’m back … or not 😉 Anyway, will see later as I’m now busy preparing my slides and demo for DockerCon among other stuff. See you there !

10 years of Project-Builder !

2017/09/24

Well, I think it’s worth to celebrate that milestone ! I’m now publishing officially the version 0.14.6 of project-builder.org for its 10th anniversary.

Everything started to solve an issue I had when I took over the maintenance of the MondoRescue project. It was very difficult in 2005 to consume the project. People had to download a tar file, follow a very succint recipe to build and install it and then being finally able to use it. The tool was supporting a lot of Linux distributions, but was difficult to use by these same distributions. So I started to build some shell scripts to help me deliver Linux distributions packages to solve that. I realized that I was duplicating stuff to address Fedora and RHEL, SuSE and SLES, Mandrake and Mandriva, … Even when building for other types such as Debian and Ubuntu or Gentoo. Typically all these distributions want a summary, a description, dependencies, changelog, … And I hate duplicating stuff. Also I realized I wasn’t the only one in need of this type of support. LinuxCOE, a peer project at HP was also in need of this, as well as many others.

So in 2007, I decided to rewrite my scripts in a more powerful language, perl (thanks Larry Wall ! and BTW I continue to find it way superior to a lot of younger ones, calling themselves more modern :-)). And version 0.5 was soon published as I used my vacation that year to code a lot for it.

It was way before I started to blog, and blog about it was among the first articles I decided to write. Of course, along the years, features have been developed, capabilities extended. I’m now able to generate project-builder.org it self and MondoRescue for 130 tuples of distributions. THat’s a proof this is possible, and that a tool like that can help achieving it.

10 years ago, I had to invent a configuration file format to support my needs. I wanted to have the least possible number of hard coded elements in project-builder.org. I created my own format, pretty simple mimicing the perl hash being: keyword param = value[,value]. At that time, YAML and JSON weren’t that popular, and I wasn’t aware of perl modules to support these, despite the fact they were existing, and my knowledge of CPAN wasn’t that accurate.
But at least I had inheritance mechanisms, and fucntions to deal with it, whch proved to be extremely useful to migrate.

So It’s now time, for the next 10 years to adopt a format, nearly as simple as mine, as easy to map to perl hashes as mine, and I chose YAML, benig lazy and finding it easier to manage manually (like mine was). Of course, this will have some consequences, as some old build environment don’t provide it so I may drop support for some very old distributions. If needed BTW something can be done as we have the source !! Version 0.15.1 is already in progress, and project-builder.org is now able to build itself, including in VMs or Docker containers for other distributions using that new format. And due to the mechanisms in place, I’ve kept easily all the features, just moving to a new format. I’ll start to work soon on migrating the conf files of MondoRescue to have another project working with that version. I’m also working in HPE with another team that is starting to use it as well, and I chose to adopt for them the new version to benefit from the way forward.

Finally, as I mentioned previously, the tool is now in use by the FOSSology project as well in the Linux Foundation build infrastructure. With the latest version 0.14.6, now FOSSology builds at least native packages for CentOS 7 on a CentOS 7 platform. Now we’ll work on the generation of packages for the other distributions the project want to support (which will be pretty easy now the infra is in place), in order to cover the topic during the Open Source Summit in Prague in October, providing I get approval to travel there (as I’m traveling quite a lot these days).

So happy birthday to project-builder.org and let’s start working on the next YAML based version, and help more projects adopt it !

It’s been a while

2017/09/12

When I look at the date of the last post, indeed it’s been a while since I took time to write here.

Many reasons to that, I took time off this summer to lead the infrastrcuture track at the RMLL, then to make music, for the 28th year in a row with 3 concerts and a diaporama, to visit Normandy with part of my family (Caen, Rouen, the Bayeux Tapestry, Honfleur, and many other places), back to work for some days and then doing another concert with the Variations ensemble I lead around François Ier in Monêtier les bains.

Then back to work. On top of my normal activities for HPE, I tried to incorporate various fixes into MondoRescue to finally publish 3.3.0, but that has again been delayed 😦 It seems I have a hard time publishing it, despite getting good patches or making them myself. I destroyed a system while doing tests, which forced me to update to Mageia 6 published meanwhile to put it back on track ! That didn’t help, and also I now need to check again but in VMs to avoid that issue and fix it for good (was too confident my modified code was working, but I was obviously wrong !). This is linked to fixes in mindi in order to fix size of initrd and its automatic computation. THat should be part of 3.3.0 as a consequence.

In order to solve support for multiple GPG keys signing my packages, I’ve had to go back to project-builder.org, and enhance it. Which lead to the publication of 0.14.6 (still not fully finished, despite code being frozen). And also I had work during my vacation time on chaging the format of project-builder.org conf files, from our own, to a YAML based one. For now, it’s just a change in format, not a semantic one, but that ill allow to improve some aspects, and also I get for free a syntax analysis with the perl YAML.pm module, which is nice (add -v to get the conf file concerned). So version 0.15.1 should come soon, once my 2 main users outside of myself won’t complain about bugs anymore.

One of these projects is FOSSology, and I should present the joint result of our common work with Michael Jaeger on the build infrastructure we put in place for the project to do Continuous Packaging at the upcoming Open Source Sumit in Prague in October, but it seems HPE doesn’t want to support my trip (I still didn’t get travel approval yet) and I may have to pay for it myself for the first time since I started talking at this conference back in …Prague in 2011. If you know anybody who would sponsor me, that would be great !!

Speaking of conferences, I’m this week at the Open Source Summit in Los Angeles, as part of my HPE job, where I lead a round table this morning on containers (are they the future of IaaS ?).

So despite having been quiet for long, Ive been busy as usual, and trying to improve the software I’m contributing to, andour global ecosystem, as well as the musical one. Hopefully you’ll get more news soon, as new releases and travel arrive !

Meet at the RMLL 2017

2017/06/07

This year the RMLL are coming back, from the 1st to the 7th of July and in my region, in St Etienne ! And this year, I am not only a speaker, but also a co-track leader for the Server track.

With my talks, You’ll be able to learn about the latest news for MondoRescue, project-builder.org, Docker, Redfish and python-redfish, Mageia and much more from my co-speakers (Ethereum, CMS, Cloud Native app, Unikernels, …) and even more from Keynote speakers as well.

Lodging is now available, so book quickly and let meet us there !

This week, gift continues with project-builder

2017/03/08

Normally, today I receive gifts, priviledge (really ?) of being older 🙂 But I can also make gifts myself ! So after python-redfish 0.4 yesterday, I’m announcing the release of project-builder 0.14.3 today (and maybe more to come later this week !)

This project seems to have a quicker pace than MondoRescue, maybe due to the fact I’m using it daily, and that it’s now used by python-redfish, MondoRescue, and also FOSSology (WIP more on that later again). I realize I didn’t announced previous versions 0.14.1 last July, and 0.14.2 last January, but lots have been added since.

I’ve worked on this version to improve deb support (no arch pkg) and especially deb repositories (with GPG keys) as well as OpenSUSE GPG signed repositories as well. Previous versions brought support for latest distros (Fedora 25, Ubuntu 16.10 e.g.), robust build using Docker in addition to VMs or chroot VEs, fixes some issues around conf files analysis, improving rpm package name for smooth upgrade between test and stable versions, renaming temporary dirs for consistency, as well as moving the main conf file outside of /etc reserved for local modifications, … Changelogs have all the nasty details.

One of the task I’d like to make now is to move the conf file format to JSON, to benefit from existing libs to manage them, and make them even easier to deal with. That would allow some other improvements afterwards, but too early to talk about.

Oh, and BTW, we decided to also publish python-redfish 0.4.1 (with its packages made with project-builder.org to keep this article’s focus 😉 ) to fix a nasty error we made when pushing to pypi. Enjoy as well !

Meet during LinuxCon EMEA

2016/09/23

After having talked on project-builder.org during the latest LinuxCon NA in Toronto, it’s now time to prepare for LinuxCon EMEA in Berlin.

I’l again talk about packaging, this time “Packaging for Linux Distributions with Docker“. And I’ll also replay the now popular (!) “101 Docker training” I’ve also delivered in Toronto, this time trying to deliver it before the one Jérome Petazzoni from Docker is doing on Docker orchestration, in order to allow for a complete overview of the Docker ecosystem.

Of course I’ll enjoy attending many sessions on various topics, and will see if in EMEA as well Docker Orchestration is the new Eldorado this year (In Toronto many talks and booth were dedicated to that theme). At least my colleague Mike Bright will cover it well !

See you around !

Project-Builder 0.14.1 is now out for LinuxConNA !

2016/08/19

Well, in fact I froze that version back in July, but didn’t officially announced it till today (vacations were just in the middle). So that version 0.14.1 is now available, it breaks compatibility with previous ones, so you’ll need to update your VMs/VEs with it as well if you want to use it correctly (that’s why it’s still version 0.x !)

But I’ve been able to build a lot using docker containers with that version, and I’m now more and more happy with these results. Especially because it’s easier to fully automate an environment from absolutely nothing except project-builder.org to working packages. I already described it when I released 0.13.1 last year through a short tutorial, but it’s really working reliably now for a large set of distributions:

$ docker images | grep pb-pb
pb   mageia-5-x86_64-pb-pb              fc479023c876        3 weeks ago         413.9 MB
pb   opensuse-13.1-x86_64-pb-pb         2c31c56e7bda        3 weeks ago         354.3 MB
pb   opensuse-13.2-x86_64-pb-pb         c8edeba0440b        3 weeks ago         438.2 MB
pb   opensuse-42.1-x86_64-pb-pb         8e91d2b584ed        3 weeks ago         461.5 MB
pb   debian-7-x86_64-pb-pb              0c133063d4be        3 weeks ago         449.8 MB
pb   debian-8-x86_64-pb-pb              8fb919480351        3 weeks ago         588.3 MB
pb   centos-5-x86_64-pb-pb              c8b6f1a0846b        3 weeks ago         706.6 MB
pb   centos-6-x86_64-pb-pb              a3c1ea318031        3 weeks ago         545.9 MB
pb   centos-7-x86_64-pb-pb              d5790ee97e13        3 weeks ago         392.1 MB
pb   ubuntu-12.04-x86_64-pb-pb          053f112ed939        3 weeks ago         398.2 MB
pb   ubuntu-14.04-x86_64-pb-pb          bc239427fc3a        3 weeks ago         501.7 MB
pb   ubuntu-15.04-x86_64-pb-pb          e6cf8d0fafd0        3 weeks ago         503.8 MB
pb   ubuntu-15.10-x86_64-pb-pb          dba3b580bdc8        3 weeks ago         550.1 MB
pb   ubuntu-16.04-x86_64-pb-pb          02a337752837        3 weeks ago         522.9 MB
pb   fedora-21-x86_64-pb-pb             07223685d24f        3 weeks ago         965.1 MB
pb   fedora-22-x86_64-pb-pb             211b865c1ec6        3 weeks ago         749.5 MB
pb   fedora-23-x86_64-pb-pb             f6012ff7b6b8        3 weeks ago         879.6 MB
pb   fedora-24-x86_64-pb-pb             6d878b489584        3 weeks ago         678.5 MB
pb   gentoo-nover-x86_64-pb-pb          b6ead3da5b85        4 weeks ago         1.392 GB

But I’ve not really worked yet on the ideas expressed when I issued 0.13.2 in February this year.

Instead I’ve done other improvements such as managing signed deb repositories (was asked for a long time), have a better naming for temporary versions (allowing easier updates between temporary versions and also with stables, but creating the incompatibility with 0.13.x), docker tests for lots of distributions (paving the way for automatic testing here from nothing to package build), adding support for pbr, standardizing temporary directory names, …

That will be the version I’ll use to make demos during my presentation next week at LinuxCon North America in Toronto as already announced. Come to see it live and so that we can discuss continuous packaging !

Meet again next week in Paris and at LinuxCon NA 2016 in Toronto

2016/06/19

I’m just back from an HPE Event (the ETSS in Gothenburg, Sweden) and will soon have to start the next ones.

First, I’ll be this week in Paris to talk during a MySQL Meetup to cover automatic deployment, in particular with Redfish, images of reference (and a bit of MondoRescue) and automation (ansible e.g.) for DBA.

Then I’m again lucky this year to be retained as a speaker for the upcoming LinuxCon North America which will be in Toronto, Canada from the 22nd of August to the 27th. I’ll have a session on Continuous Packaging, mandatory for DevOps and a 3 hour Initiation Lab on Docker. Interestingly enough at the same time, Jérôme Petazzoni, Docker, will do one on Swarm !! Maybe we should coordinate and make them one after the other (I’d like to attend his Lab BTW !!)

I have now put all my lab material that I do for HPE on GitHub, to make it easier for attendees to report errors, and for others to use them and learn. I have content for Docker (Done with René Ribaud, Yann Allandit, Michael Bright), Git (Done with Clément Poulain), Redfish (Originally written by François Donzé and done with René Ribaud) and soon for Ironic.

Hope to be able to meet with you here or there !

python-redfish reached 0.3, first interesting version for others

2016/05/12

Back in 2015, during our internal TES event in Grenoble, a group of people gathered to talk about the new Redfish DMTF standard and how we could add its support into Ironic.

Since that, we have worked first on a new Open Source python library to interact programmatically with Redfish 1.0 based systems.

We tried to promote it during OpenStack Summits but except in Vancouver when I did a short video explaining that, not enough interest from the community. Of course we had very few to show at that time.

One year after, we have now reached version 0.3 of the project. And while that doesn’t sound a lot, in fact the libray is now usable by others to interact with Redfish based systems.

What it provides:

Our next steps are in multiple directions:

Hopefully, as you’re reading this, you’re interested by the topic, and we’d be happy to get your feedback, and more over your patches in order to improve our work, and speed up the development !

And don’t forget the HPE Open Source project available as well on github which provides much more capabilities, due to its support of the iLO and iLOCM extensions to the Redfish standard.