Posts Tagged ‘vcs-pkg’

Project-Builder.org 0.12.1 is now available

2012/05/30

As promised (or threatened (c) Bryan !) I have now published a new main version of project-builder.org, the 0.12.1 !

Since my previous post, I worked a lot on it to make it available in fact, and it took finally just a bit more than a year ! But it was hopefully worth the wait 😉

In order to ease the publication, I started to write a script to make project-builder.org tests in an automatic fashion. It helped me to find lots of small or less small remain issues before the release, so I’m more confident than before that this version is a good one indeed ! This was especially needed due to the fact I received this time other contributions.

Indeed, I received great contributions from Eric Anderson but I had to revert some of them, as the project wasn’t completely ready to deal with it. the major one being to error out immediately. That broke in fact a certain number of use case, where errors should be tolerated, and even after trying to catch them, I was unsuccessful in some case, so made a new parameter in the configuration file for that. So you can benefit from Eric’s work by activating pbstoponerr (and also pbshowsudo if you want to see what pb does behind the scene with sudo commands) in the /etc/pb/pb.conf configuration file. default is to have both of them off for now. It may change in the future, when the code handles more efficiently all error cases.

Also what I missed to detail last time is the new version number. This version is not compatible with the previous one, so you’ll have to update your VEs/VMs/RMs in order to benefit from it. Especially for test versions, now we use the full version name for producing the tar files, and the old version won’t build packages in the build environment when called from a 0.12.x external pb.

In order to do that, just use pb -p myproject setupvm (or setupve, setuprm) to update all your VMs in a row.

Project is still available at ftp://ftp.project-builder.org/ and you’ll find there all the versions made for all the distributions tuples supported (120+ as of today). Use trac to report problems or enhancement requests for next version, that I promised will not be next year but earlier !!

Enjoy and feel free to join our mailing-list to discuss more about it.

Proposition of Cross-Distro Mini-Conf for Linux.conf.au 2012

2011/07/02

Time has come again to think to our friends down under ! Since I was there in 2007 for a MondoRescue conference I think this is really a place to be in the FLOSS ecosystem when possible; Too bad it’s so far away from France 😦 Travel costs are not light either.

But I thought I should propose the follwoing mini-conf, as the one I attended in Fosdem 2010 on the same topic worked very interesting, and allowed to start some joint collaboration that I think is fruitful and avoid/reduce fragmentation.

I hope it will be accepted to give me an opportunity to fly there and meet with the great community which gathers there each year.

So here it is (Thanks to the great example provided by Martin Michlmayr last year)

The “Cross-Distribution” mini-conf at LCA 2012 (Ballarat, Australia; January 16-20 2012) is for people interested in cooperation between Open Source distributions. Topics include contributors agreements, licensing policies, packaging best practices and tools, sharing patches to upstream projects, communications, working with upstream Open Source developers, translations, governance, workflow in place, version control systems, bug reporting management and sharing, vcs-pkg.org, funding, marketing, lessons from your experience, and whatever related topics people would like to bring up. Whether you are a single contributor working on a single package, or leading a full distribution, managing a full distribution infrastructure, or representing it in the press, whether it’s famous as Debian or less as AbulEdu this mini-conf hopes to bring different people together to exchange thoughts and facilitate discussion about these topics.

Presentation submitters should feel free to suggest their own topics. Here is a list of potentially interesting subjects (in no particular order):

* Building software packages on a large scale
* Choosing valid licenses and copyright policies
* Patch management
* Working with upstream
* Forking a distribution
* Experiences in distributed development
* Managing bug report and enhancement request
* Tools to support making a distribution
* Translation of a distribution
* Reuse between distributions
* Collaborationbetween distributions
* Distribution structures
* FHS
* Distribution Development Management and Governance
* Distribution Corporate Governance
* Lessons learned in building up your distribution
* Marketing
* Communication
* Copyrights
* Applying your copyright and license choices

We hope to receive proposals for:

* 50 minute expert panel discussions
* 50 minute full presentations
* 25 minute half presentations
* 5-10 minute lightning talks (e.g. success stories, …)

To submit a proposal, please contact Bruno Cornec at bruno.cornec@hp.com and include the following information:

* Your name
* Brief bio noting any previous speaking experience
* Talk title
* Brief outline of your proposed talk
* Notes of any special equipment / facilities you may require

Project-Builder.org 0.11.3 is published

2011/05/27

I recently queried users of project-builder.org to know would they would consider important to get before we could tag it as version 1.0. Of course, the community being very small, I didn’t get answers for that. So either the tool is perfect and they don’t need anything, or there is so much to do that we’re far from that point 😉

Well I think we’re in the middle of the road here. Most of what *I* wanted to have for a 1.0 version is mostly there. What remains to do is a better support of signatures (especially with my GPG agent on deb packages with debsign which timeouts :-(), some fixes (especially #93 for MondoRescue and #99 for better gentoo support with test versions). Not sure about HP-UX support, or other OSes, as that could come later.

However, as I had a couple of fixes in my tree, I thought that it would be useful to publish Project-Builder.org 0.11.3 in order to have a better version around. Hopefully, will get some more users and/or feedback that will allow me to know if I continue on this 0.x track or if other do think that we reached something nearer from 1.0.

Project-Builder.org 0.11.1 has now been published

2011/03/09

As just announced today, Project-Builder.org 0.11.1 has now been officialy published. All the latest news are detailed in the announce.

Now I’l probably release soon a 0.11.2 to fix some small annoying bugs that my colleagues using it reported in between, and in order to use the best version availble for the training I’m organizing next week (presentation ° Lab). So expect hopefully a 0.11.2 over the week-end !

Continuous Packaging Build Cloud with project-builder.org

2011/02/10

So now that I have your attention with this interesting Cloud buzzword, I can develop the idea i’m adding to project-builder.org at the moment.

Currently project-builder.org supports building in Virtual Machines (VM) or Virtual Environments (VE) aka chroot, which are all managed by the machine running the pb command. But with the expansion of the project I’m working on in our joint HP/Intel collaboration, we want to be able to support Continuous Packaging also on machine for which we can’t have a VM or a VE hosted, such as an Itanium HP-UX one or a Sparc Solaris one.

Of course, it’s always possble to log on one of these systems, install pb, give access to the VCS/CMS repository and voila, you’ve shiny new packages. But you (as I) want more no ?

So I’m realizing a new series of patches so that pb can launch build operations on a Cloud of machines, as long as you can connext to them through SSH. In fact the process is very similar to what pb currently does to build in a VM, except that this time, it can be a remote machine. I found it much easier to support than I thought in the code, proof that the design is not too bad and allows for easy improvements.

Also I made recently some good code cleanup after my stay at the FOSDEM where I attended a lot of perl sessions, that gave me energy to do that … during the 4 and half train trip I had to go back home.

And today, it took me less than 20 minutes and no cloud in the horizon to prepare 2 new VMs in order to support Debian 6.0 for future project releases. Tooling stuff is really helpful.

So expect a new version of pb RSN, in order to provide that additional support.

Project-Builder.org: 0.10.1 published and next steps

2011/01/20

So, finally Project-Builder.org 0.10.1 has been published.

A lot has been done in that version, so look at the ChangeLogs files for details. In particular, that version of pb is not backward compatible with the previous one, thus the change of version (from 0.9 to 0.10). Which also means that it is highly recommended to update your VMs/VEs with that new version of pb with the setupvm|ve command.

Another incompatibility is with the cms2* commands which now really take the content of the CMS/VCS not the sandbox status. In order to get the previous behaviour, please use the sbx2* commands instead. man pb is probably helpful 😉 In particular, if you build from files (via http/ftp) and not a VCS, you’ll have to use the new sbx commands to have your package build correctly, as there is no CMS in that case.

This version now supports parallelism during the sbx|cms2build, sbx|cms|build2vm|ve phases which can drastically reduce build time for packages. For this to work, as explained earlier, you need the perl module Parallel::ForkManager installed on your system. The VMs|VEs may benefit from it but it is not required, and only useful if you have lots of cores on your host, to enable multi-cpus VM|VEs.

A small bug has been found post-release for those of you using the announce command. What ? nobody ranted ? Well, as I’m probably the only one using that feature, sounds obvious 😉

I’ll now also work more with my FOSSology colleagues to help them produce packages for their just announced 1.3.0 version with pb.

More interesting is what will come next: As part of our joint work with Intel in our Solution Center, Project-Builder.org is part of a new solution stack we’re developing with their collaboration around RISC/Unix to IA/Linux migration. Continuous packaging was identified as a key aspect, and helping our customers support multiple platforms in parallel is indeed very useful. So it was agreed, also following an internal presentation and some surveys made during it, that our next supported platform had to be HP-UX.

So I’m working with the help of Josh Zhao on adding this, and this is expected for 0.10.2. And another colleague Nicolas Doualot is helping setting up a demo environment to showcase our Continuous Packaging approach.

Signing packages is also high on my list, but is still a problem on RPM based distro. Hopefully I’ll be able to come up with a solution after some FOSDEM discussions !

project-builder.org: nearly ready for 0.10.1

2011/01/08

Next version of project-builder.org won’t be 0.9.11 as expected ! This is due first to the incompatibilities introduced with 0.9.10. a pb 0.9.10 cannot work in a VE|VM with the current devel version of pb, and the VE|VM will have to be updated with setupve|vm.

Also on top of that quite a new set of features have been added since last version, in particular the parallelism of tasks. After the work on sbx|cms2build, I’ve done now build2pkg and also build2ve. It implied more modifications in order to have each script file generated during the preparation phase unique so that each VE could get it correctly. I’ve tested this night with 2 VEs to build pb packages, and it is now working quite well, and that’s where having a multi-core, multi-thread machine such as min base on a Core i7 Xeon helps a lot aand makes a difference.

The last remaining part is to also add that feature to VMs. I’m very near from it, I just need to handle the fact that communication based on SSH with each VM has to use a different port, whereas now it’s the same for a given project. Hopefully this will be done over the week-end, which will allow publication of the new stable version of pb in sequence.

This feature is also useful for my colleagues working on the FOSSology project to generate packages for their 1.3.0 version on their side.

project-builder.org: more parallelism

2010/12/14

Looking around to improve some of the performances of project-builder.org, I found an interesting perl module: Parallel-ForkManager.

Its man page shows how you can quickly parallelize a loop in order to benefit from the numerous cores that each system today has. It took me just a couple of minutes to transform project-builder.org, so that the generation of all build files becomes a parallel operation, thus reducing dramatically the time it was taking – at least on my Xeon X5680 @ 3.33GHz !!

Very easy as you can see in the changeset, and very efficient. And if you don’t have the module, you can still work in serial mode, but once you tested it, you don’t want to go back 😉

Next step, is to also apply that feature to the build in itself on multiple VMs or VEs, but that will require a bit more work, as the loop is not so easily usable as of now. More on that later, once the code is re-architectured.

Project-Builder 0.9.10 announce for LinuxTag 2010

2010/06/09

This year, I’ve been very happy to be selected for the first time in 10 years for conferences at 2 majors events in Europe, and on one of my preferred subject: packaging. After Fosdem in February, tomorrow, I’ll present project-builder.org during LinuxTag in Berlin.

It will be the first talk covering the newly announced 0.9.10 version, which adds Ubuntu 10.04 support, and also a more robust version of rpmbootstrap. Hopefully making these presentations will help gathering other inputs and users so that the tools become useful for more FLOSS projects.

So would be happy to meet with you to discuss around packaging, disaster recovery, cloning, HP, Linux or anything related to Open Source … and early music as well 😉

Prokect-Builder.org 0.9.9 is finaly out

2010/05/01

I decided to freeze the current state of project-builder.org to deliver it under the 0.9.9 name.

It’s more to give access externaly to the current state of the art of this project, rather than it represents a final point of features in fact. A certain number of small bugs have been fixed, so it’s useful at least for that. Also I introduced with this version a new tool (and package) called rpmbootstrap, whose role is to create a chroot of a RPM based distro. I’ve been able to create CentOS and Fedora chroot up to now. Still working on it for OpenSUSE and Mandriva. It’s largely based on ideas of rinse, and debootstrap, but tightly coupled with pb and reusing some of its interesting features around distribution detection.

And at the same time, support of debootstrap has also been added so it’s possible to easily build now with chroots for most distro.

And I have passed a lot of time trying to improve documentation. Pod doc was already quite good for functions. Now in addition, we have the Lab for easy setup, and a complete documentation of the pb.conf possibilities, which will allow to work now on the possibility to use tool to manipulate them (Target is to use Config-Model). And a website has been setup. Not as complete as I’d like it to be, but a start 😉

And I’ll be presenting the tool, and re-deliver the Lab during the upcoming HP Technology Summit in Las Vegas in June.

Hope to meet with some of you there, and also next week in Valencia in Spain for the Red Hat EMEA Partner Summit