Posts Tagged ‘Mondorescue’

Last day at LinuxCon NA 2014

2014/08/27

Today the keynotes were dedicated to Openness and Hardware. The first was from a company, Makerbot, which spoke a lot about Openness, but that I saw more as trying to sell their 3D printers, rather then anything else :-( I even learned later from an attendee that they even tend to block innovation with their patents ! So maybe the LinuxFoundation should take care of not “giving” opportunity to such actors to speak to an Open Source audience if their state of mind is nearer from the closed source business. Having a community sharing 3D design doesn’t sound sufficient to me.

Jonathan Kuniholm

The second keynote was on the topic that even is 3D printing is such a hype at the moment, disallowed people still have a hard time finding useful prothesis, modern ones, les expensive ones, or building their own. I really encourage you to listen to Jonathan Kuniholm (the keynote doesn’t seem to be online, but TED provides one very similar). This was puzzling for me to see how few technology is helping people like him :-( So I think that if you have time, you should look at helping his initiative at openprosthetics.org/ rather than inventing yet another piece of software just because the existing one doesn’t happen to please you.

IBM Keynote

Finally we had the “usual” IBM keynote, showing how Linux on Power was great, and presenting the foundation built around it. But if you look at uses cases, you see that most of them are academics, where probably the hardware was given so it weakened the talk IMO. Of course, I’m working for a competitor, so I’m not completely neutral here. Anyway having a portable Linux is extremely important, but I think it will reveal its capabilities on x86 (well it has of course!) or ARM. It had on Itanium or Sparc or Power (Linux can enable them) but the problem is that market doesn’t want such high-end platforms anymore, as they were representing a closed approach even if that has changed since. Openness is what allows mass distribution today (in processors as those mentionned, or software as Android and hopefully Linux on the desktop ;-))

UEFI Summit

After the break, I passed my day in the UEFI mini-Summit. The goal was different from last year PlugFest during LinuxCon. Instead of targetting developers, the goal was to expain the technology to potential and existing Linux sysadmin or devops. And I think it went pretty well with regards to demystifying how UEFI works woith Linux, including SecureBoot and brought back the discussion at a technical level rather than an emotional one.

An introduction talk by Dong Wei, HP served as positioning the UEFI Forum, the various groups in it (with the inclusion of ACPI), the history of UEFI, current status, and helped put everybody at the same level.

Q&A session

After that we had a (always too short IMO) round table were the audience was given the possibility to ask questions to the panelists. And there were very tough questions asked around the usefulness of UEFI, the lockdown brough by SecureBoot, … and everytime clear and honets answers were given showing why UEFI is useful, why SecureBoot help increasing Linux security without restricting users possibilties and control over their platform. All in all a lot of myths were just addressed during that Q&A session which was really interactive.

After that, we had more formal presentations:

  • UEFI Secure Boot – Strengthening the Chain of Trust – Jeff Bobzin, Insyde Software & Kevin Lane, HP
    This session was mainly about how Secureboot is working from a technology perspective, and the various solutions existing with Linux and its boot loaders to use it, benefit from it as it really increase security by providing a chain of trust from firmware up to the kernel+intrd booted, with either standard UEFI keys or its own ones.
  • Jeff Bobzin & Kevin Lane

  • UEFI Test Tools for Linux Developers – Brian Richardson, Intel & Alex Hung, Canonical
    This session was on FWTS from Canonical which provides a UEFI firmware and ACPI test suite, used alot by manufacturers to check the conformity of their platform with the UEFI and ACPI specifications. Chipsec and LuvOS were also covered which provides other areas of test with regards to respectively security and an integrated Linux distribution calling all these tools and more, both developed by Intel.
  • Brian Richardson

  • Building ARM Servers with UEFI and ACPI – Dong Wei, HP & Roy Franz, Linaro
    This session was to give a status on UEFI support for ARM architecture, and was pretty interesting for me as I had no clue on where we are on this domain. And it seems they are catching up with Intel Architecture now and should be at parity very soon. ACPI is still less advanced, but will be there for ARM servers as requested by customers, whereas device tree will probably remain what will be used on nn server platforms.
  • Dong Wei

  • Self-signing the Linux Kernel (the hobbyist approach) – Zach Bobroff, AMI

    This last session was IMHO the best of the serie, because it was demo oriented (and I like demos !) and more over, it just worked !! The goal was to show how to register its own key used to sign its own kernel with SecureBoot, and rebooting a machine with and without key loaded to demonstrate the increased security brought by that mechanism. Was very clear and illustrative of what was described during the first session of the mini-Summit by Jeff and Kevin. Zach did an excellent job explaining each step and provided great details on how all that works, and finally showed to the audience that we shouldn’t be afraid of the feature, because we have the possibility with the shim bootloade to use our own keys without issue.
  • Zach's pres

You can listen to all these presentations at the UEFI web site. And I think it’s worth doing so for those who still have questions on the SecureBoot topic, as it will enlighten you and remove and barrier you may still see there.

UEFI Summit end

The event was then over, so it was time to benefit from my speaker gift, which was the possibility to use a boat and have a cruise around Chicago, which I did with Dong and it was a very good idea from the organizers to offer that gift. Hope the pictures will give ou a good idea of how we enjoyed it.

MondoRescue 3.2.0 stabilisation

2014/06/22

It took me time for sure. I started to work on this precise branch in October 2013. At that point, I was already happy to have a compiling version as I’ve made lots of memory management changes. But of course, it was not that simple ;-)

Then in march 2014, again, I thought I was ready to publish something ASAP. Argh !! I was wrong again.

But now is time for you to test it. I’ve fixed most of the issues I had remaining at that time (isolinux menus, NFS on Mageia 4, change of network NIC name). But the main one which took me hours to fix is linked to systemd. I’ve concentrated all my tests on Fedora 20 (which is very near from RHEL 7 that is an important target of course). But it’s not before a discussion with a systemd developer during LinuxCon Japan that I understood where the problem was.

At restore time, mindi init script was mounting the /dev filesystem as tmpfs instead of devtmpfs. Once I made the change, systemd-udev was able to be launched and work correctly for me at restore time, so the restore of fedora 20 was starting to work.

Of course, after that, I had other points to fix to name a few:

  • fully support systemd and its multiple compagnon binaries
  • fully support grub2 and its installer
  • avoid some memory management issues in corner cases using a lot valgrind
  • Remove from mindi 2 functions replaced by perl code (ReadAllLink by mr-read-all-link and ProcessLDD by mr-process-ldd)
  • cleanup of old useless file in /tmp suposedly used for mondo/mindi commnuication
  • Adds lvmetad daemon to list and launch it at restore time as new LVM version seems to need it
  • Rename the conf file mondo-restore.cfg into mondorestore.cfg for homogeneity
  • NETFS-SERVER-MOUNT remains the only built file by mondo for net configuration, lots of code cleanup and mindi now uses mr-net-get-config to build the network conf
  • Add support for latest Fedora keymap file (/etc/vconsole.conf) and non us keyboard detection

So that explains the delay you’ve seen in the publication of test versions.
But now, It’s time !!!

I’ve been able to make a full backup / restore / restart of the restored fedora 20 distribution in a full automatic mode without issue anymore ! Yep, I deserve some vacation !!!

So it’s your turn now to test this version and tell me what I’ve missed, and if that doesn’t create regressions for your prefered distro.
Available as usual at ftp://ftp.mondorescue.org/test

Now this version requires in addition to the afio or buffer packages also the perl modules developed for MondoRescue (at the same place) and for project-builder.org located at ftp://ftp.mondorescue.org/

Not all distros have been built correctly yet. Let me know what is missing, I’ll try to update during my week in Barcelona. I’m particularly interested by Fedora users for now, and more generaly other systemd users.

If tests are globally positive, I’ll publish it early July. And I’ll look at porting some of these changes back into 3.0.5 and add one or two features that have been asked for some time by some customers, but we’ll have to prioritize stuff. Again give feedback here.

Still working on MondoRescue 3.2 to make it available ASAP

2014/03/14

Even if stuff do not progrees at the speed I’d like them to progress (lots of travels on HP side since early 2014 and 4 concerts to perform on the private side) I’ve tried to improve the 3.2 version I published unofficially as beta. Interestingly enough, even when I do not announce that packages are delivered, there are people who do use them !!! Which gave me some feedback (you can guess it wasn’t that positive), so in fact it’s already my second delivery :-) and it contains some interesting new features:

  • mindi now uses the new mr-kernel-get-modules perl script which allows now in mindi to just mention end modules names andd not dependencies anymore, which are now computed by the script !! This will help a lot to maintain the list of modules, which was always impacted by low-level dependencies changes at kernel level.
  • Support of symlinks for newest distributions based on systemd such as Fedora, Mageia, … is now finally working !! Again this was done exporting the existing wrong code into a separate perl script which now operates correclty. This is part of the global willingness to recode most of mindi and some of mondo in perl. This took quite a long time, as of course, we need to stay compatible (a word systemd team doesn’t care about of course) with other tools, and older distributions. Side note, this is probably one of the reason MondoRescue is still appreciated by its community :-)
  • The introduction of a dependency on a perl function was incorrect and people trying to install from packages gave feedback that they had errors dof course ue to that. This is now fixed, as project-builder.org indeed had an issue because a low level function was depending on a higher level function not part of the perl modules provided for MondoRescue. With 0.12.5 of project-builder.org this is completely solved.
  • Now I still have regressions with the isolinux menus, NFS on Mageia 4 and systemd not working anymore (change of network NIC name is the root cause). However the ldlinux.c32 issue for syslinux > 5.x is now solved.

Next week is the TES, so won’t have much time to work on it. Expect news the week after.

project-builder.org 0.12.5 is now available

2014/02/07

Finally, after having published the version 0.12.3 on time for Linux.conf.au, I found some remaining bugs in it (problem of time based realease and not enough time to test), then I made a 0.12.4 version soon after as promised to solve these, but still missed some aspects, and those are now solved with this 0.12.5 version.

So this version fixes an issue with messages printed which were done in double, and also the way we manage naming of Debian versions, following the recent change for Debian 7 (using now only the major version, and not mentioning the minor version which is now useless).

I also fixed a dependency issue, seen when trying to install the beta of MondoRescue, where a function wasn’t provided in the perl-ProjectBuilder module, while used in the code :-( This is now solved, and I’ll now start working on the next MondoRescue beta which should now be installable with yum/apt-get/urpmi.

This project-builder.org version should now be stable to use for some weeks, as I do not plan to make big changes in it, as this is perfectly fitting my needs now. Of course, more could always been done, but I can’t advance on all sides at the same time, so now it’s MondoRescue‘s turn !

Enjoy !

It starts as a happy new year indeed

2014/01/04

Well, would you really be surprised if i say that the begining of 2014 looks as promising (and as busy of course) as the end of 2013 was ?

While travelling to Linux.conf.au in Perth (28 hours door to door but could it be a better start for 2014 !) I had some time to look at Project-Builder.org, subject of my conference on Monday and also to MondoRescue.

For Project-Builder.org, I’ll publish a new version for LCA (0.12.3) which will fix rpmbootstrap for lastest Fedora (18-20), Mageia 3 and also other recent distros. Also as SVK is now broken on Mageia 3, I’m using git-svn to work on my laptop when remote, and thus pb supports it now. Some various fixes since last year release will also be done (better printing with VM/VE name for logs, improved ntp support, pbdistrogetparam command to easy shell interface…), but globally the software is pretty stable for my usage, so now it’s your turn to tell me what would be useful for you in it. I also need to make a newer presentation for Monday with a short demo.

For MondoRescue I’m not ready yet to publish a final version, so I have made for now a first beta for the 3.2.0 version. First this will be a new 3;x version as there are lots of changes in the code that may make it less reliable due to the adoption of dynamic memory allocation instead of static in a lot of functions. If I compare 3.0 and 3.2:

  • 3.0
    • total mr_asprintf usage : 646
    • total sprintf usage : 674
    • total strcpy usage : 560
    • total fgets usage : 79
    • total malloc_string usage : 367
    • total MAX_STR_LEN usage : 370
  • 3.2
    • total mr_asprintf usage : 1223
    • total sprintf usage : 166
    • total strcpy usage : 388
    • total fgets usage : 28
    • total malloc_string usage : 129
    • total MAX_STR_LEN usage : 221

Which means that a lot has been done in code changes between 3.0 and 3.2. But also a lot remains to do if we want to have no malloc_string nor MAX_STR_LEN at all. And also if I made mistakes in the mr_malloc/mr_asprintf/mr_free usage, the commands will crash. THat’s why I need your tests here ! I’ve made a full bakup restore cycle (which has revealed some issues that I fixed), but I’m far from being confident in the result.

Another consequence is that this version will probably have less fixes than previous ones as most of the time has been passed on code modernization.

And it will require new mindi and mindi-busybox versions. And these versions should finally work for systemd crazyness of removing /bin and /sbin (which broke MondoRescue for a too long time now). And you’ll need new perl modules in order to have it work correctly as I finally rewrote some bash mindi functions into perl to have what I consider better code.
Which means Fedora 18-20, Mageia 3 should work out of the box (at least my Mageia 3 now works correctly).

And expect again more pacthes to MondoRescue durnig the 28+ hours back home at the end of next week ;-)

So I wish to all readers a happy new year 2014 and hope it will start for you as it has for me !

Visit me at HP Discover in Barcelona

2013/12/09

Again a quiet active month. I’m just back from one week for an internal event in Houston US, and I’m now in Barcelona for HP Discover EMEA 13. So if you are around and want to talk about Linux, Open Source or early music, go to Pavillon 5, booth 5530. I’m demonstrating the newly announced IC4RHEV which is officially announced during the event. You can also see it on Red Hat booth if you prefer to talk to the real guys ;-)

I took time during my travels to improve the 3.2 version of MondoRescue, but as expected, it will require some tests to ensure the new memory management modifications are stable. Hopefully a beta next week.

First UEFI PlugFest for Linuxers

2013/10/31

After the 3 days dedicated to LinuxCon US 2013 in New Orleans, it was time to contribute to the UEFI Plugfest organized for the first time as a co-located event.

So what is a UEFI plugfest ? Well it’s a place where hardware manufacturers and software producers meet to check the compatibility of their implementations with regards to UEFI. So Every hardware manufacturer brings some systems, sometimes early units or prototypes, and try them with the latest operating systems available to find out potential issues, some other bring cards to see whether their UEFI driver works fine on computer manufacturer and operating system producers want to try their latest version on these often brand new systems.

UEFI PlugFest

I think it was a brilliant idea to mix the 2 populations for multiple reasons:

  • UEFI members were for sure impressed by the technical knowledge floating around, and employed in such an open fashion, which is not the standard way of working of this standard body.
  • Linux kernel members could exchange with manufacturer representatives of UEFI systems which definitely helped reducing all the FUD around this technology, in particular Secure Boot. They also had the opportunity to test some not yet available hardware platform to ensure their distributions/drivers/tools were working fine or fix them if that wasn’t the case

UEFI PlugFest - Samer El-Haj-Mahmoud, HP

So in the HP area, under the lead of Dong Wei who is UEFI Forum Vice President and HP Fellow, we tried with 2 colleagues various Linux distributions (and even Windows, but not me !) on the 4 systems that were around. And some findings were interesting !

UEFI PlugFest - Dong Wei, HP

  • Debian 7.1 had grub issue at boot and we were not able to install it
  • Mageia 3 has no UEFI support yet and we were not able to install it easily. However, support is planned for Mageia 4, and some info have been published recently to detail how to perform UEFI based installation.
  • Ubuntu 13.10 provides all what is needed to install in a UEFI compliant environment, thanks to their documentation. We were also able to test SecureBoot with success with their version of Matthew Garrett‘s shim bootloader, signed by Microsoft. They are also working on an interesting tool: FWTS aka Firmware Test Suite, which should be adopted by all distributions IMHO in order to have (for once !) a single tool able to perform firmware compliance tests for a Linux environment. Easy to use, pretty comprehensive, reports lots of useful info. Too bad that they are not providing their certification tools online anymore :-(
  • OpenSUSE 12.3+ again has what is needed for UEFI support. Same mechanism with a shim bootloader, but this time signed multiple times by Microsoft and SUSE. However, this requires a more recent implementation of the UEFI specification, which wasn’t the case on all our system during this event. SUSE provides in particular an excellent documentation on UEFI support, including the possibility to sign its own kernel with pesign in order to use it with SecureBoot.
  • Fedora 19 provides mostly all what is needed. Install worked in UEFI mode without problem. We used the updated version of the shim and shim-unsigned packages from Fedora 20 in order to avoid some issues. However, the multisign issue met with OpenSUSE was also encountered here. More over, Fedora doesn’t provide a good documentation yet for signing your own kernel, which was reported upstream and could benefit from this article. Also the usage of mokutil is broken and should be fixed for Fedora 20.
  • UEFI PlugFest - Samer El-Haj-Mahmoud, HP

    We also got visited by two Kernel Maintainers Greg Kroah-Hartman and James Bottomley who even tried some of his tools on our systems.
    UEFI PlugFest - James Bottomley, Parallels - Neill Kapron, HP

    Note that Some USB keys even correctly formated didn’t boot correctly on some platforms so if you encounter this issue, try using another USB key.

    Finally I made some tries with MondoRescue on the Last Fedora distribution installed. I thought the work done to support EFI on Itanium would be sufficient, but there are some detection problems for the boot loader in mindi need to be solved and are now tracked upstream as well.

    And on top of all what I was able to learn working with my 3 colleagues, I was pointed to a very instructive article from Ken Thomson on Trusting Trust, I hadn’t read before (and I encourage you to read it), following discussions on Secure Boot. And we had a very nice dinner downtown, a walk through Bourbon Street
    Bourbon Street

    followed by a real air of New Orleans Jazz.
    DSC_8789

    That was the end of a very rich US week. More to come on other more recent travels later.

New 3.2 development branch for MondoRescue

2013/10/01

I’ve taken over the lead for MondoRescue in September 2005. It has been 8 years these days. And since the 3rd of October 2005, I started a new branch to try to fix the first bug report (I really mean the first !) that was made !!

It took me hours to work on this branch along the years. Many times I’ve left that dev, as it wasn’t working as I wanted. Too many modifications done without testing, then a non working stuff too difficult to debug. I even included a patch from a contributor René-Marc Dolhen to add i18n support, but due to the numerous changs of branches to work on this, it never materialized in an offical MondoRescue version.

So when René-Marc contacted me back 2 weeks ago to again propose to me to add this patch renewed, I decided it was time to make something !

I’ve thus created the 3.2 branch, passed lots of hours again during my recent travel in the US to port patches from the 3.1 branch to this new one and 3.2.0 will be next MondoRescue version. I’m now testing it. trunk, stable (!) and 3.1 branches are now dead. The one after will be named devel, but that’s not for now.

3.2 now compiles, build and even runs up to the archiving point. I’ll start testing restore soon. I’ll make beta versions available as usual and will really need your help as user/contributor to test it in the various cases that you have to make it as reliable as the one we have today in 3.0.

Even if I don’t intend to maintain 3.0 for a long time in parallel with 3.2, it may be done a bit to ease the transition.

BTW, do not expect many news in 3.2. It’s 3.0 with tons of dynamic memory allocation instead of static one where it was easy to do (and done previously). More will come later on. And hopefully with the i18n patch added if René-Marc has still the force to work with me after so many years of delay !!

Interesting end of week

2013/09/13

Quite some activities for this end of week and the next one:

  1. I’m publishing on Friday mindi 2.1.7 in order to fix kernel detection issues for the most recent ones (> 3.9).
  2. I’m going by train on Saturday to Paris to attend a concert made by my daughter Ségolène singing early music with Jean Belliard in Etampes (Eglise St Gilles).
  3. I’ll sing myself on Sunday for the last concert of our yearly Académie in St Sulpice de Favière (come and talk about music !!)
  4. I’ll go back to Roissy to tke the plane on Monday morning to arrive in New Orleans for LinuxCon US 2013
  5. I’ll attend LinuxCon on Monday and Tuesday, and deliver a talk on Mageia on Wednesday, then attend the UEFI plug fest till Friday (come and talk about FLOSS !!)
  6. I’ll fly back on Saturday, arrive on Sunday in Paris pass some hours with my daughter again, and then back home in Grenoble to start working again the day after.

The real question now, is that it’s time to sleep, and I need to make the Mageia presentation for next week. Anne Nicolas Velu helped me a lot with material and pointers, now I need to make the 20+ slides I’d like to have to cover the topic in a nice and entertaining way for the audience. Luckily there is a long flight ;-)

No time to get bored as you can see.

MondoRescue 3.0.4: Probably not perfect but definitely useful

2013/08/02

Not perfect as I’d have liked to fix some more bugs before releasing, but as usual, it’s a tradeoff between delivering a version which already fixed some annoying issues users reported, and waiting more to hopefully fix more of them.

But the good aspect of being a dictator on a project, is that it’s up to you to decide !!

So here it is, version 3.0.4 is born, and on top of what was previously fixed, here are the latest news:

  • Packages are now provided for Fedora 19
  • Fix multipath support on distributions where the libraries are dynamically loaded (thanks to a patch from Dimitri Bellini)
  • Fix #656 by adding support for distributions with no /lib anymore (Fedora 17+, Mageia 3+, …) which was creating issues since quite a long time, despite some (slow !) work arounds
  • Add support for static routes by fixing #697
  • Rework the boot sceens at restore time to give access to the boot parameters docs through the keys F2-F4 which should help newcomers to understand what is possible, and also making restoration from external devices more easy thanks to suggestions of messages from SamK in #628/li>
  • Fix #640 and scratch/temp dir reletion issues for both interactive/batch modes
  • Tape block size can now be specified in the GUI as per #690
  • Files with strange chars in their names such as single quote, double quote, should’nt create a problem anymore, neither in the analysis nor with extended attributes as #673 is now fixed

Of course, please check that your issue is indeed fixed and that this version works better for you. Even if I don’t like that, as with every version, new issues may appear despite tests made on various distributions (the 2 enterprise ones RHEL and SLES being more stressed than others). Please report them so we could approach perfection with 3.0.5. Well one can always dream no ;-)

I plan on integrating more content from 3.1 in 3.0.5 as a next step, in order to manage in a better way memory. And I also have plans to work on #628 to close it completely, as well as #627 which are the 2 most annoying issues that have been reported. Now if you want to give more weight to other bur reports, feel free to add comments to this post !


Follow

Get every new post delivered to your Inbox.

Join 103 other followers