~kanarip

Another random day at the office: Can you imagine this happens to all of your users?

Solution: Tell the domain member (client machine) that the active directory domain name space is in the trusted zone. I mean, wtf... ;-)

I think I'm going to have to give you a summary of the second and third day of the Bacula Systems Administration Course, part I that I participated in late March, since apparently I've been too busy with different stuff to even finish the blog post I was originally supposed to do on the second and third day.

The second day was a more hands-on day where we practiced with backup and restores including a full meltdown of the Director and Catalog server; having the backup server go down and not be recoverable other then from your previous backup gets you in a catch-22 kind of situation. It's not at all that hard though, since Bacula specifically addresses this type of situation in its documentation, taking you through the process step-by-step, with a grant total of not even 10 steps.

You have to have done it once, even if it's just to experience this situation once before it just happens to you in real life and you don't know what to do.

Since the day was full of exercises and practicing, it was an exhausting day altogether. Though again on the second day, much like the first day, I didn't run into any trouble at all, so I could very smoothly work through the tasks assigned ;-)

Later on in the evening, we were all invited to dinner in a restaurant nearby, where -of course- we enjoyed a couple of beers and some good stories.

On day three of the course, we got to relax a little more. I think the exact words when we came in in the morning were "Welcome, glad you survived the second day."

Various types of backup strategies also were brought to our attention. A couple of case studies showed us exactly what was on Kern's mind while he discussed certain types of backup strategies along with his choice for Pools, Volumes and other settings, and his calculations on the number of Jobs per Volume, the maximum number of Volumes, and retention periods.

Although that too isn't all that difficult (it's merely a little too abstract if you will), it's great to hear the grand master himself explaining how it works.

Today was the first day of the Administration Course 1 for Bacula, by Bacula Systems. I am here sponsored by the distributor in the Netherlands, Amaziq Source, as a "thank you" for creating two sales leads which where followed up by Bacula Systems and Amaziq Source, and solidified -with two of my former Operator Groep Delft customers.

One of my former customers was using Commvault, which apparently turns out to be a big dissappointment when you decide to upgrade your backup environment. It seems Commvault only sells you the new, expensive license if you also purchase a number of days of expensive consultancy along with it. At these times, I can only grin, from ear to ear.

The other customer had decided on its own to move away from their existing Symantec backup suite, partly because it Just Didn't WorkTM, and partly because of the attractiveness of Open Source. Either way, when I worked at the customer site -on a job that had really nothing to do with backup- we were informally discussing the backup solution they were looking at at that time, and they let me know it was Zmanda. When I told them Zmanda is not(1) Open Source(2), even though they love to pretend to be, I got to explain why not (just try to download the source and you'll see for yourself) and most importantly, I got to mention an alternative. Enter Bacula, which they've now purchased a support subscription for with Bacula Systems.

Back to the original topic of this blog post, the course itself, to me at least didn't seem very advanced, since I've been occasionally operating Bacula for a number of years. I have to say that was just the first day though, and tomorrow is promosing to be way more challenging (I think the actual word used was "boggling"). I realize though the average system administrator used to any other type of Backup & Recovery software may require this course to get used to the different terminology used within Bacula, as it is not your average click-and-pray type of program.

Even in my case though, the course helps in creating a correct understanding of the features and configuration of Bacula, and as such helps me to futher increase my knowledge of Bacula. Even more so, since this course is being mentored by the original author of Bacula; Kern Sibbald. This man, for whom I've got much respect, has been working at Bacula for over 8 years already. It's one of the main reason why I'm in this course in lovely Switzerland; I might as well have followed the course in the Netherlands but I wanted to meet more of the Bacula people, Mr. Sibbald in particular.

We learned about the architecture of Bacula, which of course I thought I was pretty much familiar with already, but I did get to hear some more, new details on how in greater environments, multiple Directors, Catalogs and Storage Daemons can be aligned and balanced out -and some more details I all wrote down in the handout.

After the initial "overview", we went into the feature and configuration details of Bacula, merely scratching the surface of each configuration directive of each component. I got some things that were on my mind cleared up, as I've been working with all kinds of different backup suites throughout my carreer with Operator Groep Delft... ;-)

I can't wait 'till tomorrow, but first: dinner and drinks

I'm in Paris with Xavier Lamien this weekend, to work on RPMFusion's implementation of Koji, and to work on the implementation of Puppet, and in return I can pick Xavier's brain on dist-cvs and dist-git foo.

Today, we plan on going to Linux Solutions, where we'll probably meet a lot of Fedora and non-Fedora people ;-)

Tomorrow, the work starts. We start out with implementing Koji for RPMFusion since that has the biggest win for all of us.

Dear fellow Fedora Project contributors,

over the past few months, the Fedora Project Board as well as several special work-groups and including a Marketing FAD all seem to be headed in the same direction; Fedora's target audience, the "minimum bar" to target from a Marketing point of view, and whatever jargon I supposedly don't know about because I have not read the correct books on the subject, consists of a couple of groups;

• users (curious, new)
• users (computer-savvy)
• users (not-yet-contributor)
• users

I hope we're all users too. I'd like to think that at the very least, we are all users. Of course if this is the minimum bar then it includes everyone. Yet -while we're pinning down what exactly is our target audience and various constituencies- I'm missing one particular group in this list, which is the committed. In other words, the developers, the free software pigs. All I see anyone be concerned with is chickens.

Don't get me wrong, I think it's very important to pay a lot of attention to potential contributors and get them to come off their asses and kick some other asses. I think it starts with the computer-savvy, the curious and the new. I think it's a good thing to spread Linux to general productivity users. But I also think it's very important to explicitly rather then implicitly target those that make those users have something to aim for. Otherwise, I believe, this particular group quickly becomes the departed.

True, first-hand testimony of how no-nonsense Gets Things Done!

I came to Cardiff virtually empty-handed, except for a couple of heavy books on chemistry-foo Lydia wanted me to bring from our place in the Netherlands. Believe me, there was no room for a proper birthday present ;-) Buying flowers on the airport just seems too tacky...

Either way, I had a very enjoyable weekend with Lydia! As it was the first time in Cardiff for me, we did do some site-seeing, with the positive side-effect of getting to know the area -since we're both going to live there in some kind of house/flat some time soon, we needed to make up our minds on what area of Cardiff would be most enjoyable.

Let me first say that Cardiff is full of houses and flats that are either for sale or to let. So, there's plenty of options available! I'm sure we'll be able to find something to our liking, but let's see what the bank has to say about that before calling the shots.

Back to the original subject of this blog post... No-nonsense birthday presents ;-)

I offered to pay for anything she wanted (within a reasonable price-range of course, no huge-ass diamant ring for Lydia quite yet), and she picks up some straws, tissues and gets her mugshot taken in one of those passport picture booths in one or the other mall. Lydia's happy, so fine with me! Done!

I'm going to have to think of a better present to bring along in two weeks though. I understand chocolate is good... Maybe some flowers from the airport... ;-)

In terms of an expirement, I've been rebuilding Enterprise Linux packages, including updates, including Extra Packages for Enterprise Linux, in order to learn from it and take away a couple of notes on the subject; Here's a brief overview of what I've found so far;

• Package build requirements may be off
  Maybe some package required foo-1.0 in the buildroot, but foo has been updated to foo-1.1, obsoleting the foo-1.0 update as it is pushed to the updates channel.

  The most obvious case in a name-change of python-setuptools-devel, for which the -devel part became part of the python-setuptools package itself, without providing "python-setuptools-devel". All of the dependent packages will now fail to build because of this change. In terms of Enterprise Linux, this type of change causes everything in it's realm to no longer be reproducible, which is a key aspect of Enterprise Linux to begin with, as no reproducibility means no certainty about what's going to end up on the consumer's system.

• Koji does not allow all builds of all packages in its static repositories by default
  By default, Koji will only take the latest successful build in a tag in its static repositories. While perfectly fine in the case of FTBFS exercises, it bubbles down the inheritance all the way to the point where you don't need the latest package, but the former version. FWIW, "latest" isn't defined as "highest package NEVRA" either, it's "last build completion time". I would rather make it a configuration setting at some point, but I'll share the patch I made anyway.
• BuildNotifications are at the end of the queue
  When building a lot of packages, all of the build tasks have a higher priority then the notification tasks that let people know what the result of a build was. Initially, I thought something was wrong with my environment, but with the help of Dennis Gilmore over in #koji on FreeNode, I learned that it is in fact the priority in Koji, not my environment.

  Again I would love to make this a configurable thing some day, but here's the patch I made to let buildNotification tasks have a higher priority then anything else.

• "%patch without Patch"
  This is the most amazing sort of problem. RPM spec files will have a conditional %if statement that in a simple `rpmbuild -bs` will cause the Patch file to not be included. A very simple fix would be to never ever use the conditional inclusion of a Patch file when building the source RPM package, but instead use the conditional during the actual build.
• (No) Naming Convention
  The naming convention, or lack thereof, makes it so one needs a `yum whatprovides` or `repoquery whatprovides` to resolve build requirements. A naming convention could be to say, that if a package includes libuuid.so.*, a (meta-)package libuuid is (automatically) created, or to make yum so intelligent as to resolve a "libuuid" type of build requirement to the actual package, not unlike it does for binary packages when you use `yum install` and the package in question doesn't explicitely require libuuid.so through the original RPM spec file, but RPM inserted the library dependency automatically.
• The (lack of) use of dist-tags
  Some packages do, while others do not, include a dist-tag. For downstreams like myself, this causes the downstream to only be able to build the packages in question only once, since the N-V-R needs to be unique all across the build system. In certain circumstances, it is entirely justifiable to rebuild a large subset of packages included in the upstream distribution as well as the add-on repository; for one, being able to run Ruby 1.9.1 with a large part of its Gems causes one to re-negotiate the use of dist-tags:

1. On the end-user system, rubygem-foo for Ruby 1.9.1 needs to be distinguishible from rubygem-foo for Ruby 1.8.6. And that's just the user side of things. Failure to make it distinguishible in some or the other fashion causes a significant load on the support side of things.
2. Throughout the build- and distribution-channels, whether they be repositories or not, rubygem-foo-1.0 for Ruby 1.8.6 *must* be smaller in terms of package NEVRA then, say, rubygem-foo-1.0 for Ruby 1.9.1, in order for the Ruby 1.9.1 fast-track to be viable on any end-user system.
3. Since not using dist-tags would allow rubygem-foo-1.0-1.src.rpm (or whathaveyou) to be built only once, this cannot be a package that has any kind of binary/library linking going on with Ruby itself. In order for the package to be allowed to be rebuilt more then once, against different stacks, a dist-tag must be introduced.

• buildsys-macros is not available through the distribution
  It appears as if the buildsys-macros package is not available through the EL or EPEL repositories. Of course, this is a build system thing, and would not be used on the end-user system. For a downstream buildsystem though, having this package be part of the repository might save someone some headache.
• There's no build group
  When setting up a downstream build system environment, you use groups (or a buildroot initialization command actually), which will initially attempt to use "build" as the group name. However, EPEL only includes a buildsys-build group (of which buildsys-macros isn't part of course). It might be a little easier for downstream build environments to (re-)use an existing build group as to not modify their local koji foo all too much.
• It takes a lot of CPU cycles
  Plan for an enormous amount of CPU cycles to be spent at whatever you're building. In my particular case, a great deal of the cycles are I/O wait ;-) Maybe I should plan for better performance storage across my builders, but for now I'll have to work with what I have available. As a not-so-minor side-note, if you plan on just building a few packages every once in a while, as opposed to continuously rebuilding a lot of packages, this is a less significant parameter.
• When using external repositories...
  If you use external repositories, plan to have some memory available. Every once in a while, the `mergerepos` command executed by Koji consumes more memory then Thunderbird does at that moment, which should tell you enough.

So far so good, while you may be wondering while I'm doing this type of stuff. Well, it's a very interesting and challenging area, which makes it fun to do. Besides, it allows me to play around with different tasks I was going to try and execute, like providing some of the packages that cannot be in Fedora (mod_passenger), or cannot go into EPEL (rails3, ruby-1.9.1, you name it). I would like to return a certain amount of option value into the hands of Linux consumers, but in an efficient manner as opposed to everyone who needs certain foo doing it themselves in a million different ways (== unsustainable). That's what I'm working on, or at least some of the details concerning such.

I'm heavily experimenting with Drupal. I have 95 or so modules I want to experiment with, which is just too cool! The more modules, the more obvious it is that this thing might actually turn out to do exactly what you wish ;-)

For now, I'm looking for a good case tracker, issue tracker or you-name-it. Preferably one that integrates with organic groups, and/or user roles. It seems "Project" and "Project Issues" module are unsuitable after some initial testing -I'm not quite done with them yet, though. "Project" seems to be very specifically aimed at software projects with releases, whereas I just need a ticketing system for tasks, basically. I've done this before with "Support", so maybe that's what I'll end up using.

Just for fun, I started building ruby-1.8.6 and ruby-1.9.1 packages for Enterprise Linux 5. These would be opt-in repositories, "channels" if you will, fast-tracking an Enterprise Linux 5 box to newer versions of Ruby, and many of the packages that depend on Ruby one way or the other (such as Ruby on Rails packages, and many Ruby Gems).

I started out with the Fedora packages, obviously, and after getting a bunch of packages in said repositories, and testing and patching and rebuilding a bunch of times, it became obvious;

For the type of situation where you want two of these "fast-track" repositories, or even just multiple versions of the same packages built under different conditions, it turns out to be mandatory that a Koji build of a certain package contains a globally unique package NEVRA (Name, Epoch, Version, Release, Architecture), so they can be distinguished between the two (very much different) versions of the package. That is to say, one package NEVRA can only be built once, and can only be duplicated to other destination tags.

Example: The ruby-shadow package (a requirement for Puppet, which makes it very important to me) is binary compiled, either against ruby-1.8.6 or ruby-1.9.1. For one version (e.g. upstream's 0.9.7) to be available through both repositories, one builds ruby-shadow-0.9.7-1.el5.src.rpm.

I build the package against two Koji build targets, each one using a build tag that causes a buildroot to be created with either ruby-1.8.6 or ruby-1.9.1; in my case these build targets are feature-el5-ruby-1.8.6 and feature-el5-ruby-1.9.1.

Koji however will only allow one specific package NEVRA to be built just once. But, ruby-shadow-0.9.7-1.el5.src.rpm has to be built twice; once for ruby-1.8.6, and once for ruby-1.9.1. Since one build with the same NEVRA already exists, another cannot be built.

Ergo, you need some kind of indication of the build-root/build-tag/destionation-tag in the package NEVRA...

And we go off rebuilding everything again... ;-)

Note to self: when using external repositories and building on those, please remember that priorities in tag inheritance does matter.

Thank you.

If you have level 1 Linux Professional Institute certification, you can get Certified Linux Administrator from Novell for free:

1. Go to the LPI login-page and login with your LPI ID or email address.
2. Click on "Verify Certification" and then "certifications" near the bottom of the page.
3. Copy your LPI ID and the Verification Code for LPIC-1
4. Go to Novell Certified Linux Administrator Certification Registration Form
5. Fill out the form and press "Continue".
6. Congratulations!
7. Ask your employer for that raise he promised you in case you would obtain (yet another) certification.

Have fun!

Today is the day we come together with a bunch of Linux experts, and start knocking down some of the items on our TODO list, as well as -hopefully- share more information and responsibility on the overall Linux infrastructure inside our company, and the Linux infrastructure at a lot of out customers both!

On February 9th, Mike McGrath wrote:

Toshio Kuratomi is awesome.

That is all.

+1 Mike, and so are you!

I'm very much pleased to be able to announce that Zarafa, one of the best Linux groupware suites, is now available through the standard Fedora and EPEL repositories.

After Zarafa itself had already announced inclusion to the repositories, news that was dented through major news sites such as heise.de, Robert Scheck and myself sat together at FOSDEM and worked on the packaging until we were both sufficiently satisfied.

It's currently set up to allow the maximum amount of flexibility one could ever wish for. Normally, the packages provided by Zarafa consist of the backend server, gateway (IMAP/POP), the PHP libraries needed for the Webmail interface, and too many other things to really build up a scalable infrastructure without installing all capabilities on all servers in such infrastructure -which introduces it's own world of pain.

Now, apparently, we still need to figure out some things. For one, I get a SIGPIPE / Broken pipe when I run zarafa-server with UNIX passwd authentication. The availability of a platform like Fedora (fast-pace moving forward) allows us however to solve this kind of issues way before Enterprise Linux 6 hits public beta.

You gotta love it!

I've always been a huge fan of Zarafa, one of merely two serious competitors in the Open Source groupware market.

The other competitor is Zimbra, but I have somewhat less of an incentive to sink my teeth into that Java mess, which installs in /opt/zimbra/, and uses it's own vendored libraries rather then those available in the system stack. This, in my opinion, is just wrong, raises cost and gives you less overall options and control. But enough about Zimbra, because obviously the suite *just works* (and a lot of people are very much happy with it).

I have (and still am) running Zarafa at my company for about a thousand users, and Zarafa's headquarters are pretty close to my company's. I get to speak to Zarafa's people regularly, and most of it is while I'm wearing my Fedora hat ;-)

Either way, now that Robert Scheck and myself are attempting to package Zarafa for our dear distribution, Robert and I run into the following when using Fedora 12:

[jmeeuwen@ghandalf SPECS.mine]$ sudo zarafa-spooler -F
zarafa-spooler: symbol lookup error: /usr/lib64/libmapi.so.1: undefined symbol: _ZN10ECMemTable6CreateEP14_SPropTagArrayjPPS_

Robert has created a topic on Zarafa's forum about this some time ago.

Let me first emphasize that Zarafa is upstream for three libraries:

1. inetmapi
2. perlmapi
3. libmapi

This symbol is undefined in libmapi only, as you can see in Robert's comment.

I once succeeded (I don't know how) to make the command result in a stack trace:

[jmeeuwen@ghandalf spooler]$ gdb /usr/bin/zarafa-spooler
GNU gdb (GDB) Fedora (7.0.1-30.fc12)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/zarafa-spooler...(no debugging symbols found)...done.
(gdb) run -F
Starting program: /usr/bin/zarafa-spooler -F
[Thread debugging using libthread_db enabled]
Unable to open configuration file /etc/zarafa/spooler.cfg
Continuing with defaults
Wed 03 Feb 2010 03:53:54 AM CET: Unable to open pidfile '/var/run/zarafa-spooler.pid'
Wed 03 Feb 2010 03:53:54 AM CET: Starting zarafa-spooler version 6,30,9,18385 (18385), pid 7285
Wed 03 Feb 2010 03:53:54 AM CET: using SMTP server: localhost

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) thread apply all bt

Thread 1 (Thread 0x7ffff7fcb820 (LWP 7285)):
#0  0x0000000000000000 in ?? ()
#1  0x0000003d0da20ca9 in M4LMsgServiceAdmin::GetMsgServiceTable(unsigned int, IMAPITable**) () from /usr/lib64/libmapi.so.0
#2  0x0000003d0e259782 in CreateProfileTemp(char*, char*, char*, char*, unsigned int, char*, char*) () from /usr/lib64/libinetmapi.so.1
#3  0x0000003d0e259a3b in HrOpenECSession(IMAPISession**, char*, char*, char*, unsigned int, char*, char*, char*) () from /usr/lib64/libinetmapi.so.1
#4  0x0000003d0e259b29 in HrOpenECAdminSession(IMAPISession**, char*, unsigned int, char*, char*) () from /usr/lib64/libinetmapi.so.1
#5  0x00000000004069a1 in InitThreadData(thread_data*) ()
#6  0x00000000004075bd in ProcessQueue(char*, char*) ()
#7  0x00000000004078ff in running_server(char*, char*) ()
#8  0x000000000040d04e in main ()
(gdb)

I'm not at all too familiar with C/C++ code, and/or libtool (a more recent version of libtool in Fedora is rumored to have caused this?), and so my first step is to Google. Googling for "undefined symbol" doesn't really give you anything else then forum topics with questions and often not even solutions to very particular problems though :/

So, I turn to you, dear Lazyweb, and I'm asking you to help me wrap my head around it and put the finger on the sour spot.

SPEC: http://www.kanarip.com/custom/SPECS/zarafa.spec

SRPM: http://www.kanarip.com/custom/f12/SRPMS/zarafa-6.30.10-1.fc12.src.rpm

Thanks in advance!

I love to be able to announce the first "sysadmin-main" meeting within my company, Operator Groep Delft. You read that right, that's the exact same name the Fedora Infrastructure team - an example to us all as far as I'm concerned, uses to indicate the group of people ultimately in control of all systems and services. Read the more detailed description. I'm going to try and apply this very concept to my company's internal system administration.

Instead of taking care of the Linux infrastructure with no dedicated Linux engineering resources, we decided that utilizing all of the available Linux competencies within the company would be more efficient, more effective, more flexible and would vastly improve redundancy, collaboration and expertise between the bunch of us.

Since we're a consultancy company, most of our Linux Engineers are hired out to customers, making billable hours, earning us all money, and are thus not in a position to really sit down and take care of OGDs Linux infrastructure as well. Basically our engineers can only spend that one minute they have left at the end of a day or the hour they would otherwise waste on one of those boring meetings -they merely require the facilities to do so.

So instead we decided to try and make it a community effort of some sort, in the sense that all sysadmin-main engineers require a minimal Linux certification level of RHCT, and will have access to Life, Everything and The Universe once we've sat together and introduced them to all procedures and such.

Now, nothing is set in stone yet, but at least we're going to have a bunch of interested people show up at our meeting soon, which will hopefully lead to a large group of engineers willing to do interesting stuff.

We'd plan and assign -amongst ourselves- the tasks in our queue. If either of us needs a little more time, we save up to 8 hours a week to spend at more intrusive changes like migrations, upgrades, planning, documentation, build & test, development, that sort of thing. We (as a group) would decide what we do, how we do it, and who actually gets to execute.

Ultimately, I'd be (held) responsible (or accountable?) for the group, and my manager would be responsible for the well-being of the infrastucture as a whole, just like he is now.

Just to give you an impression of what it takes to do what we do between the two of us at this moment, in the very little time that we have available; Zarafa, HA/LB Red Hat Directory Services, Puppet (with help of puppetmanaged.org modules), Cobbler, SELinux (enforcing), Nagios, Munin (looking at Zenoss to replace both), 15 mod_security enabled webservers, 4 database servers, 6 development environment staging boxes, and a couple of workstations.

PS. For those of you who read this, and are colleagues of mine, you can find more information on https://nix-noc.ogd.nl/trac/

If I'm not mistaken, today is the birthday of not just one, but two great friends!

Congratulations both Max Spevack and Jan Wildeboer!

Overheard:

Question: "You apply a Server, Domain Controller and Client security policy requiring all network traffic to be encrypted. Some of your users report that they cannot log in or access network resources. What is the easiest way to resolve the problem?"

Answer: "Tell the user to reboot the computer."

This means, essentially, implicitly, that the policy that *all* network traffic should be encrypted, doesn't apply to *all* traffic ('cause clients can get their updated configuration without being compliant to the new configuration). *sigh* :/

Over the last couple of days it seems more and more people are stepping up to get Ruby 1.9.1 in Fedora, along with the packaging changes and all that. From a new list just two months ago, we now have 17 members working on the same problems collaboratively.

Worth noting is that I'm receiving patches from people that understand Ruby way better then I do, so it's one big happy learning experience for me too ;-) (Noted I do not use Ruby myself, nor do I program using Ruby)

Thanks to Ben Shakal, we're now over the issue I posted about earlier on the Ruby SIG mailing list, where gem install would work, but executing or requiring the gem wouldn't. Great work Ben, thanks!

From here on out, I'm going to extend the Ruby repositories for Fedora 12 and Fedora Rawhide to include some of the ruby gems with the new packaging guidelines. I'm still going to need new packaging guidelines to go along with the new packages so I'm probably going to build those packaging guidelines as I go along.

More news on Ruby later this year, hope to have some working stack early next year.

Hope you all have a merry Christmas!

I've finally recovered some of what broke down earlier this week. Long story short:

• The CPU of my web-/file-/mail-/dns- server is fubar, and the only other Pentium 4 CPU that I could find would not work
• I used the x86_64 CPU and mainboard from slevin (my girlfriend's desktop PC)
• Pulling out and stuffing back in the hardware wasn't difficult, but took me some time anyway (I did not have the right tools, go figure)
• Upgrading -slash- migrating Fedora 10 i386 to Fedora 12 x86_64 as you can imagine took me even longer, only to find out that a lot of configuration and other stuff relevant to various services were incompatible.
• The first thing I wanted to restore was email, and so I did (including external DNS), and then I left it all behind to do some other stuff.
• Now I've gotten around to rebuilding php-5.2 for Fedora 12 so that Drupal would actually work. Various kinds of functions Drupal-6.x uses are deprecated in php-5.3, and
• I had to restore name services internal to my network as well (the Pentium 4 CPU I got from this other machine made this other machine not boot up anymore)

The fun! ;-)

Dear Hyves,

Would you please stop sending me email that does not make sense?

I quote:

Hyves notification

Unfortunately you can't read this mail properly, due to the fact that our outgoing mail is
html-formatted. Your mail program probably only support plain texts.

There's a couple of things wrong with this email;

1. I can read it properly, it's just not the same contents you expect me to read when I open this email.
   You expect me to use a mail client that parses HTML right-away, without me choosing to do so when I want to. I'm sure your engineers will be familiar with the "why HTML mail is bad" meme that has been going on since the '90s.
2. It's not up to you to say anything about my ability to read anything.
3. You say my mail program probably only supports plain text, but that surely does not take into account the fact that I may have chosen to not read HTML emails.

You could, of course, include the actual message you're trying to send in plain text as well, rather then this weird looking (static) message.

I was at a round table session a few weeks ago, where consultants and advisors to local governments in the Netherlands (there's around 400 of them, and then some) discussed "Nederland Open in Verbinding", an initiative by state-secretary Frank Heemskerk with the Ministry of Economic Affairs to get government agencies of all kinds to realize the value of using Open Standards, and inherently (but not primarily) appreciate the value of Open Source software (ergo Free Software, because the rest is Crippleware).

It was the first time for me to talk on such a level, very non-technical, hypothetical and on the policy side of things. Normally, I work on design, development, implementation, administration, innovation, what-have-you, but all from the technical perspective.

Let's finally burn down the list of notes I've taken from that session.

Like within any other organisation, the main factor that influenced the atmosphere during our discussion was money. Not value. No sir! Cold hard cash, of course.

Apparently, or so I was told, government agencies wonder what the TCO on a thing such as OpenOffice.org would look like.

While I'm sure there's plenty of people out there that can whip up a comparison sheet from the top of their heads (and so I told them), that's not the issue I take offense with.

The actual issue I do take offense with is that while asking for a TCO on OpenOffice.org, it seemed that none asking for such a TCO analysis had ever done the exact same thing for either alternative product; most prominently Microsoft Office '97/2000/2003, or even a pending migration to either Microsoft Office 2007 or 2010. If you don't know what the costs were, why are you asking for comparison to what the costs might be should you use the product at zero licensing price? The next big money burning factor was considered to be migration costs.

Going down the list of examples that crossed the table; The next issue was migration off of, say, Microsoft Office (regardless what version), to OpenOffice.org. This one made me laught out loud, since such migrations are part of every single Microsoft Office release you upgrade to, with -in the case of Microsoft Office at least- the added danger of actually 1) being forced to upgrade, and 2) losing data in ways that cause you to never be able to read it again.

I emphasized that regardless of the piece of software used (you write your own if you will), the most important thing was the use of Open Standards, and every single governments agency's choice to either "use or explain". "Use Open Standards or explain why you can't or won't", is what the federal government initiative is trying to tell the other government bodies.

Either way, the choice to use Open Standards or Open Source Software is up to each individual government body. While there is no actual enforcement of the "use or explain" guidelines, the motivation to make a well-founded decision based on facts is moot. The way this is going down now merely creates awareness about the existence of an alternative technology, whether a piece of software or an open standard, you could use. The well-foundedness of the decision is quickly annihilated using false rumors and assumptions, though, and there's noone out there to tell them about the facts.

Moreover, and this is where I was truly shocked, the overall consensus seemed to be that Open Standards amongst themselves are difficult to implement and be compatible. For one, someone said, there's more then one version of the Open Document Format. This poses a problem, since Microsoft Office 2007 SP2 only implements ODF 1.1 (and so will Microsoft Office 2010 from what I can see in different preview builds), and so maybe could not deal with ODF 1.2. Not realizing that this is actually a Microsoft Office problem, I wonder if this is how it is going down;

Customer: Yeah, so, give us your offering.

SupplierA: Sure. Here's costs, they're easy. Remember that when you pay less, you get less, huh? There's also benefits, but only for Microsoft Office, primarily because we make it so extremely easy for you to get back at us and give us more of your money. In return, we'll only implement the one true version of this Open Standard, because you know multiple versions of an Open Standard are just going to confuse you, right? So, well, here's the contract, sign at the bottom please.

SupplierB: Our version of the product works wonderfully well with Open Standards, it is free of charge and you only start paying us when our phone rings. You can sign this contract, or look around for a better value proposition. On top of that, we are compatible with all versions of Open Standards and you'll never notice a thing. No transition, no migration, no hassle. If you're dissatisfied, there's no clause stopping you from getting what you get from us somewhere else.

Customer: So, SupplierB, are you saying I can take your product and use it at a zero price license? That all my problems go away? How much trouble am I going to be in and what solving those problems going to cost me? That I can just upgrade the software without a defined project involving migrations and consumer education? That I can just leave and go to your competitor? You must be freaking kidding me. SupplierA, you have a pen?

Long story short, the consultants and advisors to government bodies and other government agencies and organisations are in a position to make sure a conscious choice is being made, by matter of policy. They are in a position to emphasize the true facts and false rumors, and create some incentive to at least consider a particular implementation over another, despite the initial cost -if any more significant, in favour of the long term value. Because that's what this is about finally. Money. The tax payer's money, no less.

You may have seen Lydia's blogpost on FUDCon Toronto as she experienced it. It's the first time she's been at an event like this, while it is my trazillionth time. So, she was interested to see what kind of work we do at such events, what kind of unmess the unconference is like (I've told her many stories about Greg DeKoenigsberg throwing the markers on the floor in Brno, for example), and of course she misses Max Spevack ;-)

All and all, from her story, it may have appeared to you that I'm more of an Ambassador out of the house then I am at home ;-) Hope you all appreciate her $0.02

So now that I have this new netbook thingy, kinda like a key-chain accessory, I want the packages I have installed on my huge-ass need-a-truck laptop to be on the netbook as well. I could, of course, do a rpm -qa on one end and install the packages listed on the other end, but that's not very sustainable. Here's what I did:

1. Let Puppet collect the packages on node ghandalf.kanarip.com
2. Let Puppet install all of those packages on weee.kanarip.com

Like the snail said, riding on the back of the turtle: Weeee!

With a little help from my friends (Andreas Thiennemann in particular), I managed to get my hands on a piece of equipment I was planning on buying anyway; a netbook.

Here in Canada though, these things are particularly cheap.

Next on the list of things to do is:

1. Never boot anything but a superior version of the greatest Operating System on a piece of equipment as cool as a netbook. The first thing that Acer will attempt to boot is a Windows Vista based eRecovery Suite in order for you to install Windows XP Home. I rest my case.
2. Download Fedora 12.
3. Check the list of things you need to know before trying all kinds of things to make the foo work. In my case, I found this page very useful. For one, it helped me boot Fedora ;-)

I finally managed to come up with a bunch of proper patches that give us Ruby 1.9.1 packages again, the way we want them to be after our little HackFest at FUDCon in Toronto. At first, they would succeed in rpmbuild, but not in mock or koji, but after all I got them to build in mock. Assuming they will then build in koji too, I'm submitting a couple of scratch builds now in the background as the Internet in the Hotel isn't all that fast, and after the builds have finished we may have some packages to play around with ;-)

Only a handful people seemed interested in a FUDCon session on Ruby, Ruby-1.9.1, the Enterprise Edition, packaging foo and so forth, so it's not part of the BarCamp ;(

I'm planning to have the session tomorrow though, just after lunch, in one or the other room, so if you are interested please follow around the guy with the bad haircut wairing a Von Dutch hooded sweater ;-)

FUDCon starts! ;-)

Weeeee! Release party is ongoing!

It's more of a socializing thing then it is about Fedora 12, really, but what the heck ;-)

Also, we're speaking English rather then Dutch ;-)