~kanarip

In terms of an expirement, I've been rebuilding Enterprise Linux packages, including updates, including Extra Packages for Enterprise Linux, in order to learn from it and take away a couple of notes on the subject; Here's a brief overview of what I've found so far;

• Package build requirements may be off
  Maybe some package required foo-1.0 in the buildroot, but foo has been updated to foo-1.1, obsoleting the foo-1.0 update as it is pushed to the updates channel.

  The most obvious case in a name-change of python-setuptools-devel, for which the -devel part became part of the python-setuptools package itself, without providing "python-setuptools-devel". All of the dependent packages will now fail to build because of this change. In terms of Enterprise Linux, this type of change causes everything in it's realm to no longer be reproducible, which is a key aspect of Enterprise Linux to begin with, as no reproducibility means no certainty about what's going to end up on the consumer's system.

• Koji does not allow all builds of all packages in its static repositories by default
  By default, Koji will only take the latest successful build in a tag in its static repositories. While perfectly fine in the case of FTBFS exercises, it bubbles down the inheritance all the way to the point where you don't need the latest package, but the former version. FWIW, "latest" isn't defined as "highest package NEVRA" either, it's "last build completion time". I would rather make it a configuration setting at some point, but I'll share the patch I made anyway.
• BuildNotifications are at the end of the queue
  When building a lot of packages, all of the build tasks have a higher priority then the notification tasks that let people know what the result of a build was. Initially, I thought something was wrong with my environment, but with the help of Dennis Gilmore over in #koji on FreeNode, I learned that it is in fact the priority in Koji, not my environment.

  Again I would love to make this a configurable thing some day, but here's the patch I made to let buildNotification tasks have a higher priority then anything else.

• "%patch without Patch"
  This is the most amazing sort of problem. RPM spec files will have a conditional %if statement that in a simple `rpmbuild -bs` will cause the Patch file to not be included. A very simple fix would be to never ever use the conditional inclusion of a Patch file when building the source RPM package, but instead use the conditional during the actual build.
• (No) Naming Convention
  The naming convention, or lack thereof, makes it so one needs a `yum whatprovides` or `repoquery whatprovides` to resolve build requirements. A naming convention could be to say, that if a package includes libuuid.so.*, a (meta-)package libuuid is (automatically) created, or to make yum so intelligent as to resolve a "libuuid" type of build requirement to the actual package, not unlike it does for binary packages when you use `yum install` and the package in question doesn't explicitely require libuuid.so through the original RPM spec file, but RPM inserted the library dependency automatically.
• The (lack of) use of dist-tags
  Some packages do, while others do not, include a dist-tag. For downstreams like myself, this causes the downstream to only be able to build the packages in question only once, since the N-V-R needs to be unique all across the build system. In certain circumstances, it is entirely justifiable to rebuild a large subset of packages included in the upstream distribution as well as the add-on repository; for one, being able to run Ruby 1.9.1 with a large part of its Gems causes one to re-negotiate the use of dist-tags:

1. On the end-user system, rubygem-foo for Ruby 1.9.1 needs to be distinguishible from rubygem-foo for Ruby 1.8.6. And that's just the user side of things. Failure to make it distinguishible in some or the other fashion causes a significant load on the support side of things.
2. Throughout the build- and distribution-channels, whether they be repositories or not, rubygem-foo-1.0 for Ruby 1.8.6 *must* be smaller in terms of package NEVRA then, say, rubygem-foo-1.0 for Ruby 1.9.1, in order for the Ruby 1.9.1 fast-track to be viable on any end-user system.
3. Since not using dist-tags would allow rubygem-foo-1.0-1.src.rpm (or whathaveyou) to be built only once, this cannot be a package that has any kind of binary/library linking going on with Ruby itself. In order for the package to be allowed to be rebuilt more then once, against different stacks, a dist-tag must be introduced.

• buildsys-macros is not available through the distribution
  It appears as if the buildsys-macros package is not available through the EL or EPEL repositories. Of course, this is a build system thing, and would not be used on the end-user system. For a downstream buildsystem though, having this package be part of the repository might save someone some headache.
• There's no build group
  When setting up a downstream build system environment, you use groups (or a buildroot initialization command actually), which will initially attempt to use "build" as the group name. However, EPEL only includes a buildsys-build group (of which buildsys-macros isn't part of course). It might be a little easier for downstream build environments to (re-)use an existing build group as to not modify their local koji foo all too much.
• It takes a lot of CPU cycles
  Plan for an enormous amount of CPU cycles to be spent at whatever you're building. In my particular case, a great deal of the cycles are I/O wait ;-) Maybe I should plan for better performance storage across my builders, but for now I'll have to work with what I have available. As a not-so-minor side-note, if you plan on just building a few packages every once in a while, as opposed to continuously rebuilding a lot of packages, this is a less significant parameter.
• When using external repositories...
  If you use external repositories, plan to have some memory available. Every once in a while, the `mergerepos` command executed by Koji consumes more memory then Thunderbird does at that moment, which should tell you enough.

So far so good, while you may be wondering while I'm doing this type of stuff. Well, it's a very interesting and challenging area, which makes it fun to do. Besides, it allows me to play around with different tasks I was going to try and execute, like providing some of the packages that cannot be in Fedora (mod_passenger), or cannot go into EPEL (rails3, ruby-1.9.1, you name it). I would like to return a certain amount of option value into the hands of Linux consumers, but in an efficient manner as opposed to everyone who needs certain foo doing it themselves in a million different ways (== unsustainable). That's what I'm working on, or at least some of the details concerning such.