Omnibus Cache Exploitation

I've been playing with Omnibus a bit lately, in hopes of making our product installable in locations where internet access is not an option. We have a decent list of dependencies that we install at deploy time, and we need some consistent way to get those items put in place when public repos aren't available.

If you've never seen Omnibus before, it's pretty great (though there is a bit of a learning curve).

Expectedly, our dependency list gets hairy when we start pulling in the dependencies of our dependencies. That's not so bad, by itself. What sucks is that any change to your project definition (omnibus-my-project/config/projects/my-project.rb) - even whitespace - causes Omnibus to invoke its HopelesslyPessimistic mechanism which in turn invalidates your software cache faster than you can say, "but it's just a pip install!".

This is done for perfectly rational reasons, sure, but it's annoying when you're working in a container whose only purpose in life is to run Omnibus builds, and your build times start to creep into the tens of minutes.

Fortunately you can route around this pessimism while you're getting your dependency list ironed out by adding your dependencies to a dependency. To elaborate...

You normally add your run-time dependencies to your project definition (my-project.rb), but you can also add those dependencies to another software definition that you depend on in omnibus-my-project/config/software.

So for example, instead of:

You'd have a more minimal definition:

Then your pycrypto definition might look something like this:

Now you can add dependencies to the bottom of pycrypto.rb and take full advantage of the Omnibus software cache! Saves a lifetime of waiting on builds to complete.

NOTE: You really shouldn't leave your project like this, it'd be terribly irresponsible of you, and then you'll say that I told you to do it, and I'll catch all kinds of grief for it. So don't do that.

When you're done adding all your dependencies, you can simply shovel all the dependency "<name>" lines out of (in this example) pycrypto.rb back into my-project.rb, and kick off one more (very long) build to get a sane final package.

Happy packaging!

Omnibus for Offline Installs

Software repos are wonderful things, man.

Until your servers are offline, of course... that's no good.

Or if they're online, but not allowed to update but once a year.

Or worse yet, they only get whatever updates are considered mission-critical by IT, and there's no way in hell you're going to get the right versions of your dependencies installed at deploy time.

At the end of the day, trying to get anything installed at a customer site that has externally-hosted dependencies is almost guaranteed to be more of a pain than it's really worth.

So what can be done? Well, a few ideas come to mind...

• Tar up your top-level dependencies, push it out at deploy time, and hope for the best. Then watch as all the installs fail because dpkg or rpm can't satisfy the dependencies of your dependencies
• Script out something like  'apt-get --print-uris --yes install <stuff>' and wget the results so you can tar 'em up, push 'em, and install 'em. Then watch as your next deploy fails miserably because the customer did their quarterly security patches and broke your dependency chain all to pieces
• Containerize all the things! This actually might work just dandy depending on what your software does. In some cases though, your software needs more intimacy with the hardware than a container can offer (mostly ultra-low-latency IO and the like). Plus it might mean introducing technologies that your customer is unwilling to accept on their systems...
• Stick it all in a single package and jail it all in /opt. Winner!

Enter Omnibus. You get to include every dependency above libc in the correct order and specify how to build each one, jailing them in /opt/<name> (or wherever you want, really). Then you get a nice shiny package out of it (pretty much any platform you want) and can confidently install it nearly anywhere without having to worry about whether or not you can get to any external repos.

I'll see about putting together a Cody-fied (read: spoon-fed) how-to on getting started with Omnibus shortly.

In the meantime, if you're already using Omnibus and your builds are taking forever due to your software cache getting nuked every time you try to add a dependency, check this out.