Uncompatible Systems

There is a wealth of resources available now through Amazon Web Services. One of which is the EC2 compute cloud. But EC2 is really just a bunch of virtual machines with finite resources. To enable these to act as a render farm, performing asyncronous jobs, there needs to be some kind of management front end. Here’s what I’ve found so far.

Scalr has nice process documents, a screencast and come flow charts. Main problem is it’s written in object oriented PHP. Suprisingly, this decision didn’t influence the developers enough to use some kind of ORM, and I’m seeing SQL in functions as high level as RunInstance. I’m going to write off Scalr for now. I wish them the best.

On the level of further abstraction, Morph is a company who’s attempting to completely abstract Ruby on Rails and offer a set of management interfaces for a complete production deployment scenario. Unfortunately this makes details like dependency management and base OS choice impossible, as well as making version conflict resolution complicated within a community (Ruby on Rails) that is currently not well coordinated, to put it nicely.

So…survey says…NOTHING. I’m off to generate some Rails scaffolding. Of course it will be open source. I’m actually sort of excited.

After working with Xen for a while, I’m packing it away after the Citrix aquisition. I’m now more interested in hardware virtualization with KVM, which is an extension of the QEMU project to run with hardware CPU virtualization. The userland with KVM is by no means as rad and the networking also requires some manual intervention, though it’s nothing out of hand.

The first problem in running a kvm host on an existing server. The host needs a kernel greater than 2.6.20. This led me to Debian testing. As always, testing is not stable, especially with low level packages like kernels. I settled on kernel package linux-image-2.6.22-3-amd64 as of this writing. The intel VT instructions had to be enabled in the server’s BIOS. After that the kvm and kvm-intel modules could be inserted into the running kernel safely.

The second problem was installing a guest OS without a monitor. While possible, I abandoned this in favor of going home to my desktop and making an image from a host with a monitor.

The third problem is managing multiple serial terminals via the host OS. This requires some manual changes once you make that guest OS on the desktop computer. They are well documented.

The fourth is configuring bridged networking for multiple virtual ethernet adaptors. Setting up a bridge is simple, though setting up multiple guest OS’s appear to have some problems, since I’m getting address overlap errors.

I spent a few hours trying to compile Darwin Streaming Server on Debian Stable (Etch) x86_64. It appears I am doing something unique. I find this surprising since I rarely if ever have done anything unique when it comes to software. I’ll be documenting my progress here and eventually forking over a Debian package if all goes well.

A good example of why companies who primarily build closed source proprietary software have no business pretending to be members of the open source community. Apple’s Darwin Streaming Server is documented as “supporting Linux” and has source code available. The source code is available under the Apple Public Source License. The release notes in the latest source release are dated 2002. As of this writing, it is 2008. The public mailing lists from Apple are filled with would-be developers who began talk about forking the code two years ago because Apple isn’t merging changes in to the project and they won’t give non-employees commit access for CVS. Then there are the details…

There is no documentation for building on GNU/Linux. In fact, the build script produces errors on an x86_64, 2.6 kernel, GCC 4.1 system. To add insult to injury, the installer script doesn’t even use the proper ps(1) syntax. It’s clear that Apple has abandoned true cross-platform support for their open source projects and are not even interested in working with the public who has patched the code to address these issues. Let’s hope Darwin Streaming Server will eventually become a member of the open source community rather than a poser riding off some popular press from years ago.