November 2009 Archives

Here are two quick notes (and a bonus meta-note) about the quickly-evolving SPQR project:

1. SPQR now includes a gem target (courtesy of jeweler). Pull a recent version and sudo rake install to try it out. Once the project has stabilized a little more, I will publish gems to gemcutter.
2. I'm mirroring the fedorahosted SPQR repository on github; if you're already a github user, you might rather follow the repository there.

I'm pretty excited about SPQR, but this site won't become "all SPQR, all the time." Since interested hackers can easily follow the day-to-day project status on github or fedorahosted, I will reserve future SPQR-related blog posts for substantial announcements.

In a previous post, I introduced SPQR and presented a couple of examples of how one could use SPQR to publish Ruby objects over QMF. Sometimes, though, you aren't starting from an application -- instead, you're starting from an XML QMF schema document. SPQR includes spqr-gen, a tool designed to automatically generate a skeleton SPQR application from a QMF schema. In this post, we'll see an example of spqr-gen in action.

First, let's look at a simple QMF schema for a class that exposes one method, echo, which returns its argument (note that the code examples may not show up if you're viewing this in a feed reader):

Running spqr-gen on this example produces two files: agent-app.rb and examples/codegen/EchoAgent.rb. As you can see, these two files contain all of the boilerplate we need to start implementing these QMF methods --- and, by chance, the boilerplate methods actually have the behavior that our agent is meant to!

spqr-gen is included in the SPQR repository; the system as documented in the last two posts is tagged "introducing-spqr".

SPQR is a framework to make it almost painless to create QMF agents in the Ruby language, and thus to write Ruby applications that can be managed remotely. I built it to serve as infrastructure for some new development at work, but am pleased to announce that it is now functional enough to be useful for a range of applications, and you can download it from its git repository on fedorahosted.org and try it out. If you know what a "QMF agent" is, then I hope you're already interested in learning more --- please skip ahead to the examples to see it in action! If you're confused, fear not and read on.

Background

AMQP is an open standard for interprocess messaging --- and an excellent basis for distributed applications with even particularly demanding requirements. I'm most familiar with the Apache Qpid implementation of AMQP, since many of my colleagues at Red Hat are Qpid committers. Qpid is scalable, high-performance, and open-source --- it also features a variety of language bindings.

One great feature of Qpid that isn't in other AMQP implementations is the Qpid Management Framework. QMF is essentially a protocol so that applications can allow themselves to be managed remotely. In this way, it is similar to familiar remote procedure call mechanisms: a server application publishes an interface specification, and client programs are able to interact with that interface by invoking proxy methods that forward their arguments to the server machine and ferry return values back to the client. Unlike RPC, however, QMF is built upon Qpid messaging, and so offers low-latency, high throughput, and great resiliency.

In the QMF world, "server" applications --- those that can be managed --- are called agents, and "client" applications are called consoles. (Consoles and agents communicate by connecting to the same Qpid message broker.) The interface to an agent is called a schema; while schemas are instantiated programmatically in the applications that use them, there is also a well-defined format for recording schema information in XML.

In older versions of Qpid, it was only possible to develop QMF agents in C++; with more recent versions (including the one packaged in Fedora 12), it is possible to use Ruby as well. Since I am currently developing a rather substantial QMF agent application in Ruby, I wanted to have robust infrastructure that would allow me to avoid repeating myself, skip writing boilerplate as much as possible, and get on with the interesting work of my actual application. SPQR is built on top of the existing Ruby QMF engine and allows developers to publish instances of standard Ruby classes (with minimal and unobtrusive annotation) over QMF.

Examples

The first example we'll run through is (of course) a very simple application that publishes one method, which merely greets the user. Ours will be a little more interesting than classic "Hello, world!" programs, though, since we'll provide a personalized greeting (and publish our answer to another process). Here's what our application would look like in plain Ruby:

Note that we're using keyword-style arguments for the hello method, since we'd like to be able to support out-parameters (that is, not merely a single return value). Otherwise, this is pretty straightforward. Here's what it would look like if we used SPQR to publish Hello.hello over QMF:

The core of the class is basically the same (although we do collect statistics now, primarily to demonstrate SPQR's support for QMF statistics). The only differences are in the annotations we've added in order to enable SPQR to manage this class (viz., mixing in the SPQR::Manageable module), and to tell SPQR which methods to expose over QMF (and the types of their arguments). We've also added find and find_all methods to make it possible for QMF console applications to query for a particular Hello object (or for all of them). (Since the Hello class has no state, we simply make it a singleton class and the find methods just return the sole instance.)

The second example is a little more interesting: it shows the intersection of SPQR's capability to publish objects to QMF and my Rhubarb library's capability to store specially-declared Ruby classes in SQLite databases. (You'll need SQLite and its Ruby bindings installed to run this example.) This example is a simple logging agent that exposes two classes: LogService, which is a singleton class that can generate log records, and LogRecord, which is a class backed by a database table that models these records.

Conclusion

SPQR is still under development, and is only available via git for the moment. However, if you're interested in developing QMF agent applications in Ruby, it supports a useful subset of QMF and presents a clean, simple way to expose your applications over QMF. I welcome comments, bug reports, feature requests, and patches.