Earlier today, I released version 0.4.0 of Rhubarb, a little object-graph persistence library for Ruby built on top of SQLite. Rhubarb 0.4.0 adds no new features, unless you consider “working with recent releases of the sqlite3 gem” to be a feature. The case of Rhubarb 0.4.0 is interesting, though, because it exposes a fundamental incompatibility between the package-dependency model of RubyGems and that of operating-system distributions.
The sqlite3 gem (formerly known as ruby-sqlite3) introduced some nontrivial semantic and interface incompatibilities between versions 1.2 and 1.3; as far as I can tell, 1.3 is a near-total rewrite. The main problems I ran into with Rhubarb were related to prepared statements and to the (now deprecated and subtly broken) “type translation” functionality that is meant to take untyped SQLite values and convert them to Ruby objects of an appropriate type.1
In some circles, backwards incompatibility is anathema, especially if (as in this case) it entails deprecating useful functionality and providing no replacements or workarounds. However, in the Ruby world, it essentially doesn’t matter: the RubyGems package manager is designed to let you install multiple versions of a gem package simultaneously; gems can specify that they will only work with a particular minor-version series of a dependency; and there are a whole host of external tools to make it easier to install an application in a private, application-specific local environment with the correct versions of its dependencies.2
Since production Ruby applications can run indefinitely against an older version of a dependency without affecting the rest of the system, there is really little disincentive to breaking backwards compatibility. Unless, that is, someone wants to package production Ruby applications as part of Fedora or another Linux distribution. In that case, the packager is likely stuck with just the most recent stable version of the dependency — and this version must work for every application packaged with the distribution. Since it is typically not an option for downstream packagers to install “local” versions of packages, this means that they are faced with an unappealing choice: break (and thus don’t ship) some software or package unnecessarily conservative versions of dependencies.
This problem is especially interesting since, due to the impedance mismatch between these packaging cultures, there is no obviously correct solution.
Footnotes
I observed several related problems that led to a lot of test suite and application failures: (1) In the 1.2 series, automatic type translation caused field values to be converted based on the given type of the column; in newer versions (in which automatic type translation is officially deprecated), conversion proceeds based on properties of the string representation of the value. So, for example, a
varcharfield containing the value"fooblitz"would become a RubyString, but avarcharfield containing the value"1234"would become a RubyFixnum. (2) Enabling (deprecated) automated type-translation support in the version ofsqlite3-rubyin Fedora 15 breaks theresults_as_hashfacility that allows addressing rows by column name and not merely column index. (I understand this is fixed in a more recent release.) (3) Attempting to install explicit type translations results in a deprecation warning, leaving no reasonable way to work around the deprecation of automatic type translation. (4) Foreign-key triggers are apparently now incompatible with prepared statements, resulting in confusing failures upon database close if both are enabled.↩︎- While the local-environment idea is well-known, notably in the Java world and anywhere that people ship software that uses Boost, some environments take the basic philosophy further still. Consider the build process for applications using the webmachine REST framework, which clones and checks out particular versions of webmachine’s dependencies!
↩︎