One of the great things about Wallaby is that it’s a platform, not merely a tool. Put another way, if it doesn’t do exactly what you want, you can use its API to build tools that benefit from configuration validation and deployment. We’ve talked in the past about a number of useful tools built on the Wallaby API. (Another cool Wallaby API client that I hope to talk about more in the future is Erik Erlandson’s Albatross project, which programmatically generates and changes pool configurations in order to test Condor scale and functionality.)
The Wallaby API is designed to be sufficiently general to allow developers to do just about anything with configuration data, not to unnecessarily restrict users to a few use cases that we thought of. Because of this generality, some tasks might require adopting application-level conventions. In this article, we’ll cover one such convention and see how the Wallaby API is flexible enough to handle an interesting use case — namely, tagging nodes with various keywords, perhaps as supplied by a user or generated by an agent like sesame or matahari.
First, though, we’ll review how configurations are generated and applied to nodes. Recall that a node is a member of several groups. These memberships are ordered: a node’s lowest-priority membership is always in the special default group (which includes every node), and its highest-priority membership is always in a special, node-specific identity group (which only includes a single node). Zero or more memberships in explicit groups may occupy the priority space between the default group and a node’s identity group. In the illustration below, node.local is a member of two explicit groups, which have blue backgrounds: “EC2 submit nodes,” and “Execute nodes.”
When Wallaby calculates the configuration for node.local., it will begin with a copy of the default group’s configuration. It will then repeatedly apply the configurations for the explicit groups and identity group in order of increasing priority, so that parameter-value mappings from higher-priority groups take precedence over lower-priority ones (and thus either replace these or are appended to them, depending on the mapping kind). The condor_configd, which takes a node’s configuration from Wallaby and installs it on a node, will also cause the Wallaby groups a node is a member of (as well as the Wallaby features it has installed) to be advertised for use in Condor matchmaking. So a Condor job could specify that it wanted to match against a node that was a member of the “EC2 submit nodes” group; this would translate into a preference for node.local.
Because group names appear in machine ClassAd attributes, the list of explicit node memberships is a natural place to put tag information. Nodes would simply be members of “dummy” groups like “Desktop workstations” or “Machines with more than 8GB RAM,” and these keywords could be used in matchmaking or in searching for particular nodes. However, the list of explicit groups is not necessarily suitable for automatic manipulation: users will not necessarily expect their changes to be overridden, other API clients won’t necessarily expect users to rearrange or remove groups corresponding to tags, and in general it is impossible to determine whether a group membership should be interpreted as a tag or as an explicit membership.
We can adopt a convention to partition the space of group memberships. Say we create a special sentinel group to partition the membership space: every node will be a member of the sentinel group. All memberships that are of a lower priority than the sentinel group will be managed by tagging agents, and all memberships that are of a higher priority will be explicitly managed by the user. In the example below, “—EXPLICIT GROUPS” is the sentinel group, and groups in yellow correspond to tags.
This approach demonstrates the generality of the Wallaby API, and allows users to supply tag-specific configurations by installing parameters or features on tag groups, for example, ensuring that every desktop workstation has a policy that favors its owner, or overprovisioning high-memory nodes. However, it is not free of shortcomings. Firstly, Wallaby will present all of these groups — sentinel, tag, and explicit groups — so a user-facing tool that puts a friendly interface on node memberships and tagging will need to use the sentinel group to partition which groups it displays to the user. Secondly, the sentinel groups must be added to all nodes (in practice, we can assume that the absence of a sentinel group implies the absence of tags); in general, we are relying on clients to enforce invariants in this case. Finally, altering a node’s membership list to insert a tag will cause that node’s configuration to be validated at the next activation. If the node has a complex configuration, this could be expensive.