Migrating legacy Condor configurations to Wallaby

htcondor
mrg
wallaby
Published

September 27, 2010

Wallaby provides a great way to manage Condor configurations, and if you’re just starting out with Condor, it’s easy to do things the Wallaby way from the start. However, many installations have their own systems for managing Condor features and configuration files — perhaps with separate files representing common, node-specific, and feature-specific configurations. In this post, we’ll talk about wallaby feature-import, a command that lets you import Condor configuration-file snippets into the Wallaby database as Wallaby features, complete with metadata and relationships with other features.

The basic syntax of the file you’ll pass to wallaby feature-import is the same as the basic syntax of a Condor configuration file; in fact, you can import many Condor configuration files without modification. (Only the NAME = VALUE syntax is supported at this time; you can’t use the NAME = +VALUE syntax for DC_DAEMON_LIST.) However, there are some special directives that you can pass to the configuration file as comments:

#name NAME
Indicates that this feature should be named NAME. (Note that you must supply a name for the feature, either on the command line, via -n NAME, or in the file itself.)
#includes FEATURE
Indicates that this feature should include FEATURE, which must exist in Wallaby.
#dependson FEATURE
Indicates that this feature should depend upon FEATURE, which must exist in Wallaby.
#default PARAM
Indicates that this feature should include PARAM, and set it to its default value.

Because Condor ignores these comments, it is possible to use the same file both as a Condor configuration file and as a Wallaby feature description. (Other comments will be ignored by wallaby feature-import, but they may generate harmless warnings.) wallaby feature-import will automatically create any parameters referenced in the file that don’t already exist in Wallaby’s database; this can be a great time-saver for migrating to Wallaby from a custom configuration-management system.

The following example file is a feature that defines eight startds on a single node:

#name EightStartds #depends Master #depends NodeAccess USE_PROCD=FALSE COLLECTOR_HOST=condor-collector.freevariable.com ALLOW_WRITE=.freevariable.com ALLOW_READ=.freevariable.com SEC_DEFAULT_AUTHENTICATION_METHODS=CLAIMTOBE MASTER.PLUGINS = $(LIB)/plugins/MgmtMasterPlugin-plugin.so STARTD.PLUGINS = \((LIB)/plugins/MgmtStartdPlugin-plugin.so STARTD_ST1=\)(STARTD) STARTD_ST1_ARGS=-f -local-name ST1 STARTD.ST1.STARTD_NAME=st1 STARTD.ST1.ADDRESS_FILE=\((LOG)/.startd1-address STARTD.ST1.STARTD_LOG=\)(LOG)/Startd1-Log STARTD_ST2=\((STARTD) STARTD_ST2_ARGS=-f -local-name ST2 STARTD.ST2.STARTD_NAME=st2 STARTD.ST2.ADDRESS_FILE=\)(LOG)/.startd2-address STARTD.ST2.STARTD_LOG=\((LOG)/Startd2-Log STARTD_ST3=\)(STARTD) STARTD_ST3_ARGS=-f -local-name ST3 STARTD.ST3.STARTD_NAME=st3 STARTD.ST3.ADDRESS_FILE=\((LOG)/.startd3-address STARTD.ST3.STARTD_LOG=\)(LOG)/Startd3-Log STARTD_ST4=\((STARTD) STARTD_ST4_ARGS=-f -local-name ST4 STARTD.ST4.STARTD_NAME=st4 STARTD.ST4.ADDRESS_FILE=\)(LOG)/.startd4-address STARTD.ST4.STARTD_LOG=\((LOG)/Startd4-Log STARTD_ST5=\)(STARTD) STARTD_ST5_ARGS=-f -local-name ST5 STARTD.ST5.STARTD_NAME=st5 STARTD.ST5.ADDRESS_FILE=\((LOG)/.startd5-address STARTD.ST5.STARTD_LOG=\)(LOG)/Startd5-Log STARTD_ST6=\((STARTD) STARTD_ST6_ARGS=-f -local-name ST6 STARTD.ST6.STARTD_NAME=st6 STARTD.ST6.ADDRESS_FILE=\)(LOG)/.startd6-address STARTD.ST6.STARTD_LOG=\((LOG)/Startd6-Log STARTD_ST7=\)(STARTD) STARTD_ST7_ARGS=-f -local-name ST7 STARTD.ST7.STARTD_NAME=st7 STARTD.ST7.ADDRESS_FILE=\((LOG)/.startd7-address STARTD.ST7.STARTD_LOG=\)(LOG)/Startd7-Log STARTD_ST8=\((STARTD) STARTD_ST8_ARGS=-f -local-name ST8 STARTD.ST8.STARTD_NAME=st8 STARTD.ST8.ADDRESS_FILE=\)(LOG)/.startd8-address STARTD.ST8.STARTD_LOG=$(LOG)/Startd8-Log DAEMON_LIST=MASTER,STARTD_ST1,STARTD_ST2,STARTD_ST3,STARTD_ST4,STARTD_ST5,STARTD_ST6,STARTD_ST7,STARTD_ST8 SLOT_TYPE_1=cpus=100%,disk=100%,swap=100% SLOT_TYPE_1_PARTITIONABLE=TRUE NUM_SLOTS=1 NUM_SLOTS_TYPE_1=1