Chapeau - Misfeature recovery and APIs

My team recently agreed that it would improve the usability of our main Trello board if we moved lists containing cards we’d completed in previous years to archival boards. The idea was that those lists and cards would still be searchable and accessible but that they wouldn’t be cluttering our view of our current work. I moved very old lists late in November, moved all of our lists from 2017 at the beginning of this week, and prepared to bask in a web page containing only recent virtual index cards.

My basking ended abruptly, as baskings are wont to do. In this case, the abrupt end was occasioned by an offhand question from a colleague:

“By the way, what’s the deal with me getting removed from all of my old cards?”

I looked at the Trello board I’d created to archive activity from 2017 and saw that the only cards that had a member attached were my cards. Even though I’d made the archive board visible to the whole team, every other person on the team was removed from her cards when I moved the lists.

Now, I’m not a Trello expert and hope I’ll never become one. It may be that removing users from cards on boards they don’t belong to is actually the correct behavior. However, having such a drastic side effect occur without warning is absolutely user-hostile.¹

Since software is rarely content to injure without also insulting, Trello also insinuated that I had explicitly removed my colleagues from their cards, like this:

So not only had I screwed up our team’s task history, but I looked like a jerk with too much free time.

Fortunately, all of those spurious explicit removals gave me a way to start unwinding the mess. Those member removals were captured in the actions log for each card as removeMemberFromCard actions; I was able to see them by exporting cards as JSON:²

    "actions": [
      {
        /* ... */
        "type": "removeMemberFromCard",
        "date": "2018-01-11T15:57:16.652Z",
        "member": {
          "id":  /* ... */,
          "avatarHash":  /* ... */,
          "fullName": "Erik Erlandson",
          "initials": "EJE",
          "username":  /* ... */
        },
        "memberCreator": {
          "id":  /* ... */,
          "avatarHash":  /* ... */,
          "fullName": "William Benton",
          "initials": "WB",
          "username":  /* ... */
        }
      }
    ],

Trello provides a pretty decent API, so I got to work. (The official Trello Python client appears to lack support for Python 3; I used py-trello instead.) My basic approach was to look for removeMemberFromCard actions that had happened since just before I moved the lists, identify the removed members from each card, and then add them back to the card.

I was able to get our history restored pretty quickly. Here are some of the minor snags I hit with the Trello API and how I worked around them:

By default, querying for actions on cards only returns card-creation actions and comments. You will need to specify an explicit action type filter to the API (e.g., removeMemberFromCard or all) in order to get all relevant actions.
Even though I cached results and thought I took adequate care to avoid Trello’s rate limits, I found myself regularly getting rate-limited at the /1/members endpoint while resolving from member IDs to py-trello Member objects to pass to the add_member function on a card. I was able to work around this by converting the dict corresponding to the member in the action to a namedtuple instance, which acted enough like a Member object to do the trick.³
Some cards didn’t have the removeMemberFromCard actions. This actually seems like a Trello bug, but I was able to work around it by adding everyone who had ever been added to a card but wasn’t currently on it. This means that there may be some people spuriously ascribed to cards now (i.e., people who should have been explicitly removed from cards), but I think it’s better to have slightly lower precision than virtually zero recall in this application. (Also, our team’s practice is to only add members to cards when they’re in progress or complete, which minimizes the potential impact here.)

My code, which is quick, dirty, and profoundly underengineered, is available for your review. To use it, you’ll need a Trello API key, OAuth secret, and token, all of which you can get from Trello’s developer site.

The code is certainly not that broadly useful but hopefully the takeaway lesson is: you can recover from a lot of application bugs and misfeatures if your data model explicitly tracks state changes.⁴ It may even be worth going to a data representation that explicitly allows rollback in some cases. Finally, If you expose a way to inspect history with your API, users can even recover from your bugs without your help.

Footnotes

Could Trello have asked if I wanted to invite users to the board? Told me I’d be removing member ascriptions from the cards before moving? It certainly seems like it should have.↩︎
I’ve redacted unnecessary information, including usernames, member IDs, and other IDs.↩︎
Hooray for the Wild West of untyped languages, eh?↩︎
It’s almost as if those wacky functional programming zealots have a point about their persistent data structures.↩︎