Reduce, Reuse, Refactor: On The Relationship Between Automation & Code

Welcome! This is a four-part series that explores the deep fundamentals of Object-oriented programming using a mostly platform-agnostic lens:

• On The Relationship Between Automation & Code (this post)
• What Is An Object?
• Object Variables
• Interfaces & Object Inheritance

When in the course of human events it becomes necessary to optimize processes, code has increasingly become the mechanism by which these optimizations are realized. Whereas in the past, a mill might have done the work of dozens — and in some cases still works in that same function — code provides an engineer with the tools with which to apply that same savings to an effectively limitless number of corporate & everyday applications. While the term “digital transformation” has (at least in my mind) outlived its usefulness as the kind of phrase that excites people, the true digital transformation has been the process by which actual physical inputs and actual physical outputs have been superceded by the ability to take digital inputs and produce those same physical outputs. That code is also the superior tool for transforming physical inputs to digital outputs, and certainly digital inputs to other digital outputs, should also be clear. Automation — the process by which code can both enhance and in some cases replace — manual operations is the calling card of the effective engineer.

And yet. And yet, and yet, and yet. For all its much-touted transformative effects, the digital age has also largely become one where we live at the mercy of bugs and other code-related defects. In exchange for being able to fill out forms online; for being able to e-sign documents; conduct interviews; and order food online, we’ve also paid a different price — learning to live with conveniences that don’t always work. And it makes sense. “To err is human,” after all, and to expect perfection from a system built by humans is, as it turns out, an effective lesson in humility. Now layer on things like deadlines, modern corporate culture, internal turf wars and other such net-negatives prevalent throughout much of our modern working lives, and is it any wonder that the code that gets produced and run by applications globally occasionally suffers in the same way that we do?

So that’s the current state of affairs, when it comes to programming. You might be asking yourself questions, questions like:

• but I thought this was a series about programming on the Salesforce platform?
• what does this have to do with me?
• how do low-code solutions factor into the above?

And those are all good questions; questions that this series will certainly make the attempt to answer. But I think that it’s important to set the stage on our shared starting points; to remind you that whether you work on a team, by yourself, or for yourself, we all have much more in common than we tend to admit. Put another way — if you want to do good work it doesn’t really matter whether or not the motivation for doing so matches that of somebody else. People have plenty of reasons to work hard:

• to improve themselves
• to earn more money
• to gain satisfaction
• to waste less time
• to compare better to others
• to give themselves something to do

Our ways of being, motivations, conflicting emotions and desires are all woven together in a tough-to-isolate social and personal matrix that defines our lives. Introspection is a tool, just like programming is. But that’s not what brings you here today, and my point is: that’s OK. Regardless of how or why you got here, I appreciate it, and furthermore I promise you that everybody can learn something from this series. Originally, I was going to publish this series as a book, but I had philosophical issues with the idea of charging money for this content when everything else on the Joys Of Apex is free, and (in my experience) most of the paid book content out there about Salesforce suffers from the same software ills I’ve listed above — not enough time spent in quality control, not enough time spent on fundamentals, rushed to meet a deadline, etc…

There are so many myths about programming that are propagated for reasons as manifold as the list of motivations above. In my experience, the microcosm that Salesforce development represents isn’t any less or more prone to this than any other programming practice, but it’s important to be mindful about how — and who — you choose to learn from. You may have heard the phrase, “trust but verify” before, and the first explicit lesson of this series, if I can be so bold as to term it that, is to apply that motto to how you choose to learn.

Which brings me to my next point in this intro: who I am, and why you should (or shouldn’t!) trust me. For the last five years, I’ve been writing at The Joys Of Apex about Salesforce development. Five years before that, I did my first Salesforce implementation. I’ve worked on dozens of Salesforce projects over the last decade: as an in-house product owner turned developer; as a small business owner; as a consultant to some of the largest brands in the world; and lastly (at the time of this writing) as a Principal Software Engineer at Salesforce itself. Throughout that time, my focus has been almost entirely on giving back to the community, open sourcing work that I believe others can benefit from and doing pro bono work for nonprofits & small businesses.

I’m a lifelong learner, and my goal is to provide you with something of lasting value that empowers you to build better, faster, and more secure code by turning one specific programming tenet on its head: the idea that fast & good are two sides of the same coin. The truth is that baking quality into your programming practice through simple, repeatable patterns will always speed you up, and I can show you how. And it all begins with objects!

Object-oriented Programming

Lot of words in a series about code with no code shown so far, right? Let’s begin by jumping ahead to a code snippet so that we can come back to talk about Object-oriented Programming — a loaded phrase that we’ll be unwinding throughout this series. Don’t worry if it feels like this example is showing off concepts, terminology, or syntax that you’re unfamiliar with (and feel free to skip over it for now, if so — just make sure to return here after finishing the series!) — while we’ll be diving right in, it’s only to show how powerful objects can be so that the real beginning to the series can start from the absolute basics. This series is intended for learners of all experience levels; whether you’ve never written code before or consider yourself to be an expert nonpareil. At times, that will mean that if you’re in the former category, you may find the subject material daunting and/or unexplained at first. I promise that each concept is meant to build on itself such that by the end of each post you’ve had the chance to learn something new, even if to completely understand each segment means having to refer back to previous sections to unearth new understanding(s).

With that out of the way, let’s dive in.

A Naive Approach To Implementing Retry Mechanisms

// let's assume we're in some Apex class looking at some method:
public static void thingDoer(List<SObject> records) {
  for (SObject record : records) {
    try {
      someOperationThatCanFail(record);
    } catch (Exception ex) {
      // error handling
    }
  }
}

private static void someOperationThatCanFail(SObject record) {
  if (System.now().getTime().format().endsWith('5')) {
    throw new IllegalArgumentException('You hit the wrong lotto jackpot today!');
  }
}

Guard clauses like the try/catch above are common in code, even if someOperationThatCanFail is … hopefully unique, even amongst examples. They tend to get repeated. That’s OK! One of the bigger mistakes that I tend to see when people first learn about programming principles like DRY (Don’t Repeat Yourself) is the tendency to use objects to abstract away common language syntax. But now, let’s consider a new requirement for our application, given the current flakiness associated with thingDoer: the desire to implement a retry mechanism.

In a typical setting, the person or team responsible for this functionality will be tasked with improving the current functionality — especially because something that works but also occasionally fails is very typical when working with something like an API. As a concrete example, consider the Slack chat.postMessage documentation, which has a corresponding link to the Rate Limits page. Here, we get to learn about what it means for an API to be in the “Special Tier”:

Rate limiting conditions are unique for methods with this tier. For example, chat.postMessage generally allows posting one message per second per channel, while also maintaining a workspace-wide limit. Consult the method’s documentation to better understand its rate limiting conditions.

In both cases (the code snippet above, and in the case with sending messages to Slack), a retry mechanism serves to improve the resilience of the code and, ultimately, the end user experience. A simple take on implementing a retry might look something like this:

private static final Integer RETRY_CAP = 5;

public static void thingDoer(List<SObject> records) {
  for (SObject record : records) {
    attemptToDo(record);
  }
}

private static void attemptToDo(SObject record) {
  Integer currentRetryCount = 0;
  while (currentRetryCount < RETRY_CAP) {
    try {
      someOperationThatCanFail(record);
      break;
    } catch (Exception ex) {
      // error handling
    }
    currentRetryCount++;
  }
}

private static void someOperationThatCanFail(SObject record) {
  if (System.now().getTime().format().endsWith('5')) {
    throw new IllegalArgumentException('You hit the wrong lotto jackpot today!');
  }
}

If I run this on a single record in a simple script 100 times:

// using Anonymous Apex
for (SObject unused : new SObject[100]) {
  thingDoer(new List<SObject>{ unused });
}
/**
 * The output might look something like the below - 6 exceptions out of 100
  System.IllegalArgumentException: You hit the wrong lotto jackpot today!
  System.IllegalArgumentException: You hit the wrong lotto jackpot today!
  System.IllegalArgumentException: You hit the wrong lotto jackpot today!
  System.IllegalArgumentException: You hit the wrong lotto jackpot today!
  System.IllegalArgumentException: You hit the wrong lotto jackpot today!
  System.IllegalArgumentException: You hit the wrong lotto jackpot today!
*/

So the 94 out of 100 attempts succeed immediately. But we know from experience that while someOperationCanFail is a ridiculously “compressed” version of an error condition, error conditions like that method are routine when we consider the day-to-day operations of a business. Which brings us back to the Slack API, and how rate-limiting differs from other failure states. In real world conditions, that might mean we have many different code paths that can all benefit from retry mechanisms, but suffer from taking slightly different approaches to handling them. I say “suffer” purposefully, because the naive approach that we’ve just seen to implementing retry mechanisms in multiple places in a codebase leads to a lot of code like attemptToDo above.

This is an important principle that we’ll be returning to again and again in this series: concepts (retrying something, in this case) map directly to shapes in code. Code encapsulates concepts.

How Concepts Multiply Through A Codebase

Let’s expand our “simple” retry example to show how “implementing retries” by copying and pasting can go wrong:

// first, some rather vanilla callout framework:
public class Api {
  private static final Integer RETRY_CAP = 5;

  public static HttpResponse makeRequest(HttpRequest req) {
    Integer currentRetryCount = 0;
    HttpResponse res;

    while (currentRetryCount <= RETRY_CAP && res == null) {
      try {
        res = new Http().send(req);
      } catch (Exception ex) {
        // error handling
      }
      currentRetryCount++;
    }
    return res;
  }
}

// and then a more specific example for Slack:
public class SlackMessenger {
  private final Slack.BotClient client;
  private static final Integer RETRY_CAP = 5;

  public SlackMessenger(String appName, String slackWorkspaceId) {
    this.client = Slack.App.getAppByName(appName).getBotClientForTeam(slackWorkspaceId);
  }

  public Slack.ChatPostMessageResponse sendMessage(String slackChannelId, Slack.ViewReference formattedMessage) {
    Integer currentRetryCount = 0;
    Slack.ChatPostMessageResponse response;
    while (currentRetryCount <= RETRY_CAP && response == null) {
      try {
        response = this.client.chatPostMessage(
          new Slack.ChatPostMessageRequest.builder()
                .channel(slackChannelId)
                .viewReference(formattedMessage)
                .build()
        );
      } catch (Exception ex) {
        // error handling
      }
      currentRetryCount++;
    }
    return response;
  }
}

Both of these examples purposefully have several subtle bugs in them. Can you spot them? Don’t worry if you can’t, and don’t worry if it’s frustrating that you can’t. The reality of the situation is that copying and pasting code creates several different problems at the same time:

1. Updates to a particular code block without searching for other usages of the same paradigm (in this case, retrying) leads to code drift; sometimes, that’s intentional, but without explanatory comments or meaningful source control messages, code drift is challenging to reason about. Unless the person that authored or changed the code is immediately available to answer questions, we can only guess (or wait) as to their motivations and intentionality
2. Blindly copying + pasting allows these sorts of subtle bugs to proliferate throughout a system. Sometimes that’s not a problem; other times, it leads to increased running costs (extra API calls, anyone?) and increased maintenance costs (the costs to find and update everything increases).

The best engineers in the world build up the equivalent of muscle memory for situations in which they’ve been burned in the past, and drawing from those experiences allows them to more confidently manage risks when modifying code and writing net-new code. Everybody makes mistakes though. Repeat that to yourself! I make mistakes every day, and I doubt that will ever change. The only way that I can confidently exclude mistakes from happening is by meticulously curating test cases that ensure that the possible error states that I know about become impossible.

In the above example, copying and pasting might look like it’s saved time on the surface. Instead, it’s created several different problems:

• the first issue is that the RETRY_CAP variable has itself become misleading due to a simple typo. Because the while loop is comparing the currentRetryCount variable using the less than or equals comparison operator. Technically we’re retrying one too many times. This goes back to the question of cost before — maybe it’s fine to retry one-too-many times. Maybe it’s an issue that causes your program to exceed your licensed number of API calls per day! Only time will tell.
• try/catching in the case of the Api class assumes that all HTTP-related errors lead to explicit exceptions, but that’s not uniformly true
• as well, Slack’s documentation shows that this is also true for the SlackMessenger: while there are explicit exceptions that can be thrown while using the Slack SDK, there are also error conditions that are represented on the Slack.ChatPostMessageResponse object itself

Objects To The Rescue

As I said earlier, objects really shine as a way to map concepts to code. When it comes to representing what a retry mechanism looks like, it doesn’t take much code at all to do this:

public abstract class RetryMechanism {
  private final Integer retryCap;
  private static final Integer DEFAULT_RETRY_CAP = 5;

  protected Object potentialNext;

  public RetryMechanism(Integer retryCap) {
    this.retryCap = retryCap;
  }

  protected RetryMechanism() {
    this(DEFAULT_RETRY_CAP);
  }

  public abstract Boolean hasNext();
  public abstract Object next();

  protected Object getOrRetry() {
    Integer currentRetryCount = 0;
    while (currentRetryCount < this.retryCap && this.hasNext()) {
      try {
        this.potentialNext = this.next();
      } catch (Exception ex) {
        // error handling
      }
    }
    return this.potentialNext;
  }
}

Let’s transform each of the above examples to take advantage of RetryMechanism:

// this example is also an example of why it can be easier to refactor
// static methods prior to doing something like this, but I've chosen
// the more straightforward approach here to keep it simple
public class Api extends RetryMechanism {

  private static HttpRequest req;

  public override Boolean hasNext() {
    return this.potentialNext == null;
  }

  public override Object next() {
    return new Http().send(req);
  }

  public static HttpResponse makeRequest(HttpRequest req) {
    req = req;
    return (HttpResponse) new Api().getOrRetry();
  }
}

public class SlackMessenger extends RetryMechanism {
  private final Slack.BotClient client;

  private String slackChannelId;
  private Slack.ViewReference formattedMessage;

  public SlackMessenger(String appName, String slackWorkspaceId) {
    super();
    this.client = Slack.App.getAppByName(appName).getBotClientForTeam(slackWorkspaceId);
  }

  public override Boolean hasNext() {
    if (this.potentialNext == null) {
      return true;
    }

    Slack.ChatPostMessageResponse res = ((Slack.ChatPostMessageResponse) this.potentialNext);
    if (res.isOk() == false) {
      Integer retryAfterSeconds = Integer.valueOf(res.getHttpResponseHeaders().get('Retry-After')) ?? 0;
      Datetime nowish = System.now();
      // wait however many seconds
      while (nowish.addSeconds(retryAfterSeconds) <= System.now());
      return true;
    }

    return false;
  }

  public override Object next() {
    return this.client.chatPostMessage(
      new Slack.ChatPostMessageRequest.builder()
        .channel(this.slackChannelId)
        .viewReference(this.formattedMessage)
        .build()
    );
  }

  public Slack.ChatPostMessageResponse sendMessage(String slackChannelId, Slack.ViewReference formattedMessage) {
    this.slackChannelId = slackChannelId;
    this.formattedMessage = formattedMessage;
    return (Slack.ChatPostMessageResponse) this.getOrRetry();
  }
}

Wrapping Up

I’ve only refactored the Slack-based example to show how, suddenly, hasNext() can become a domain-specific focal point for applying business logic to retries. There’s a bit more boilerplate (though that would change if generics are introduced to Apex!) but the SlackMessenger and Api classes are much more flexible. One of them has already implemented logic that’s specific to the use-case at hand to improve retries. The other has the possibility to do something similar by expanding HTTP header responses, but hasn’t thus far. A third case might only need to retry a single time, in which case it can pass 1 as a value to the call to super.

This flexibility — the flexibility that objects create in a codebase by allowing for consistent but dynamic customization — is magic. But maybe your eyes are glossing over; maybe this is too much too fast. I’ll close this intro by saying that whether the above seems routine or wizardry, returning to the core fundamentals is a chance for everyone to learn something new and exciting about Apex, and about Object-oriented programming languages in general. We learned that code encapsulates concepts directly — there are a lot of other, more subtle lessons to pluck out from the examples above. Be sure to revisit this code after reading the rest of the series. What stands out at you? What questions does it bring to the surface?