Retryable operations

In every project that I’ve worked on, there’s always a need of a certain piece of functionality: retrying an operation. Normally it’s about calls over the network that can fail once, but then succeed. It can be about a lot of other stuff, mostly including communication with another system (be it over the network or not). It is functionality that you absolutely need in most applications, especially if you want them to be highly available (as pointed out here, for example).

And every time I have to introduce this functionality in a project, I check the standard libraries we have imported, and there’s no such thing. So I always end up copy-pasting the same piece of code from previous projects of mine. I don’t even remember when was the first time I introduced it, but it’s “traveling” with me ever since. So here it is:

/**
 * Class that provides retrying functionality. Example:
 * <p></p>
 * <code>
 * Callable&lt;String&gt; callable = new Callable&lt;String&gt;() {..};
 * String result = RetryableOperation.create(callable).retry(5, IOException.class);
 * </code>
 *
 * @param <T> the return type of the operation
 */
public class RetryableOperation<T> {
    private Callable<T> callable;
    private Runnable runnable;
    private boolean exponentialBackoff;
    private int backoffInterval = 500;

    /**
     * Create a retryable operation based on a Callable instance. The return
     * type of retry(..) is the type parameter of the Callable instance.
     *
     * @param callable
     * @return
     *      a new instance of RetryableOperation
     */
    public static <T> RetryableOperation<T> create(Callable<T> callable) {
        return new RetryableOperation<T>().withCallable(callable);
    }

    /**
     * Creates a retryable operation based on a Runnable instance. In this case
     * the retry(..) method always returns null.
     *
     * @param runnable
     * @return
     *      a new instance of RetryableOperation
     */
    public static RetryableOperation<?> create(Runnable runnable) {
        return new RetryableOperation<Object>().withRunnable(runnable);
    }

    /**
     * Retries the operation. Retrying happens regardless of the exception thrown.
     *
     * @param retries
     *      number of retries before the exception is thrown to the caller
     * @param exceptions
     *      the operation will be retried only if the exception that occurs is one of the
     *      exceptions passed in this array
     * @return
     *      the result of the operation (null if Runnable is used instead of Callable)
     * @throws Exception
     *      the exception that occurred on the last attempt
     */
    public T retry(int retries, Class<? extends Exception>... exceptions) throws Exception {
        if (callable == null && runnable == null) {
            throw new IllegalStateException("Either runnable or callable must be set");
        }
        Set<Class<? extends Exception>> retryFor = new HashSet<Class<? extends Exception>>();
        retryFor.addAll(Arrays.asList(exceptions));
        for (int i = 0; i < retries; i++) {
            try {
                if (exponentialBackoff && i > 0) {
                    int sleepTime = (int) ((Math.pow(2, i) - 1) / 2) * backoffInterval;
                    Thread.sleep(sleepTime);
                }
                if (callable != null) {
                    return callable.call();
                } else if (runnable != null) {
                    runnable.run();
                    return null;
                }
            } catch (Exception e) {
                if (retryFor.isEmpty() || retryFor.contains(e.getClass())) {
                    if (i == retries - 1) {
                        throw e;
                    }
                } else {
                    // if the exception is not the expected one, throw it immediately
                    throw e;
                }
            }
        }
        // can't be reached - in case of failure on the last iteration the exception is rethrown
        return null;
    }


    private RetryableOperation<T> withCallable(Callable<T> callable) {
        this.callable = callable;
        return this;
    }

    private RetryableOperation<T> withRunnable(Runnable runnable) {
        this.runnable = runnable;
        return this;
    }

    public RetryableOperation<T> withExponentialBackoff() {
        this.exponentialBackoff = true;
        return this;
    }
}

It is very simple, and yet works pretty well. You can either retry every failure, or you can retry a specific exception (you don't want to retry NullPointerException, but you have to retry network failures, having configured the proper timeouts, of course):

   Result result = op.retry(3);
   ...
   Result result = op.retry(3, IOException.class);

I had even proposed this piece to guava for inclusion, and then saw some other similar proposals, but to my knowledge there is no such functionality - neither in guava, nor in apache commons yet. And I'm not creating a new github project, because that would require managing an entry in maven central, and it's too much of an effort for just a single utility class.

Of course, there are other attempts to solve this, which have a bit bigger APIs and footprints - the retry guava extensions and the recently extracted as a separate project spring-retry. They are worth checking, and they have maven dependencies ready to import.

Whatever option you choose, check if it supports anonymous functions (since Java 8). It probably does automatically, but check anyway.

The point is to have this functionality available and with a very easy API, so that you can spare the users avoidable failures - retrying calls to external systems a few times is a must.

P.S. Depending on the complexity and the requirements, this can be an underlying component, or used in conjunction with a "Circuit breaker" (an implementation here)

P.P.S The reddit discussion brought some other nice utilities

5 thoughts on “Retryable operations”

  1. When supporting additional runnable objects, you can wrap the given runnable in an internal callable that returns Void? You’d minimize state.

  2. I’d argue that Callable and Runnable are not the best choice here, but rather some form of transactions that offer rollback functionality should be used – in case the call fails mid-way and leaves things in an inconsistent state.

  3. Why not use an interceptor or a dynamic proxy for that matter? Did that for the dreaded ORA-8177 database error. An interceptor is even nicer to do in Spring than in CDI/JEE, because you can define the relationship with the transaction boundaries (inside or outside).

Leave a Reply

Your email address will not be published. Required fields are marked *