Code

A functional Retry pattern

Functional programming

A few weeks ago, I finally took the time to (try to) find out what the fuss about functional programming was about. I had a long look at F#, played around with it to get a feel for it, and (of course) came away with a few new insights. Since I usually create applications that contain a lot of UI code and F#'s forte seems to be algorithmic, I couldn't see any immediate benefits to F# over C# or VB.NET, but I did make a resolution to write more "pattern-like" code. By that I mean functions that loop, iterate, test, etc., but where the inner body of the function is supplied using a lambda. F# is riddled with them, and so is Javascript: map(), reduce(), fold(), filter()...

A Retry pattern

The first real use came along when I ran into an issue opening an Access database on a DFS volume. Sometimes the database would be locked - I presume by DFS' volume replication functionality - and retrying after a short pause would solve the problem. There are many of those types of operations: opening a file across a network, opening a database connection (especially with a Failover Partner!), downloading a file from the Internet.

Basically, what I had in mind was:

Retry(() => DoSomething(), 5, 1000);

This should call DoSomething() at most five times, and sleep for 1000 between occasions where DoSomething() would throw an exception. Then, I ran into another case: opening the clipboard using P/Invoke needs to be retried until it returns a non-zero value.

Retry(() => return (OpenClipboard(IntPtr.Zero) != IntPtr.Zero), 5, 1000);

Since the two are different - the first method relies on exceptions, the second on return values - I wrote a static C# class called Retry that encapsulates the behaviour in two methods:

Retry.UntilTrue(int retries, int delayMs, Func<bool> operation)

and

Retry.WhileException<T>(int retries, int delayMs, Func<T> operation, Type exceptionType, bool allowDerivedExceptions)

They both take the following parameters:

retries: the maximum number of times to retry the operation
delayMs: the delay between retries, in milliseconds
operation: a lambda specifying the operation to retry

In the case of Retry.UntilTrue the operation takes no parameters and returns a bool indicating if the operation succeed. This is also the return type of Retry.UntilTrue: if true, the operation succeeded (possibly after one or more retries). If false, none of the attempts succeeded.

Retry.WhileException has two more optional parameters:

exceptionType: a Type that specifies which type of exceptions are "expected". These will be ignored and will cause a retry. If an exception is not expected, it is immediately rethrown - no retries. If this parameter is not specified, all exceptions are considered expected and a retry will occur whenever an exception is thrown
allowDerivedExceptions: specifies if exceptions that derive from the specified exception type will also be ignored. The default is false

Additionally, you need to provide the generic type parameter T, which should match the return type of your operation. Retry.WhileException returns the value from the first successful call (i.e. the first one that doesn't throw an exception). If no retries succeed, the exception thrown is passed through, and you'll need to catch it.

So, with this class retrying an operation is literally one line of code: wrap it in Retry.WhileException with a maximum retry count and a delay, and presto: retries are fully automated.

No rocket science

This is by no means rocket science. The takeaway for me was that functional code does not need to be complicated - ideally it isn't at all, really - but even then it can be extremely useful. Every line in the retry pattern is a possible bug. There's try/catch involved, and some type testing, which you can all do wrong - especially if your operation is itself part of a try/catch block. Generally you won't introduce any bugs when implementing a retry mechanism, but the chance is always there. Plus there's the convenience of single-line retries and the resulting improved readability. What's not to like?

For me, this type of function looks like a pretty good hammer - now, off to find more nails!

The Code

Here's the C# version of the class:

using System;
using System.Threading;

namespace Mobzystems
{
  public static class Retry
  {
    /// <summary>
    /// Retry the specified action at most retries times until it returns true.
    /// </summary>
    /// <param name="retries">The number of times to retry the operation</param>
    /// <param name="delayMs">The number of milliseconds to sleep after a failed invocation of the operation</param>
    /// <param name="operation">The operation to perform. Should return true</param>
    /// <returns>true if the action returned true on one of the retries, false if the number of retries was exhausted</returns>
    public static bool UntilTrue(int retries, int delayMs, Func<bool> operation)
    {
      for (var retry = 0; retry < retries; retry++)
      {
        if (operation())
        {
          return true;
        }
        Thread.Sleep(delayMs);
      }
      return false;
    }

    /// <summary>
    /// Retry the specified operation the specified number of times, until there are no more retries or it succeeded
    /// without an exception.
    /// </summary>
    /// <typeparam name="T">The return type of the exception</typeparam>
    /// <param name="retries">The number of times to retry the operation</param>
    /// <param name="delayMs">The number of milliseconds to sleep after a failed invocation of the operation</param>
    /// <param name="operation">the operation to perform</param>
    /// <param name="exceptionType">if not null, ignore any exceptions of this type and subtypes</param>
    /// <param name="allowDerivedExceptions">If true, exceptions deriving from the specified exception type are ignored as well. Defaults to False</param>
    /// <returns>When one of the retries succeeds, return the value the operation returned. If not, an exception is thrown.</returns>
    public static T WhileException<T>(
      int retries, 
      int delayMs, 
      Func<T> operation, 
      Type exceptionType = null, 
      bool allowDerivedExceptions = false
    )
    {
      // Do all but one retries in the loop
      for (var retry = 1; retry < retries; retry++)
      {
        try
        {
          // Try the operation. If it succeeds, return its result
          return operation();
        }
        catch (Exception ex)
        {
          // Oops - it did NOT succeed!
          if (
            exceptionType == null || 
            ex.GetType().Equals(exceptionType) ||
            (allowDerivedExceptions && ex.GetType().IsSubclassOf(exceptionType))
          )
          {
            // Ignore exceptions when exceptionType is not specified OR
            // the exception thrown was of the specified exception type OR
            // the exception thrown is derived from the specified exception type and we allow that
            Thread.Sleep(delayMs);
          }
          else
          {
            // We have an unexpected exception! Re-throw it:
            throw;
          }
        }
      }

      // Try the operation one last time. This may or may not succeed.
      // Exceptions pass unchanged. If this is an expected exception we need to know about it because
      // we're out of retries. If it's unexpected, throwing is the right thing to do anyway
      return operation();
    }
  }
}

And for good measure, here's the VB.NET equivalent:

Option Strict On
Option Explicit On
Option Infer Off

Imports System.Threading

''' <summary>
''' Class Retry. Contains Shared methods to retry operations until they return True, or they do NOT throw
''' exceptions of an expected type.
''' </summary>
Public Class Retry
  ''' <summary>
  ''' Retry the specified operation at most retries times until it returns True.
  ''' </summary>
  ''' <param name="retries">The number of times to retry the operation</param>
  ''' <param name="delayMs">The number of milliseconds to sleep after a failed invocation of the operation</param>
  ''' <param name="operation">The operation to perform. Should return True</param>
  ''' <returns>True if the action returned True on one of the retries, False if the number of retries was exhausted</returns>
  Public Shared Function UntilTrue(retries As Integer, delayMs As Integer, operation As Func(Of Boolean)) As Boolean
    For retry As Integer = 0 To retries - 1
      If (operation()) Then
        Return True
      End If
      Thread.Sleep(delayMs)
    Next
    Return False
  End Function

  ''' <summary>
  ''' Retry the specified operation the specified number of times, until there are no more retries or it succeeded
  ''' without an exception
  ''' </summary>
  ''' <typeparam name="T">The return type of the operation</typeparam>
  ''' <param name="retries">The number of times to retry the operation</param>
  ''' <param name="delayMs">The number of milliseconds to sleep after a failed invocation of the operation</param>
  ''' <param name="operation">The operation to perform</param>
  ''' <param name="expectedExceptionType">If not null, ignore any exceptions of this type (and possibly subtypes, see <paramref name="allowDerivedExceptions"/></param>
  ''' <param name="allowDerivedExceptions">If true, exceptions deriving from the specified exception type are ignored as well. Defaults to False</param>
  ''' <returns>When one of the retries succeeds, return the value the operation returned. If not, an exception is thrown.</returns>
  Public Shared Function WhileException(Of T)(
    retries As Integer,
    delayMs As Integer,
    operation As Func(Of T),
    Optional expectedExceptionType As Type = Nothing,
    Optional allowDerivedExceptions As Boolean = False
  ) As T

    '  Do all but one retries in the loop
    For retry As Integer = 1 To retries - 1
      Try
        '  Try the operation. If it succeeds, return its result
        Return operation()

      Catch ex As Exception
        '  Oops - it did not succeed!
        If (
          expectedExceptionType Is Nothing OrElse
          ex.GetType().Equals(expectedExceptionType) OrElse
          (allowDerivedExceptions AndAlso ex.GetType().IsSubclassOf(expectedExceptionType))
        ) Then
          ' Ignore exceptions when expectedExceptionType is not specified or
          ' the exception thrown was of the specified exception type (or a derived one if allowed)
          Thread.Sleep(delayMs)
        Else
          ' We have an unexpected exception! Re-throw it
          Throw
        End If

      End Try
    Next

    ' Try the operation one last time. This may or may not succeed.
    ' Exceptions pass unchanged. If the exception is expected, we need to know about it because
    ' we're out of retries. If it's unexpected, throwing is the right thing to do anyway
    Return operation()
  End Function
End Class

Happy retrying ;-)

Code

A functional Retry pattern

Functional programming

A Retry pattern

No rocket science

The Code

Code

Tools

Online

Code