Skip to content

Pattern 10: Error Handling

Retry for TCP request-response

For request-reply adapters built on TcpProtocolBase/ProtocolChannel, prefer SendOptions.RetryPolicy and keep retry behavior at the channel level.

Avoid wrapping Send(...) in custom retry loops. That duplicates queue/timeout behavior and can produce hard-to-debug retry storms.

var response = await Send(request, new SendOptions
{
    Timeout = TimeSpan.FromSeconds(2),
    RetryPolicy = RetryPolicy.Exponential(
        maxAttempts: 4,
        initialDelay: TimeSpan.FromMilliseconds(100))
}, ct);

Choose policy by failure mode:

  • RetryPolicy.None for strict single-attempt commands
  • RetryPolicy.Fixed(...) for stable but occasionally busy links
  • RetryPolicy.Exponential(...) for bursty load or temporary device pressure
  • new RetryPolicy(maxAttempts, attempt => ...) for protocol-specific delay rules

Circuit Breaker

internal sealed class CircuitBreaker
{
    private int _failureCount;
    private DateTimeOffset _lastFailureTime;
    private readonly TimeProvider _timeProvider;
    private readonly int _threshold = 5;
    private readonly TimeSpan _resetTimeout = TimeSpan.FromMinutes(1);

    public CircuitBreaker(TimeProvider timeProvider)
    {
        _timeProvider = timeProvider;
    }

    public async Task<T> Execute<T>(Func<Task<T>> operation)
    {
        if (_failureCount >= _threshold)
        {
            if (_timeProvider.GetUtcNow() - _lastFailureTime < _resetTimeout)
                throw new InvalidOperationException("Circuit breaker open");

            // try reset
            _failureCount = 0;
        }

        try
        {
            var result = await operation();
            _failureCount = 0; // success - reset counter
            return result;
        }
        catch
        {
            _failureCount++;
            _lastFailureTime = _timeProvider.GetUtcNow();
            throw;
        }
    }
}

Graceful Degradation

Handle partial failures without stopping the adapter:

protected override async Task OnEventReceived(DeviceEvent evt)
{
    try
    {
        var door = FindDoor(evt.DoorId);
        if (door == null)
        {
            _logger.LogWarning("Door {DoorId} not found for event {Code}",
                evt.DoorId, evt.Code);
            return; // continue processing other events
        }

        await ProcessEvent(door, evt);
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Failed to process event {Code} for door {DoorId}",
            evt.Code, evt.DoorId);
        // don't rethrow - continue processing
    }
}

Best Practices

  1. Log at appropriate levels:
  2. Trace: Detailed flow
  3. Debug: Diagnostic info
  4. Info: Normal operations
  5. Warning: Recoverable issues
  6. Error: Failed operations

  7. Critical: System failures

  8. Preserve context: 

    catch (Exception ex)
{
    _logger.LogError(ex, "Operation failed for device {DeviceId}, door {DoorId}",
        deviceId, doorId);
    throw; // or handle
}

  9. Use structured logging: 

    _logger.LogInformation("Door {DoorId} opened by {PersonId}",
    door.Address, personId); // not string interpolation

  10. Timeout long operations: 

    using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30));
await operation(cts.Token);

See Also