Timeouts, Retries & Backoff

Per-adapter timeout configuration, exponential backoff on transient failures, maximum retry count, and dead-letter handling for permanently failed runs.

Timeout and Retry Configuration

What you'll learn

Configure adapter timeouts, understand the automatic retry mechanism, and decide when to escalate vs retry a failed issue.

Adapter Timeout

Each adapter accepts a timeout_ms config. If the adapter call takes longer than this, it is killed and the issue is marked failed:

{
  "adapter_config": {
    "timeout_ms": 45000   // 45 seconds
  }
}

Defaults by adapter:

What Happens on Timeout

1. Adapter throws a TimeoutError
2. Issue status set back to 'open', attempt_count++
3. Error recorded in heartbeat_runs.error_detail
4. Next tick will retry (if attempt_count < max_attempts)

The max_attempts Field

Per issue, default 3:

-- More forgiving for flaky external services
UPDATE issues SET max_attempts = 5 WHERE id = 'issue-id';

-- One-shot only (no retry on failure)
UPDATE issues SET max_attempts = 1 WHERE id = 'issue-id';

When attempt_count >= max_attempts on a failure, the issue moves to 'escalated' instead of back to 'open'.

Global Retry Config

Set a company-level default for max_attempts:

{
  "default_max_attempts": 3,
  "retry_delay_ms": 0    // 0 = retry immediately next tick
}

A non-zero retry_delay_ms requires the heartbeat to wait before retrying — useful with a 1-minute heartbeat to avoid hammering a flaky API.

Implementing Exponential Back-off

For HTTP adapters calling external services, implement back-off in the adapter:

const delays = [1000, 5000, 15000]; // ms

for (let i = 0; i < 3; i++) {
  try {
    return await callExternalService(task);
  } catch (err) {
    if (i === 2) throw err;
    await sleep(delays[i]);
  }
}

Keep total retry time under timeout_ms or the outer timeout will fire.

When to Escalate Instead of Retry

Set max_attempts = 1 for tasks where retrying is pointless:
- Deterministic transformations (a CSV format error won't self-heal)
- Uniqueness-constrained writes (retrying will just fail again)
- Human-in-the-loop tasks (always route to approval)

Checkpoint ✓

• [ ] You can set per-adapter timeout and explain what happens on expiry
• [ ] You understand max_attempts and when escalation replaces retry
• [ ] You know how to implement back-off inside an HTTP adapter