Debugging Failed Heartbeat Runs

Reading run error logs in the dashboard, replaying a failed run via the API, using the --dry-run flag, and a reference table of common failure signatures per adapter type.

Debugging Heartbeat Runs

What you'll learn

Systematic approach to diagnosing heartbeat failures, reading error details from the database, and reproducing adapter errors locally.

The Debugging Checklist

When a heartbeat run fails or an issue stays stuck in 'open':

1. Check heartbeat_runs.status and error_detail
2. Check issues.attempt_count vs max_attempts
3. Check agent status (active / paused)
4. Check company paused flag
5. Check API key environment variables
6. Reproduce the adapter call locally
7. Check adapter logs (for shell/http adapters)

Step 1 — Read the Run Error

SELECT
  id,
  status,
  error_detail,
  skip_reason,
  started_at,
  finished_at
FROM heartbeat_runs
WHERE company_id = 'your-company-id'
  AND status IN ('failed', 'skipped')
ORDER BY started_at DESC
LIMIT 5;

error_detail is a JSONB array — each element represents one agent failure:

[
  {
    "agent_id": "...",
    "agent_name": "Researcher",
    "issue_id": "...",
    "error": "TimeoutError: adapter invoke exceeded 60000ms",
    "attempt": 2
  }
]

Step 2 — Check Issue State

SELECT
  id, title, status, attempt_count, max_attempts
FROM issues
WHERE company_id = 'your-company-id'
  AND status IN ('open', 'escalated')
ORDER BY attempt_count DESC;

Issues with attempt_count = max_attempts and status = 'escalated' have given up and need human review.

Step 3 — Reproduce the Adapter Call

For claude-code:

# Get the system prompt and a failing issue's description
SYSTEM=$(psql $DATABASE_URL -t -c "SELECT system_prompt FROM agents WHERE name='Researcher' LIMIT 1")
TASK=$(psql $DATABASE_URL -t -c "SELECT description FROM issues WHERE id='failing-issue-id'")

# Invoke directly
echo "${SYSTEM}\n\n${TASK}" | claude --model claude-opus-4-5 --output-format json

For http adapter:

curl -X POST http://your-endpoint/api/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "test prompt", "model": "llama3"}'

Step 4 — Read the Server Logs

In development, the API server logs each heartbeat tick:

[HEARTBEAT] company=acme tick=started
[HEARTBEAT] agent=Researcher issue=pcl-001 status=claimed
[HEARTBEAT] agent=Researcher adapter=invoke started
[ADAPTER:claude-code] spawn pid=12345
[ADAPTER:claude-code] exit code=1 stderr="ENOENT: claude not found"
[HEARTBEAT] agent=Researcher issue=pcl-001 status=failed attempt=1/3
[HEARTBEAT] company=acme tick=completed agents=1 resolved=0 failed=1

Common Errors and Fixes

Checkpoint ✓

• [ ] You can query heartbeat_runs.error_detail and interpret the JSON
• [ ] You can reproduce a failing claude-code adapter call in the terminal
• [ ] You know the five most common errors and their fixes