Scaling to 10+ Agents

PostgreSQL connection pooling requirements at scale, staggering heartbeat schedules to spread API load, memory growth patterns, cost explosion risks, and the practical upper limits.

Scaling Agent Teams

What you'll learn

Strategies for scaling a Paperclip company from 5 agents to 50, understand the database and API rate limit constraints, and plan for horizontal scaling.

What Scales Automatically

In Paperclip, the following scale without configuration changes:

• Issue volume: PostgreSQL handles millions of rows; FOR UPDATE SKIP LOCKED stays efficient up to hundreds of concurrent claims
• Agent count: more agents = more parallel invocations per tick
• Companies: each company is isolated; adding 100 companies doesn't affect another
• Historical data: heartbeat_runs and cost_events grow indefinitely; archive strategy covers this (pcl-50)

The API Rate Limit Ceiling

The real scaling constraint is LLM API rate limits:

Strategy: distribute agents across multiple API keys / providers:

// Some agents use claude-code, others use openai, others use groq
// Each uses a different API key → different rate limit bucket
{ "agent_1_adapter": "claude-code", "agent_2_adapter": "openai", "agent_3_adapter": "groq" }

Database Scaling

For high-volume production (>10K issues/day):

1. Add indexes:

CREATE INDEX CONCURRENTLY idx_issues_status_company
  ON issues (company_id, status, priority, created_at)
  WHERE status = 'open';

CREATE INDEX CONCURRENTLY idx_cost_events_month
  ON cost_events (company_id, date_trunc('month', created_at));

1. Partition cost_events by month (once >10M rows):

-- Range partitioning
CREATE TABLE cost_events_2024_01 PARTITION OF cost_events
FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');

1. Read replicas for reporting queries — never run analytics against the primary.

Horizontal Scaling the API Server

Run multiple instances of the Paperclip API server:

• Each instance runs its own heartbeat scheduler
• The FOR UPDATE SKIP LOCKED pattern ensures no double-work across instances
• Use a load balancer (Nginx, Traefik) in front

Load Balancer
├── API Server 1 (heartbeat + API requests)
├── API Server 2 (heartbeat + API requests)
└── API Server 3 (heartbeat + API requests)
         ↓
    PostgreSQL (single primary, read replicas)

Scaling Checklist

□ Indexes on (company_id, status) for issues table
□ Indexes on (company_id, created_at) for cost_events
□ Multiple API key providers to spread rate limits
□ Read replica for reporting dashboards
□ Heartbeat interval tuned to task volume (not "every 1 min")
□ max_concurrent_agents set per company
□ Archive strategy for old runs and cost events

Checkpoint ✓

• [ ] You can describe the database indexes needed for high-volume operation
• [ ] You understand how to spread load across multiple LLM API providers
• [ ] You know the horizontal scaling architecture for multiple API server instances