Parallel Work Distribution

Seeding N identical task structures with different inputs so N agents process them in parallel during the same heartbeat window. Load balancing and result collection patterns.

Parallel Work: Running Agents Simultaneously

What you'll learn

Design tasks that run in parallel, understand the practical concurrency limits, and build a parallel research pattern.

Parallelism in Paperclip

Within a single heartbeat tick, Paperclip invokes all active agents simultaneously (up to max_concurrent_agents). This means parallel execution is free — you don't need special configuration. Simply assign work to multiple agents, and the heartbeat runs them in parallel.

The Parallel Research Pattern

Goal: research 10 topics simultaneously, then merge results.

DO $$
DECLARE
  cid      uuid;
  parent   uuid;
  res_id   uuid;
  topics   text[] := ARRAY[
    'LangGraph', 'CrewAI', 'AutoGen', 'LlamaIndex', 'Haystack',
    'LangSmith', 'Weights & Biases', 'Helicone', 'PromptLayer', 'Arize'
  ];
  t        text;
BEGIN
  SELECT id INTO cid FROM companies WHERE slug = 'research-co';
  SELECT id INTO res_id FROM agents WHERE name = 'Researcher' AND company_id = cid;

  -- Create a parent "meta-research" issue
  INSERT INTO issues (company_id, title, status)
  VALUES (cid, 'AI Tooling Landscape Research', 'in_progress')
  RETURNING id INTO parent;

  -- Create one subtask per topic
  FOREACH t IN ARRAY topics LOOP
    INSERT INTO issues (company_id, title, description, assigned_agent_id, parent_issue_id, status)
    VALUES (
      cid,
      'Research: ' || t,
      'Produce a 2-paragraph summary of ' || t || ' covering: purpose, key features, typical use case, and licensing.',
      res_id,
      parent,
      'open'
    );
  END LOOP;
END $$;

With max_concurrent_agents = 10, all 10 research tasks run in the same heartbeat tick (~30 seconds vs 300 seconds sequential).

Parallel with Different Agents

Assign different topics to different specialist agents for true parallel specialisation:

-- Assign API-focused topics to API Specialist, OSS to OSS Specialist
INSERT INTO issues (company_id, title, assigned_agent_id, status)
SELECT
  'cid',
  'Research: ' || t.topic,
  CASE
    WHEN t.category = 'api_service' THEN (SELECT id FROM agents WHERE name = 'API Specialist')
    ELSE                                  (SELECT id FROM agents WHERE name = 'OSS Specialist')
  END,
  'open'
FROM (VALUES
  ('LangSmith',       'api_service'),
  ('Helicone',        'api_service'),
  ('LangGraph',       'oss'),
  ('CrewAI',          'oss')
) AS t(topic, category);

Monitoring Parallel Progress

-- See parallel subtask completion in real time
SELECT
  i.title,
  i.status,
  a.name  AS agent,
  i.resolved_at
FROM issues i
JOIN issues parent ON parent.id = i.parent_issue_id
LEFT JOIN agents a ON a.id = i.assigned_agent_id
WHERE parent.title = 'AI Tooling Landscape Research'
ORDER BY i.status, i.created_at;

Rate Limit Considerations

With 10 parallel Claude invocations, you may hit API rate limits. Options:
- Use different models for different agents (separate rate limit buckets)
- Set max_concurrent_agents = 5 and let the remaining 5 run in the next tick
- Use the http adapter pointing at a self-hosted model for unlimited parallelism

Checkpoint ✓

• [ ] You can create 10 parallel subtasks that run in the same heartbeat tick
• [ ] You understand that parallel execution is automatic, not special configuration
• [ ] You know the rate limit issue and how to work around it