Skip to main content
AI & Innovation

AI Agent Swarms: When More Agents Stop Paying Off

AI agent swarms promise speed, but returns flatten fast. See what Replit's 32-agent numbers and a Google study reveal about when more agents stop paying off.

Nora Whitfield
November 22, 20268 min read
Share
AI Agent Swarms: When More Agents Stop Paying Off
Table of contents

Your manager forwards a vendor demo with one line: "Can we run 32 of these?" On screen, a swarm of AI agents splits a coding job into pieces, works on all of them at once, and finishes in an afternoon. The pitch is simple. More agents, more speed.

AI agent swarms went from a research curiosity to a product category this month. Replit launched Missions on November 17, a tool that sends teams of agents after a goal you set, inside a budget you cap. The demos are real. The question nobody puts on the slide is where the extra agents stop earning their cost.

The numbers to answer it now exist, partly from the vendors themselves. Here is how to read them before you sign off on a bigger swarm.

What AI Agent Swarms Actually Are

A single AI agent is a large language model that can use tools, run code and check its own results in a loop. A swarm is many of those running at once. Usually one agent acts as the planner. It breaks the job into tasks, hands them out, and merges what comes back.

Replit describes Missions in these terms. You set a goal, define what success looks like, set a spending cap and choose which actions need your approval. An independent AI judge checks the result against your criteria. If the budget runs out first, the work simply stops, unresolved.

Why the idea is catching on now

Three things changed. Models got good enough to finish multi-hour tasks alone. Coding tools added ways to run agents side by side in separate workspaces. And companies under pressure to show returns on AI want something that looks like a team, not a chatbot. If you are still weighing the basics, our explainer on why businesses are racing to deploy AI agents covers the wider push.

The Scaling Curve in Replit's Own Numbers

Replit published a benchmark alongside the launch. It asked agents to speed up 13 tasks in NumPy, a widely used Python library for number crunching. It reported results for two settings, which it calls Auto (Max) and Auto (Power).

  • 1 agent: 4x faster code in Max mode, 2x in Power mode.
  • 16 agents: 17x in Max, 7x in Power.
  • 32 agents: 19x in Max, 9x in Power.

Read the first jump and the swarm looks magical. Read the second and the picture changes.

A worked example: what each extra agent buys

Take the Max numbers and divide the gain by the agents added.

  1. From 1 to 16 agents: speed rises from 4x to 17x. That is 13 extra "x" for 15 extra agents, or about 0.87x per agent.
  2. From 16 to 32 agents: speed rises from 17x to 19x. That is 2 extra "x" for 16 extra agents, or about 0.125x per agent.

Each agent in the second batch is worth roughly one seventh of an agent in the first. If cost grows with the number of agents, doubling from 16 to 32 roughly doubles the bill for about 12% more speed. The analysis site FourWeekMBA ran the same arithmetic and reached the same 12% figure.

None of this means Replit's product is bad. A 17x speedup is impressive. It means the sweet spot sits well below the biggest number on the slide, and the vendor's own chart shows it.

Why Returns Flatten: Coordination Costs

The best independent evidence comes from a paper by researchers at Google Research and MIT, posted to arXiv in December 2025. They tested 180 agent configurations across five designs, three model families and four benchmarks. Their findings explain the flattening curve.

  • Sequential work gets worse. On tasks where each step depends on the last, every multi-agent design cut performance by 39% to 70% compared with a single agent.
  • Parallel work gets better. On tasks that split into independent pieces, a centrally coordinated team improved results by 80.8%.
  • Errors compound. Agents working side by side with no coordinator amplified mistakes 17.2 times. A central coordinator held that to 4.4 times.
  • Strong models gain less. Once a single agent already succeeded around 45% of the time, adding more agents gave shrinking or negative returns.

Think of it like a kitchen. Ten cooks can prep ten salads at once. Ten cooks cannot make one sauce faster, because each step waits on the one before, and every extra voice adds a chance to oversalt it.

Even builders of these tools are cautious

The skepticism is not only academic. The Decoder reported that Eric Provencher, a developer on OpenAI's Codex team, argued that running more than two parallel sub-agents usually burns tokens without improving quality. That is one engineer's view, posted on X, not an OpenAI policy. Still, it matches what the Google study measured.

Where AI Agent Swarms Do Pay Off

Swarms earn their keep when the job breaks into many pieces that rarely touch. Large test suites, codebase migrations, and checking hundreds of documents against the same rule all fit that shape.

The clearest public case is Anthropic's compiler experiment from February. Researcher Nicholas Carlini set 16 Claude agents to write a C compiler in the Rust language, from scratch, capable of building the Linux kernel. According to Anthropic's engineering write-up, and to independent reporting by The Register and InfoQ, the agents ran nearly 2,000 sessions over about two weeks and cost about $20,000 in API fees.

The unit economics behind that bill

The result was a compiler of roughly 100,000 lines that could build a bootable Linux kernel on three chip architectures. Divide it out:

  • Per session: $20,000 across 2,000 sessions is about $10 each.
  • Per line of code: $20,000 across 100,000 lines is about 20 cents a line.

That is cheap next to human engineering time, but it came with heavy human design work. The agents stalled on the Linux kernel until the team built a harness that used GCC, a mature existing compiler, as a known-good reference to isolate failures. The swarm worked because people shaped the problem into parallel pieces, not because 16 agents figured that out alone.

A Budget Test Before You Add More Agents

Most work inside a company is more sauce than salad. Before you scale a swarm, run it through these five questions.

  1. Can the job split into pieces that do not depend on each other? If step two needs step one's answer, start with one strong agent.
  2. Is there an automatic way to check each piece? Tests, a reference output or a clear rubric. Without one, errors multiply instead of cancelling out.
  3. Does a single agent already succeed often? If it clears the task most of the time, extra agents may add cost and noise rather than quality.
  4. Where does your own curve bend? Run the task with 1, 4, 8 and 16 agents. Stop adding agents when the last doubling buys less than a quarter of the gain the first one did.
  5. Is there a hard spending cap? Replit's own demo app, a monitoring tool called DataCat, was built in two days on a $3,000 budget. Set yours before the first run, not after the invoice.

Skipping these checks is a common route into the pattern described in why most AI pilots never pay off: a dazzling demo, a scaled rollout, and a cost line nobody can tie to an outcome.

Frequently Asked Questions

What is an AI agent swarm?

It is a group of AI agents working on one goal at the same time. A planner agent usually splits the job, other agents handle the pieces in parallel, and the results are merged and checked.

Are multi-agent systems better than a single agent?

Only for some tasks. A Google Research and MIT study found multi-agent teams improved parallel tasks by up to 80.8% but cut performance on step-by-step reasoning tasks by 39% to 70%.

How many AI agents should I run in parallel?

Fewer than vendors suggest, for most work. Test your own task at a few team sizes and stop when doubling the agents adds little. In Replit's benchmark, going from 16 to 32 agents added only about 12% more speed.

How much does it cost to run AI coding agents?

It depends on the model and the length of the job. Anthropic's 16-agent compiler build cost about $20,000 across nearly 2,000 sessions, roughly $10 per session.

The Smart Swarm Is Usually the Smaller One

AI agent swarms are real, and on the right job they are fast. Replit's 17x speedup and Anthropic's compiler both show it. But the same data shows a curve that bends early, and an independent study shows teams of agents making step-by-step work worse.

So when someone asks for 32 agents, ask a different question back. Does this task split cleanly, and can a machine check each piece? If yes, find the point where your curve flattens and stop there. If no, one good agent and a human reviewer will likely beat the swarm on both cost and quality.

Share

Written by

Nora Whitfield

Technology & Science

Covers technology and science for Quick Trend Insights, with a focus on what new AI tools, consumer hardware and space missions actually change for the people following them.

More articles

Related Articles

View all