Claude Code vs Codex vs Devin (2026)

The founder's field guide to the three AI coding agents everyone is arguing about, and the ten others you should know before you pick one.

By mid-2026, one product alone (Claude Code) crossed a $2.5 billion annual run-rate, and a rival agent (OpenAI Codex) passed 5 million weekly users, roughly a fifth of them not even developers - The Next Web. In under two years, "AI that writes a suggestion" turned into "AI that opens the pull request, runs the tests, and pings you on Slack when it is done." The three names at the center of every engineering-team debate are Claude Code from Anthropic, Codex from OpenAI, and Devin from Cognition.

But here is the problem most comparison posts miss: these three are not the same kind of thing. One is a terminal companion you supervise, one is a review-gated agent that drafts pull requests, and one is a fully autonomous "software engineer" you assign work to and walk away from. Picking by benchmark score is the wrong instinct, because in 2026 they all cluster near the top of every saturated leaderboard. The real decision is about autonomy, control, and what the meter does when you run it hard.

This guide breaks down exactly what each tool is in 2026, the current models powering them, real pricing (including the runaway-bill stories), the benchmark truth (including the one independent study that found AI made experienced developers slower), and ten more players worth knowing. It is written for founders and builders, not compiler engineers, so the language stays plain even when the topic gets deep. If you are a non-technical founder who wants the outcome and not the diff, there is a category for you too, and we treat it as one option among many.

Contents

1. From autocomplete to autonomous agents: what actually changed
2. The master comparison: eleven tools, scored
3. Claude Code in depth
4. OpenAI Codex in depth
5. Devin in depth
6. Head to head: how the big three actually differ
7. Benchmarks and the uncomfortable truth about them
8. Pricing and the real cost of running an agent
9. The wider field: every other player that matters
10. Where each tool wins and where each one fails
11. Security and the operational risks nobody budgets for
12. How founders should actually choose, and the road ahead

The master comparison: eleven tools, scored

Before the deep dives, here is the whole field in one place. This table scores eleven AI coding tools (the three principals plus the eight contenders that founders keep asking about) against the five criteria that actually decide the purchase. Each cell carries the score and the reason for it, because a bare number is worthless. The table is sorted by final score, highest first, and the category column exists so you can still see that a browser-based app builder and a terminal-native agent are different animals even when their scores are close.

The five criteria, and why they carry the weight they do, appear right below the table. The short version: autonomy and reliability matter most because they define how much real work you get and how much of it you have to redo, while cost, accessibility, and ecosystem decide whether the tool fits your budget, your skill level, and your existing stack.

How to read the scores. Each tool was rated 0 to 10 on five axes, and the final is the weighted average. Autonomy (25%) measures how much real work the tool can complete without a human in the loop, from a single edit to a multi-hour session. Reliability and control (25%) is the balance between output quality and your ability to steer, verify, and roll back, because an agent that does a lot but cannot be trusted or corrected is a liability, not an asset. Cost and predictability (20%) weighs both the sticker price and how the bill behaves under heavy use, since every tool here has quietly shifted toward usage-based billing. Setup and accessibility (15%) captures how technical you need to be to get value, which matters enormously for founders who are not engineers. Ecosystem (15%) rewards breadth of surfaces and integrations, because a coding agent that lives everywhere your team already works compounds in value.

Notice what the ranking is not. It is not a smartness leaderboard. Devin is the most autonomous tool in the table (a 9) yet lands eighth, precisely because its autonomy comes at the cost of predictability and control for the typical buyer. That is the whole thesis of this guide in one row: in 2026, more autonomy is not automatically better, and the winner for you depends on how much of the work you are able and willing to verify.

1. From autocomplete to autonomous agents: what actually changed

To choose well, you have to understand what shifted between 2023 and 2026, and the shift is bigger than "the models got smarter." The first generation of AI coding help was autocomplete: you typed, and a model finished the line. That was GitHub Copilot's original 2021 pitch, and it kept the human firmly in the driver's seat. The second generation was chat: you asked a question in a side panel and pasted the answer back. Useful, but still a conversation, not a colleague. The third generation, the one that defines 2026, is agentic: the tool reads your files, runs commands, edits across many files, executes the tests, and iterates toward a goal while you watch, redirect, or step away entirely. Anthropic describes Claude Code in exactly those terms, as an environment that can "autonomously work through problems while you watch, redirect, or step away" - Anthropic.

The structural change underneath is that intelligence became cheap and long-horizon. When a model can only reliably do a two-minute task, you keep it on a tight leash. When it can run a coherent multi-hour session, you start handing it whole tickets. METR, an independent evaluation lab, quantified this trajectory: the length of task an AI can complete with 50% reliability has been doubling roughly every seven months, climbing from about four seconds of equivalent human work in 2019 to more than sixteen hours by 2026, with the 2024-to-2025 stretch accelerating to a four-month doubling - METR. That single curve is why "AI software engineer" stopped sounding absurd. It is not that any model became a senior engineer. It is that the horizon over which a model stays coherent got long enough to matter.

The consequence for buyers is that the interesting axis is no longer capability, it is where the human sits on the spectrum. As agents take over more of the typing, the person's job migrates from writing code to reviewing it, and eventually to orchestrating several agents at once. This reframing is the single most useful lens for the entire market, and it is worth seeing visually before we go tool by tool.

Adoption confirms the field went mainstream, even as trust did not keep pace. The 2025 DORA report from Google found that 90% of surveyed technology professionals now use AI at work, up roughly fourteen points year over year, and that over 80% believe it lifted their productivity - Google Cloud. Yet the same report found a negative relationship between AI adoption and software delivery stability: teams shipped faster and broke more, because AI amplifies whatever discipline (or lack of it) a team already had. The Stack Overflow 2025 survey, with more than 49,000 responses, is even blunter on trust: only about a third of developers trust AI accuracy, favorability slid from over 70% to under 60%, and 66% named "solutions that are almost right, but not quite" as their top frustration - Stack Overflow.

The executives running the largest engineering organizations have put hard numbers on the shift, and they are big enough to reframe the stakes. Google's CEO said on a late-2024 earnings call that more than a quarter of all new code at Google is AI-generated and then reviewed by engineers - Entrepreneur. Microsoft's CEO put the share of AI-written code in its own repositories at roughly 20 to 30% in early 2025, and Anthropic's CEO went further, predicting AI would write the large majority of code within months. Analysts size the AI code tools market at somewhere between roughly $7 billion and $10 billion in 2025 depending on definition, compounding near 27% a year - Fortune Business Insights. Treat those dollar figures as directional estimates that disagree by billions, but the direction is not in doubt: Gartner projects 75% of enterprise software engineers will use AI code assistants by 2028, up from under 10% in early 2023 - Gartner.

That gap between eager adoption and shaky trust is the emotional center of the 2026 market, and it explains why the "which agent" question is really a "how much do I trust it to run unsupervised" question. Founders coming from the no-code world will recognize the pattern from our own writing on building software with AI: the tooling raced ahead of the guardrails, and the winners are the teams that added the discipline back. Keep that tension in mind as we profile each tool, because every strength below has a matching failure mode that only shows up when you stop watching.

2. Claude Code in depth

Claude Code is Anthropic's agentic coding tool, and in 2026 it is the commercial pace-setter of the category. It began in May 2025 as a terminal-first command-line program and has since spread across surfaces while keeping the same engine underneath. Anthropic's own framing is that it "reads your codebase, edits files, runs commands, and integrates with your development tools," available "in your terminal, IDE, desktop app, and browser," with your project settings, memory files, and connected tools working identically across all of them - Anthropic. The reason it matters commercially is scale: Anthropic disclosed that Claude Code reached $1 billion in run-rate revenue within six months of general availability, naming Netflix, Spotify, KPMG, L'Oreal, and Salesforce among its users - Anthropic. Reuters later reported the figure had climbed past $2.5 billion by February 2026.

The surfaces are the widest in the category. You can run Claude Code as a terminal CLI (the full-featured original), a VS Code or JetBrains extension with inline diffs, a desktop app that runs several sessions side by side and schedules recurring tasks, and a web version at claude.ai/code for long-running or parallel work with no local setup. It also reaches into GitHub Actions and GitLab CI, posts automatic pull-request reviews, answers a bug report tagged in Slack with a PR, and can be driven from the Claude mobile app on iOS and Android - Anthropic. For teams that want to build on top of it rather than just use it, the Agent SDK exposes Claude Code's tools and permission model so you can wire custom agents, which is exactly the kind of infrastructure a product company builds its own workflows on.

The feature set is where Claude Code earns its reliability score, because these are the levers that let you keep an autonomous agent honest. The most important ones are worth stating plainly before we interpret them.

• Subagents and background agents spawn multiple coordinated sessions, each with its own context window, so a lead agent can fan out work and merge results
• MCP (Model Context Protocol) connects Claude Code to external data and tools like Jira, Slack, databases, and browsers through one open standard
• Hooks run deterministic shell commands before or after actions, so you can auto-format after every edit or block a risky command as a security checkpoint
• Plan mode forces a read-only analysis and a written plan before any file is touched, which you can edit before approving
• Sandboxing and checkpoints isolate command execution at the OS level and snapshot state so you can roll back a bad change

The through-line of that list is control, and it maps directly to Anthropic's own number-one best practice: "Give Claude a way to verify its work," because "Claude stops when the work looks done" and, without a check it can run, "looks done" is the only signal it has - Anthropic. This is the honest weakness of the tool stated by its own maker. Anthropic names the failure pattern "the trust-then-verify gap," where Claude produces a plausible implementation that quietly mishandles an edge case, and the prescribed fix is to always attach a test, a linter, a build, or a screenshot comparison. A founder who ignores that advice will ship confident-looking bugs, which is the recurring theme of our guide to building a live app with Claude Code.

On models, Claude Code runs on Anthropic's current Claude 5 generation, and the default is plan-dependent rather than fixed. Claude Sonnet 5 became the default in mid-2026 for most users, described by Anthropic as the best balance of speed and intelligence - Anthropic. After Claude Opus 5 launched on July 24, 2026, the Opus alias began resolving to Opus 5 on Claude Max, Team Premium, and Enterprise, while Pro and Team Standard defaulted to Sonnet 5 - Anthropic. Above both sits the Mythos-class Claude Fable 5, the most capable widely released Anthropic model, with an invitation-only sibling, Claude Mythos 5, reserved for defensive cybersecurity work. On the independent Terminal-Bench 2.1 leaderboard, which grades real command-line agent tasks, the Claude Code plus Fable 5 pairing leads at 83.8%, narrowly ahead of the top Codex configuration - Terminal-Bench. If you want the deeper capability picture, our breakdown of the best AI model to build your app argues that the harness usually matters more than the model, and Claude Code is a strong illustration of why.

The best way to understand the day-to-day feel is to watch Anthropic engineers actually use it rather than read a keynote, and their own workflow video is the most current official demo.

Pricing is the part founders should study hardest, because Claude Code's economics are a genuine advantage with a genuine trap. Access is bundled into every paid Claude plan at no extra per-use charge: Pro at $20 a month, Max at $100 and $200 a month for five and twenty times the usage, and Team and Enterprise seats on top - Anthropic. For sustained coding, the subscription is dramatically cheaper than paying per token, because heavy sessions are dominated by cache reads and a Max plan folds those into a flat fee. The trap is the API path, which has no usage wall: you keep paying per token until you stop it, which is the root of the runaway-bill stories we cover in the pricing section. Anthropic's own 2025 rate-limit episode, where it imposed weekly caps to curb users running Claude Code "continuously around the clock," is the clearest sign that even the vendor found the meter hard to manage - Winbuzzer.

Here is a look at Claude Code running in its native terminal, which remains where its most advanced features live.

The verdict on Claude Code for a founder is that it is the most capable and best-instrumented agent for real, complex codebases, provided you can live near a terminal and you set up verification. It rewards discipline and punishes the "let it run and hope" approach. If your instinct is to skip the terminal entirely, the tools later in this guide are a better fit, but if you want the deepest control over an autonomous coder, this is the one to beat.

3. OpenAI Codex in depth

Codex in 2026 is not the deprecated 2021 code model that shared the name, and getting that straight is the first step to understanding it. The modern Codex is OpenAI's agentic software-engineering system, relaunched in 2025 as an open-source terminal agent and a cloud agent, then consolidated into a family that spans CLI, cloud, IDE, ChatGPT, GitHub, Slack, and mobile. By mid-2026 OpenAI positions it as an "agent for work" rather than a pure coding tool, a framing borne out by adoption: Codex passed 5 million weekly active users by June 2026, with roughly 20% of them non-developers using it for tasks well beyond code - The Next Web. Those usage numbers are OpenAI-reported, so treat them as directional marketing rather than audited fact, but the growth curve is steep by any measure.

The defining architectural idea of Codex is the cloud task. You hand it a job, and it spins up an isolated, OpenAI-provisioned sandbox preloaded with your repository, where it reads and edits files, runs tests, and returns command logs plus a diff for you to review. Most tasks finish in one to thirty minutes, and they run in parallel, so you can fire off several at once. The reasoning time is dynamic, from seconds for a small edit up to many hours for a hard problem, and the flagship Codex models added "compaction" so a single session can run over millions of tokens across a full day - Simon Willison. This is the "review-gated agent" model: the work happens unattended, but the intended control point is the diff or the pull request, not a live prompt-by-prompt conversation.

Codex's surfaces have consolidated in a way that matters for how you would actually adopt it. The pieces you would touch, and what each is for, break down cleanly.

• Codex CLI is the local terminal agent, an MCP client that can also launch cloud tasks
• Codex Cloud runs the parallel, sandboxed tasks that return diffs and logs
• The IDE extension works in VS Code, Cursor, JetBrains, and Xcode
• GitHub code review lets you comment @codex review on a pull request or enable automatic reviews, and it flags only high-severity issues to stay high-signal
• ChatGPT, Slack, Linear, and mobile let you assign and monitor work from where your team already talks

The practical meaning of that spread is that Codex meets you at the surface you already live in, which is a real edge for teams standardized on ChatGPT and GitHub. Its signature behavior, though, cuts both ways. Codex defaults to "act, don't ask," which practitioners describe as its strength on bulk work and its weakness on scope-sensitive work: it "will install packages without asking, edit adjacent files you didn't mention, and make architectural assumptions to push the task to completion" - nxcode. It is excellent at extending a mature codebase that already has clear patterns, and less reliable when the task is ambiguous or the change is architectural.

The model story is where careful founders should pay attention, because the naming moved. Codex now runs on OpenAI's GPT-5.6 generation, which launched on July 9, 2026 across ChatGPT, Codex, and the API in three tiers named Sol, Terra, and Luna - TechCrunch. The important nuance is that OpenAI folded the old dedicated "-codex" model line into the general family, so the current catalog no longer lists a separate Codex model string, and the flagship GPT-5.6 Sol is what powers agentic coding. If a comparison you read still cites a "GPT-5.3-Codex" as the current engine, it is out of date. On the independent Terminal-Bench 2.1 leaderboard, the top Codex configuration scores 83.1%, effectively tied with Claude Code at the very front of the field - Terminal-Bench.

The most current official product video is OpenAI's launch of the Codex desktop app, which shows the cloud-task and review workflow in context.

Codex's growth as a product is easiest to appreciate as a curve, and the trajectory OpenAI has disclosed over 2026 is genuinely steep, even discounted for self-reporting.

On pricing, Codex is bundled into ChatGPT plans rather than sold separately: Free, Go at $8, Plus at $20, Pro at $100 and $200 for five and twenty times the limits, and Business seats, with API usage billed separately - learn.chatgpt.com. The billing model changed in a way that surprised users: on April 2, 2026, OpenAI moved Codex from per-message pricing to token-based credits, and a heavy agentic session can burn credits far faster than a message count implies - uibakery. The other 2026 wrinkle was limit whiplash: OpenAI temporarily removed the five-hour usage window in July, then reinstated it weeks later, prompting a backlash from users who had briefly enjoyed the unlimited stretch - eesel.

Codex's autonomy also creates a real attack surface that founders should note. In March 2026, a disclosed vulnerability let a maliciously crafted GitHub branch name inject commands during a task's setup and exfiltrate GitHub authentication tokens, a flaw OpenAI subsequently patched - Wikipedia. It is a reminder that the agent's execution environment, not just its code output, is something you are trusting. The wins, on the other hand, are concrete: practitioners describe migrating multiple production coding agents in a single weekend, including a multi-file refactor bot running against a 400,000-line monorepo, which is exactly the bulk, bounded work Codex handles better than any interactive tool - MindStudio. Here is the cloud-task dashboard where that parallel work is queued, monitored, and reviewed.

The takeaway is that Codex is a superb review-gated agent embedded in the tools most teams already use, as long as you respect its eagerness to act and keep the diff review tight. Our founder's guide to Codex goes deeper on what the ChatGPT bundle does and does not include for someone building a company.

4. Devin in depth

Devin is the tool that defined the phrase "AI software engineer," and it remains the purest expression of the autonomous end of the spectrum. Built by Cognition and led by CEO Scott Wu, Devin does not live in your editor. You give it a task, and it works independently in its own cloud environment, a Linux sandbox with a shell, a code editor, and a browser, planning the approach, writing code, running tests, and iterating until it finishes or reports back. It launched publicly in March 2024 as "the first AI software engineer," a claim that generated both enormous attention and immediate skepticism - Cognition. The current product is Devin 2.2, released February 24, 2026, with continuous feature drops layered on top rather than a numbered "3.0," despite third-party posts that claim otherwise - Cognition.

The operating model is worth understanding because it is genuinely different from the other two. Work happens in Devin sessions, each running an isolated cloud VM. Devin first produces an interactive plan, researching the codebase and proposing the files it will touch, then executes. You can run parallel Devins, multiple concurrent sessions each with its own cloud IDE, and since early 2026 you can orchestrate child sessions where one Devin manages several sub-Devins. It indexes your repositories into a Devin Wiki with architecture diagrams and summaries, so new sessions read the index instead of crawling the code cold, and it exposes a searchable, cited codebase Q&A through Devin Search. You assign and monitor all of this from Slack, Linear, Jira, GitHub, or Microsoft Teams, which is why teams describe the Devin workflow as "assign and forget."

Cognition uses Devin to build Devin, and the most credible evidence for its 2026 usefulness is that internal dogfooding number. In one week the company merged 659 Devin-authored pull requests into its own codebase, up from 154 in its best week of 2025, with Devin handling bug fixes, test-coverage improvements, CI-failure investigations, CVE remediation, and design-system enforcement - Cognition. Cognition also built its own fast coding models to power Devin, culminating in SWE-1.7, released July 8, 2026 and served at up to a thousand tokens per second through Cerebras hardware, which is what makes Devin's parallel sessions economical to run at scale - Winbuzzer.

The corporate story around Devin is as dramatic as the product. In July 2025, after OpenAI's roughly $3 billion deal to buy the AI IDE Windsurf collapsed and Google paid $2.4 billion to license its tech and hire its leadership, Cognition acquired the remaining Windsurf team, product, and brand, which brought roughly $82 million in annual recurring revenue and 350-plus enterprise customers - Cognition. Cognition then raised more than $1 billion in May 2026 at a $26 billion post-money valuation, led by Lux Capital, General Catalyst, and 8VC, with enterprise customers including Mercedes-Benz, NASA, Goldman Sachs, and Santander named in the coverage - TechCrunch. The company also reported a $492 million annualized revenue run-rate in that round, a figure that is self-reported and not independently audited, so it belongs in the "impressive if true" column.

The foundational Devin demo is the 2024 launch video, and it belongs here as the historical anchor for what the product set out to be, with the caveat that it drew criticism for staged elements and does not reflect current capability.

Now the honest part, because Devin's autonomy is both its differentiator and its weakness. The most-cited independent evaluation, Answer.AI's month-long hands-on test in January 2025, ran twenty real tasks and logged three successes, fourteen outright failures, and three inconclusive results - Answer.AI. The failures were instructive: spaghetti "integration" code, getting stuck in loops, hallucinating security vulnerabilities in a 700-line repository, and pursuing impossible approaches for hours rather than recognizing a fundamental blocker. The reviewer's verdict cut to the core tradeoff: "Tasks it can do are those so small and well-defined I may as well do them myself, faster, my way." That test was on an earlier version, and the 2026 product is materially better, but the structural point survives: when a fully autonomous agent goes off-track, there is less opportunity to course-correct than with an interactive tool, and it can spend hours (and dollars) heading the wrong way.

Cognition itself is refreshingly clear about the boundaries. Its own guidance says Devin "performs best on tasks with clear, upfront requirements and verifiable outcomes that would take a junior engineer 4 to 8 hours," and that it "can't independently tackle an ambiguous coding project end-to-end like a senior engineer could," and specifically warns that it "usually performs worse when you keep telling it more after it starts the task" - Cognition. Read that as the instruction manual, not the fine print. Devin is a force-multiplier for scoped, verifiable, junior-level work at volume: migrations, CVE fixes, test coverage, small tickets. It is not a drop-in senior engineer, and buying it as one is the fastest way to be disappointed. Here is Cognition's own announcement art for the 2.2 release that reframed it as a team productivity tool rather than a demo.

5. Head to head: how the big three actually differ

With the profiles in hand, the differences between Claude Code, Codex, and Devin resolve into a clean picture, and it is not about which one writes better code in the abstract. On the independent leaderboards they are separated by fractions of a point, so capability is close to a wash at the top. What genuinely differs is the default posture toward the human: how much the tool expects you to be present, and where it hands control back to you. Getting this right is the difference between a tool that fits your working style and one you fight every day.

Claude Code assumes you are present and want the wheel. Its default mode is interactive and terminal-native, built for a developer who watches the agent work, interrupts with Escape when it drifts, and uses plan mode and checkpoints to steer. It scales toward async through cloud sessions and non-interactive runs, but its heart is a supervised pair-programmer that happens to be extremely capable. Codex assumes you want to review, not watch. Its center of gravity is the cloud task that runs unattended and returns a diff, so the human control point is the pull request rather than the live session. Devin assumes you want to delegate. You scope a task, assign it from Slack, and come back to a finished (or failed) result, with the least moment-to-moment involvement of the three. None of these is better in the abstract. They are answers to different questions about how you want to spend your attention.

The verification burden shifts accordingly, and this is the part founders underestimate. Every one of these tools moves work off your plate and onto your review queue, and in 2026 that queue became the actual bottleneck. Industry data is stark: one analysis found teams "generating 98% more pull requests while experiencing a 91% increase in PR review time," with AI-authored PRs waiting far longer before a human picks them up - Codacy. Simon Willison, who runs several agents in parallel, put the ceiling bluntly: "the natural bottleneck on all of this is how fast I can review the results" - Simon Willison. The more autonomous the tool, the larger and less familiar the change it hands you, and the heavier your review load per unit of output. That is the hidden cost that never shows up on a pricing page.

A concrete way to feel the difference is to imagine the same task, "add rate limiting to our API," on each. With Claude Code, you would likely run it interactively, watch it read the middleware, approve its plan, and catch a wrong assumption in real time. With Codex, you would fire it as a cloud task, go do something else, and review the diff it opens, accepting or sending it back. With Devin, you would write a precise ticket in Linear, let it produce a plan and a PR unattended, and review the result knowing you had little visibility into the middle. The first gives you the most control and the most interruption, the last the least of both. Which trade you want depends on how much you trust the agent on this class of task, and how costly a wrong turn would be.

There is also a workflow trend that all three enable and that changes the calculus: running several agents at once. Senior engineers increasingly kick off parallel agents on isolated copies of the repo, treating themselves as an orchestrator of a small team of tireless juniors. Claude Code ships explicit primitives for this (worktrees, a multi-session desktop view, and coordinated agent teams), Codex recommends git worktrees for parallel tasks, and Devin's parallel sessions are native to its design. The catch, per OpenAI's own guidance, is that "multiple agents running without clear deliverables produce overlapping, conflicting changes," so parallelism multiplies output only if you scope each agent tightly - getmaxim. The practical ceiling most teams report is a handful of concurrent agents per person before review, not the model, becomes the wall.

6. Benchmarks and the uncomfortable truth about them

Every vendor will wave a benchmark at you, and a founder who takes those numbers at face value will make a bad decision. The most famous coding benchmark, SWE-bench Verified, a set of 500 real GitHub issues, has become nearly useless for separating top models because it is both saturated and contaminated. Scores now cluster in the high 80s and 90s, and OpenAI publicly stopped reporting it in early 2026, explaining in a post titled "Why we no longer evaluate SWE-bench Verified" that a large share of the audited problem tasks were effectively unsolvable as written, and that frontier models could reproduce the gold-patch solution from the task ID alone, a fingerprint of training-data leakage - Tessl. An independent academic analysis put the contamination at roughly a third of successful patches showing solution leakage. When the leading lab abandons the industry-standard benchmark for being broken, that is the story, not the score.

The healthier signal comes from harder, contamination-resistant benchmarks, and here the picture is more honest and more humbling. SWE-bench Pro, built by Scale AI across 41 actively maintained repositories with a held-out private set, shows top scores around 61% on its independent public leaderboard, not the 80s and 90s the vendors report on their own slices - Scale AI. The gap between the two benchmarks for the same model is the single most clarifying data point in this entire field, and it is worth seeing directly.

For a "which agent" decision specifically, the most relevant leaderboard is Terminal-Bench, from Stanford and the Laude Institute, because it grades agent-and-model pairs doing real command-line work in isolated sandboxes, judged by the end state rather than the transcript. On the 2.1 board, the results are astonishingly close at the top: Claude Code paired with Fable 5 leads at 83.8%, Codex paired with GPT-5.5 sits at 83.1%, and Cursor's CLI, Terminus, and various other harnesses fill out a tightly bunched top ten - Terminal-Bench. The lesson is not "Claude Code wins by 0.7 points." The lesson is that the harness matters as much as the model, and that the two leading agents are within a rounding error of each other on the most realistic public test.

Two more benchmarks add useful color before the sobering one. The Aider polyglot leaderboard, which tests real code editing across six languages, is valuable less for its accuracy numbers than for its cost column: it showed one top model costing five to fourteen times more than another for a marginal accuracy edge, a direct reminder that the expensive model is rarely the rational default - Aider. OpenAI's own SWE-Lancer benchmark asked a blunter question, whether models can earn real freelance money, by testing them on more than 1,400 actual Upwork tasks worth a million dollars in total payouts. The verdict in the original run was humbling, with even the best model claiming only a fraction of the available money and the authors concluding that frontier models "are still unable to solve the majority of tasks" - arXiv. Tellingly, Cognition stopped reporting SWE-bench numbers for Devin back in 2024, arguing that benchmark performance often does not represent the real experience of using an agent, which is itself an admission that the leaderboard race and real usefulness have drifted apart.

Now the study that every buyer should know and almost no vendor will cite. METR, an independent lab, ran a randomized controlled trial with sixteen experienced open-source developers on 246 real tasks in their own mature repositories in early 2025. The result was the opposite of the marketing: when allowed to use AI tools, the developers took 19% longer to complete their issues. Worse, the perception gap was enormous. The same developers forecast beforehand that AI would make them 24% faster, and even after experiencing the slowdown they still believed it had sped them up by about 20% - METR. This is the most important reliability data in the field, because it shows that self-reported productivity gains are systematically unreliable.

The honest reading of METR requires nuance, and METR itself provided it. A February 2026 follow-up found the original result was partly a product of selection effects, and that a broader sample landed closer to neutral rather than a clear slowdown - METR. So the correct takeaway is not "AI makes everyone slower." It is that the uplift on large, familiar codebases is much smaller and noisier than the hype, and that AI helps most on greenfield and unfamiliar work and least on the mature systems where you already know your way around. That maps perfectly onto where each tool shines. A founder building something new will feel enormous speedups; a team maintaining a decade-old monorepo should expect modest, uneven gains and budget review time accordingly. This is exactly why we argue in our piece on what it costs to build an app with AI that measuring your own before-and-after beats trusting any leaderboard.

7. Pricing and the real cost of running an agent

The pricing pages for these tools are a study in how the whole industry quietly shifted from flat subscriptions to metered usage, and if you do not understand the shift, you will be surprised by a bill. On the surface, the entry prices are almost identical: Claude Code, Codex, Devin, Cursor, and Replit all start at $20 a month, GitHub Copilot undercuts them at $10, and the individual power tiers converge on $200. Underneath, though, the billing models diverge sharply, and that divergence is where the money actually goes. Here is the landscape at a glance.

The single most important concept on that table is that the sticker price is a floor, not a ceiling, for every tool except the bundled ones. Claude Code's subscription is genuinely great value for sustained work, because heavy sessions are dominated by cache reads that a Max plan folds into a flat fee, and one analysis estimated a Max 20x user can consume the equivalent of $600 to $1,500 a month in API tokens for a flat $200 - CloudZero. But that value only exists on the subscription. On the API path, which many teams wire into CI, there is no cap, and the horror stories are real: one client was reportedly billed roughly $500 million in a single month after failing to set any usage limits, another developer left Claude Code running overnight and woke up to a $6,000 charge - MakeUseOf. Those are extreme anecdotes, not typical outcomes, but they exist precisely because agentic workflows burn tokens far faster than chat.

Devin's model deserves its own paragraph because it is the only tool here priced by work performed rather than tokens or seats. Cognition bills in Agent Compute Units, where one ACU is roughly fifteen minutes of active Devin work and costs about $2.25 on the entry plan - Lindy. That sounds cheap until you remember Devin's failure mode: an open-ended task where it heads the wrong way for hours consumes ACUs the entire time, so the bill is tightly coupled to how well-scoped your task is. Cognition's own best practice, to keep sessions under ten ACUs because performance degrades in long runs, is as much a cost-control tip as a quality one. For a founder, the mental model is simple: Devin charges you for the agent's time, so vague instructions cost real money, and the discipline that improves output also lowers the bill.

The real-world monthly spend, once you account for actual usage rather than the sticker, tells a more useful story than any pricing page. Enterprise estimates put heavy Claude Code developers at $150 to $250 a month on average, with the heaviest agentic users reaching much higher, while Codex tends to run somewhat lower and Copilot lower still because its completions stay free - morphllm. These are modeled estimates, not audited figures, so treat the exact numbers loosely, but the shape is right.

Two patterns should shape how you budget. First, usage-based billing is now universal and it breeds distrust when handled badly. Cursor's June 2025 change from a flat plan to a metered credit pool triggered a user revolt severe enough that the CEO publicly apologized and issued refunds - TechCrunch. Every vendor here has some version of this scar, and the lesson for buyers is to model your heavy month, not your average one. Second, the subscription almost always beats the API for sustained coding, because the vendors price the flat plans to be a bargain for power users and make their margin on the long tail of light ones. If you are going to code with an agent daily, buy the subscription, set organization-level spend limits on any API keys, and never wire an uncapped key into an automated loop. Our cost breakdown for AI-native companies walks through building a real stack for under a few hundred dollars a month using exactly this discipline.

8. The wider field: every other player that matters

Claude Code, Codex, and Devin dominate the conversation, but they are three tools in a field of dozens, and for many founders the best answer is not one of the three. The wider market splits cleanly by where the tool sits on the autonomy spectrum, and knowing the categories helps you avoid comparing an editor to an autonomous agent as if they competed for the same job. Below are the players worth knowing, grouped by what they fundamentally are, with the honest one-line reason each exists.

The AI-native IDEs put the agent inside an editor and keep you as the driver. Cursor, from Anysphere, is the category-defining one and, by revenue, the fastest-scaling B2B software company on record, reportedly crossing $1 billion in annual recurring revenue by late 2025 on its way to a valuation around $29 billion - The Next Web. It ships its own agent model, Composer, alongside a picker for every frontier model, and its edge is the tight, incremental control an editor gives you, which is exactly what the METR study suggests matters on real codebases. GitHub Copilot is the other giant, now spanning inline completions, an in-IDE agent mode, an async coding agent that turns an issue into a pull request, and a natural-language app builder called Spark. Its moat is integration: the deepest GitHub and Azure ties, a marketplace of more than twenty-five user-selectable models, IP indemnity, and enterprise compliance, all starting at $10 - GitHub. Microsoft has said Copilot passed 20 million all-time users, though it pointedly does not report active users - TechCrunch.

The async cloud agents take a task and return a pull request with no editor at all, which is the same posture as Codex Cloud and Devin. Google Jules is the notable one here, a fully autonomous agent that runs in cloud VMs on Google's flagship Gemini 3.1 Pro, bundled into Google AI subscriptions rather than sold standalone, with a generous free tier of fifteen tasks a day - Google. Factory's Droids push further into enterprise, autonomous agents that own whole software-lifecycle tasks across CLI, IDE, and cloud, backed by a $150 million round in April 2026 at a $1.5 billion valuation - TechCrunch. These tools are for teams that want to delegate scoped work at volume and are comfortable living in the pull-request review loop.

A distinct and interesting branch is the spec-driven and app-builder approach, which changes the interaction model rather than just the autonomy level. Amazon's Kiro writes a structured specification before it writes any code, then plans and builds against it, an attempt to reduce the drift that plagues free-form agents, powered by Claude through Bedrock - Kiro. Replit Agent goes the other direction toward accessibility, an agentic app builder in the browser that scaffolds, codes, provisions a database, and deploys, aimed squarely at non-traditional builders, with effort-based pricing that scales to the actual work done. For founders who want to see this whole builder category ranked, our top AI app builders guide covers it in depth, as does the broader AI website builders market map.

The open-source, bring-your-own-key tools are the option for builders who want zero lock-in and full auditability. Aider is a terminal pair programmer that treats git as the source of truth and auto-commits every change, works with any model, and costs nothing beyond the API tokens you spend - Aider. Cline and its fork Roo Code bring the same philosophy to VS Code as approval-gated agents, with Cline alone reporting more than five million installs - Cline. These trade polish and hand-holding for control and cost transparency, and they are a genuinely good fit for technical founders who want to understand and own every part of their stack.

Finally, there is a category one level up from all of these that matters most for founders who are not developers, and it is worth naming honestly as one option among many. Every tool above produces code, and a code artifact is not a company. A non-technical founder does not want to review a diff; they want a working product with a website, an app, billing, and an admin panel, and they want it to keep running. That is the gap that company builders address, and it is where a platform like Founden sits: rather than handing you a terminal or a pull request, it takes a plain-English description of a business and builds and operates the whole thing, orchestrating coding agents underneath so the founder never touches the code layer at all. It scores a 10 on accessibility in our table for exactly this reason, and a modest 4 on ecosystem because it is deliberately not a developer-tool integration play. It is the right answer for a specific person, the founder who wants the outcome and not the craft, and the wrong answer for an engineer who wants to control every line, which is exactly the kind of honest trade the rest of this guide is built to help you make. Our guide to the autonomous business unpacks how far that model can currently go.

9. Where each tool wins and where each one fails

Benchmarks and pricing tell you what a tool can do and what it costs, but the decision hinges on fit: matching the tool to the actual work and to your tolerance for cleanup. The single most sobering number for setting expectations is that, by credible practitioner synthesis, even the best agents in 2026 resolve only 30% to 50% of well-defined tickets end to end without human intervention, and they fail hardest on ambiguous, judgment-heavy work - getbeam. Internalize that before you internalize any marketing. These are powerful junior collaborators with uneven judgment, not autonomous seniors, and every "win" below assumes a human is verifying the output.

Claude Code wins on complex, existing codebases where understanding the code is the hard part. Its strength is interactive, multi-file work with heavy verification, and in blind comparisons cited by multiple reviews, engineers preferred its output on complex refactors by roughly two to one. It is also an excellent onboarding tool, because you can ask it the same questions you would ask a senior colleague about how a system works. It fails when you over-trust it: the "trust-then-verify gap" where a plausible implementation quietly mishandles an edge case, and the "infinite exploration" where it reads hundreds of files and fills its context without converging. The fix, from Anthropic's own best-practices, is to always give it a way to verify and to clear its context aggressively between unrelated tasks - Anthropic.

The concrete wins, when the discipline is present, are real and measurable. Anthropic's own security team reported cutting incident-resolution time from fifteen minutes to five by pointing Claude Code at stack traces and infrastructure plans, and its data-science team treats the tool "like a slot machine," committing first, letting it run for about thirty minutes, then accepting the result or discarding it and starting fresh - Anthropic. That commit, run, accept or reset loop is the single most practical habit to copy from teams that get value from these agents, because it makes every run cheap to undo and removes the temptation to argue an agent out of a hole it has dug.

Codex wins on bulk, bounded, independently testable work in a mature codebase. Dependency upgrades, expanding test coverage, style migrations, documentation, and bug fixes from clear issue descriptions are its sweet spot, and its cloud parallelism makes it excellent for fanning out many small tasks at once. It fails on scope-sensitive and architectural work, because its "act, don't ask" default means it will make assumptions and touch files you did not mention to push the task to completion, and its fluent output invites over-reliance on code that sounds production-ready while missing validation or security depth - nxcode. The mitigation is to prompt it with an explicit goal, context, constraints, and a "done when" criterion, and to run tests and type checks before accepting anything.

Devin wins on scoped, verifiable, junior-level work at volume, which is precisely what Cognition uses it for internally. Migrations, CVE remediation, test writing, and small tickets that would take a junior four to eight hours are its documented strength, and running many Devins in parallel on that kind of work is where the "659 PRs in a week" number comes from. It fails on ambiguity, on large unfamiliar codebases where it can modify the wrong file or duplicate existing utilities, and on mid-task requirement changes, because piling on instructions after it starts degrades its performance - Cognition. The reviewer consensus is that Devin is a supervised force-multiplier, and the failure mode is treating it as unsupervised.

The unifying principle across all three is the greenfield-versus-legacy split, and it is the most reliable predictor of whether you will love or hate these tools. On new code and unfamiliar territory, where there is no accumulated context to get wrong, agents feel magical and the productivity gains are large and real. On mature systems you know intimately, the gains shrink and can even invert, exactly as METR found, because the agent lacks the tacit knowledge you carry in your head and its confident wrong turns cost you review time. A founder building a new product from scratch is in the best case for these tools; a team grafting AI onto a sprawling legacy system is in the hardest one. Match your expectations to which situation you are actually in, and revisit our analysis of what software is left to build for where the greenfield opportunities still are.

10. Security and the operational risks nobody budgets for

The failures that actually hurt companies in 2026 were not model mistakes, they were operational disasters, and this is the section founders skip at their peril. When you give an autonomous agent a shell, a database connection, and the permission to act without asking, you have created a new and powerful attack surface, and the incidents are no longer hypothetical. Understanding the risk starts with a concept the security researcher Simon Willison named the lethal trifecta: an agent becomes dangerous when it combines access to private data, exposure to untrusted content, and the ability to communicate externally, because a model "follows instructions in content," and it cannot reliably tell your instructions from a malicious one hidden in a file it reads - Simon Willison.

That is not a theoretical concern for a coding agent. As one security analysis put it, "a single malicious README in your dependency tree can redirect Claude mid-task, and the default configuration does nothing to stop it," and a successful prompt injection can escalate to remote code execution on the developer's machine - UpGuard. Between January 7 and 15, 2026, the security firm PromptArmor disclosed indirect-prompt-injection vulnerabilities in four production AI tools used at Fortune 500 scale, each following the same private-data-plus-untrusted-content-plus-outbound-channel pattern - Airia. The uncomfortable truth is that this class of attack cannot be fully patched away, because the property that makes agents useful (following instructions in content) is the same property that makes them exploitable. The defense is architectural, not a magic filter.

The most visceral risks, though, come from a mode every one of these tools offers and every vendor warns about: running the agent with all permission prompts disabled. In Claude Code this is the flag literally named --dangerously-skip-permissions, sometimes called "YOLO mode," and the damage is documented. In one case an agent in that mode deleted every user-owned file on the machine, with only system file permissions preventing total destruction. The canonical disasters are two production-database wipes that should be pinned above every founder's desk. In February 2025, a developer asked Claude Code to run a Terraform operation without the correct state file, and the agent executed terraform destroy on production, wiping roughly two and a half years of data before AWS restored it from an internal snapshot - Hacker News. In another, an agent running through Cursor "deleted the production database and all volume-level backups in a single API call in nine seconds" - Tom's Hardware.

The crucial insight from those incidents, and the reason they belong in a founder's guide rather than a security whitepaper, is that they were operational failures, not model failures. In each case a human gave an agent production credentials and destructive tools with no staging environment, no deletion protection, no least-privilege restrictions, and no backups in a separate account. The top-voted lesson from the community post-mortem was blunt: "If you give a robot the ability to delete production, it is going to delete production." Agents amplify your existing operational discipline, or your existing lack of it. The mitigations are unglamorous and non-negotiable, and they are the same controls you should already have.

• Least privilege: never give an agent credentials that can touch production or delete backups
• Isolation: run unattended agents in a container or VM, ideally without network access
• A human gate on destructive operations: no automated terraform apply or database drops
• Separate backup accounts: so a compromised or confused agent cannot reach them
• Tight sandbox and approval defaults: loosen them only for repositories you trust

Those five controls are worth more than any benchmark score when you are deciding how much autonomy to grant. The safest way to run these tools aggressively is to make the blast radius of a mistake small: sandboxed execution, no live credentials, and reversible changes. This is also, incidentally, an argument for the managed and higher-abstraction tools, because a platform that never hands you a raw shell also never hands a confused agent your production database. The more autonomy you grant, the more your architecture, not the model, is what keeps you safe.

11. How founders should actually choose, and the road ahead

After all the depth, the decision is simpler than the market makes it look, because it reduces to two questions: how technical are you, and how much of the work can you actually verify. The benchmarks are a wash at the top, so the choice is about fit, not capability. Here is the decision path, stripped to its essentials.

For a technical founder or an engineering team that wants maximum control, Claude Code or Cursor is the pick, because they keep you close to the change and give you the strongest verification tooling, which the evidence says matters most on real codebases. For a team that wants to delegate scoped work and review the results, Codex and Devin are the async workhorses, with Codex better embedded in existing ChatGPT and GitHub workflows and Devin better for high-volume junior-level tasks. For a founder optimizing for value and ecosystem, GitHub Copilot at $10 with its model marketplace and enterprise guardrails is genuinely hard to beat. And for a non-technical founder who wants the product, not the process, an app builder like Replit or a company builder like Founden meets you where you are, at the cost of the granular control a developer would want. There is no universal winner, only a right answer for your situation, which is the entire reason the scored table at the top has a category column.

Two disciplines matter more than the tool choice, and they are the same two the whole guide keeps returning to. First, budget your review time, not just your subscription, because the review queue is the real bottleneck and the hidden cost that no pricing page shows. Second, make the blast radius small, because the worst outcomes in 2026 came from permissions and missing backups, not from bad code generation. A founder who gets those two things right will succeed with almost any tool on this list, and a founder who gets them wrong will struggle with even the best one. As we argue throughout our coverage of building software with AI, the discipline is the differentiator now, not the model.

A note on the author. This guide was written for founders by the team behind Founden, and it reflects a bias we will admit to: a preference for measuring things against live reality rather than leaderboards. Yuma Heymans (@yumahey), who builds Founden and the AI-agent workforce platform O-mega and co-founded the AI recruiter HeroHunt.ai, spends a chunk of every week re-running agent-benchmark claims against actual live sessions before he trusts them enough to route real work through them - O-mega. That habit is the right posture for this entire category, and it is the one we would urge on you too: trust your own before-and-after over any vendor's number.

Where does this go from here? The clear trajectory is toward longer-horizon autonomy and multi-agent orchestration, with METR's task-horizon curve suggesting agents will handle progressively longer stretches of coherent work, and the parallel-agent workflow already normalized among senior engineers. But the equally clear counterweight is that the human review bottleneck does not scale the way the agents do, and until verification gets automated as thoroughly as generation has, the practical ceiling on all of this is how fast a person can check the results. The most interesting frontier, then, is not a smarter coding model, it is the orchestration layer that sits above the coding agents and manages the verification, the deployment, and the operation of what gets built. That is where the company-builder category is placing its bet, and it is a reasonable one: as raw code generation becomes a commodity that every tool here does well, the value migrates up to whoever can turn that generation into a running, verified, operating business with the least human toil.

Conclusion

The honest answer to "Claude Code versus Codex versus Devin" is that you are asking about three different jobs, not three versions of one job. Claude Code is the supervised expert for complex existing code, the tool with the deepest control and the strongest verification story, best for people who want the wheel. Codex is the review-gated agent embedded in the tools most teams already use, best for delegating bounded, testable work at scale. Devin is the most autonomous of the three, a genuine force-multiplier on scoped junior-level work and a genuine liability when pointed at ambiguity, best for teams that can scope tightly and verify rigorously. On raw capability they are separated by fractions of a point, so the decision is about autonomy, control, and cost behavior, not about which one is smartest.

The framework to carry away is the one we opened with. Decide where you want to sit on the autonomy spectrum, from watching every change to delegating whole tickets to describing an outcome and getting a running product. Then budget for the two things the pricing pages hide: the review time that every agent shifts onto your plate, and the blast radius of a mistake made by a tool with real system access. Get those two right, keep your credentials scoped and your backups separate, and measure your own results rather than trusting a leaderboard, and any of the tools in this guide can be a strong choice. The market will keep producing new models and new agents at a dizzying pace, but the discipline that makes them pay off will not change, and that discipline, not the model, is what turns an impressive demo into a shipped, working company.

This guide reflects the AI coding agent landscape as of July 2026. Models, pricing, and usage limits in this category change almost weekly, so verify the current details on each vendor's own page before you commit.