Last updated: 11/21/2026Last verified: 2026-11-15

Coarena

Coarena Overview: Blind Agent Battles, Benchmarks and a Free API

11
0

Coarena is a public arena and leaderboard for computer-use AI agents from Coasty Systems (YC S26): two agents race the same task on a computer or an Android phone, people vote blind on the winner, and the results feed a Bradley-Terry ranking plus four benchmarks covering capability and safety. It also exposes the same models through an OpenAI- and Anthropic-compatible API with $5 of model usage a day free, and licenses the resulting trajectory data by contract.

Freemium

What is Coarena?

Coarena is a public arena and leaderboard for computer-use agents, run by Coasty Systems, which its YC company page lists as S26. Two agents attempt the same task on a computer or an Android phone and people vote blind on the result; the votes feed a Bradley-Terry ranking that reports pTop1 and standard error, next to four tracks: Computer Use Arena, Phone Use Arena, CUA KnowledgeBench and CUA SafetyBench. The same models are exposed through an OpenAI- and Anthropic-compatible API with a free daily allowance, and the trajectory data behind the arena is licensed by contract. This listing read the home page, /docs, /leaderboard, /governance, /data and the benchmark methodology page on 2026-11-15, but did not register, vote or call the API.

Key Features

  • Blind pairwise battles: two agents run the same task on a computer or an Android phone, and the voter does not see which model produced which result.
  • A public Bradley-Terry leaderboard reporting pTop1 and standard error per model, ordered by tokens processed.
  • Four tracks: Computer Use Arena, Phone Use Arena, CUA KnowledgeBench and CUA SafetyBench; /mission states 101 public metrics and 24 ranked models.
  • Model API for OpenAI SDKs at https://coarena.ai/api/v1 and for Anthropic's SDK at https://coarena.ai/api, plus public /api/v1/models, /api/metrics and /api/leaderboard endpoints.
  • The free allowance the /docs states: '$5 a day, and Coarena pays', 30 requests a minute, 2 at once, and a key that needs no card.
  • One self-service call to ask for the next step up: $10 a day, or up to 600 requests per minute.
  • Trajectory data licensing by contract under /data rather than a public download.

Best For

Researchers comparing computer-use and Android agents on human preference rather than a single benchmark score.Developers calling several agent models through an OpenAI or Anthropic SDK with one key.Teams weighing completion, speed, recovery and cost when shortlisting an interactive agent.Analysts pulling /api/metrics and /api/leaderboard into dashboards or papers.Organizations that need licensed computer-use trajectories under contract terms.

Pricing

There is no price list: /pricing returns 404. What the /docs does publish is the free tier, '$5 a day, and Coarena pays' with 30 requests a minute, 2 concurrent requests and a key that needs no card, plus a self-service request for the next step up, $10 a day or up to 600 requests a minute. Trajectory data and evaluation work are priced by contract, and this listing found no published rates. Posting tasks, voting and labelling need a Google sign-in; reading the leaderboard does not.

Pros & Cons

Pros

  • Human blind voting plus four benchmarks, instead of one headline score.
  • Covers both a real computer and an Android phone.
  • Publishes pTop1 and standard error, so the firmness of a ranking is visible.
  • The same models are reachable through OpenAI and Anthropic SDKs, with a free daily allowance and no card.
  • Public models, metrics and leaderboard endpoints for programmatic use.
  • Documents its own conflict of interest and keeps the parent company's agent off the board.

Cons

  • No published pricing page: /pricing 404s, and data licensing is contract-only.
  • Default API access is capped at $5 a day of model usage, 30 requests a minute and 2 concurrent requests.
  • Higher limits need a request, and the docs do not say how approval is decided.
  • The board moves: it mixes unreleased preview models, and the methodology page says part of the phone comparison uses a model as judge.
  • Vendor usage figures for the site are self-reported, and this listing did not call the API or verify them.

Alternatives

AA-AgentPerf-Local

Artificial Analysis's open-source benchmark that replays recorded agent trajectories against your own model servers to measure throughput and latency locally.

Plurai Simulation

Generates scenarios from your own agent flow graph and plays them with a simulated user, for testing one agent you built rather than ranking public models.

TesterArmy

Browser agents that run your web and mobile app's end-to-end tests, for QA of a product instead of a public leaderboard.

Handit.ai

Monitors production agent traffic for hallucinations and schema breaks and ships fixes, for reliability after launch.

FAQ

Details

Platform

WebAPI

Features

  • Blind pairwise battles: two agents attempt the same real task on a computer or an Android phone and a person votes on the winner, which feeds a public Bradley-Terry leaderboard with pTop1 and standard error per model
  • Four benchmarks — Computer Use Arena, Phone Use Arena, CUA KnowledgeBench and CUA SafetyBench — with published measures for completion, speed, recovery and cost
  • Free model API at https://coarena.ai/api/v1 for OpenAI SDKs and https://coarena.ai/api for Anthropic’s: $5 of model usage a day paid by Coarena, 30 requests a minute, 2 concurrent, free key with no card, with self-service requests to step up to $10 a day and up to 600 requests a minute
  • Public JSON endpoints for /api/v1/models, /api/metrics and /api/leaderboard, plus a /data programme that licenses computer-use trajectories, human judgments and provenance by contract

Languages

en

Known limitations

  • No published prices: https://coarena.ai/pricing returns 404. The only cost figures on the site are the free API allowance (“$5 of model use a day, and Coarena pays”), the $10-a-day step-up request, and “Terms are scoped per contract” for datasets and evals — there are no plan names, seats, credit packs or dated discount
  • A Google account is required for the productive actions; /governance states that “Posting a task, voting, annotating and giving step verdicts require an account”, while watching is open to everyone
  • Self-disclosed conflict of interest: the parent Coasty Systems also builds a computer-use agent, and the /governance page says the company “owns the arena and every dataset assembled from it” while keeping its own agent off the roster
  • The traffic figures (62.5K people, 6.11M requests, 245B tokens in a 7-day window), the “biggest crowdsourced benchmark” claim and the 24-strong model roster are the vendor’s own numbers; the roster includes unreleased preview models, and the Phone board has comparisons judged by a model rather than a person, as /benchmark/methodology discloses. Third-party bots exist to farm the free tier, and coarena.com is an unrelated for-sale domain

Official pricing source

https://coarena.ai/docs

Rate This Tool