arlen/benchOpen benchmarks for agentic consumers
SNAPSHOT web_search-2026-q3
11 JUN 2026 · BERKELEY, CA
Vendor Profile · Illustrative prototype

Exa

Exa on the arlen/bench leaderboards — agent web-search accuracy, extraction fidelity, freshness, cost per verified-correct answer, and agent-readiness, scored against golden truth.

Answer firstillustrative

Exa ranks #1 of 6 for agent web-search accuracy at 87.6% hit@5; #5 of 7 for extraction fidelity at 0.74, and is agent-ready — an autonomous agent obtained a working API key in live trials with zero humans.

hit@5
87.6%
web search
cost / correct
$0.0064
web search
extraction fidelity
0.74
web extraction
agent-ready
Yes
onboarding harness
§ A

Web Search — Exa

snapshot web_search-2026-q3
Metric ExaLeaderboard bestDirection
hit@1 %71.2leadshigher is better
hit@5 %87.6leadshigher is better
fresh<30d %81.0Tavily 86.4higher is better
retrievability h11.2Tavily 7.8lower is better
cost/correct $$0.0064Serper $0.0041lower is better
p50 latency ms412Brave 301lower is better
§ B

Web Extraction — Exa

snapshot web_extraction-2026-q2
Metric ExaLeaderboard bestDirection
fidelity 0-10.74Firecrawl 0.91higher is better
JS gap Δ0.22Firecrawl 0.03lower is better
block rate %8.8Apify 3.3lower is better
cost/correct $$0.0058Jina $0.0009lower is better
schema validity %84.0Firecrawl 96.2higher is better
§ C

Compare Exa

Exa vs Tavily Exa vs Serper Exa vs Brave Exa vs Firecrawl Exa vs SerpAPI

Illustrative prototype. No verified vendor run has been published yet; every figure here is a placeholder and must not be cited as a measured result. Numbers are replaced when a snapshot’s first full run lands.