ForgeApply · Job listing
Research Engineer, Benchmarks
Clera
Apply in about a minute — without sacrificing quality.
ForgeApply autofills this application and tailors your resume to this exact posting. You review everything before it's sent. Free trial, no card required.
About this role
ABOUT THE ROLE
Join a small, technically elite team — including International Olympiad medalists and published AI researchers — building high-quality benchmarks to evaluate frontier AI agents on realistic, domain-specific workflows. As a Research Engineer, Benchmarks, you'll own the design and implementation of evaluations that frontier labs and enterprise customers trust. This role is central to ensuring our benchmarks are rigorous, credible, and tightly aligned with real-world agent performance.
This is an on-site role based in San Francisco, CA. Visa sponsorship is available.
WHAT YOU'LL DO
- Design, implement, and own the quality of internal benchmarks for evaluating frontier agents on domain-specific tasks.
- Partner with subject-matter experts to define realistic workflows and tasks for domain-specific evaluations.
- Build reliable infrastructure to run models and agents against benchmark tasks at scale.
- Develop metrics and analyses that measure benchmark difficulty, reliability, and failure modes.
- Validate that benchmark performance correlates with real-world evaluations, customer needs, and frontier lab expectations.
- Write clear documentation and benchmark reports that make results legible and credible to technical audiences.
WHAT WE'RE LOOKING FOR
Required
- 2–4 years of experience in software engineering, ML engineering, or research roles.
- Strong proficiency in Python, Docker, and Linux environments.
- Experience building environments, evaluations, or benchmarks for AI systems.
- Published research or technical writing on topics such as public benchmarks, model failure modes, or evaluation methodology.
- Deep understanding of what makes a benchmark realistic, reliable, and practically useful.
- Curiosity and genuine ability to understand how real-world workflows operate across diverse domains.
- Strong attention to detail — a habit of spotting subtle inconsistencies and edge cases in task design.
- Ability to reason from first principles about task design, scoring, and failure modes.
- Comfort thriving in unstructured problem spaces and working independently in fast-paced, early-stage environments.
- Excellent communication skills for collaborating across time zones and with technical teams.
COMPENSATION & BENEFITS
- Salary: $150,000 – $250,000 USD annually, depending on experience.
- Visa sponsorship available.
- Opportunity for significant early-stage equity and career growth within a high-impact, research-driven team.
LOCATION
This is a full-time, on-site position in San Francisco, CA. Candidates must be willing and able to work in-office.
Salary insight
The midpoint of this range ($200k) is right around the median disclosed salary for San Francisco roles listed on ForgeApply ($203k across 6,469 jobs).
Based on live postings with disclosed pay on ForgeApply; refreshed daily. Not an estimate of this employer's offer.
Ready to apply to Clera?
Apply in about a minuteSimilar jobs
- Research Engineer, Benchmarks — Hud · San Francisco
- Research Scientist, Benchmarks & Evaluations — Protege · Remote
- Research Scientist - Frontier Benchmarks — Snorkelai · San Francisco, CA (Hybrid)
- Research Engineer – Benchmarking, Evals & Failure Analysis — Mercor · San Francisco
- Performance Engineer - Benchmarking — Supabase · Remote
- Director, Research - Frontier Benchmarks — Snorkelai · San Francisco, CA (Hybrid)
- Principal Engineer - Perf and Benchmarking — Coreweave · Sunnyvale, CA / Bellevue, WA
- Research Engineer — Bespokelabs · Mountain View
More like this: More jobs at Clera · Browse all jobs