Research Engineer / Scientist, Data & Evals
Job description
Member of Technical Staff, Data & Evals in Harvey Research, building the data and measurement systems behind post-training, benchmarking, and agent capability efforts for legal AI models.
At a glance
- Work mode
- Remote
What you'll do
- Build synthetic data pipelines that generate, filter, and curate training and evaluation samples for complex legal reasoning and agentic workflows
- Design evaluations and verifiers that reflect real legal work, calibrate them against expert judgment, and investigate where they fail
- Build and maintain capability benchmarks like the Legal Agent Benchmark (LAB), along with product-focused evals that track how models perform on real customer workflows
- Work with legal domain experts to define quality metrics, rubrics, and other grading mechanisms for training and evaluation
- Own training and evaluation data quality, including provenance, coverage, and safeguards against contamination
- Close the loop from product usage and customer feedback to new evaluations and training data
- Partner with researchers to investigate model failures, distinguish real gains from misleading metrics, and use findings to guide training and release decisions
What we're looking for
- Experience building data pipelines, ML evaluation systems, or experimentation tooling
- Strong Python skills and ownership of production-quality code
- A rigorous approach to measurement, including statistical reasoning, experimental design, and understanding when a metric can mislead
- The ability to translate ambiguous goals or expert judgment into structured datasets, rubrics, and measurable criteria
Nice to have
- Experience with synthetic data, LLM evaluations, automated raters, or preference data
- Strong data and measurement engineering experience close to ML model development
Your team
Harvey Research
A small, high-bar team of hands-on engineers and researchers, where each person owns a large part of the work.
Apply
About Harvey
Transforming how legal and professional services operate with frontier agentic AI
Harvey combines frontier agentic AI, an enterprise-grade platform, and deep domain expertise to reshape how critical knowledge work gets done. The company has strong product-market fit and world-class investor support and is scaling fast to define a new category.
Mission
Transforming how legal and professional services operate by combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise to reshape how critical knowledge work gets done.
Key facts
- Recently post-trained its first model built for legal work
- Building the team to scale up post-training efforts
- Hiring for this role across multiple levels
What they value
- Decisiveness
- Simplicity
- Job's Not Finished