AI Benchmark Engineer | Native Language Specialist - French (France) - Remote
Job description
Freelance, remote role for native-speaking software engineers to design, build, and validate Terminal-Bench evaluation tasks that test large language models on multilingual software challenges.
This is an application to a talent network
You are applying to be considered for upcoming projects. It is not a single open position, and work is offered as projects arise.
At a glance
- Employment
- Freelance · Independent contractor
- Work mode
- Remote
- Languages
- English – fluent
What you'll do
- Evaluating Coding Agents
- Build realistic task environments using datasets and files in your native language, keeping assets in the target language to genuinely measure multilingual handling
- Find failure points where AI does not work in your native language through prompting and translation
- Support the development of robust reference implementations and write highly reliable, deterministic verifier scripts (using rubric-based judging only when strictly necessary)
- Analyze execution logs and calibrate task difficulty (Easy to Very Hard) using standard Terminal-Bench run configurations against various model tiers (Haiku, Sonnet, Opus)
- Participate in a rigorous 4-layer human quality control process (creation, human review, calibration review, and audit) alongside automated LLM-based checks to ensure fairness, grammatical accuracy, and benchmark integrity
What we're looking for
- 1+ years of industry experience in software engineering
- Proven track record at leading technology companies and/or graduation from top-tier engineering universities
- Native or near-native fluency in the target language, with a deep understanding of its grammar, register, and phrasing rules; high English proficiency
- Strong proficiency in Python, standard shell scripting, and data processing
- Extensive experience with Terminal/CLI-based development workflows and a working familiarity with coding agents
- Deep technical understanding of multilingual text processing pitfalls, including encoding/decoding robustness and Unicode normalization, locale-dependent conventions (collation, casing, non-Gregorian dates), and text I/O, toolchain interoperability, and safe string operations
- For specific languages: bidirectional/RTL handling, font fallbacks, and rendering/typography in UI or artifacts
- Reliable availability and commitment: most tasks require a minimum of 2 hours per day or 15-20 hours per week
Benefits
Your schedule, your rules: work when you want, as much or as little as you want, no fixed hours, no check-ins, no micromanaging
- Get paid quickly and fairly: competitive rates, prompt payments, no chasing invoices
- Work on projects that actually matter: contribute to cutting-edge AI and language technology
- Be part of a global community of linguists, subject matter experts, and language professionals
- Grow without limits: access to diverse, innovative projects that expand your portfolio and sharpen your skills
- Have fun doing what you love
How to get hired
- 1Submit your application including an updated copy of your CV in English
- 2Complete a GenAI assessment to evaluate your skills
- 3Finalize onboarding and profile set-up in our system, and become eligible for Applied AI projects
Apply
Direct apply at LILT (Production)
Opens the job posting at LILT (Production)
About LILT
AI is changing how the world communicates — and LILT is leading that transformation.
LILT delivers multilingual AI and human-verified services to Enterprises, Governments, and AI Developers around the world.
Mission
LILT's mission is to make the world's information available to everyone, no matter the language they speak.
Key facts
- Delivers multilingual AI and human-verified services to Enterprises, Governments, and AI Developers around the world
- Engages independent contractors (1099) globally
- Uses AI and automated tools to assist in evaluating applications
What they value
- Fair, inclusive, and transparent hiring process
- Innovation and excellence