AI Security Engineer
Job description
Data & AI / ML Evaluation Engineer at Menlo Security responsible for owning data collection, mining, curation and continuous evaluation for AI security products. The role builds automated data ingestion pipelines, dataset curation and labeling workflows, model training, and benchmark/evaluation harnesses.
At a glance
- Work mode
- Remote
What you'll do
- Design and operate automated data collection, mining, and web-scraping pipelines to gather threat samples from surrogate traffic and web sources.
- Build data processing, annotation, and dataset curation workflows, training and re-training various models, e.g. Prompt Injection (PI) SLMs, Shadow-AI detection models, Malicious Macro classifiers, and PII detection.
- Build mechanisms to capture false-positive (FP) samples from live user/DDR flows and represent / integrate them into model retraining pipelines.
- Develop automated benchmarking and evaluation harnesses to continuously test model detection quality, runtime performance, and accuracy against baseline models (e.g., Vertex AI).
- Build automated pipelines for model training for various models.
- Work closely with research and software engineering teams to turn raw detection telemetry into ongoing actionable model improvements.
What we're looking for
- 2–4 years of experience in software engineering, data engineering, or automated QA/evaluation roles.
- Strong programming proficiency in Python. Adv: C++, SQL.
- Experience in model training and various ML, AI algorithms.
- Experience building data pipelines, web scrapers, or automated data processing systems.
- Familiarity with data structures, storage formats, and database configurations.
- Methodical approach to dataset labeling, validation, bug logging, and metric tracking.
Nice to have
- Hands-on experience with ML evaluation frameworks, dataset curation, or annotation tools.
- Familiarity with text extraction, NLP preprocessing, or document parsing techniques.
- Understanding of cybersecurity concepts (Prompt Injection, Malicious Macros, DLP/PII).
- Basic experience with cloud storage, object stores, and basic containerization (Docker).
- Experience with AWS, GCP, Terraform, Kubernetes, and Linux.
Apply
Direct apply at Menlo Security
Opens the job posting at Menlo Security
About Menlo Security
The leader in Browser Security for human and agentic workforces
Menlo Security protects organizations from cyberattacks by stopping threats across the web, documents, and email before they reach the user. With Menlo Agent Runtime Security (MARS), that protection now extends to the AI agents working alongside every employee.
Mission
To enable humans and agents to connect, communicate, and collaborate securely, without compromise.
Key facts
- Browser Security
- Trusted by major global businesses, including Fortune 500 companies and government agencies
- Backed by top-tier investors
- Menlo Agent Runtime Security (MARS) extends protection to AI agents
What they value
- Proactive Leadership
- Straight Talk
- United Impact
- Elevated Talent
- Customer-Compelled