Performance Engineer
Job description
A Performance Engineer role at CuspAI bridging frontier AI models and high-performance computing infrastructure, optimizing large-scale training and inference workloads. The role focuses on profiling training runs to identify Model FLOP Utilization bottlenecks and building optimization layers.
At a glance
- Employment
- Full-time
- Work mode
- Hybrid · Cambridge, United Kingdom · London, United Kingdom
- Languages
- English
What you'll do
- Analyze model architectures to optimize how mathematical operations map onto hardware, focusing on NVIDIA GPU kernels and collective communication layers (NCCL)
- Implement efficiency techniques such as quantization, model pruning, and distillation to lower latency for materials search engine APIs
- Streamline data pipelines to ensure high-performance accelerators are never starved for data during massive training runs
- Use advanced observability tools to trace end-to-end execution of training jobs, pinpointing processing clogs or interconnect contention
- Build and maintain real-time dashboards tracking MFU, memory bandwidth utilization, and cluster uptime
- Monitor and justify utilization of AI accelerators (e.g. H100 clusters), preventing idle time and memory leaks
- Develop scaling laws and internal tools to predict system behavior across compute, memory, and networking as models grow
- Work closely with AI Researchers to adapt model architectures for better hardware compatibility without sacrificing scientific accuracy
- Partner with the Infrastructure team to influence the design of future compute clusters based on performance data and profiling results
What we're looking for
- 3+ years of experience in performance engineering, site reliability engineering, or low-level software development, specifically focused on high-performance computing (HPC) or AI workloads
- Expert-level knowledge of GPU architecture and experience optimizing CUDA kernels or using libraries like Triton
- Deep understanding of distributed training frameworks (e.g. PyTorch Distributed, DeepSpeed, or Megatron-LM) and networking protocols
- Proficiency in profiling tools such as Nsight Systems, Nsight Compute, or Weights & Biases for performance tracking
- A builder mindset with the ability to work autonomously in an early-stage, fast-paced startup environment
- Personal interest in the potential applications of the technology CuspAI is building
Nice to have
- Experience working with specialized AI hardware such as TPUs or alternative accelerators
- An academic background or personal interest in Materials Science, Chemistry, or Physics
- Knowledge of MLOps practices, specifically regarding the deployment of models at scale in cloud environments (AWS/GCP/Azure)
- Contributions to open-source performance optimization or machine learning libraries
Benefits
- Competitive salary
- Equity in CuspAI
- 28 days holiday (DE, NL, UK) or 21 days holiday (JP, SG, US), in addition to local public holidays
- 'Gold Standard' parental leave: 26 weeks (primary caregiver) and 12 weeks (secondary caregiver) at full pay
- Professional development budget
- Solve meaningful problems with direct impact on advancing materials science and sustainability
- True interdisciplinary teamwork bridging AI research, computational chemistry, and experimental science
Apply
Direct apply at Cuspai
Opens the job posting at Cuspai
About CuspAI
The frontier AI company on a mission to solve the breakthrough materials needed to power human progress
CuspAI is a frontier AI company building a 'search engine' for next-generation materials, working on challenges including energy, clean water, the future of compute, and carbon capture. Its founding team is comprised of world-class researchers in AI, chemistry and engineering.
Mission
To solve the breakthrough materials needed to power human progress by harnessing AI to unlock trillion-dollar materials breakthroughs in months, not millennia
Key facts
- AI / Materials Science
- Founding team is the most cited in the world, comprised of world-class researchers in AI, chemistry and engineering
- Working on energy, clean water, the future of compute, and carbon capture
- Building a 'search engine' for next-generation materials
What they value
- Diverse and inclusive workplace
- Collaboration
- Impact and growth