T
Senior Software Engineer — ML Inference & Numerics
Contract type
Ongoing
Work mode
Remote · Munich, Germany
Experience
Senior · Not stated
Level inferred from job title
Job description
Software engineering role focused on making large language models run efficiently on Tensordyne's custom AI inference silicon, from model bring-up to numerical optimization. The engineer will also build agentic workflows that automate exploration, experimentation, and validation of implementations.
At a glance
- Work mode
- Remote · Munich, Germany
What you'll do
- Build agentic workflows: give agents tools to inspect models, change implementations and numerical settings, run emulation and hardware experiments, and diagnose failures; build feedback and evaluation that make results trustworthy
- Bring up new models quickly and make them performant, taking unfamiliar architectures from reference code to validated, performant execution on the accelerator
- Turn learnings into reusable model support and SDK improvements
- Optimize numerics: explore precision, accumulation, and quantization choices across the model and measure gains in latency, throughput, and memory use against model quality on silicon
- Build conversion and emulation software to test numerical choices quickly, compare activations and outputs against reference models, trace accuracy regressions, and validate changes with end-to-end model evaluations
- Own design, implementation, testing, and delivery; build clear APIs that are easy to use for agents; make difficult bugs reproducible
- Work with hardware architects to test how numerical formats and arithmetic choices affect real models and turn findings into better hardware and algorithm designs
What we're looking for
- Understand LLM inference end to end: read a new model's implementation and reason about prefill, decode, attention, KV caches, MoE, batching, and parallelism, and how these translate into compute, memory, and communication on an accelerator
- Build excellent software: have owned substantial systems and delivered complex features from design through production
- Strong in Python, thoughtful about interfaces, rigorous about abstractions and testing
- Fluent with agentic tooling: prefer building the agent that runs the optimization, comfortable running numerous agents in parallel, know what agents need to work effectively in a large software organization, and judge when an agentic loop needs a human
- Take ownership and bring others along: form clear hypotheses, test them, drive problems to resolution across software boundaries, communicate clearly through specs, design docs, and reviews, explain complex trade-offs, give direct feedback
Nice to have
- Expertise in improving inference systems, such as debugging and optimizing the runtime of complex models and reasoning about accuracy-performance trade-offs
- Experience with ML compilers, accelerator SDKs, or model compression libraries, particularly for non-NVIDIA accelerators such as TPU, Trainium/Inferentia, Tenstorrent, Groq, or Cerebras
- Experience with model quantization, sparsity, codebooks, low-rank factorization or other compression techniques
- Contributions to distributed inference or serving frameworks such as vLLM or SGLang
Benefits
- Comprehensive benefits
- Competitive compensation
- Flexible spending options
- Recognition programs
Apply
Direct apply at tensordyne
Opens the job posting at tensordyne
About Tensordyne
Tensordyne is an AI system solution company that builds very high-performance, low-power generative AI inference systems. It is a well funded, fast-paced startup with headquarters in Sunnyvale, CA, and Munich, Germany, and team members working remotely across North America and Europe.
Mission
Through the creation of custom silicon, hardware and software, enable multimodal Generative AI inference acceleration at scale, with safe, sustainable, high-performance systems for hyperscaler and neocloud data center customers.
Key facts
- Sunnyvale, CA, and Munich, Germany
- AI inference systems / custom silicon
- Builds custom silicon, hardware and software for multimodal Generative AI inference acceleration
- Serves hyperscaler and neocloud data center customers
- Well funded, fast-paced startup
- Team members work remotely across North America and Europe
What they value
- Put people first. We only succeed when our people succeed.
- Ethics and integrity always. Be open, honest, and respectful of everyone.
- Think Big. Be ambitious and have audacious goals.
- Aim for excellence. Quality and excellence count in everything we do.
- Own it and get it done. Results matter!
- Make each person better together than they would be as an individual.
- Embrace each other's differences, and embrace that there will be differences.