Senior Software Engineer, Inference Performance
Level inferred from job title
Job description
Builds the software stack (Python SDK and Rust programming stack, kernels, libraries, tooling) that takes AI models from code down to Tensordyne's custom silicon, with a focus on performance. The role owns substantial parts of one layer and works closely with models, compilers, runtime, and hardware engineers.
At a glance
- Work mode
- Remote · Munich, Bavaria, Germany
What you'll do
- Design APIs, kernel abstractions, and libraries in the Python SDK or Rust stack so model engineers and agents can reach high performance at the layer that fits their problem.
- Take priority workloads, from LLMs to multimodal models, to peak performance on real silicon; profile, find the bottleneck across kernels, compiler, runtime, and parallelism, and fix it at the right layer.
- Compare measured performance against analytical models and simulation, explain every material gap, and turn the explanation into a fix or a better model.
- Give agents the profilers, traces, counters, and benchmark harnesses to form a hypothesis, change code, and measure the result in minutes; make results trustworthy enough to act on.
- Define how performance is measured, track regressions, and report results.
- Design abstractions that carry forward to future hardware generations and work with hardware architects to test how architectural choices affect real workloads.
What we're looking for
- You've optimized AI workloads on GPUs or custom accelerators and can reason about compute utilization, memory hierarchy, latency hiding, synchronization, and multi-chip scaling.
- You've owned substantial systems from design through production; strong in Python and/or at least one systems language such as Rust, C++, or C, and rigorous about interfaces, abstractions, and testing.
- You understand LLM inference end to end: can read a new model's implementation and reason about prefill, decode, attention variants, KV caches, MoE, and parallelism, and how they map to compute, memory, and communication.
- Agentic development is how you work, not an experiment; you'd rather build the loop that finds the optimization than find it by hand, and you know when the loop still needs a human.
- You've designed APIs, libraries, or performance tooling that other engineers adopted.
- You form clear hypotheses, measure, and follow the evidence across software and hardware boundaries; you take ownership of the result.
Nice to have
- Experience with embedded, bare-metal, or low-level runtime programming.
- Experience with MLIR, other compiler frameworks, or kernel DSLs such as Triton, Pallas, NKI, or CuTe.
- Contributions to LLM serving stacks such as vLLM or SGLang, or a track record of taking end-to-end inference workloads to competitive performance on a hardware target.
Benefits
- Comprehensive benefits
- Competitive compensation
- Flexible spending options
- Recognition programs
Apply
Direct apply at Tensordyne
Opens the job posting at Tensordyne
About Tensordyne
Tensordyne is an AI system solution company that builds very high-performance, low-power generative AI inference systems. It is a well funded, fast-paced startup with headquarters in both Sunnyvale, CA, and Munich, Germany, and many team members working remotely across North America and Europe.
Mission
Through the creation of custom silicon, hardware and software, to enable multimodal Generative AI inference acceleration at scale, with safe, sustainable, high-performance systems for hyperscaler and neocloud data center customers.
Key facts
- Sunnyvale, CA, and Munich, Germany
- AI inference systems / custom silicon
- Builds very high-performance, low-power generative AI inference systems
- Creates custom silicon, hardware and software
- Serves hyperscaler and neocloud data center customers
- Well funded, fast-paced startup
- Headquarters in both Sunnyvale, CA, and Munich, Germany
- Many team members working remotely across North America and Europe
What they value
- Think big. Pursue ambitious technical and business goals.
- Aim for excellence. Quality matters in everything we build and deliver.
- Own it and get it done. Take responsibility and drive results.
- Operate with integrity. Be direct, transparent, and respectful.
- Win as a team. Make the people around you more effective.
- Value different perspectives. The strongest teams challenge assumptions and bring diverse experience to difficult problems.