Senior Software Engineer - Analytics Data Platform
Job description
Platform engineering role on Datadog's Analytics Data Platform Batch team, owning the infrastructure and tooling that powers batch data processing (Spark, PySpark, Ray, Python) for internal data engineers and scientists. The role focuses on AI-driven migration automation, engine-agnostic batch execution, and platform reliability at scale.
At a glance
- Employment
- Full-time · Employee (employment agreement)
- Work mode
- Hybrid
- Languages
- English – fluent
What you'll do
- AI-driven version migration automation: Spark and JVM
- Building agentic tooling that analyzes compatibility, generates PRs, and orchestrates multi-datacenter testing
- Engine-agnostic batch execution
- Platform reliability and scale
- Spark Pod Autosizing - CPU and memory allocations
- Understanding tradeoffs across execution models
What we're looking for
- 6+ years of experience
- Significant backend programming experience in one or more languages (Go, Java, Rust, Python, Scala)
- Spark internals/infrastructure - executor lifecycle, memory model, dynamic allocation, resource tuning
- Experience building platform or infrastructure that other engineers depend on
- Distributed systems engineer with experience building high performance large scale systems
- Kubernetes fundamentals
- Excellent communication skills (English)
- High proactivity and cross-team collaboration experience
- Excited about leveraging AI tools to enhance how you code, solve problems, and build - or eager to learn how
- Demonstrated ability to use AI coding tools in day-to-day workflows and validate, critique, and refine AI-generated output
Nice to have
- AI/LLM integration in engineering workflows
- Observability and cost attribution
Your team
Analytics Data Platform Batch team
Owns the platform that powers batch data processing across Datadog's internal data infrastructure, executing 800k+ Spark jobs daily plus PySpark, Ray, and Python-based workloads, serving 50+ internal teams across billing, cost management, security intelligence, applied AI and more.
Benefits
- New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
- Continuous professional development, product training, and career pathing
- Intradepartmental mentor and buddy program for in-house networking
- An inclusive company culture, ability to join our Community Guilds (Datadog employee resource groups)
- Access to Inclusion Talks, our Internal panel discussions
- Free, global mental health benefits for employees and dependents age 6+
- Competitive global benefits
Apply
About Datadog
The leading observability and security platform for the AI era
Datadog provides businesses with unified visibility across the technology stack to manage complexity at scale. It brings applications, infrastructure, data, models, and security into one place, using AI to detect and resolve issues before they impact customers. Trusted globally by Fortune 500 companies and high-growth AI leaders, Datadog enables businesses to move faster with clarity and confidence.
Key facts
- Observability and security platform
- Trusted globally by Fortune 500 companies and high-growth AI leaders
- Operates as a hybrid workplace
What they value
- Inclusive company culture
- Community Guilds (Datadog employee resource groups)