Senior Analytics Engineer

Imagine Worldwide

Contract type

Ongoing

Work mode

Remote · Africa

Experience

Senior · 7+ years

Job description

The Senior Analytics Engineer builds, runs and improves Imagine Worldwide's analytics platform, covering infrastructure, pipelines and core data models that reporting and analysis depend on. The role focuses on scaling the platform as data volumes grow while keeping costs under control.

At a glance

Employment: Full-time
Work mode: Remote · Africa

What you'll do

Manage infrastructure as code in Terraform, keeping environments reproducible, version-controlled and reviewable.
Administer the ClickHouse Cloud warehouse, including service configuration, user and role management, and monitoring of performance and capacity.
Host and maintain the team's self-hosted tools, including Apache Superset for BI and Apache Airflow for orchestration, covering deployment, upgrades, authentication and availability.
Contribute to architectural decisions on how the platform evolves, setting out options and trade-offs for review with the Director of Analytics.
Build and maintain ingestion pipelines from the platform's sources, including application databases, APIs and operational systems, using tools such as AWS Glue, Airbyte and AWS Lambda.
Implement incremental loading patterns that stay efficient and recoverable as volumes grow, landing raw data in the lake in a form that preserves its full history.
Develop and optimise Spark jobs that process billions of device events into learning sessions and higher-level aggregates, paying close attention to partitioning, skew and compute cost.
Bring in and model the programme context that gives event data its meaning, such as schools, terms, device allocations and enrolment, and handle how that context changes over time.
Design and maintain Iceberg tables for performance and cost, including partitioning strategy, file sizing, compaction and snapshot management.
Validate data at ingestion and surface quality issues early, working with the Analytics Team to trace them back to their source.
Contribute to evaluating new processing approaches as volumes grow, including moving more work upstream and assessing where streaming would add value.
Build and maintain Airflow workflows that coordinate ingestion, processing and transformation, with clear dependencies, retries and alerting.
Develop and maintain dbt staging and core models in ClickHouse, following consistent conventions for naming, structure and materialisation.
Implement incremental models for high-volume tables, choosing materialisation strategies and table designs that balance freshness, correctness and cost.
Write and maintain dbt tests and model contracts for core datasets, so that changes that would break downstream reporting are caught before they reach production.
Document core models such as learning sessions as agreed definitions, including their lineage, and make clear where a model applies an approximation rather than a direct transformation.
Work with the Senior BI & Data Analyst on the boundary between core and reporting models, and with the Data Scientist on datasets for research and evaluation.
Build and maintain CI/CD pipelines in GitHub Actions for infrastructure, pipeline and dbt code, including automated checks that stop unsafe changes from reaching production.
Implement monitoring and alerting across pipelines, the warehouse and hosted tools, so that failures and delays are visible quickly and reach the right people.
Respond to platform incidents and data issues, carry out root cause analysis, and contribute to the team's data quality incident process.
Apply and maintain access controls and security practices across the data lake, warehouse and BI layer, in line with Imagine's data governance policies.
Monitor platform costs across AWS and ClickHouse, track the cost and performance of individual workloads over time, and use that evidence to identify savings and prioritise optimisation work.
Maintain and periodically test backup and recovery procedures for the platform's core components.
Take part in code review and write clear runbooks and technical documentation, following the team's documentation standards, so that knowledge of the platform is shared across the team.

What we're looking for

A bachelor's degree in Computer Science, Software Engineering or a related field, or equivalent practical experience.
At least 7 years in data engineering, software engineering or platform engineering roles, including at least 3 years building and running distributed data processing (Spark or equivalent) in production.
A track record of taking responsibility for a production data platform, or major parts of one, including its reliability, performance and day-to-day operation.
Experience working with datasets in the hundreds of millions to billions of records, and designing processing that stays efficient as volumes grow.
Experience running data workloads within real budget constraints: making deliberate trade-offs between compute, storage and performance, and being able to show the effect of those decisions.
Experience managing the cloud infrastructure that data workloads run on, not only using managed services.
Experience in small teams where engineers cover a broad scope and work with limited supervision.
Experience working with remote teams across multiple countries and time zones.
Strong Python, written to production standards, with testing and modular design.
Advanced SQL, including writing and optimising queries against analytical databases.
Spark (PySpark), including performance tuning for partitioning, shuffles and data skew.
A solid understanding of data lake design, including columnar formats such as Parquet, partitioning strategy, and when to process in the lake rather than the warehouse.
Hands-on AWS experience with data services and the infrastructure around them, such as S3, Glue or EMR, IAM and EC2.
Infrastructure as code, using Terraform, CloudFormation, CDK or similar.
Linux administration and Docker, to the level needed to deploy and maintain self-hosted services.
Dbt in production, including incremental models and testing.
Workflow orchestration with Airflow, Dagster, Prefect or similar.
Git, CI/CD and code review practice.
Must be based in Africa.
Comfortable working independently in a virtual environment and coordinating with colleagues across global teams.

Nice to have

Apache Iceberg, or another open table format such as Delta Lake or Hudi.
ClickHouse, or a comparable columnar warehouse such as Snowflake, BigQuery or Redshift.
Terraform specifically.
Apache Airflow specifically.
AWS Glue and Lake Formation.
Data modelling for analytics, including handling reference data that changes over time.
Managed Spark platforms such as Databricks.
Streaming technologies such as Kafka, Flink, Kinesis or Spark Structured Streaming.
Event, telemetry or IoT data, particularly from devices that sync intermittently.
Ingestion tools such as Airbyte.
Administering BI tools such as Apache Superset.
Software development beyond data pipelines, such as building APIs or system integrations.
Experience in the non-profit, international development or social sector.

Your team

Analytics Team

A small team; the role reports to the Director of Analytics, who sets the platform's architecture and technical standards, and works alongside the rest of the Analytics Team.

Benefits

Salary is competitive and commensurate with experience.

Save

Direct apply at Imagine Worldwide

Opens the job posting at Imagine Worldwide

About Imagine Worldwide

Imagine Worldwide provides an innovative education technology solution and implementation model to the global literacy/numeracy learning crisis using the onebillion application and various toolkits and systems to support implementation. Its model is massively scalable, evidence-based, and can deliver foundational learning to millions of the most marginalized children.

Mission

Imagine Worldwide believes that every child should be empowered with the basic right to literacy and numeracy.

Key facts

Data comes from tablets in more than 1,500 schools across Malawi, Tanzania and Sierra Leone, recording learning activity as individual events adding up to billions of records.
The analytics platform runs on AWS and ClickHouse Cloud, with infrastructure managed as code.
The role collaborates with operations teams across Africa, the US, and the UK.