Data Platform Engineer – Cloud

Senior Data Platform Engineer – Cloud Platform Migration

Location – Hybrid Leeds of London 3 days

Drive a major cloud data modernisation programme, migrating legacy Databricks pipelines and workloads to a modern AWS infrastructure. Working closely with Architects and stakeholders, you will build automated, large-scale data solutions to secure and transform critical healthcare data assets. Requirement: (SC Clearance or eligibility is mandatory).

Key Responsibilities

Data Engineering & Migration

Support migration of data pipelines, notebooks, and data processing workloads between cloud environments.

Analyse existing data engineering solutions and identify migration requirements.

Develop and maintain data processing pipelines using modern cloud data platform technologies.

Support testing, validation, and reconciliation of migrated datasets.

Work with structured and semi-structured data stored in cloud environments.

Troubleshoot and resolve data processing issues across development, test, and production environments.

Cloud Data Platform Engineering

Support the operation and enhancement of cloud-based data platforms.

Assist with onboarding datasets, users, and workloads onto target platforms.

Collaborate with platform and architecture teams to implement scalable data solutions.

Support platform optimisation, monitoring, and operational readiness activities.

Infrastructure as Code

Develop and maintain Terraform code supporting data platform deployments.

Support deployment of data platform components, jobs, permissions, and environment configurations.

Assist with automation of deployment and release processes.

Maintain infrastructure consistency across environments.

Platform Modernisation

Support cloud migration initiatives and platform transformation programmes.

Contribute to platform standards, engineering practices, and reusable deployment patterns.

Produce technical documentation and implementation guides.

Essential Skills

Data Engineering

• Python
• PySpark
• SQL
• Data Pipeline Development
• ETL / ELT Design
• Data Modelling fundamentals
• Data Quality and Validation

Cloud Platforms

• AWS
• S3
• IAM
• Cloud-based Data Platforms

Infrastructure as Code

• Terraform
• Infrastructure Automation
• Git

Engineering Practices

• CI/CD concepts
• Agile Delivery
• DevOps principles

Desirable Skills

• Databricks
• Palantir Foundry
• Foundry Pipeline Builder
• Ontology concepts
• Delta Lake
• Healthcare or NHS data experience
• Secure Data Environment (SDE) experience
• Cloud migration programmes

Experience

5+ years of Data Engineering experience.

Experience delivering cloud-based data platforms.

Experience supporting migration of data workloads between environments.

Experience working with Infrastructure as Code using Terraform.

Experience working within regulated or enterprise-scale environments.

Role Senior Data Platform Engineer – Cloud Platform Migration

Type – Hybrid Leeds or London – 3 days a week £70,000 to £80,000 ++ depending on experience

Candidates must be SC Clearance or have SC eligibility is mandatory

SC eligibility - Security Check (SC) clearance in the UK, candidates cannot apply independently; the process must be sponsored by an employer or government department. You must be aa UK citizen or have the right to work in the UK and have resided in the UK for at least 5 years.