Data Engineer

ESSENTIAL ROLES & RESPONSIBILITIES

• Identify and understand customer data-centric use cases within regulated financial services environments
• Design and implement data ingestion, processing, and transformation pipelines on Azure
• Build and maintain data pipelines for cleaning, normalisation, enrichment, and preparation
• Apply appropriate data modelling techniques and architecture patterns, with a strong focus on medallion architecture
• Orchestrate, monitor, and optimise Azure Databricks jobs and Azure Data Factory pipelines across development, UAT, and production environments
• Operationalise workflows developed by analysts and data scientists
• Support customers in adopting Azure data, analytics, and machine learning services

GOVERNANCE & REPORTING

• Maintain accurate documentation of data pipelines, schemas, transformations, and deployment processes
• Support data governance initiatives including lineage, metadata management, and access control

TECHNOLOGY STACK (AZURE)

Data Engineering & Analytics:

• Azure Databricks (development, UAT, and production)
• Azure Data Factory
• Azure Synapse Analytics (where applicable)

Databases:

• Microsoft SQL Server / Azure SQL Database (primary platforms)
• PostgreSQL (limited use)
• MySQL (limited use)

Security & Governance:

• Role-based access control (RBAC)
• Data encryption and key management
• Audit logging and monitoring

CRITICAL COMPETENCIES – TECHNICAL FIT

Essential:

• Strong SQL skills
• Programming experience with Python and/or Scala
• Hands-on experience with Azure-based data platforms
• Experience designing, building, and maintaining data pipelines
• Strong understanding of data modelling (relational and analytical), including medallion architecture
• Experience orchestrating and optimising Databricks and Data Factory workloads
• Experience using CI/CD pipelines for data and analytics solutions
• Strong awareness of security, networking best practices, GDPR, and PII handling

Desirable:

• Experience with Azure Databricks in production environments
• Familiarity with Azure Machine Learning and AI services

SHIFT & WORKING PATTERN

• Standard business hours, with participation in an on-call rota as required
• Occasional weekend engineering coverage will be required, typically limited to a small number of planned weekends per year to support business continuity, resilience testing, or disaster recovery activities