Databricks Solution Architect in London, Greater London

Posted by GIS Recruiter Ltd

£90,000 – £120,000 a year

London · Permanent

Consultancy client are looking to recruit a Databricks Solution Architect to define and deliver enterprise-scale data platform architectures using Azure Databricks.

Candidates must possess deep hands-on Databricks engineering knowledge alongside experience designing secure, scalable and highly available data platforms for UK Public Sector organisations. This is a technical architecture role requiring the ability to move between architecture design, technical implementation and engineering leadership.

Essential Technical Skills

  • 5+ years of hands-on Databricks platform experience.
  • Extensive Azure Databricks architecture experience.
  • Expert-level PySpark, Spark SQL and Python knowledge.
  • Deep understanding of Apache Spark processing architecture.
  • Extensive Delta Lake and Lakehouse architecture experience.
  • Strong experience designing Medallion Architecture.
  • Expert knowledge of Unity Catalog, governance, permissions and data lineage.
  • Experience designing Databricks workspaces, clusters, compute policies and networking.
  • Knowledge of private endpoints, managed identities, Key Vault and secure connectivity.
  • Experience with Azure Data Lake Storage Gen2, Azure Data Factory and Azure services.

Public Sector Experience

  • Extensive experience delivering enterprise data architectures
  • Understanding of UK Government security principles and data governance requirements.
  • Experience architecting platforms handling sensitive and restricted government data.
  • Knowledge of UK GDPR, Data Protection Act 2018 and relevant government security standards.
  • Experience producing architecture documentation, HLDs, LLDs and technical design authorities.
  • Experience working with public sector enterprise architecture and governance boards.

Security Clearance Requirements

  • Be eligible and willing to undergo SC clearance
  • Previous experience working on sensitive government projects is advantageous.

Responsibilities

  • Own the end-to-end Databricks solution architecture.
  • Define scalable and secure Lakehouse architecture.
  • Establish engineering and architecture standards.
  • Design data ingestion, transformation and consumption patterns.
  • Define Unity Catalog and enterprise data governance models.
  • Provide technical direction to data engineering teams.
Apply Now →

Application opens at the source listing. Free for jobseekers.

Apply Now →

Similar jobs

Application Support Developer

ISR RECRUITMENT LIMITED

£43,000 – £47,000 a year

Apply

Northwich · Permanent · 2 hours ago

Position: Senior Application Support Developer. Location: Northwich, Cheshire. Contract: 12-month Fixed-Term Contract. Salary:..

Business Excellence & Improvement Lead

CWA

Pay not stated

Apply

Ipswich · Permanent · 6 hours ago

Help Shape the UK's Energy Future. Business Excellence & Improvement Lead Opportunity with the Civil Works Alliance (CWA),..

Information Security Team Leader

Lenacre Group Ltd

£78,000 a year

Apply

Canary Wharf · Permanent · 8 hours ago

Information Security Team Leader – Hybrid, Canary Wharf, London – to £78,000 plus13% discretionary bonus and salary review..

Business Development Executive

MMC Workforce Solutions

£65,000 – £85,000 a year

Apply

Nationwide · Permanent · 8 hours ago

Business Development Manager – Civil Engineering. Location: Head Office North West. UK based, with travel to clients and sites..