Senior Databricks Specialist / Data Architect

About the role

We are seeking a Senior Databricks Specialist / Data Architect to lead the design, implementation and optimisation of an enterprise‑scale data platform on Microsoft Azure. The role will own the end‑to‑end data architecture, integration with Dynamics 365 CRM and other systems, and ensure high performance, security and governance.

Key responsibilities

• Design and implement enterprise‑scale data solutions using Azure Databricks.
• Define and maintain Databricks architecture, standards and Medallion (Bronze‑Silver‑Gold) design.
• Build scalable ETL/ELT pipelines with Apache Spark, PySpark and Delta Lake.
• Integrate Databricks with Microsoft Dynamics 365 / Dataverse and other enterprise applications.
• Implement data quality, monitoring, lineage and governance using Unity Catalog.
• Optimise Spark jobs, clusters, SQL queries and Delta tables for performance and cost.
• Support Power BI and other reporting tools with curated data models.
• Establish CI/CD, DevOps and version‑control practices for Databricks solutions.
• Mentor data engineers and conduct technical reviews.

Required profile

• Minimum 8 years of experience in data engineering or data architecture.
• At least 4 years of hands‑on experience with Azure Databricks.
• Proven track record designing enterprise data platforms and pipelines.
• Strong understanding of data governance, security and disaster‑recovery requirements.

Required skills

• Azure Databricks
• Apache Spark / PySpark
• Delta Lake
• Unity Catalog
• SQL (large‑scale processing)
• Azure Data Factory
• Azure Data Lake Storage Gen2
• Microsoft Dynamics 365 / Dataverse integration
• Power BI
• CI/CD, Git, Azure DevOps