Senior Databricks Specialist / Data Architect

About the role

A company based in Qatar is looking for a Senior Databricks Specialist / Data Architect with 5+ years of experience and Arabic as a mandatory requirement. The role covers the architecture, development, optimization, governance, integration and performance of the organization's Databricks data platform and its integration with Dynamics 365 CRM and other enterprise systems.

Key responsibilities

• Design and implement enterprise-scale data solutions on Azure Databricks, including scalable ETL/ELT pipelines and Medallion Architecture (Bronze, Silver, Gold)
• Integrate Databricks with Microsoft Dynamics 365 / Dataverse and other applications
• Build batch and near-real-time pipelines, and optimize Spark jobs, clusters, SQL queries and Delta tables for performance and cost
• Implement data quality, validation, monitoring, lineage and governance, including Unity Catalog
• Support Power BI and other reporting platforms, and establish development, testing and production standards
• Support CI/CD and DevOps practices, conduct technical reviews and mentor data engineers

Required profile

• Minimum 8+ years of experience in data engineering or data architecture, with at least 4 years of hands-on Azure Databricks work
• Proven experience in production enterprise environments, able to design, develop, troubleshoot, optimize and govern Databricks solutions independently
• Strong SQL skills and experience with large-volume enterprise datasets
• Preferred: Databricks Certified Data Engineer Professional or Associate, and Microsoft Certified: Azure Data Engineer Associate

Required skills

• Azure Databricks, Apache Spark and PySpark
• Delta Lake and Medallion Architecture
• Unity Catalog, data governance and lineage
• Azure Data Factory and Azure Data Lake Storage Gen2
• Data modeling, dimensional modeling and data quality
• CI/CD, Git, Azure DevOps and deployment automation