Senior Data Engineer
other jobs Hays Specialist Recruitment Limited
Added before 7 Days
- Wales,Cardiff
- Full Time, Contract
- £500 per day
Job Description:
Contract Length: Initial 6 months (extension likely)
Day Rate: £500 per day (Outside IR35)
Location: Hybrid in South Wales
Role Overview
We are seeking an experienced Senior Data Engineer contractor to support and extend the Azure Databricks Lakehouse platform, ensuring the reliable delivery of new data pipelines alongside ongoing operational stability.This is a hands-on engineering role focused on implementation, optimisation and support of Delta Lake pipelines across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework.
Key Responsibilities
*Develop and maintain data pipelines in Azure Databricks using PySpark and SQL
*Build incremental processing using Delta Lake and Change Data Feed
*Extend and maintain watermark-driven CDC frameworks
*Implement and maintain Slowly Changing Dimension (SCD Type 2) logic
*Engineer time-aware joins and dimensional transformations
*Support optimisation of Spark workloads and Delta tables
*Solid understanding of dimensional modelling and star schema
Medallion Architecture Delivery
*Implement ingestion patterns into Bronze
*Build Silver state, CDC and SCD2 tables
*Develop Gold dimensional models aligned to Kimball principles.
*Maintain clean separation of Bronze, Silver and Gold usage and responsibilities
Delta Lake & Performance Optimisation
*Work extensively with Delta Lake features including: *ACID transactions
*Merge-based upserts
*Partitioning and data layout strategies
*Tune Spark jobs and clusters to improve performance and cost efficiency
What you need to do now
If you’re interested in this role, click ’apply now’ to forward an up-to-date copy of your CV, or call us now.
If this job isn’t quite right for you, but you are looking for a new position, please contact us for a confidential discussion about your career.
Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept the T&C’s, Privacy Policy and Disclaimers which can be found at hays.co.uk
Day Rate: £500 per day (Outside IR35)
Location: Hybrid in South Wales
Role Overview
We are seeking an experienced Senior Data Engineer contractor to support and extend the Azure Databricks Lakehouse platform, ensuring the reliable delivery of new data pipelines alongside ongoing operational stability.This is a hands-on engineering role focused on implementation, optimisation and support of Delta Lake pipelines across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework.
Key Responsibilities
*Develop and maintain data pipelines in Azure Databricks using PySpark and SQL
*Build incremental processing using Delta Lake and Change Data Feed
*Extend and maintain watermark-driven CDC frameworks
*Implement and maintain Slowly Changing Dimension (SCD Type 2) logic
*Engineer time-aware joins and dimensional transformations
*Support optimisation of Spark workloads and Delta tables
*Solid understanding of dimensional modelling and star schema
Medallion Architecture Delivery
*Implement ingestion patterns into Bronze
*Build Silver state, CDC and SCD2 tables
*Develop Gold dimensional models aligned to Kimball principles.
*Maintain clean separation of Bronze, Silver and Gold usage and responsibilities
Delta Lake & Performance Optimisation
*Work extensively with Delta Lake features including: *ACID transactions
*Merge-based upserts
*Partitioning and data layout strategies
*Tune Spark jobs and clusters to improve performance and cost efficiency
What you need to do now
If you’re interested in this role, click ’apply now’ to forward an up-to-date copy of your CV, or call us now.
If this job isn’t quite right for you, but you are looking for a new position, please contact us for a confidential discussion about your career.
Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept the T&C’s, Privacy Policy and Disclaimers which can be found at hays.co.uk
Job number 3448658
Increase your exposure to recruiters with ProJobs
Thousands of recruiters are looking for you in the Job Master profile database, increase your exposure 4 times with a ProJob subscription
You can cancel your subscription at any time.
metapel
Company Details:
Hays Specialist Recruitment Limited
At Hays we invest in lifelong partnerships that empower people and businesses to succeed. With over 50 years? success under our belts and a workforce ...