Data Engineer - AWS - SC Cleared
other jobs SR
Added before 4 Days
- England,London,City of London
- Full Time, Contract
- £450 - £500 per day
Job Description:
Full job descriptionData Engineer - AWS - SC / NPPV3 Cleared
We are seeking an experienced Data Engineer to support the delivery of an AWS-based data solution within a UK Government programme. The role involves building scalable data pipelines, transforming complex datasets, and implementing data-matching and record-linkage solutions.
You will work closely with the Technical Lead and Lead Data Scientist to deliver reliable, production-ready data solutions.
Key Responsibilities*Design, develop and maintain scalable ETL/ELT pipelines using AWS Glue, Redshift, Python and SQL.
*Ingest, transform and validate data from multiple sources and formats.
*Implement data matching, deduplication and record-linkage processes.
*Develop automated testing, monitoring and data quality controls.
*Optimise data pipelines for performance, scalability and cost.
*Apply engineering best practices, including Git, CI/CD and secure development.
*Collaborate with technical teams and stakeholders within an Agile environment.
Essential Skills & Experience*Active SC clearance (NPPV3 clearance desirable)
*Strong hands-on experience with AWS Glue, Amazon Redshift, Python and SQL.
*Proven experience building production-grade data pipelines.
*Strong understanding of ETL/ELT, data modelling and data quality.
*Experience with data matching, record linkage or complex data transformations.
*Experience with automated testing, Git and CI/CD.
*Experience working within Agile/Scrum delivery teams.
We are seeking an experienced Data Engineer to support the delivery of an AWS-based data solution within a UK Government programme. The role involves building scalable data pipelines, transforming complex datasets, and implementing data-matching and record-linkage solutions.
You will work closely with the Technical Lead and Lead Data Scientist to deliver reliable, production-ready data solutions.
Key Responsibilities*Design, develop and maintain scalable ETL/ELT pipelines using AWS Glue, Redshift, Python and SQL.
*Ingest, transform and validate data from multiple sources and formats.
*Implement data matching, deduplication and record-linkage processes.
*Develop automated testing, monitoring and data quality controls.
*Optimise data pipelines for performance, scalability and cost.
*Apply engineering best practices, including Git, CI/CD and secure development.
*Collaborate with technical teams and stakeholders within an Agile environment.
Essential Skills & Experience*Active SC clearance (NPPV3 clearance desirable)
*Strong hands-on experience with AWS Glue, Amazon Redshift, Python and SQL.
*Proven experience building production-grade data pipelines.
*Strong understanding of ETL/ELT, data modelling and data quality.
*Experience with data matching, record linkage or complex data transformations.
*Experience with automated testing, Git and CI/CD.
*Experience working within Agile/Scrum delivery teams.
Job number 4351406
The jobs on site are for both men and women