Full job descriptionSenior Data Engineer & Technical Delivery Lead (Python / Databricks)
Location: London (Hybrid – 2-3 days onsite)
Contract: Initial 6 months


We’re currently supporting a major Investment Bank that’s expanding its Data Engineering function and is looking to hire multiple contractors across Senior Data Engineer and Technical Delivery Lead level.
Whether you’re a highly technical Senior Data Engineer looking to remain hands-on, or an experienced Technical Lead who enjoys combining delivery leadership with coding, we’d be keen to speak with you.
You’ll be joining a large-scale cloud data transformation programme, helping design, build and optimise enterprise data platforms using Python, Databricks and Spark within a modern Lakehouse environment.


Key Responsibilities
*Design, develop and maintain scalable data pipelines using Python and Databricks.
*Build, optimise and support enterprise ETL/ELT workflows using Apache Spark and Delta Lake.
*Design and develop robust data models and Lakehouse architectures.
*Implement and manage data workflows within the Databricks ecosystem.
*Ensure high levels of data quality, governance and pipeline reliability.
*Optimise performance across large-scale distributed data processing platforms.
*Collaborate with architects, product owners, analysts and engineering teams to deliver high-quality data solutions.
*Implement monitoring, logging and alerting across critical data platforms.
*Drive engineering best practice through code reviews, mentoring and technical collaboration.
*For Technical Delivery Lead opportunities, provide technical leadership, architectural oversight and end-to-end delivery of complex data engineering initiatives.


Required Skills & Experience
*Strong commercial experience developing data engineering solutions using Python.
*Extensive hands-on experience with Databricks, including Workflows, Notebooks and Delta Lake.
*Strong experience with Apache Spark / PySpark.
*Proven experience building scalable ETL/ELT pipelines.
*Strong SQL and data modelling skills.
*Experience designing modern Lakehouse architectures.
*Experience working with cloud platforms including AWS, Azure or GCP.
*Experience with modern data warehousing technologies such as Snowflake, Redshift or BigQuery.
*Experience with orchestration tools such as Airflow or Databricks Workflows.
*Previous experience working within Agile delivery environments.
Desirable Experience
*Databricks certifications.
*Kafka or Structured Streaming.
*CI/CD and DevOps practices.
*Docker and Kubernetes.
*Exposure to Machine Learning pipelines.
*Experience within Financial Services or other highly regulated environments.


About You
You’ll be an experienced Data Engineer or Technical Lead with a passion for building modern, scalable data platforms. Comfortable working in a collaborative Agile environment, you’ll enjoy solving complex technical problems and delivering high-quality engineering solutions. If you’re applying at Technical Delivery Lead level, you’ll also bring strong leadership, stakeholder management and mentoring experience whilst remaining hands-on technically.