Sam Pyzer
Associate DirectorJob Description
For our customer we are looking for a Data Engineer to migrate SAS workloads to the Databricks Lakehouse Platform. You will translate legacy SAS data processing into PySpark and SparkR, design scalable pipelines, and optimize Delta Lake tables for performance and reliability. Collaborate with analysts and data scientists to ensure functional equivalence between SAS and Databricks implementations, implement CI/CD with Git and Azure DevOps, and validate data through automated testing and data quality checks.
Key Responsibilities
Migrate SAS workloads to the Databricks Lakehouse Platform.
Translate legacy SAS data processing into PySpark and SparkR
Design scalable pipelines
Optimize Delta Lake tables for performance and reliability
Collaborate with analysts and data scientists to ensure functional equivalence between SAS and Databricks implementations
Implement CI/CD with Git and Azure DevOps, and validate data through automated testing and data quality checks.
Required Experience Mandatory:
Databricks
Good experience to have:
PySpark
SAS, Basics
ETL
SQL
Additional Information
Language requirements:h
Finnish mandotary
English
Start date: As soon as possible after candidate is choosen
By applying for this role, you consent to SGI contacting you by telephone regarding recruitment services, market updates, and relevant business opportunities
We believe in equal opportunity for all and actively encourage applications from diverse backgrounds, experiences, and perspectives.
Source Group International Ltd is acting as an Employment Business in relation to this vacancy