Data Engineer

Responsibilities- Design, develop, and maintain ETL pipelines using PySpark, ensuring data quality and performance- Utilize AWS services (EMR, Glue) for data processing and storage optimization- Implement dimensional modeling and manage data lakehouse solutions- Develop and maintain CI / CD pipelines with AWS CDK and GitHub Actions- Actively participate in SCRUM processes and use JIRA for task management- Work with stakeholders to understand data requirements and deliver solutions- Mentor junior engineers and stay updated on industry trendsRequirements- Experience with PySpark- At least 7 years of total experience in building ETL pipelines, with preference given to experience in Spark (Scala / Python)- At least 5 years of experience with Amazon AWS Data & Analytics Toolset (EMR / Glue) OR Databricks, with preference for AWS Glue- Strong understanding of Data Lakehouse / Data warehouse principles, such as Dimensional Modelling- A working knowledge of Continuous Integration / Continuous Deployment practices is required, with practical experience in DevOps principles, for example, AWS CDK and GitHub Actions- Experience with the agile SCRUM delivery model, along with proficiency in JIRA- Strong interpersonal skills are essentialUseful links