We are looking for an experienced Data Engineer to join a leading and major financial institution in Israel. In this role, you will play a key part in developing and implementing advanced data-driven solutions for regulatory compliance and strategic risk management in the financial sector.
Role Overview:
As a Data Engineer, you will be responsible for designing and building scalable data pipelines, optimizing data migration processes, and contributing to the development of high-performance data solutions in a Data Lake environment. Your work will support critical business areas such as compliance, anti-money laundering, capital adequacy, and overall risk management initiatives.
Key Responsibilities:
- Develop and maintain data pipelines in a Data Lake environment, ensuring seamless data processing from source to destination.
- Design and implement data migration processes in relational databases and distributed systems.
- Develop and optimize Python scripts, with a strong focus on PySpark for big data processing.
- Write and optimize complex SQL queries for efficient data retrieval and manipulation.
- Conduct unit and integration testing to ensure the accuracy and reliability of data systems.
- Work with CI/CD pipelines to automate data solution deployments and ensure continuous delivery.
Required Skills and Experience:
- 2+ years of hands-on experience with Python, including work with PySpark.
- 2+ years of experience with Big Data technologies, specifically Spark and Hive.
- Proficiency in writing and optimizing complex SQL queries.
- 3+ years of experience with Microsoft Data technologies (SSIS, SQL Server).
- Strong understanding of DWH, DL, and ETL/ELT processes.
- Experience with CI/CD processes and automation of deployments.
- Familiarity with CTRLM or similar risk and compliance management tools.
- Experience with TERADATA databases – advantage.
- Familiarity with Qlik Sense and Qlikup – advantage.
- Understanding of the banking and financial services sector – advantage.
Key Competencies:
- Strong problem-solving and analytical skills, with a focus on handling complex data workflows.
- Proactive, independent, and organized, with excellent time management abilities.
- Eagerness to learn, adapt, and stay updated with the latest technologies.
- Excellent communication and collaboration skills, with the ability to work in cross-functional teams.
