Job Description
Job Summary:
The Data Engineer is a core part of the Engineering team and is responsible for supporting the continued development and maintenance of our client's data models and pipelines. This role is a key part of building a world-class data platform that provides access to clean and efficient data across all teams.
Key Responsibilities
- Building data pipelines to move and transform data from various sources to storage systems as per ongoing business objectives and requirements.
- Developing and documenting solutions with clear, well-defined flowcharts, layouts, diagrams, and/or charts.
- Support and maintain any data platform failures or issues in the data systems by troubleshooting, proposing, and implementing solutions.
- Collaborate closely with other teams across the business, ensuring clear communication and understanding of business requirements.
- Continually staying up to date with the latest technologies and methodologies within data engineering to propose and implement innovative solutions.
- Identifying and implementing process improvements.
In addition to the above, a Senior Data Engineer also:
- Supporting the Lead Data Engineer with coaching and mentoring junior team members.
- Collaborating with analysts and stakeholders to understand data requirements
- Developing frameworks to support data systems and assist with extending functionality as our client continues to grow.
- Contributing to and supporting the strategic planning of data architecture as part of quarterly planning.
- Leading data projects end-to-end, ensuring ongoing collaboration with relevant stakeholders, and ensuring timelines are met.
- Promoting and presenting data engineering work both within the team and with external teams
- Implementing Data Quality and Validation checks to improve the reliability of the data platform, ensuring optimal performance of data systems
- Implementing and maintaining data security and compliance measures
Requirements:
- Knowledge and understanding of Data Warehousing concepts and methodologies
- Working knowledge of Python and SQL
- Experience and familiarity with data modelling techniques such as Star and Snowflake Schemas
- Experience with platforms such as Snowflake, dbt, and Airflow
- Familiarity with cloud platforms - Azure, AWS, GCP
- Experience with code version control tools like GitHub, GitLab
- Strong verbal and written communication skills to work effectively within cross-functional teams.
- Ideally, experience working within the financial services domain or similar.
- Experience working in an agile environment.
