The Role: We are looking for an experienced
Data Engineer with expertise in
Snowflake and
DBT. You will design and optimize data models, implement DBT workflows, ensure data quality, and build scalable pipelines to make data reliable and accessible for stakeholders.
Key Responsibilities
- Design, develop, and implement scalable and efficient data pipelines using Apache Spark, Kafka, and AWS Kinesis for real-time and batch data processing.
- Optimize and maintain data pipelines to ensure timely delivery of data for critical business needs.
- Build, maintain, and optimize data models in data warehouse systems (Snowflake, AWS Redshift), ensuring high data quality and consistency.
- Leverage DBT for data transformations and modeling workflows.
- Develop and implement data cleansing and transformation routines to convert raw data into actionable insights.
- Collaborate with data scientists, analysts, and business stakeholders to translate data requirements into robust technical solutions.
- Monitor and maintain data infrastructure health, troubleshoot issues, and enforce data security measures.
- Automate routine tasks and implement DevOps practices to streamline data engineering processes.
- Document data pipelines, data models, and processes for future reference and knowledge sharing.
- Stay up to date with the latest trends and technologies in data engineering, proactively suggesting improvements to the data infrastructure.
Requirements
- At least 3+ years of hands-on experience with Snowflake and DBT.
- Strong knowledge of data modeling techniques and best practices.
- Excellent communication and collaboration skills, with the ability to work effectively with both technical and non-technical teams.
- Strong problem-solving and analytical skills to troubleshoot complex data issues.
- Passion for continuous learning and staying updated with emerging technologies.
Technical Expertise
- Data Warehousing: Expertise in Snowflake and/or AWS Redshift.
- DBT: Proven experience in implementing and managing transformation workflows.
- Streaming Services: Proficient in Kafka, AWS Kinesis, or similar streaming technologies.
- Big Data Processing: Strong understanding of Apache Spark and its ecosystem (Spark SQL, DataFrames, Spark Streaming, PySpark).
- Programming Languages: Proficiency in Python, Scala, or Java.
- Cloud Technologies: Familiarity with AWS or Azure is a plus..
Bonus Skills
- Experience with CI/CD pipelines and DevOps practices.
- Knowledge of containerization technologies like Docker and Kubernetes.
