The Ideal Candidate
We are looking for a pragmatic, production-focused individual to join our Data team.
What you will be doing
- Design, develop, and support data pipelines for a hybrid data-platform (mix of batch and real-time data processing).
- Monitor, troubleshoot, and resolve issues in production data pipelines, ensuring high availability and reliability.
- Hands-on approach with Data Governance, including SLA metric capture, anomaly detection and reporting to key stakeholders, and a commitment to ensuring optimum data quality.
- Ownership of several key data platform infrastructure components.
- Actively research, evaluate, and prototype new technologies, especially in support of the department’s new AI/ML initiatives.
- Actively participate in the design, scoping, and architectural discussions for new data solutions and features.
- Translate business requirements and product goals into scalable and efficient technical data solutions.
Qualifications and Experience Required
- Hold a Bachelor’s Degree in Computer Science or equivalent.
- 3+ years experience in any of the following OOP programming languages (Python, Java, Go).
- Experience with Clickhouse and similar columnar technologies. Possess a strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures. A deep understanding of Clickhouse inner workings is desirable.
- Experience in RDBMS databases (Aurora, MariaDB, MySQL), SQL scripting skills and SQL optimization capabilities.
- Experience with streaming practices and technologies (Kafka, Kafka Connect, Kafka Streams)
- Experience with Apache Airflow for data orchestration and pipelining.
- Exposure to Prometheus and experience working with Grafana based dashboards.
- Exposure within microservice orchestration systems (Docker, Kubernetes).
- Experience with developing cloud native applications on AWS (EC2, Lambda, S3, Serverless, MWAA, MSK).
Skills Required
- Have a data-ops driven mindset, and exposure to hands-on on-prem deployments.
- Functional programming-oriented mindset in real time streaming architectures. Familiarity with distributed processing engines (e.g., Spark, Flink, Dask, or Presto/Trino).
- Unix scripting knowledge and capabilities.
- Fluency in English, both written and spoken.
Start Date: As soon as we find a match!
