Role Overview
The organization is seeking Python Web Scraping Engineers to design, build, and operate reliable web data collection systems. This role focuses on production-ready Python scrapers and browser automation, particularly across dynamic and JavaScript-heavy websites. The successful candidate will contribute to the reliability, scalability, and quality of Deep Future’s data pipelines.
Important Requirement: Previous AWS experience is essential for this role.
Key Responsibilities
- Design, develop, and maintain Python-based web scrapers.
- Build and maintain browser automation workflows using Playwright and Selenium.
- Extract data from dynamic, JavaScript-heavy websites.
- Maintain and improve scraping systems to handle routine target-site changes.
- Containerise scraping services using Docker.
- Deploy and operate scraping workloads on AWS.
- Monitor scraper performance, reliability, and data quality.
- Troubleshoot common failures, performance issues, and data collection inconsistencies.
Required Skills & Experience
- Working knowledge of AWS services such as EC2, ECS, Lambda, S3, and SQS.
- Commercial experience using Python in production environments.
- Experience building, maintaining, and improving web scrapers.
- Practical experience with Playwright and/or Selenium.
- Experience containerising applications with Docker.
- Ability to write clear documentation and communicate effectively within a team.
- Experience collaborating using tools such as Slack, Google Workspace, Jira, and Confluence.
Desirable / Nice to Have
- Exposure to sites protected by Cloudflare or similar web application firewalls.
- Familiarity with CAPTCHA and bot-detection challenges.
- Basic understanding of browser fingerprinting concepts.
- Experience with higher-volume or distributed scraping systems.
- Familiarity with monitoring and logging using AWS CloudWatch.
Organization Offers
- Ownership of meaningful parts of core data acquisition systems.
- Opportunity to work on real-world data intelligence problems.
- Support and mentorship from experienced engineers.
- Flexible working arrangements.
- Competitive remuneration aligned with experience.
Ideal Candidate Profile
The ideal candidate is a hands-on Python engineer with strong experience in production scraping, browser automation, AWS-based deployment, and reliable data pipeline operations. They should be comfortable working with changing web targets, debugging failures, documenting technical decisions, and collaborating closely with engineering teams to maintain high-quality data acquisition systems
