SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
iCapital is seeking an experienced Python Developer with deep expertise in web scraping, browser automation, and cloud-native deployment on AWS. This is a hybrid role (4 days in office, 1 day remote) based in Jaipur, India.
You will design and build robust, secure, and scalable scraping solutions that interact with complex web applications and secured enterprise portals. This involves architecting end-to-end data extraction pipelines using Python and Playwright for browser automation, with deployment on AWS serverless infrastructure.
Key responsibilities include:
- Designing and implementing advanced scraping solutions using Python, Playwright, and AWS services
- Automating interactions with JavaScript-heavy and authentication-secured websites, handling MFA, CAPTCHAs, and session/token-based login flows
- Architecting serverless scraping pipelines using AWS Lambda, Step Functions, S3, CloudWatch, and Secrets Manager
- Building fault-tolerant systems that scale to high-volume data extraction with intelligent logging and retry mechanisms
- Integrating complex workflows across multiple portals, APIs, and data sources
- Contributing to architectural decisions, tooling, and best practices
Required qualifications:
- 5+ years of Python development experience with focus on automation and data extraction
- Proven expertise in web scraping tools (Playwright, Selenium, Scrapy, BeautifulSoup, requests)
- In-depth experience with multi-step authentication flows, MFA, CAPTCHA solving, and session management
- Proficiency deploying scraping workloads on AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager)
- Experience with asynchronous programming, headless browsers, and JavaScript-rendered content
- Strong understanding of web protocols (HTTP/HTTPS, cookies, headers) and ability to reverse-engineer authentication mechanisms
- Comfortable working with APIs, JSON/XML, and data transformation
Preferred: CI/CD pipelines, Docker, infrastructure-as-code (CloudFormation/Terraform), data pipeline orchestration (Airflow/Step Functions), experience with secure enterprise portals (financial/healthcare/legal), data engineering/ETL background, Python testing frameworks.
Education: Bachelor's or Master's in Computer Science, Software Engineering, or equivalent professional experience.
