SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 150,000 - 180,000 / annual
iCapital seeks an experienced Python developer to design and build advanced web scraping and automation solutions. The role focuses on architecting robust, secure, and scalable systems that interact with complex web applications and secured financial portals using Python, Playwright, and AWS cloud services.
Key responsibilities include designing and implementing sophisticated scraping solutions that handle JavaScript-heavy websites, multi-factor authentication (MFA), CAPTCHAs, and session/token-based login flows. You will architect serverless scraping pipelines using AWS Lambda, Step Functions, S3, CloudWatch, and Secrets Manager, building systems that scale to high-volume data extraction with fault tolerance, intelligent retries, and comprehensive logging. The role involves integrating and managing complex workflows across multiple portals, APIs, and data sources while contributing to architectural decisions, tooling, and best practices.
Required qualifications include a bachelor's or master's degree in computer science or related field (or equivalent professional experience) and 10+ years of Python development with strong focus on automation and data extraction. You must have proven expertise in web scraping tools (Playwright, Selenium, Scrapy, BeautifulSoup, requests) and in-depth experience handling multi-step authentication flows including MFA and CAPTCHA solving. Proficiency in deploying and managing scraping workloads in AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager) is essential, along with experience in asynchronous programming, headless browsers, and JavaScript-rendered content.
Additional requirements include solid understanding of web protocols (HTTP, HTTPS, cookies, headers), ability to reverse-engineer network calls and authentication mechanisms, comfort with APIs and data transformation (JSON/XML), experience with CI/CD pipelines, Docker, and infrastructure-as-code tools (CloudFormation, Terraform), and familiarity with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions. Prior experience scraping from secure enterprise-level portals in financial, healthcare, or legal sectors is highly valued. Background in data engineering or ETL workflows and exposure to Python testing frameworks are pluses.
The role is hybrid, requiring in-office presence Monday-Thursday with remote flexibility on Fridays. iCapital offers competitive compensation including equity for all full-time employees, annual performance bonuses, employer-matched retirement plans, comprehensive healthcare coverage (100% employer-paid dental and vision), telemedicine, virtual mental health counseling, parental leave, and unlimited PTO.
