SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
iCapital is seeking an experienced Python developer to design and build advanced web scraping and automation solutions. The role focuses on architecting robust, secure, and scalable systems that interact with complex web applications and secured enterprise portals, particularly in the financial services domain.
Key responsibilities include designing and implementing scraping solutions using Python and Playwright for browser automation; automating interactions with JavaScript-heavy and authentication-secured websites, including handling multi-factor authentication (MFA), CAPTCHAs, and session/token-based login flows; architecting serverless scraping pipelines using AWS Lambda, Step Functions, S3, CloudWatch, and Secrets Manager; building fault-tolerant systems that scale to high-volume data extraction with intelligent retries and logging; integrating complex workflows across multiple portals, APIs, and data sources; contributing to architectural decisions and best practices; and conducting code reviews and managing deployments.
Required qualifications include a bachelor's or master's degree in computer science or related field (or equivalent professional experience); 10+ years of Python development experience with a strong focus on automation and data extraction; proven expertise in web scraping tools such as Playwright, Selenium, Scrapy, BeautifulSoup, and requests; in-depth experience handling multi-step authentication flows including MFA and session management; proficiency in deploying scraping workloads on AWS (Lambda, S3, IAM, CloudWatch, Secrets Manager); experience with asynchronous programming and headless browsers; solid understanding of web protocols (HTTP, HTTPS, cookies, headers) and ability to reverse-engineer network calls; comfort with APIs, JSON/XML, and data transformation; experience with CI/CD pipelines, Docker, and infrastructure-as-code tools (CloudFormation, Terraform); familiarity with data pipeline orchestration tools like Apache Airflow or AWS Step Functions; prior experience scraping from secure enterprise portals in financial, healthcare, or legal sectors; and exposure to Python testing frameworks.
The role operates on a hybrid schedule with four days in-office and one day remote (Friday). iCapital offers competitive salary, annual performance bonus, equity for full-time employees, 100% employer-paid health and dental insurance, and generous paid time off.
