SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Heidi Health is building an AI Care Partner platform for clinicians, currently supporting 2.8 million patient sessions weekly across 190+ countries in 110 languages and 200+ specialties. The company has grown ARR from $1M to $50M in two years and recently secured $340M in funding (Series C led by Blackbird plus $240M growth investment from General Catalyst).
As a Clinical AI Engineer, you will join the Clinical AI R&D team within a product pod, bridging the gap between clinical practice and AI engineering. You will own the evidence generation for product features: designing rigorous evaluations to academic standards before development begins, building and validating LLM-based judges and rubrics at scale, executing evaluations with proper statistical analysis, and publishing findings in peer-reviewed literature. You will also conduct post-market surveillance, monitoring output quality in production and responding to regulatory feedback with data.
Key responsibilities include:
- Designing evaluations for new products with academic rigor: defining endpoints, comparators, ground truth, sample size, and analysis plans
- Building, optimizing, and validating LLM judges and rubrics that score product outputs at scale, measuring agreement with clinician ground truth
- Executing evaluations, performing statistical analysis, and writing reports for the product pod, regulatory files, and academic submissions
- Running post-market surveillance to monitor quality in production and working with the clinical safety team to interpret findings and drive product improvements
- Making code changes in the pod's codebase (Python/TypeScript) through normal review and release processes
- Contributing to peer-reviewed publications on clinical AI translational benefit
- Serving as the clinical expert in the room, catching clinically incorrect outputs and maintaining clinical nuance in product decisions
The role requires deep medical knowledge combined with engineering capability and research rigor. You will work with light direction on multi-week projects, maintain clear communication through reports and manuscripts, and stay with products throughout their market lifecycle.
Heidi operates on a philosophy of building for the next decade, following evidence relentlessly, owning outcomes end-to-end, and shipping iteratively. The team is composed of clinicians, engineers, builders, and designers focused on real-world healthcare impact.
Requirements:
- Medical degree with clinical practice experience and independent clinical judgment (non-negotiable)
- Academic depth in study design, inferential statistics, agreement/reliability measures, sample size calculation, and multiple comparisons handling; thesis or publication-level work required
- Experience evaluating medical software or clinical AI, including building or validating LLM-as-judge pipelines
- Engineering capability to work in production codebases: read code, understand changes, write in Python or TypeScript, and pass code review
- Independence to run projects for weeks with minimal direction and communicate when stuck
- Strong written communication for evaluation reports, surveillance summaries, and manuscripts
Nice to have:
- Post-market surveillance or vigilance experience under medical device frameworks (TGA, MHRA, FDA, EU MDR)
- Peer-reviewed publications on clinical AI or LLM evaluation
- Machine learning background beyond prompting (fine-tuning, classical model evaluation)
- Specialty depth in areas Heidi is expanding into
- Academic or clinical-site networks able to host studies
