SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 170,000 - 210,000 / annual
Health Note is building AI-powered digital assistants that function as trusted members of the care team, handling workflows from simple reschedules to complex intake, referrals, medication refills, and patient access management. The company aims to deliver a concierge experience for patients while providing clinics with reliable, transparent, and continuously improving automation.
As a Senior AI Product Engineer, you will own improvements to the Access Agent—Health Note's core AI system—from investigation through production rollout and measurement. This is a product engineering role focused on making AI systems measurably better in production, not pure ML research or dedicated platform work.
Key responsibilities include: building practical measurement systems (evals, dashboards, review workflows, logs, production metrics) to understand agent performance; analyzing transfer patterns, containment rates, task success, and operational outcomes to identify improvements; connecting prompts, configurations, interventions, customer workflows, and outcomes into coherent systems; creating regression-detection checks for pre- and post-deployment validation; partnering with Product, Operations, and Engineering to prioritize improvements; designing lightweight experiments for model, STT, prompt, workflow, and configuration changes; and building visibility through dashboards and analytics.
You will use AI-assisted development tools and agentic workflows to accelerate experimentation and delivery. Success is measured by whether the systems you build make the Access Agent more reliable, measurable, and effective in production.
Required: 5+ years shipping production systems; experience deploying agentic systems and measuring their effectiveness; strong analytical skills translating ambiguous problems into measurable outcomes; ability to use metrics, logs, qualitative review, or experiments to validate changes; strong product instincts connecting technical decisions to business outcomes; demonstrated ability to independently own initiatives from problem identification through impact; experience with modern AI-assisted engineering workflows (Claude, Cursor, Codex, or similar).
Nice-to-haves: eval framework experience; familiarity with Langfuse, observability platforms, analytics, or experimentation tools; RAG, agent orchestration, prompt management, or workflow automation experience; healthcare technology or regulated-industry background; startup experience with ambiguity.
