SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Health Note is seeking an AI Operations Specialist to own customer AI implementations end-to-end, from discovery and configuration through launch and ongoing optimization. This is a hands-on, customer-facing technical individual contributor role.
You will be responsible for translating customer workflows and requirements into clear AI behavior specifications and implementation plans. You'll design and refine prompts and workflow logic for customer-facing AI agents, build and execute evaluations and test plans to improve reliability and consistency, and investigate production issues across prompts, workflows, integrations, data, APIs, and application behavior.
Key responsibilities include:
- Owning customer AI implementations from discovery through launch and ongoing optimization
- Translating customer workflows into AI behavior, implementation plans, and testing criteria
- Designing and improving prompts and workflow logic for AI agents
- Building evaluations, test plans, and regression checks to improve reliability
- Investigating production issues and determining whether they can be resolved directly or require escalation to product or engineering
- Facilitating customer testing and troubleshooting with clear communication of status, risks, and blockers
- Driving open issues to resolution across customers and internal teams
- Using recurring customer and production issues to improve implementation practices and product behavior
This role requires comfort moving fluidly between AI behavior analysis, implementation details, customer workflows, and production troubleshooting. You will work directly with customers and internal teams to ensure implementations work reliably in real-world environments.
Benefits include health, dental, and vision insurance; 401(k) with company match; flexible PTO and company-paid holidays; fully remote work within US hours; and company-provided laptop.
Requirements:
- Hands-on experience working with LLM-powered products or workflows in production
- Experience building evaluations and improving unreliable or nondeterministic AI behavior
- Strong technical troubleshooting skills and ability to investigate complex systems independently
- Experience owning customer implementations from discovery through launch and optimization, including direct customer engagement throughout the process
- Experience with healthcare technology and/or conversational or voice AI is helpful but not required
