SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Hims & Hers, a public healthcare platform (NYSE: HIMS), is seeking a Senior Staff Machine Learning Systems Engineer to own the end-to-end evaluation and data infrastructure for AI/ML models in a regulated healthcare environment. This is a leadership role spanning data pipelines, evaluation methodology, statistical rigor, and safety assurance for AI products deployed to patients.
You will set technical direction for evaluation systems including metric design, judge/scorer calibration, statistical regression gates, and failure tracking. You'll design and scale data pipelines for ingestion, transformation, dataset versioning, and labeling workflows that support both evaluation and downstream data science. The role involves leading multi-team initiatives to replace manual review processes with statistically sound automated gates, designing adversarial and red-team evaluation as a risk-reduction program, and architecting core platform improvements with org-wide impact.
Key responsibilities include: defining evaluation approaches for new AI services from scratch; owning adversarial testing and failure taxonomies to catch safety issues before patient deployment; resolving cross-team technical disagreements and driving alignment across engineering, product, and AI leadership; originating reusable methodologies and standards rather than one-off fixes; and mentoring engineers across the organization while raising the technical and statistical bar.
Required qualifications: 10+ years in ML infrastructure, data engineering, or evaluation/testing systems with demonstrated impact beyond a single team; hands-on expertise in LLM judge/scorer design, statistically sound regression testing (paired significance testing, multiple comparison correction), human label agreement measurement, and adversarial evaluation; deep experience with dataset versioning, feature pipelines, labeling workflows, and high-throughput data systems; proven track record of building standards adopted across teams; experience leading multi-team projects through technical disagreement; mentoring history including senior engineers; excellent communication across audiences; strong Python and statistical fluency for production-grade testing frameworks.
