SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 193,400 - 290,000 / annual
Harvey is building an AI platform for legal and professional services, combining frontier agentic AI with enterprise-grade infrastructure. As a Staff Software Engineer on the Model Infrastructure team, you'll lead the design and development of systems powering every AI request at Harvey.
You'll own the full stack of model infrastructure, including:
**Model Reliability & Operations**: Build health monitoring, automated failover and recovery, capacity provisioning, and operational tooling to ensure production AI systems run flawlessly.
**Unified Model Controller (UMC)**: Design policy-based model routing, intelligent model selection, traffic management, and reliability/latency optimization across multiple AI providers.
**Provider Platform**: Architect multi-provider integrations with OpenAI, Anthropic, Azure OpenAI, Fireworks, Baseten, and emerging providers. Enable rapid adoption of frontier models through clean APIs and SDKs.
**Observability & Cost Platform**: Build token usage analytics, cost attribution, latency dashboards, capacity forecasting, and utilization optimization to drive efficiency.
**AI Platform Foundation**: Develop infrastructure for model evaluation, deployment, operations, and future training platforms. Build agent infrastructure and CaaS capabilities.
You'll partner closely with AI Research, Product Engineering, Infrastructure, and external model providers. Key responsibilities include leading technical design, ensuring high availability and low latency, improving observability, optimizing infrastructure costs, and mentoring engineers across the organization.
Harvey operates with intensity and decisiveness, moving fast while maintaining high standards. The company has strong product-market fit, world-class investor backing, and is scaling rapidly in a newly defined category.
About Harvey
Legal / Compliance / Risk; AI / Data / Infrastructure — AI platform for legal and professional services work.
