SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 180,000 - 250,000 / annual
Clear Street is modernizing the brokerage ecosystem with a cloud-native clearing and custody system supporting billions in trading volume daily. The Active team is building a state-of-the-art, cloud-based trading platform for sophisticated investors with high-performance access to liquidity venues, advanced charting, and sophisticated order handling.
In this role, you will own the reliability and quality bar for an AI copilot embedded in the trading platform. You will design and build comprehensive evaluation systems measuring correctness, safety, latency, and regression risk across market analysis, portfolio/risk reasoning, and trading workflows including order placement.
Key responsibilities include:
- Developing and maintaining benchmarks: curated "golden sets," scenario suites, stress/adversarial cases, and continuously refreshed market/regime-based test corpora
- Building automated quality gates and regression workflows that block releases when key metrics degrade
- Partnering with engineering and product to define safe tool/action contracts (deterministic previews, confirmations, auditability) and ensure predictable assistant behavior
- Owning model improvement loops tied to evals: data collection/labeling strategies, error taxonomy, prompt/tooling changes, and fine-tuning or preference optimization to measurably improve benchmark performance
- Designing and operating monitoring and incident response for AI systems: telemetry, alerting, RCA, and fix-forward processes
- Developing deep understanding of trading concepts (margin, shorting, portfolio margin, risk, execution) and expressing them accurately to users
Tech stack includes Rust, TypeScript, Postgres, React (Web), React Native (Mobile), observability/telemetry tooling, LLM APIs, model serving, and eval/training pipelines.
Requirements:
- At least 8 years of experience shipping production software with strong proficiency in any programming language
- Strong knowledge of computer science fundamentals, testing methodology, and systems design
- Experience building evaluation frameworks, test harnesses, and benchmark suites for complex systems (LLMs, agents, search, retrieval, ranking, recommenders)
- Experience running model improvement cycles: dataset curation, labeling/QA, offline experimentation, and deploying changes with measurable impact on benchmarks
- Ability to define metrics, build measurement pipelines, and drive engineering/product decisions from data
- Comfort working across the stack: debugging model/tooling failures, instrumenting services, and partnering with frontend/product on UX patterns that improve safety and trust
- High degree of self-motivation and willingness to jump into unfamiliar areas to solve problems
Bonus qualifications:
- Experience with fine-tuning, preference optimization, distillation, or prompt/compiler-style techniques for improving tool-use reliability
- Experience creating domain-specific benchmarks and adversarial suites for high-stakes applications
- Deep experience with trading across asset classes and margin types
- Experience with Rust and performance-sensitive services
- Experience designing incident response and SLOs for ML/AI systems
