SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Code Metal is building AI-driven code transpilation, modernization, optimization, modeling and simulation solutions. The AI Platform team provides the foundational infrastructure that enables these products: GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management.
As a Senior Software Engineer on the AI Platform team, you will design, build, and operate core components of the AI enablement stack. You'll own one primary focus area—such as model serving, agentic infrastructure, or observability and experimentation—while contributing across the broader platform.
Model serving involves deploying, benchmarking, and tuning elastic production inference for open-weight models using runtimes like vLLM, SGLang, and TensorRT-LLM, as well as operating model gateways that product teams use to access self-hosted and commercial models.
Agentic infrastructure focuses on building agent harnesses, orchestration primitives, and context-engineering services (memory, retrieval, data discovery) that product teams compose into reliable, verifiable workflows.
Observability and experimentation involves instrumenting the stack end-to-end with tools like OpenTelemetry, building evaluation harnesses, and creating experiment-tracking and artifact layers for reproducibility and comparison.
Core responsibilities include owning one or more platform components end-to-end from design through production operation; building and operating services in your focus area while contributing across the stack; writing clean, well-tested code and holding AI-generated code to the same standard; debugging hard production issues such as tail latency, GPU memory pressure, or agent failures; scoping and estimating efforts with internal customers; reviewing code and designs; and running focused benchmarks and experiments to inform platform decisions.
This is fundamentally an engineering role. Most of your time goes to designing, building, and operating production systems. You'll work closely with the Applied AI Research team and occasionally run experiments when platform decisions need evidence. You'll need solid data science and AI research fundamentals alongside strong systems engineering skills.
REQUIREMENTS:
- Production-grade Python and solid platform engineering fundamentals: API and service design, distributed systems, containers and orchestration (Kubernetes), CI/CD, and testing
- Production experience in at least one focus area: (1) Model serving—deploying and tuning LLM inference or running an LLM/API gateway; (2) Agentic infrastructure—shipping agentic systems or building context-engineering services like RAG, embeddings, vector/hybrid search; (3) Observability and experimentation—instrumenting and operating services against SLOs or building evaluation and experiment-tracking systems
- Solid data science and AI research fundamentals: understanding how transformers and LLM inference work, experiment design, benchmarking, and model evaluation; working familiarity with PyTorch and Hugging Face
- Experience owning a service or component in production, including debugging and improving reliability
- Experience writing design docs for focused projects, reviewing code, and mentoring or onboarding teammates
- Typically 4+ years of software engineering experience, including 2+ years building and operating ML/AI or LLM systems in production
PREFERRED QUALIFICATIONS:
- Experience in more than one focus area
- Production experience running an LLM gateway or proxy (SMG, Bifrost) or building an API gateway with routing, auth, rate limiting, quotas, failover, and cost attribution
- Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, GPU profiling
- Experience with durable workflows, agent frameworks, or tool protocols such as MCP
- Experience fine-tuning language models, including distributed training across multiple GPU nodes
- Experience with experiment-tracking or artifact management systems such as MLflow or Weights & Biases
- Experience deploying AI systems on-prem or in regulated domains such as defense or aerospace
