SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
webAI is building distributed AI infrastructure that enables enterprises to run private, large-scale AI models on their own hardware—whether on-prem Apple-silicon fleets, Linux servers, or customer cloud environments—while keeping data and IP entirely private.
As Staff Platform/Infrastructure Engineer, you will own the deployment substrate that makes this repeatable across bare metal and cloud. You'll inherit a production deployment serving a global enterprise customer, architect its evolution to a leaner next-generation system, and transform it into a self-service deployment kit that forward-deployed engineers can use to onboard new customers from documentation alone. This is a hands-on staff role: you make architecture decisions, write code, and carry the pager.
Key responsibilities:
- Own end-to-end deployment for the webAI kernel and B2B SDK, delivering single-machine installs, multi-node clusters, and cloud deployments (AWS, Azure/AKS) from a unified model.
- Evaluate and evolve orchestration strategy (Nomad, Kubernetes, or config-managed native services like Ansible + launchd), prioritizing simplicity and eliminating unnecessary custom infrastructure.
- Operate Apple-silicon fleets in production: launchd services, firewalling (pf), Lima/Docker VMs, GPU/unified-memory tuning for local LLM inference, and recovery procedures.
- Build the inference serving layer with the AI team: OpenAI-compatible model gateway, conversation-pinned load balancing, per-model and per-persona routing, and performance testing (throughput, time-to-first-token, concurrency).
- Own the software supply chain: GitLab CI, container and OCI model artifacts, signing (cosign/minisign), pinned dependencies, mirrored registries for air-gapped deployments, and release processes.
- Secure the platform: secrets management (Vault, KMS, cloud secret stores), PKI and mTLS, enterprise identity integration (Entra ID/OIDC, SCIM), and network policy for on-prem sites.
- Build observability and audit: metrics, logs, alerting (Prometheus, Loki, Grafana), and export of audit/telemetry into customer monitoring tools.
- Define high availability and reliability for the platform: set SLOs and reliability levels committable in customer contracts, then implement them.
- Create runbooks, operational recipes, and RCAs that enable forward-deployed engineers and customers to operate independently. Lead incident response when escalation is needed.
- Collaborate with kernel engineers on networking (QUIC, peer-to-peer connectivity, relays, NAT traversal) and deployment simplification.
- Mentor engineers and raise standards for infrastructure-as-code, testing, and operations across the team.
Requirements:
- 8+ years building and operating production infrastructure, with meaningful experience on bare metal or on-prem systems (not only managed cloud).
- Deep expertise with at least one workload orchestrator (Nomad or Kubernetes), plus judgment to know when not to use one.
- Strong Terraform and configuration management (Ansible or equivalent); clean state and environment management across staging and production.
- Production experience with AWS, Azure, or both (identity/IAM, KMS, container registries, managed Kubernetes).
- Solid networking fundamentals: VPN overlays (WireGuard/Tailscale), firewalls, NAT, load balancers, L4/L7 proxies (e.g., Envoy), DNS, TLS.
- Secrets and identity: Vault or equivalent, PKI/mTLS, OIDC.
- CI/CD and release engineering in GitLab: container builds, artifact signing, reproducible pinned builds.
- Observability: Prometheus/Grafana/Loki or equivalent, alert design, on-call experience.
- Proficiency in at least one systems language (Rust or Go) plus scripting (Python/Bash).
- Clear technical writing: runbooks, design docs, and RCAs that others can act on.
Nice to have:
- macOS fleet administration at scale (launchd, pf, MDM-managed or locked-down environments, FileVault constraints).
- Local LLM inference experience (llama.cpp, MLX, vLLM), GPU/unified-memory capacity planning, model packaging.
- Air-gapped or disconnected delivery: mirrored registries, offline installers.
- Peer-to-peer or QUIC networking (iroh, libp2p), relays, NAT traversal.
- Kubernetes operators/controllers or Nomad tooling (kube-rs, controller-runtime).
- Customer-facing deployment experience: installing and supporting software at customer sites.
