SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
SpaceXSI is building supercomputer infrastructure and AI models (Grok) to advance human discovery. This role leads the Site Reliability Engineering organization for a hyperscale GPU data center operation spanning multiple campuses.
You will own >99% compute uptime, drive unplanned facilities compute losses to zero, and manage efficient allocation of >3GW site power. You are responsible for both compute and infrastructure performance, including continuous site monitoring, cross-discipline incident command, deep hardware root-cause analysis, and software systems that orchestrate repair velocity. Key outcomes include robust fault-free megapack failover systems (160 MWhr per hall, >4 hours duration), hardware inventory systems, and repair velocity optimization.
RESPONSIBILITIES:
- Serve as senior operational authority for the SRE organization; represent production reliability in engineering and leadership discussions
- Lead integrated supercompute systems engineering reliability team for reactive and proactive cluster and facilities health initiatives
- Build out SRE sub-teams: compute hardware failure analysis, software tools, cluster monitoring, and integrated site reliability engineering
- Define SRE organizational structure and develop 24/7 operational excellence culture
- Define on-call responsibilities, incident management protocols, leadership structure, and communication discipline
- Develop training schemes for monitoring and response teams
- Establish operational interfaces with Datacenter Operations, engineering teams, vendors, and service owners
- Own vendor oversight and accountability; drive mean time to repair toward physics limits
- Define and continuously improve operational procedures and analysis documentation with emphasis on intelligent automation
- Lead robustification of documentation workflow and ensure efficient signal-to-noise in execution
- Define operational targets, health indicators, alerting standards, and incident severity definitions
- Drive high technical standards for hardware root cause analysis and remediation teams
- Integrate with platform engineering to develop sophisticated hardware and system understanding
- Drive repeat hardware issues to minimum and lead automation efforts to eliminate manual overhead
REQUIREMENTS:
- Bachelor's degree and 3+ years experience in large-scale operations with 2+ years leading people/technical teams, OR 10+ years in large-scale operations with 5+ years leading people/technical teams
- Proven ability to lead large, multi-site, 24/7 operations organizations in fast-paced, high-responsibility settings
- Deep expertise in server hardware, cluster reliability, and data center technologies from deployment through lifecycle management
- Experience supporting compute-heavy environments (AI, machine learning, high-performance computing) at scale
- Track record of owning uptime, SLAs, or reliability metrics for large compute clusters
- Experience leading site reliability engineering or equivalent reliability-focused teams, including root cause analysis and procedure ownership
- Strong analytical skills and ability to explain technical concepts to diverse audiences
- History of partnering with vendors at scale, driving mean time to repair down, and scaling operations across multiple sites
- Familiarity with tooling and automation (Jira, Python, Bash) for cluster health monitoring and team efficiency
- Willingness to travel frequently to data center locations
- Physical capability to handle data center tasks: lifting up to 50 lbs unassisted, standing for long periods, occasional ladder use
- Willingness to work extended hours and/or weekends as needed
