SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Snowflake is seeking a hands-on Engineering Manager to lead the Spark Runtime Engineering team, which is responsible for bringing Snowflake's performance, efficiency, and reliability to Spark workloads. The team builds compiler and runtime integration that translates Spark workloads into efficient execution on Snowflake's cloud-native query engine, tackling challenges in compiler infrastructure, query-plan lowering, Spark compatibility, execution integration, and end-to-end observability.
You will lead a team of approximately 5–8 engineers building production compiler and runtime capabilities for Spark workloads on Snowflake. You will guide architectural decisions, review critical designs and code, and help investigate complex correctness, performance, and reliability issues. In partnership with compiler, query engine, runtime, and product teams, you will turn customer needs into a focused roadmap and measurable engineering outcomes.
Key responsibilities include:
- Setting technical direction for Spark query compilation and runtime integration, including intermediate representations, type resolution, semantic compatibility, query-plan transformations, and efficient SQL generation
- Improving end-to-end workload performance and cost-efficiency through profiling, compilation-latency reduction, metadata-access improvements, and collaboration with query-engine teams
- Establishing strong correctness and compatibility practices, including differential testing against Apache Spark, regression coverage, and clear handling of unsupported behavior
- Owning the production quality of the team's components, including observability, incident response, fault recovery, and safe rollout practices
- Hiring, coaching, and developing engineers; providing actionable feedback, managing performance, and supporting career growth across experience levels
- Partnering with senior technical leaders and Product Management to prioritize investments, manage dependencies, and deliver a realistic roadmap
- Validating changes against representative customer workloads and tracking measurable improvements in compatibility, latency, reliability, and cost-efficiency
REQUIREMENTS:
- 2+ years of direct engineering management experience, with demonstrated success developing engineers and delivering production systems
- 5+ years of hands-on software engineering experience, including substantial work on compilers, query-processing systems, or runtime infrastructure
- Deep expertise in at least one relevant area: compiler architecture and IR transformations, relational query processing, language semantics and type systems, or runtime integration
- Proficiency in Scala, Java, C++, or a comparable implementation language, with the ability to review production code and guide technical decisions
- Strong systems fundamentals, including concurrency, memory behavior, and performance analysis, together with experience debugging complex production issues
- Ability to translate technical and customer requirements into engineering priorities and collaborate effectively across teams and locations
PREFERRED QUALIFICATIONS:
- Experience with Apache Spark internals, Catalyst, Spark Connect, or DataFrame APIs
- Experience with relational algebra, SQL generation, query optimization, or database execution systems
- Experience with Scala/JVM performance, native runtime functions, metadata caching, or compilation-performance optimization
- Experience building compatibility-test frameworks, differential testing, performance benchmarks, or production observability for data systems
- Experience leading systems through customer adoption, production hardening, and staged releases
- Degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience
