SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Normal Computing is an applied AI company building foundational hardware and software for the semiconductor industry and critical AI infrastructure. The company operates across New York City, Silicon Valley (Mountain View), London, Copenhagen, and Seoul.
You will own the design and implementation of Normal's runtime layer—the software that makes their specialized AI hardware programmable. The runtime is the critical interface through which compiled programs are loaded, scheduled, and executed on the company's memory-bound inference hardware. This is a senior, hands-on role where you will set the technical direction for this foundational component.
Key responsibilities include:
- Runtime Architecture: Design the runtime's core systems including device management, memory allocation and host-device transfer, executable loading, and command submission. Your work ensures the compiler and serving layers have a stable, unified interface to the hardware.
- Hardware Abstraction: Create abstraction layers that hide differences between simulators, FPGA prototypes, silicon, and future hardware generations behind a single interface.
- Device State Management: Manage state resident on the hardware between requests, such as model weights, enabling the serving layer to rely on consistent state.
- Performance Optimization: Keep runtime overhead off the critical path for LLM inference, measured by end-to-end latency and throughput.
- Cross-Team Interfaces: Define and own boundaries with the compiler (above) and driver/firmware (below), collaborating with hardware teams.
- Technical Leadership: Set the runtime's technical direction, review dependent designs, and work with compiler and serving engineers to ensure the interface meets their needs.
The hardware is pre-silicon, so you will initially build against simulators and FPGA prototypes behind one interface, ensuring software layers remain stable when silicon arrives.
Requirements:
- Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware
- Strong systems programming in C++ or Rust, including memory management, concurrency, and low-level performance optimization
- Track record of owning a complex systems component end-to-end, from design through production, and driving technical decisions across teams
- Understanding of LLM inference workloads and their runtime demands
- Comfort building software for evolving hardware, from simulation through bring-up
Bonus qualifications:
- Experience with open runtime designs such as IREE's HAL, or with CUDA or ROCm runtime internals
- Experience with in-memory compute or other non-von-Neumann hardware
- Experience integrating a runtime with inference servers such as vLLM
- Contributions to open-source runtime or systems infrastructure
