SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
P-1 AI is building Archie, an AI engineer agent for the physical world designed to work seamlessly within existing engineering teams. Archie currently operates at the level of a junior mechanical and electrical engineer with quantitative intuition over product design and the ability to use complex engineering tools. The company recently closed a $50M Series A led by NEA, with former GE CEO Jeff Immelt joining the board, alongside investors from Anthropic, Nominal, Google, and OpenAI.
In this role, you will sit at the forefront of AI evaluation for physical engineering. You'll work directly with subject matter experts, customers, partners, and internal technical teams to challenge and evaluate Archie on real engineering work, understand failure modes, and capture those insights in evaluations that guide product development. Your core responsibilities include:
- Study how engineers and customers use Archie in the field, identifying successes and failures from an engineering perspective, and translating observations into actionable evaluations.
- Recreate real-world engineering tasks and failure modes in repeatable environments for development teams.
- Build and refine evaluation methodologies for Archie, including automated judges and human evaluation processes.
- Assess whether automated judges align with expert human judgment and improve them when misaligned.
- Collaborate across engineering, product, customers, SMEs, and external partners to ensure evaluations reflect real engineering workflows rather than abstract benchmarks.
- Use evaluation results to guide the team on where Archie needs improvement and which problems matter most to users.
You'll need strong technical background in a physical engineering domain (power systems, power electronics, mechanical engineering, aerospace, robotics, or related field) and deep understanding of agentic AI systems. Experience designing and reasoning about AI evaluations—including creating evaluation tasks, designing judges, and comparing automated evaluation against human judgment—is essential. You must be comfortable extracting actionable insights from ambiguous workflows by working directly with customers, engineers, SMEs, and technical teams. Strong product judgment and the ability to turn observations about system usage or failure into concrete evaluation work are critical. You should be motivated by building superintelligence for engineering and excited by the challenge of making AI systems genuinely capable in the physical world.
Location: Remote (US/Canada) or San Mateo, CA. Remote employees spend one week out of six on-site in San Mateo. Relocation support available.
REQUIREMENTS:
- Technical background in physical engineering domain (power systems, power electronics, mechanical engineering, aerospace, robotics, or closely related field)
- Strong understanding of agentic AI systems sufficient to design and reason about evaluations
- Experience with AI evaluations: creating evaluation tasks, designing/crafting judges, comparing automated evaluation against human judgment
- Ability to work directly with customers, engineers, subject matter experts, partners, and internal technical teams
- Strong product judgment and ability to translate observations into concrete evaluation work
- Motivation to build superintelligence for engineering and make AI systems capable in the physical world
