SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Salary: USD 215,000 - 240,000 / annual
Huntress, founded in 2015 by former NSA cyber operators, is building an Agentic SOC—a system where LLM-based agents investigate security signals end-to-end, gathering telemetry, reconstructing events, and determining whether activity is malicious. The system augments human analysts by handling high-volume, repetitive work and pre-processing tool outputs, freeing analysts to focus on complex cases and active intrusions.
As Principal Software Engineer, you will own the architecture and technical direction of this agentic investigation system. This is not a chatbot; the system processes thousands of signals daily across thousands of customer organizations, makes real decisions (like closing investigations or isolating hosts), and must be reliable, auditable, and secure. You will decide how investigations are structured, where deterministic code ends and model reasoning begins, what the agent can and cannot touch, and how to measure its performance.
Key responsibilities include:
- Architecting agentic investigation: defining how signals flow from detection into investigation, managing investigation state, designing tools exposed to agents, and structuring verdicts with timelines, scope, and evidence.
- Building agents that investigate like strong analysts by learning from SOC workflows and turning that into agent behavior.
- Reducing analyst toil through LLM-powered pre-processing of tools and data sources analysts use daily.
- Embedding security into the agent itself: designing for untrusted input, multi-tenant isolation, and auditability, recognizing that attackers will try to manipulate it.
- Treating evaluation as core to the product: defining correctness metrics, ensuring all changes are evidence-backed, and measuring quality over time.
- Deciding autonomy levels based on measured performance, recognizing when problems don't need agents.
- Building for production reliability, latency, and cost at scale; systems must work when models are slow, wrong, or unavailable.
- Designing human-agent handoffs so analysts are faster and better informed when cases reach them.
- Prototyping through hard problems and derisk uncertain approaches.
- Being the engineering voice to SOC leadership, other teams, product, and executives.
- Balancing technical rigor with pragmatism and bias for action.
- Raising team performance through code review, pairing, and creating leverage through better patterns and tooling.
You will report to the Director of Engineering and work closely with SOC analysts, detection engineers, product, and other engineering teams. The company uses AI coding tools extensively and expects responsible, full use of them to accelerate development.
Requirements:
- 15+ years developing complex software products, with significant time as technical lead on production systems.
- Hands-on production experience designing, shipping, and operating LLM-based agents, including tool use, context management, structured output, guardrails, and evaluation; clear understanding of failure modes.
- Track record building evaluation for non-deterministic systems: labeled datasets, offline evals, regression gates, and measuring quality over time.
- Security mindset: experience with untrusted input handling, multi-tenant isolation, and audit trails that withstand scrutiny.
- Experience building systems that make automated decisions with real consequences and judgment about what to automate vs. delegate to humans.
- Curiosity about security analyst workflows and willingness to learn from them deeply.
- Deep backend expertise in distributed systems, queues, durable workflows, and concurrency; track record designing for throughput and correctness under load.
- Strong skills in one or more backend languages with ability to pick up new ones quickly; primary stack is Ruby on Rails (fluency expected).
- Experience with AWS, Azure, or other public cloud; data stores like Postgres and Redis.
- Experience with AI coding tools (e.g., Claude Code).
- BS or MS in Computer Science or Engineering, or equivalent experience.
Bonus: SOC, incident response, threat hunting, or detection engineering experience; familiarity with endpoint and identity telemetry and attacker tradecraft; experience attacking or adversarially testing ML/LLM systems; Ruby on Rails experience.
