SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
ServiceNow is seeking a Staff Software Engineer to lead a team of software and AI engineers in developing AI-native software built around LLMs, agents, and retrieval systems. This role combines deep technical expertise with people leadership and requires full ownership of code from development through deployment, testing, and validation.
You will architect and develop intuitive AI-native solutions, create technical specifications including model selection, prompt/agent design, and data/retrieval architecture. You'll build evaluation harnesses for AI features (accuracy, regression, hallucination checks), troubleshoot complex coding and model-behavior issues, and implement secure software practices including AI-specific safeguards (prompt-injection defense, data privacy, PII handling, responsible-AI guardrails). You'll monitor model performance and drift in production, maintain software regularly, and work with IaaS platforms (Google Cloud, AWS, Azure) and their managed AI/ML services.
Leadership responsibilities include managing individual contributors' daily tasks, mentoring software and AI engineers, performing detailed code reviews (including prompts and agent logic), evaluating performance, setting and tracking quarterly goals, and assisting with training and recruitment. You'll communicate effectively with internal and external customers, translate AI capabilities for both technical and non-technical audiences, participate in root cause analysis for customer issues, and collaborate with cross-functional teams (product, domain specialists, engineering) to define requirements and develop scalable AI-driven solutions. You'll stay current with AI advancements and proactively identify opportunities to apply new models, frameworks, and techniques.
Requirements:
- 8–10 years of practical Java experience
- 2 years of prompt engineering and prompt/agent scripting experience
- 2 years of hands-on experience integrating third-party AI/ML services and platform APIs (OpenAI, Anthropic, Salesforce, etc.)
- Proven ability to design and manage domain-specific languages, prompt templates, and agent instruction frameworks
- Proficiency in Python, Java, and JavaScript (strong Python skills for ML/AI ecosystem)
- Experience developing and optimizing microservices for high-performance data and embedding pipelines (up to 10 million rows, feeding vector stores)
- Knowledge of cloud computing and managed AI services (AWS Bedrock/SageMaker, Azure OpenAI, Google Vertex AI)
- Experience with RESTful and streaming APIs, model/inference endpoints, web services, and integration technologies
- Experience with LLM orchestration and retrieval frameworks (LangChain, LlamaIndex) and vector databases (Pinecone, Weaviate, pgvector)
- Ability to develop unit tests and AI evaluation harnesses (accuracy, regression, hallucination checks)
- Ability to debug and isolate root causes in probabilistic and distributed systems
- Experience with Agile and Scrum methodologies
- Experience with Spring Boot or similar frameworks
- Experience with APIs, REST standards, JSON, SQL, and NoSQL/vector stores
- Outstanding collaboration and communication skills
- Strong problem-solving and debugging skills, including working with non-deterministic AI outputs
- Ability to learn quickly, take direction, work independently and as part of a team
- Bachelor's degree in computer science or related field (ML/AI coursework or experience a plus)
- Experience with Kubernetes and containerization welcome; experience serving models or deploying inference/GPU workloads a plus
