SlipstreamJobs tracks this role from the company's public career site. Apply directly on the employer's site.
Lovable is a platform that enables anyone to build software using AI agents, serving millions of users from solopreneurs to Fortune 100 teams. The company is building a generation-defining product from Stockholm with a small, talent-dense team focused on extreme ownership and high velocity.
In this role, you will own how Lovable measures and improves its AI agent. You are not a passive analyst—you actively drive agent quality metrics upward through evaluation systems, experimentation, and continuous improvement. You will define success metrics (completion rates, error rates, task success), build the eval frameworks that gate agent changes, and turn agent telemetry and traces into concrete fixes shipped by the engineering team.
Key responsibilities include:
- Defining and owning agent quality metrics: success rates, completion rates, error rates, and behavioral drivers
- Building evaluation systems and experiment frameworks (A/B testing) that determine whether agent changes ship safely
- Converting agent traces and telemetry into actionable fixes in close collaboration with the agent engineering team
- Creating tooling and agents that produce evaluations continuously as the agent evolves
- Setting standards for judging agent behavior in ambiguous scenarios where no clean answer key exists
You will work with SQL, Python, applied statistics, and experimentation design. The tech stack includes BigQuery, PubSub, Braintrust for LLM evaluation, OTEL tracing, Hex for analytics, and GCP. You should have strong instincts for what "good" looks like in agent behavior and how to measure it when outcomes are noisy or subjective. This is a role for someone entrepreneurial who thrives in ambiguity and wants to make a measurable dent in how AI agents work.
