Software Engineer, Agent Optimization & Evaluation

Job Description

Lead development of Reddit’s agent optimization and evaluation platform, including evaluation environments, benchmarking, offline replay, synthetic task generation, regression detection, and production feedback loops. Design model routing and orchestration, define productivity and velocity metrics, investigate agent behavior, and collaborate with platform, infrastructure, and developer productivity teams.

Requirements: Requires 5–12+ years of engineering experience and a record of building and shipping large-scale production systems. Candidates should have experience with evaluation or measurement systems, strong knowledge of LLMs and AI agents, the ability to optimize quality, latency, reliability, and cost, and the independence and collaboration skills to lead strategic work.

Key Skills: Software Engineering, AI Agent Optimization, AI Evaluation, Benchmarking, Large Language Models, Model Orchestration, Model Routing, Distributed Systems, Production Systems, Experimentation, Regression Detection, Synthetic Task Generation, Offline Replay, Human-in-the-Loop Feedback, Developer Productivity, Cross-Functional Collaboration

Benefits: Comprehensive Healthcare Benefits, Income Replacement Programs, 401(k) With Employer Match, Global Benefits Programs, Professional Development Support, Caregiving Support, Family Planning Support, Gender-Affirming Care, Mental Health and Coaching Benefits, Flexible Vacation, Paid Volunteer Time Off, Paid Parental Leave, Equity in the Form of Restricted Stock Units