Senior Software Engineer, LLM Platform

Job Description

Build and operate shared LLM infrastructure, including LiteLLM gateways, provider integrations, routing, access controls, observability, cost attribution, and hosted or self-hosted inference. Improve reliability and developer enablement across the broader data and ML platform, partnering with engineering, data science, product, and FDE teams while participating in on-call support.

Requirements: Candidates should have strong production software engineering experience and experience operating high-throughput gateways or multi-tenant services, with hands-on knowledge of Kubernetes, Terraform, CI/CD, and production service ownership. They should also have practical experience with LLM APIs, access and credential management, distributed-systems observability, incident response, and usage metering or capacity planning.

Key Skills: Python, TypeScript, Go, LLM Platform Engineering, API Gateways, Kubernetes, Terraform, CI/CD, LLM Provider APIs, Authentication and Access Controls, Observability, Incident Response, Cost Attribution, Capacity Planning, Model Serving, Workflow Orchestration

Benefits: Equity, 401(k) Match, Paid Parental Leave, Fertility Benefits, Medical Insurance, Dental Insurance, Vision Insurance, Mental Health Support, Wellness Stipend, Learning Stipend, Internet Allowance, Commuting Benefits, Free Lunch, Gym Access, Flexible Paid Time Off, Paid Holidays, Team Outings, Referral Bonuses