Software Engineer, Inference

Job Description

Design and operate Sierra’s inference architecture across self-hosted models and third-party providers, including serving, routing, failover, capacity, and quota systems. Run and optimize production inference infrastructure for latency, reliability, throughput, and cost, while collaborating with research, model, runtime, and provider teams.

Requirements: Candidates should have strong distributed-systems fundamentals and experience designing, building, and operating large-scale production infrastructure. The role also calls for sound tradeoff judgment and ownership of complex systems; experience with ML infrastructure, LLM serving, GPU operations, vLLM or SGLang, post-training, or inference optimization is advantageous.

Key Skills: Distributed Systems, Large-Scale Production Systems, Inference Architecture, Inference Routing, Capacity Management, Quota Management, Reliability Engineering, Latency Optimization, Cost Optimization, GPU Infrastructure, Self-Hosted Inference, LLM Serving, Inference Frameworks, vLLM, SGLang, Post-Training Infrastructure

Benefits: Flexible Paid Time Off, Medical Insurance, Dental Insurance, Vision Insurance, Life Insurance, Disability Benefits, Retirement Plan, Parental Leave, Fertility And Family-Building Benefits, Lunch, Snacks And Coffee, Discretionary Benefit Stipend, Equity Plans, Alphorn Lessons