Staff + Senior Software Engineer, Inference

Job Description

Design, build, and operate distributed inference systems that serve Claude at scale, including intelligent routing, load balancing, autoscaling, accelerator orchestration, and reliable model deployment. Improve compute efficiency and performance while providing researchers with infrastructure for developing and serving new models and architectures.

Requirements: Requires significant software engineering experience, particularly with distributed systems, along with flexibility, a results-oriented approach, and interest in machine learning infrastructure and its societal impact. Preferred qualifications include large-scale distributed systems, production machine learning, routing or traffic management, LLM inference optimization, Kubernetes and cloud infrastructure, and proficiency in Python or Rust; a bachelor's degree or equivalent education, training, or experience is the stated minimum.

Key Skills: Distributed Systems, Machine Learning Systems, High-Performance Computing, Inference Optimization, Request Routing, Load Balancing, Traffic Management, Autoscaling, Fleet Orchestration, Deployment Pipelines, Kubernetes, Cloud Infrastructure, Python, Rust, Observability, Multi-Region Deployments

Benefits: Competitive Compensation, Equity Donation Matching, Generous Vacation, Parental Leave, Flexible Working Hours