Staff Software Engineer, Inference

Job Description

Lead the architecture, performance, and reliability of a Kubernetes-native inference platform for large-scale AI workloads. Drive cross-team design initiatives focusing on request routing, GPU utilization, and system-wide optimizations.

Requirements: Requires 8-12+ years of experience in large-scale distributed systems and deep expertise in production-scale Kubernetes. Candidates must have a strong background in low-latency, high-throughput systems and experience with AI inference frameworks.

Key Skills: Go, Python, C++, Kubernetes, Distributed Systems, GPU Resource Management, Performance Optimization, Low-latency Systems, Inference Systems, Networking, Scheduling, Batching, Memory Optimization, CUDA, NCCL, RDMA

Benefits: Medical insurance, Dental insurance, Vision insurance, Company-paid Life Insurance, Voluntary supplemental life insurance, Short and long-term disability insurance, Flexible Spending Account, Health Savings Account, Tuition Reimbursement, Employee Stock Purchase Program (ESPP), Mental Wellness Benefits, Family-Forming support, Paid Parental Leave, Childcare support, 401(k) with employer match, Flexible PTO, Catered lunch