Senior Software Engineer I, Inference

Job Description

Lead the design and architecture of a Kubernetes-native inference platform to improve latency, throughput, and reliability. Mentor junior engineers and implement advanced optimizations like speculative decoding and KV-cache reuse to meet strict SLAs.

Requirements: Requires 3-8 years of experience building distributed systems with proficiency in Python or Go and production-scale Kubernetes. Candidates should have practical knowledge of inference internals and a track record of improving tail latency.

Key Skills: Python, Go, Kubernetes, Distributed Systems, Inference Optimization, C++, Prometheus, Grafana, OpenTelemetry, CI/CD, CUDA, NCCL, RDMA, vLLM, Triton, TensorRT-LLM

Benefits: Medical insurance, Dental insurance, Vision insurance, Company-paid Life Insurance, Voluntary supplemental life insurance, Short and long-term disability insurance, Flexible Spending Account, Health Savings Account, Tuition Reimbursement, Employee Stock Purchase Program (ESPP), Mental Wellness Benefits, Family-Forming support, Paid Parental Leave, Childcare support, 401(k) with employer match, Flexible PTO, Catered lunch