Principal Software Engineer, Inference

Job Description

Define and own the technical architecture for LLM serving, including engine integration, batching, and distributed execution strategies. Partner with performance engineering teams to optimize latency and throughput while mentoring engineers and presenting technical direction to stakeholders.

Requirements: Requires at least 12 years of software engineering experience with a minimum of 1 year working directly on LLM inference runtimes. Candidates must possess expert-level proficiency in Kubernetes, Go, Python, and deep knowledge of GPU memory hierarchies and inference internals.

Key Skills: LLM inference, vLLM, SGLang, TensorRT-LLM, Kubernetes, Go, Python, C++, CUDA, Distributed execution, Continuous batching, KV cache management, Quantized execution, GPU scheduling, Performance engineering, Architecture design

Benefits: Health and wellbeing benefits, Professional development programs, Flexible work arrangements