Principal Software Engineer, Inference

Job Description

Define and own the technical architecture for LLM serving deployment, including engine integration, batching, and distributed execution strategies. Partner with performance engineering teams to optimize latency, throughput, and GPU utilization on customer-owned infrastructure.

Requirements: Requires a minimum of 12 years of software engineering experience, including at least one year working directly on LLM inference runtimes. Candidates must possess expert-level proficiency in Kubernetes, Go, Python, and deep knowledge of GPU memory hierarchies and inference internals.

Key Skills: LLM inference, vLLM, SGLang, TensorRT-LLM, Kubernetes, Go, Python, C++, CUDA, Continuous batching, KV cache management, Distributed execution, Quantized execution, GPU scheduling, Performance engineering

Benefits: Health & Wellbeing, Personal & Professional Development