Principal Software Engineer, Inference

Job Description

Define and own the technical architecture for LLM inference runtimes, focusing on execution efficiency and distributed inferencing strategies. Lead design reviews, mentor engineers, and partner with performance teams to optimize latency and throughput on customer-owned hardware.

Requirements: Requires at least 12 years of software engineering experience with a minimum of 1 year working directly on LLM inference runtimes. Candidates must possess expert-level proficiency in Kubernetes, Python, Go, and deep knowledge of GPU memory hierarchies and inference internals.

Key Skills: LLM Inference, Kubernetes, Python, Go, C++, CUDA, TensorRT-LLM, vLLM, Distributed Systems, GPU Optimization, Continuous Batching, KV Cache Management, Architecture Design, Performance Engineering, Machine Learning

Benefits: Health & Wellbeing, Personal & Professional Development