Staff Software Engineer - GenAI inference

Job Description

Lead the architecture, development, and optimization of Databricks’ GenAI inference engine and model-serving stack, improving latency, throughput, memory efficiency, and hardware utilization. Build scalable and reliable inference capabilities—including routing, batching, scheduling, instrumentation, and distributed orchestration—in collaboration with researchers and platform teams.

Requirements: Requires a bachelor’s, master’s, or doctoral degree in computer science or a related field, along with at least six years of experience in performance-critical software systems and a record of owning complex architecture end to end. Candidates should have deep ML inference knowledge, hands-on CUDA and GPU programming experience, distributed systems expertise, and experience with profiling and performance optimization.

Key Skills: Software Engineering, GenAI Inference, LLM Model Serving, CUDA, GPU Programming, Distributed Systems, Performance Optimization, Inference Engine Architecture, Machine Learning Inference, Quantization, Sparse Operations, Profiling and Tracing, Memory Management, Scheduling and Batching, Fault Tolerance, Cross-Functional Leadership