Staff Software Engineer- Foundation Model Inference

Job Description

Build and operate infrastructure for large-scale LLM inference across partner and self-hosted models, improving the reliability, latency, and efficiency of distributed AI workloads. Collaborate with platform, infrastructure, and machine learning teams to deliver end-to-end AI experiences and shape how developers and data scientists use AI on Databricks.

Requirements: Candidates should have at least eight years of backend or infrastructure engineering experience and experience with distributed systems, scalable APIs, or cloud-native infrastructure. Experience with real-time serving, machine learning infrastructure, or GPU orchestration is sought; familiarity with service-oriented architecture, deployment pipelines, and observability is also relevant.

Key Skills: Backend Engineering, Infrastructure Engineering, Distributed Systems, Scalable APIs, Cloud-Native Infrastructure, Real-Time Serving, Machine Learning Infrastructure, GPU Orchestration, Service-Oriented Architecture, Deployment Pipelines, System Observability, LLM Inference, AI Infrastructure, Developer Platforms