Senior System Software Engineer, DGX Cloud AI Infrastructure

Job Description

Lead the design, development, launch, and operation of resilient AI infrastructure services, tools, and open-source projects for large-scale training on Kubernetes and Slurm. Guide team planning and mentoring, build systems for failure recovery and automated job restarts, develop agentic troubleshooting tools, and establish production support, monitoring, testing, and on-call practices.

Requirements: Requires a bachelor's, master's, or doctoral degree in computer science or a related field, or equivalent experience, and 12 or more years of experience delivering large systems from design through production. Candidates should have large-scale Kubernetes and/or Slurm experience, hands-on familiarity with a major ML framework, knowledge of NVIDIA GPUs and networking, and experience operating production services.

Key Skills: Kubernetes, Slurm, PyTorch, TensorFlow, JAX, Ray, NVIDIA GPUs, GPU Networking, AI Infrastructure, Distributed Training, Fault Tolerance, Checkpointing, Production Services, CI/CD, Monitoring and Alerting, LLM-Based Agents

Benefits: Equity, Benefits