Staff AI Software Engineer

Job Description

Job Description

Staff AI Software Engineer

Salary up to $275K + Equity

San Mateo | High Impact AI Platform

US Citizen or GreenCard holder

A fast-growing AI startup is building LLM-powered systems transforming healthcare operations.

They’re looking for a Staff AI Software Engineer to design and scale the backend infrastructure powering production LLM applications used in real healthcare workflows.

This role is ideal for someone who enjoys taking LLM systems from prototype to reliable production infrastructure. You’ll be building APIs, data pipelines, and distributed systems that make AI dependable in mission-critical environments.

What You’ll Be Doing

Design and build scalable backend services and APIs that orchestrate LLM systems in production
Develop RAG pipelines and data ingestion systems that ground LLMs in healthcare data
Build LLMOps infrastructure including monitoring, evaluation, cost tracking, and guardrails
Optimize performance through caching, batching, and distributed workloads
Implement secure and compliant systems designed for healthcare environments
Partner with product and ML teams to bring AI into real operational workflows
Provide technical leadership and system design guidance across AI infrastructure

What They’re Looking For

10+ years of software engineering experience building backend or distributed systems
Proven experience working with LLMs on the backend in production environments
Strong coding ability in Python plus one of Go, Java, or TypeScript
Experience building systems using LangChain, Hugging Face, LlamaIndex, OpenAI APIs, Anthropic or similar frameworks
Deep understanding of distributed systems and scalable API architecture
Cloud infrastructure experience with AWS, GCP, or Azure
Experience with Docker, Kubernetes, and cloud-native infrastructure
Familiarity with LLMOps / MLOps practices including monitoring and model evaluation

Nice to Have

Experience building RAG pipelines and working with vector databases
Exposure to healthcare or regulated environments (FHIR, HL7, HIPAA)
Background building enterprise SaaS or mission-critical platforms

Why This Role

Work on real-world LLM systems used in healthcare
High ownership role shaping core AI infrastructureSalary up to $275K + equity
Opportunity to help define how AI is deployed in mission-critical environments

If you’re a senior engineer building real backend systems for LLMs, feel free to reach out.