Principal Data Engineer, Data Quality & Datasets Pipelines

Job Description

Build and operate petabyte-scale pipelines and systems for selecting, moving, filtering, retaining, offloading, and deleting driving data across storage tiers. Collaborate with dataset strategy, storage, and data infrastructure teams to improve dataset quality and training coverage while reducing storage costs through early, low-compute decisions.

Requirements: Applicants should be senior data or software engineers with experience running data ingestion or data quality pipelines at petabyte scale, strong Python and Spark skills, and cloud data engineering experience. They should exercise sound judgment about filtering and data quality, ship practical solutions quickly, and be comfortable building systems around models; experience in autonomous vehicles, robotics, fleet data, high-volume IoT, or data lifecycle policies is a bonus.

Key Skills: Python, Apache Spark, Cloud Data Engineering, Data Ingestion, Data Quality, Petabyte-Scale Data Pipelines, Data Filtering, Data Retention, Data Offloading, Storage Lifecycle Policies, Data Movement, Pipeline Engineering, System Design, Technical Leadership, Data Infrastructure, Storage Cost Optimization

Benefits: Equity, Relocation Support, Visa Sponsorship, Hybrid Working, Learning And Development Budget, Health Insurance, Dental Insurance, Enhanced Maternity And Paternity Leave, Retirement Or Pension, Therapist Access, Wellbeing Partner Discounts