Principal Data Engineer, Data Quality & Datasets Pipelines

Job Description

Build and operate petabyte-scale driving-data pipelines, including filtering, retention, offloading, and early selection processes that reduce storage costs while preserving valuable data. Partner with dataset strategy, storage, and data infrastructure teams to monitor data quality and ensure datasets provide useful behavioral and environmental coverage for model training.

Requirements: Applicants should be senior data or software engineers with experience operating data ingestion or data quality pipelines at petabyte scale, strong Python and Spark skills, and cloud data engineering experience. They should have sound judgment about data filtering and quality, be comfortable iterating on live systems, and ideally bring experience in autonomous vehicles, robotics, fleet data, high-volume IoT, or data lifecycle policies.

Key Skills: Python, Apache Spark, Cloud Data Engineering, Data Ingestion, Data Quality, Petabyte-Scale Data Pipelines, Data Filtering, Data Retention, Data Offloading, Data Lifecycle Management, Storage Cost Optimization, Data Selection Models, System Design, Technical Leadership, Software Engineering, Autonomous Driving

Benefits: Equity, Relocation Support, Visa Sponsorship, Hybrid Working, Learning And Development Budget, Health Insurance, Dental Insurance, Enhanced Maternity Leave, Enhanced Paternity Leave, Retirement Or Pension, Therapist Access, Wellbeing Partner Discounts