Software Engineer – Presto / Spark Development

Job Description

As a Software Engineer, you will design, develop, test, and deliver connectors and performance improvements for the watsonx.data query layer. You will work in an Agile environment to enhance query throughput, latency, and reliability at petabyte scale.

Requirements: Candidates should have 6+ years of software engineering experience, including 2 years with Presto/Trino or Apache Spark internals. Strong Java or Scala skills and experience with connectors, UDFs/UDAFs, and performance optimization are essential.

Key Skills: Presto, Spark, Java, Scala, UDFs, UDAFs, Optimizer Extensions, Performance Optimization, CI/CD, Kubernetes, Data Skew, Memory Pressure, Open Table Formats, Agile, Debugging, Documentation