Lead Data Platform Engineer
jade globalSan Jose, CA
Design, deploy, and operate data platform services on Kubernetes in Microsoft Azure.
Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS, including catalog configuration, table maintenance, schema evolution, and partitioning strategy.
Operate and tune Trino and Spark clusters for performance, reliability, and cost efficiency.
Develop and maintain data transformation pipelines using DBT, with sound modeling, testing, and documentation practices.
Define and enforce standards for data access, security, governance, and multi-tenant resource management across ADLS and query engines.
Implement observability for the platform, including monitoring, alerting, capacity planning, and incident response.
Partner with data engineers, analysts, and platform teams to onboard workloads and resolve performance bottlenecks.
Automate infrastructure and deployments using infrastructure-as-code and CI/CD.
Required Qualifications:
Strong hands-on experience with Kubernetes, including deploying and operating stateful and distributed workloads.
Strong hands-on experience with Microsoft Azure Cloud, including ADLS
Hands-on experience with Apache Iceberg and Iceberg catalogs REST
Expertise in distributed query and compute technologies, specifically Trino and Spark.
Hands-on experience with DBT for data transformation and analytics engineering.
Solid understanding of data lake and lakehouse architectures, file formats (Parquet, ORC), and query optimizatio
Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries of the position across all US locations. Within the range, individual pay is determined by work location and additional job-related factors, including knowledge, skills, experience, tenure and relevant education or training. The pay scale is subject to change depending on business needs. Your recruiter can share more about the specific salary range for your preferred location during the hiring process. Additional compensation may include benefits, discretionary bonuses, and equity.
Matching similar jobs
Level
Lead L6
Location
San Jose, CA
Occupation
Data Warehousing Specialists
Industry
Computer Systems Design Services
Posted
yesterday