Responsibilities and duties:
Serve as architecture point of contact for DPaaS (Databricks + Unity Catalog) during coverage, maintaining continuity on in-flight decisions.
Support and unblock the SQL Server-to-Databricks migration, incl. sequencing tied to the SAP HANA cloud (RISE) migration deadline.
Apply and enforce the team's Databricks governance framework (5 pillars): duplicate-data prevention, environment segregation, compute selection, engineering standards, cost visibility.
Review/advise on schema, pipeline, and compute design to prevent recurrence of known anti-patterns (e.g., duplicated department-level table copies).Partner with DevOps, DataOps, and platform support leads to keep architecture-dependent workstreams moving.
Advise on open platform decisions (e.g., BI tooling, compute-tier questions); final calls remain with the permanent architecture lead and director.
Document decisions, rationale, and open issues for smooth handoff back to the returning architect.
Requirements:
Hands-on data/platform architect experience on Databricks, incl. Unity Catalog governance, at enterprise scale.
Strong background in data platform migrations, ideally SQL Server-to-lakehouse or similar legacy-to-cloud.
Solid grasp of data governance: environment segregation, compute cost management, engineering standards, duplicate-data prevention.
Able to ramp quickly into an in-flight environment given the fixed-term engagement.
Strong written communication for documenting decisions and a clean handoff.
Hands-on Spark (PySpark or Scala) and Delta Lake for pipeline design, performance tuning, troubleshooting at production scale.
Advanced SQL: query optimization and schema design across multi-schema, multi-workspace environments.
Working knowledge of cloud infra under Databricks (Azure or AWS): networking, storage, access control.
Experience with orchestration/CI-CD tooling for data pipelines (Databricks Workflows/Jobs, Airflow, Azure DevOps, or similar).Familiarity with Unity Catalog access controls, lineage, and metadata management, incl. auditability and least-privilege design.
Understands Databricks compute cost drivers (cluster sizing, job vs. all-purpose compute, autoscaling) to advise on cost visibility.
Experience integrating/migrating ERP-adjacent source systems (e.g., SAP/HANA) into a lakehouse architecture.
Nice to Have:
Prior experience in a large enterprise data platform org spanning DevOps, DataOps, FinOps, and platform support.
Familiarity with BI tool ecosystems (Power BI, Tableau) and platform rationalization efforts.
Experience with SAP HANA or ERP-adjacent source systems a plus.
For a further discussion or to find out more, contact Snehita on (909) 867-0939 or apply now.
