Skip to main content

Data Engineer

wiproWashington, DCSenior L5
Overview: As a Data Engineer, you will design and optimize scalable data pipelines using Python and Azure Databricks to support data-driven solutions. You will build ETL/ELT workflows, work with large datasets, and apply Spark-based processing within Databricks. You will ensure data quality, security, and governance while collaborating with cross-functional teams to deliver data-driven insights. This role offers a chance to contribute to cloud-based data platforms and CI/CD-enabled deployments in a growing digital transformation environment.
Responsibilities: Design, develop, and maintain scalable data pipelines using Python and Azure Databricks Build and optimize ETL/ELT workflows for ingesting, transforming, and processing large datasets Develop data solutions using Apache Spark within the Databricks environment Integrate data from multiple structured and unstructured sources Ensure data quality, reliability, security, and governance across data platforms Collaborate with Data Scientists, Analysts, Architects, and Business stakeholders to deliver data solutions Implement performance tuning and optimization for large-scale data processing workloads Develop and maintain data models, metadata management, and documentation Monitor and troubleshoot production data pipelines and resolve data-related issues Support CI/CD implementation and deployment automation for data engineering solutions
Key requirements: Strong expertise in Python and Azure Databricks Hands-on experience with cloud-based data engineering and big data processing Data transformation using Azure services and Databricks Experience with Apache Spark in Databricks Ability to build scalable data pipelines and ETL/ELT workflows Knowledge of data quality, security, and governance Collaboration with cross-functional teams Strong communication Problem-solving mindset Python Azure Databricks Apache Spark
APPLY NOW (opens in a new tab)

Matching similar jobs

Level

Senior L5

Location

Washington, DC

Occupation

Data Warehousing Specialists

Industry

Computer Systems Design Services

Posted

yesterday

APPLY NOW (opens in a new tab)