Lead Data engineer

Job Description: We are seeking a Lead Software Engineer to join our Enterprise Data Platform Services team. This team plays a central role in the data backbone of client’s., building and operating self-service capabilities that empower data scientists and engineers to build and deliver with greater speed, confidence, and autonomy. Our mission is to accelerate the development and delivery of scalable, trustworthy data solutions and insights that drive business outcomes, while laying the groundwork for AI-ready data systems and intelligent observability in the future. As a Lead Software Engineer, you will help shape the future of our enterprise data platform. Our platform plays a key role in integrating the enterprise data value chain, from ingestion and transformation to insight delivery, supporting teams across the organization as they build, test, and deliver high-impact data products. These products enhance data quality, observability, and overall data health at scale. You’ll work closely with product managers, data scientists, and engineers across the enterprise, transforming strategic goals into tangible solutions that simplify complex workflows and unlock innovation. You will also ensure our services are thoughtfully integrated with broader platform initiatives, contributing to a cohesive and extensible data product development platform. Qualifications, Skills, and Experience•10+ years of professional software development experience, including ownership of production systems•Proficiency in Python and experience building APIs using FastAPI or comparable frameworks•Strong SQL skills and experience working with structured data•Experience working in cloud environments, especially Microsoft Azure (Function Apps, Service Bus, AKS, etc.)•Experience with Databricks and PySpark for data-intensive applications•Experience with Domain-Driven Design and event-driven architecture•Experience with version control systems (Git, SVN)•Deep familiarity with automated testing frameworks (e.g., Pytest, Unittest)•Understanding of Agile methodologies (Scrum, XP, etc.)•Knowledge of RESTful API design and integration•Experience with performance tuning, debugging, and dependency management in production environments•Understanding of CI/CD pipelines and object-oriented design principles (e.g., SOLID)•Familiarity with front-end technologies such as Angular, React, and TypeScript is preferred•Exposure to platform engineering or self-service data tooling is a plus, especially in contexts preparing for AI-driven data applications or agentic automation frameworks Preferred Certifications·Databricks certified Data Engineer Professional·Cloud provider certifications, such as Azure Certified Solutions Architect

Key Responsibilities: •Drive product development: Collaborate with product managers and stakeholders to define, scope, and lead new feature development that meets evolving user needs and aligns with enterprise priorities.•Deliver platform-scale services: Contribute to the implementation and operation of core data platform capabilities using technologies such as Python, FastAPI, Azure Kubernetes Service (AKS), and Databricks.•Promote engineering excellence: Lead by example in writing well-tested, maintainable code. Champion unit and integration testing and contribute to automated validation pipelines.•Mentor and grow others: Provide coaching, feedback, and technical guidance to junior and mid-level engineers, fostering a culture of learning and continuous improvement.•Shape how we work: Lead retrospectives and technical design discussions, identifying opportunities to improve delivery pipelines, team workflows, and system reliability.•Drive estimation and execution: Partner with product managers to scope, estimate, and plan releases for mid- to large-scale initiatives, balancing technical constraints with business value.•Innovate with peers: Collaborate across engineering teams to bring new perspectives to shared problems, drive reusability, and contribute to a modern, flexible data management platform.•Elevate platform engineering: Evolve engineering best practices across the team and platform, contributing to a culture of craftsmanship, experimentation, and operational excellence.•Anticipate future needs: Contribute to the evolution of our platform as we prepare for AI-readiness, including foundational support for intelligent observability and autonomous agents.