Data Engineer

Duration: 6 months contract to Perm Location: New York, NY 10001, (onsite)

Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD.

About the Role: We are seeking a Lead Software Engineer – Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence.

Responsibilities: Design and develop large-scale data pipelines and data platforms. Build cloud-native solutions using AWS services and Spark. Lead data modeling, warehousing, migration, and analytics initiatives. Improve system performance, resilience, monitoring, and automation. Collaborate with cross-functional teams and mentor junior engineers.

Required Qualifications: Strong proficiency in Python and/or Java with Spark/PySpark. Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration. Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation).Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles. Experience with data formats such as Parquet, Iceberg, Avro, or JSON.Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager.

Preferred Qualifications: Experience building REST APIs using Java. Knowledge of Hybrid Cloud architectures. Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD.Strong leadership, problem-solving, and communication skills.