Software Engineer, Data Platform
consultnetBrooklyn, NY
Software Engineer, Data Platform
consultnetBrooklyn, NYLead L6
Overview:
In this backend/systems role, you will own the design of schemas, migrations, and real-time pipelines that support hundreds of concurrent ML jobs. You will build real-time ingestion and consolidate existing schemas into a scalable architecture powering an AI-enabled data layer for engineers. This role offers greenfield ownership and the freedom to choose the right tools, with a focus on delivering a robust data platform for AI products. You will work closely with cross-functional teams to shape scalable data infrastructure that enables fast, reliable queries.
Responsibilities:
Design schemas, migrations, and real-time pipelines for high-volume ML workloads Build real-time ingestion to support concurrent ML outputs Consolidate messy schemas into a clean, scalable data architecture Power the data layer behind an AI agent product and enable easy querying by engineers
Key requirements:
5+ years of engineering experience CS or Computer Engineering degree production experience with data-intensive systems deep database knowledge (Postgres/Aurora preferred) Schema design Database optimization Real-time data pipelines PostgreSQL/Aurora Streaming or event-driven pipelines (nice to have) OLAP/columnar engines (DuckDB, ClickHouse) (nice to have)
Matching similar jobs
Level
Lead L6
Location
Brooklyn, NY
Occupation
Software Developers
Industry
Computer Systems Design Services
Posted
today