Senior Software Engineer, GenAI Evaluation
googleBrooklyn, NY
Senior Software Engineer, GenAI Evaluation
googleBrooklyn, NYLead L6
Overview:
As a Software Engineer, you design, build, and maintain reusable evaluation infrastructure for scalable AI workflows. You will work on streaming evaluation, autorater pipelines, and metric aggregation to support large-scale experiments. You’ll help product teams adopt self-serve features, ensure data isolation and access control, and transition from offline to online evaluations and live experiments. This role blends systems engineering with AI workflows to enable fast, reliable, and secure evaluation at scale. You will join a fast-paced team that shapes how evaluation informs production AI systems. Compensation / Benefits bonus target equity benefits
Responsibilities:
Design, build, and maintain reusable eval infra blocks (streaming evaluation, autorater pipelines, metric aggregation) Improve scalability, reliability, and efficiency of large-scale eval workflows across compute, storage, and serving backends Onboard product teams, debug issues, and turn recurring needs into self-serve features Implement secure per-customer data isolation and access control for eval data and results Facilitate moving from offline evals to online evals and live experiments/production
Key requirements:
Bachelor's degree or equivalent practical experience 5 years of Python coding experience 3 years of experience developing large-scale infrastructure, distributed systems or networks, or with compute technologies, storage, or hardware architecture 3 years of experience working with Generative AI agents 2 years of multithreading experience team collaboration problem solving communication Python programming large-scale infrastructure / distributed systems Generative AI agents
Matching similar jobs
Level
Lead L6
Location
Brooklyn, NY
Occupation
Software Developers
Industry
Computer Systems Design Services
Posted
today