Sr Data Scientist
amazonSanta Clara, CA
Sr Data Scientist
amazonSanta Clara, CALead L6
Other Scientific and Technical Consulting ServicesComputer Systems Design ServicesCustom Computer Programming Services
Overview:
In this role, you will design evaluation and benchmark datasets and the metrics to gauge Quick’s AI features. You will run end-to-end human annotation programs and apply model-based evaluation to ensure alignment with customer needs. You’ll analyze performance to identify gaps and drive high-impact improvements in collaboration with science, engineering, and product teams. Your work will shape product direction and model improvements at enterprise scale, contributing to Safe and effective AI in Quick. This position sits on the Quick Data team, which focuses on high-quality data and responsible AI practices. Compensation / Benefits health insurance (medical, dental, vision) 401(k) matching RSUs paid time off parential leave sign-on payments
Responsibilities:
Design and build evaluation and benchmark datasets Define quality metrics and evaluation frameworks (human evaluation, LLM-as-a-Judge, correlation checks) Run human annotation programs end-to-end (guidelines, inter-annotator agreement, data quality) Analyze model/agent performance to identify gaps and high-impact improvements Partner with science, engineering, and product teams to translate evaluation results into roadmap decisions Present findings and recommendations to senior leadership
Key requirements:
5+ years of data querying languages (SQL), scripting languages (Python) or statistical/mathematical software 4+ years of data scientist experience Experience as a leader and mentor on a data science team cross-functional collaboration strong communication to senior leadership mentorship and coaching SQL Python R/SAS/Matlab
Matching similar jobs
Level
Lead L6
Location
Santa Clara, CA
Occupation
Data Scientists
Industry
Other Scientific and Technical Consulting Services
Posted
yesterday