Skip to main content

Sr Data Scientist

amazonSanta Clara, CALead L6
Overview: In this role, you will design evaluation and benchmark datasets and the metrics to gauge Quick’s AI features. You will run end-to-end human annotation programs and apply model-based evaluation to ensure alignment with customer needs. You’ll analyze performance to identify gaps and drive high-impact improvements in collaboration with science, engineering, and product teams. Your work will shape product direction and model improvements at enterprise scale, contributing to Safe and effective AI in Quick. This position sits on the Quick Data team, which focuses on high-quality data and responsible AI practices. Compensation / Benefits health insurance (medical, dental, vision) 401(k) matching RSUs paid time off parential leave sign-on payments
Responsibilities: Design and build evaluation and benchmark datasets Define quality metrics and evaluation frameworks (human evaluation, LLM-as-a-Judge, correlation checks) Run human annotation programs end-to-end (guidelines, inter-annotator agreement, data quality) Analyze model/agent performance to identify gaps and high-impact improvements Partner with science, engineering, and product teams to translate evaluation results into roadmap decisions Present findings and recommendations to senior leadership
Key requirements: 5+ years of data querying languages (SQL), scripting languages (Python) or statistical/mathematical software 4+ years of data scientist experience Experience as a leader and mentor on a data science team cross-functional collaboration strong communication to senior leadership mentorship and coaching SQL Python R/SAS/Matlab
APPLY NOW (opens in a new tab)

Matching similar jobs

Level

Lead L6

Location

Santa Clara, CA

Occupation

Data Scientists

Industry

Other Scientific and Technical Consulting Services

Posted

yesterday

APPLY NOW (opens in a new tab)