Research Scientist, AI Benchmarks
virtual vocationsNew York, NY
Research Scientist, AI Benchmarks
virtual vocationsNew York, NYSenior L5
Computer and Information Research ScientistsData ScientistsMarket Research Analysts and Marketing Specialists
Other Scientific and Technical Consulting ServicesAll Other Professional, Scientific, and Technical ServicesResearch and Development in the Physical, Engineering, and Life Sciences
APPLY NOW (opens in a new tab)To support the development of AI benchmarks, the full-time remote Research Scientist, AI Benchmarks will design public benchmarks for image, video, and document models, build private test sets from real data, and run data-value experiments to measure the impact of Kled data on model performance.
Key responsibilities:
Design and ship public benchmarks for various AI models
Build private test sets using real, opt-in data
Run experiments to evaluate the value of Kled data in training modelsRequired qualifications 5-7+ years of ML research or research engineering experience
Proven experience creating benchmarks or evaluations used by others
Strong hands-on experience with training and fine-tuning image or video models using PyTorch
Deep understanding of experimental design and statistics
Experience running end-to-end experiments from GPUs to final reporting
Matching similar jobs
Level
Senior L5
Location
New York, NY
Occupation
Computer and Information Research Scientists
Industry
Other Scientific and Technical Consulting Services
Posted
3 days ago