К содержанию
RabotaYou
Ко всем вакансиям
C

Clera

Expert Talent Pool, Evals and Benchmarks (Associate)

Со всего мира
Где
Удалённо, весь мир
Зарплата
$30–60 в час
Уровень
Senior
Занятость
Полная занятость
Направление
HR и рекрутинг
Опубликовано
42 мин назад

Коротко о вакансии

Join a specialist talent pool supporting human-data projects that evaluate AI models. You will help design and run assessments that measure model capabilities and contribute to AI safety and capability research.

Кто может откликнуться

В поле «локация» указано: Worldwide (Himalayas)

Описание вакансии

Описание на английском: работодатель ждёт отклик на английском.

About the Role

Join a specialist talent pool supporting human-data projects that evaluate AI models. You will help design and run assessments that measure model capabilities and contribute to AI safety and capability research.

What You'll Do

  • Write questions and tasks that isolate and test specific model capabilities.

  • Develop rubrics that define strong, partial, and poor answers.

  • Create reference answers and edge cases to challenge AI models.

  • Grade model outputs against rubrics and explain scoring decisions.

  • Analyze failed answers, group error patterns, and document findings.

  • Compare multiple models using identical prompts.

  • Validate benchmarks and flag compromised or ineffective questions.

  • Participate in small user studies evaluating ratings or interactions with model outputs.

What We're Looking For

  • Zero to five years of professional experience.

  • Experience in QA or test engineering, assessment design, psychometrics, UX research, data analysis, ML evaluation, research support, or relevant subject-matter work involving question writing.

  • Experience writing test cases, exam questions, rubrics, or evaluation tasks, and reviewing work consistently against written criteria.

  • Comfort using spreadsheets or Python to organize and analyze results, and identifying and documenting error patterns.

  • Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo is useful.

  • Clear written communication and the ability to compare AI model outputs or participate in user and rating studies.

Compensation & Benefits

Compensation is $30 to $60 per hour, depending on project, field, and experience. Payment is for hours worked.

Location

Fully remote. Projects typically last from one week to several months.

Compensation Range: $62.4K - $124.8K

Originally posted on Himalayas

Похожие вакансии