Responsibilities

Evaluate AI-generated coding interactions end to end.
Judge whether outputs are useful, correct at a high level, and aligned with strong engineering practices.
Assess the quality of explanations, reasoning, preambles, and engineering judgment.
Distinguish and define different levels of response quality.
Provide clear, opinionated written feedback and record short video explanations.
Help define what excellent interaction quality looks like for Cursor, Codex, and Claude Code.

Requirements

Senior, staff, or principal-level software engineering experience, or equivalent experience.
Strong background in TypeScript/JavaScript or Python.
Hands-on experience with at least one of OpenAI Codex, Claude Code, or Cursor.
Deep familiarity with modern AI-assisted development workflows.
Ability to evaluate code without executing it or reviewing every line.
Strong written and spoken English at B2 level or above.
Comfort giving direct, opinionated feedback and maintaining a high engineering-quality bar.

Remote worldwide contract engagement.
Flexible workload of 10–20 hours per week.
Ongoing projects generally lasting about two weeks to a few months.
New projects may be offered to evaluators who perform well.
Start as soon as the take-home exercise is completed and a project seat is available.
Selection process consists of one take-home evaluation exercise with a recorded Loom walkthrough.