Backend Engineer (Python/JavaScript) – Remote, $15/hr

About the role

We are looking for a Backend Engineer to create multi‑agent benchmark tasks that evaluate AI coding agents. This short‑term contract (4 weeks) is fully remote and offers $15 per hour for 20‑40 hours per week.

Key responsibilities

• Build benchmark tasks based on real‑world open‑source code changes (bug fixes, migrations, refactors).
• Use the Harbor evaluation framework to run and validate tasks inside Docker containers.
• Write clear task instructions, including file paths, function signatures, expected behavior, and constraints.
• Develop Python verification scripts to check the correctness of agent‑generated code.
• Create decomposition strategies that split complex changes across independent sub‑agents.
• Run, debug, and refine tasks in containerised environments to ensure reproducibility.
• Evaluate performance signals and continuously improve task quality, clarity, and difficulty.

Required profile

• Strong experience in Python and JavaScript development.
• Familiarity with AI coding benchmarks such as SWE‑bench or Terminal‑Bench.
• Proven ability to read and navigate large open‑source codebases (e.g., Django, Flask, FastAPI, Node.js).
• Comfortable working independently in a remote setting.

Required skills

• Python
• JavaScript
• Docker (Dockerfiles, image building, debugging)
• Git workflows (pull requests, diffs, cherry‑picking)
• Testing frameworks (pytest, unittest or custom assertions)
• Experience with Django, Flask, FastAPI, or Node.js

What we offer

• Remote work with flexible hours (20‑40 h/week) and a 4‑hour overlap with PST.
• Competitive hourly rate of $15 USD.
• Short‑term contract providing focused, high‑impact experience.