Mindrift1 follower72 jobs

Freelance Agent Evaluation Engineer

Mid levelPart-timeRemoteVictoria, AustraliaSoftware engineeringQuality engineeringPosted 8 hours agoVerified 16 hours ago
Pending fitX of Y criteria met

About the job

This job offers an exciting opportunity to work on cutting-edge AI projects, focusing on evaluating coding agents through realistic developer tasks. Contributors will play a vital role in shaping the future of AI by creating and testing challenges that push the boundaries of current models. The team values collaboration, innovation, and a flexible work environment, allowing contributors to choose their hours and work at their own pace.

Meet the team

About the company

  • Explore
    • Jobs
    • Companies
    • People
    • Communities
    • Hatch Hotlist 2025
  • Hiring
    • Permanent hires
  • Resources
    • Blog
    • Community stories
    • Career advice
    • Customer stories
    • Help centre
  • Hatch
    • About
    • Careers
    • Contact
    • Hatch updates
    • Media enquiries
  • © 2026 Hatch
  • Privacy
  • Terms
Think you're a good fit?See what the hiring team are looking for

You'll be responsible for

📝

Creating tasks and evaluation criteria

Develop challenging tasks and criteria for evaluating AI coding agents within realistic simulated environments.
🏢

Designing isolated environments

Set up emulations of a developer's workstation to accurately assess AI performance on coding tasks.
🤖

Iterating with AI agents

Collaborate with AI agents to refine tests, ensuring they accurately evaluate solutions and identify issues.

Key criteria

💻

Software development experience

5+ years in software development, primarily with Python.

View

🔍

Test writing skills

Proven experience writing functional and integration tests.

View

🌐

Full-stack development background

Experience building React-based interfaces and robust back-end systems.

View

View more

A meaningful career starts with a match

View your fit

5 criteria for this job
Software development experience
5+ years in software development, primarily with Python.

View

Test writing skills
Proven experience writing functional and integration tests.

View

Full-stack development background
Experience building React-based interfaces and robust back-end systems.

View

Docker familiarity
Experience with Docker containers and related infrastructure tools.

View

CI/CD understanding
Experience using GitHub Actions for CI/CD processes.

View

Similar jobs

View all
Mindrift
Mindrift
Mechanical Engineer & Python Expert - Freelance AI TrainerNSW · Part-time
This job offers an exciting opportunity to connect with leading tech companies through project-based AI tasks. Contributors will play a vital role in designing and solving computational engineering problems, making a meaningful impact on AI systems. The team values collaboration and innovation, providing a flexible environment where engineers can thrive.
Mindrift
Mindrift
Freelance Agent Evaluation EngineerNSW · Part-time
This job offers an exciting opportunity to work on project-based AI tasks that evaluate coding agents for leading tech companies. Contributors will engage in creating realistic developer tasks, reviewing AI-generated code, and iterating based on expert feedback. The team values collaboration and flexibility, allowing contributors to choose their working hours while ensuring high-quality outcomes.
Mindrift
Mindrift
Freelance Agent Evaluation EngineerAustralia · Part-time
This job is an exciting opportunity to work with Mindrift, connecting specialists with project-based AI opportunities for leading tech companies. You'll play a crucial role in evaluating AI coding agents by creating realistic tasks and criteria, all while collaborating with cutting-edge technology. The team values flexibility and encourages contributors to choose their own working hours while meeting project deadlines.
Mindrift
Mindrift
Freelance Agent Evaluation EngineerQLD · Part-time
This job offers an exciting opportunity to work with Mindrift, connecting specialists with project-based AI opportunities. Contributors will engage in creating and evaluating tasks for AI coding agents, making a significant impact on the development of AI systems. The team values collaboration and innovation, allowing contributors to choose their working hours while meeting project deadlines.