Mindrift4 followers18 jobs

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

SeniorPart-timeRemoteVictoria, AustraliaPosted 4 days agoVerified 2 weeks ago
Pending fitX of Y criteria met

About the job

This job offers an exciting opportunity to connect with leading tech companies while working on project-based AI initiatives. By creating realistic developer environments and challenging tasks, you will play a crucial role in evaluating AI coding agents. The team values flexibility, allowing you to set your own schedule while ensuring tasks meet acceptance criteria.

You'll be responsible for

🧩

Creating challenging tasks

Designing tasks and evaluation criteria within realistic simulated environments to assess AI coding agents.
🏢

Building realistic developer environments

Constructing a virtual company with codebase, infrastructure, and context to simulate a believable development history.
🔄

Iterating on tasks and tests

Reviewing agent solutions, analyzing failures, and refining tasks based on QA feedback to ensure fair evaluations.

Key criteria

🐍

Python development experience

8+ years of experience with Python, particularly FastAPI.

View

🛠️

Testing skills

Proven experience writing functional and integration tests.

View

🌐

JavaScript/TypeScript knowledge

Experience with JavaScript/TypeScript, especially React.

View

View more

Meet the team

About the company

  • Explore
    • Jobs
    • Companies
    • People
    • Communities
    • Hatch Hotlist 2026
  • Hiring
    • Permanent hires
    • Student placements
  • Resources
    • Blog
    • Community stories
    • Career advice
    • Customer stories
    • Help centre
  • Hatch
    • About
    • Careers
    • Contact
    • Hatch updates
    • Media enquiries
  • © 2026 Hatch
  • Privacy
  • Terms

A meaningful career starts with a match

Get started
Apply on Mindrift
You'll apply on an external website
Think you're a good fit?See what the hiring team are looking for

View your fit

4 criteria for this job
Python development experience
8+ years of experience with Python, particularly FastAPI.

View

Testing skills
Proven experience writing functional and integration tests.

View

JavaScript/TypeScript knowledge
Experience with JavaScript/TypeScript, especially React.

View

Containerization expertise
Experience with Docker for application deployment.

View

Similar jobs

View all
Canva
Canva
Senior Python Software Engineer (Backend Workflows Team) - open to remote across ANZMelbourne, VIC · Full-time
This job is about joining a dynamic team at Canva that is redefining how the world experiences design through innovative Python development. The impact of this job is significant, as it directly contributes to enhancing the engineering experience and supporting Canva's AI strategy. The team thrives on collaboration, creativity, and a commitment to excellence, ensuring that every engineer can do their best work.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)NSW · Part-time
This job is about connecting specialists with exciting project-based AI opportunities, specifically focused on evaluating AI coding agents. Your expertise will play a crucial role in creating realistic tasks that challenge AI models, ensuring they can handle real-world developer tasks effectively. The team values flexibility and encourages you to set your own schedule while meeting deadlines.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)QLD · Part-time
This job offers an exciting opportunity to connect with leading tech companies while working on project-based AI initiatives. You'll play a crucial role in evaluating AI coding agents by creating realistic tasks and environments that challenge their capabilities. The team values flexibility and encourages you to set your own schedule while ensuring high-quality outcomes.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)Australia · Part-time
This job offers an exciting opportunity to connect with leading tech companies and work on project-based AI initiatives. By creating realistic tasks and evaluation criteria, you will play a crucial role in assessing AI coding agents and their capabilities. The team values flexibility, allowing you to choose your own schedule while ensuring tasks meet acceptance criteria and deadlines.