Mindrift4 followers18 jobs

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

SeniorPart-timeRemoteAustraliaPosted 4 days agoVerified 2 weeks ago
Pending fitX of Y criteria met

About the job

This job offers an exciting opportunity to connect with leading tech companies and work on project-based AI initiatives. By creating realistic tasks and evaluation criteria, you will play a crucial role in assessing AI coding agents and their capabilities. The team values flexibility, allowing you to choose your own schedule while ensuring tasks meet acceptance criteria and deadlines.

You'll be responsible for

🎯

Creating challenging tasks

Designing tasks and evaluation criteria within realistic simulated environments to assess AI coding agents.
🏢

Building realistic developer environments

Crafting a virtual company setup with codebase, infrastructure, and context to simulate real-world scenarios.
🔄

Iterating on tasks and tests

Reviewing agent solutions, analyzing failures, and refining tasks based on feedback to ensure fair evaluations.

Key criteria

🐍

Python development experience

8+ years of experience with Python, particularly FastAPI.

View

🛠️

Testing expertise

Proven experience writing functional and integration tests.

View

📦

Containerization skills

Experience with Docker for application deployment and management.

View

View more

Meet the team

About the company

  • Explore
    • Jobs
    • Companies
    • People
    • Communities
    • Hatch Hotlist 2026
  • Hiring
    • Permanent hires
    • Student placements
  • Resources
    • Blog
    • Community stories
    • Career advice
    • Customer stories
    • Help centre
  • Hatch
    • About
    • Careers
    • Contact
    • Hatch updates
    • Media enquiries
  • © 2026 Hatch
  • Privacy
  • Terms

A meaningful career starts with a match

Get started
Apply on Mindrift
You'll apply on an external website
Think you're a good fit?See what the hiring team are looking for

View your fit

4 criteria for this job
Python development experience
8+ years of experience with Python, particularly FastAPI.

View

Testing expertise
Proven experience writing functional and integration tests.

View

Containerization skills
Experience with Docker for application deployment and management.

View

Database knowledge
Experience with Postgres, Kafka, and Redis for data handling.

View

Similar jobs

View all
Canva
Canva
Senior Python Software Engineer (Backend Workflows Team) - open to remote across ANZMelbourne, VIC · Full-time
This job is about joining a dynamic team at Canva that is redefining how the world experiences design through innovative Python development. The impact of this job is significant, as it directly contributes to enhancing the engineering experience and supporting Canva's AI strategy. The team thrives on collaboration, creativity, and a commitment to excellence, ensuring that every engineer can do their best work.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)NSW · Part-time
This job is about connecting specialists with exciting project-based AI opportunities, specifically focused on evaluating AI coding agents. Your expertise will play a crucial role in creating realistic tasks that challenge AI models, ensuring they can handle real-world developer tasks effectively. The team values flexibility and encourages you to set your own schedule while meeting deadlines.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)QLD · Part-time
This job offers an exciting opportunity to connect with leading tech companies while working on project-based AI initiatives. You'll play a crucial role in evaluating AI coding agents by creating realistic tasks and environments that challenge their capabilities. The team values flexibility and encourages you to set your own schedule while ensuring high-quality outcomes.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)VIC · Part-time
This job offers an exciting opportunity to connect with leading tech companies while working on project-based AI initiatives. By creating realistic developer environments and challenging tasks, you will play a crucial role in evaluating AI coding agents. The team values flexibility, allowing you to set your own schedule while ensuring tasks meet acceptance criteria.