Mindrift4 followers18 jobs

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

SeniorPart-timeRemoteNew South Wales, AustraliaPosted 4 days agoVerified 2 weeks ago
Pending fitX of Y criteria met

About the job

This job is about connecting specialists with exciting project-based AI opportunities, specifically focused on evaluating AI coding agents. Your expertise will play a crucial role in creating realistic tasks that challenge AI models, ensuring they can handle real-world developer tasks effectively. The team values flexibility and encourages you to set your own schedule while meeting deadlines.

You'll be responsible for

🛠️

Creating tasks and evaluation criteria

Designing challenging tasks and criteria within realistic simulated environments to evaluate AI coding agents.
🏢

Building developer environments

Constructing a virtual company with a codebase, infrastructure, and context to simulate real-world development scenarios.
🔄

Iterating on tasks and tests

Reviewing agent solutions and refining tasks based on QA feedback to ensure fair and robust evaluations.

Key criteria

🐍

Python development experience

8+ years of experience in Python development, especially with FastAPI.

View

☕

JavaScript/TypeScript skills

Proven experience with JavaScript/TypeScript, particularly in React.

View

🧪

Testing experience

Experience writing functional and integration tests for software.

View

View more

Meet the team

About the company

  • Explore
    • Jobs
    • Companies
    • People
    • Communities
    • Hatch Hotlist 2026
  • Hiring
    • Permanent hires
    • Student placements
  • Resources
    • Blog
    • Community stories
    • Career advice
    • Customer stories
    • Help centre
  • Hatch
    • About
    • Careers
    • Contact
    • Hatch updates
    • Media enquiries
  • © 2026 Hatch
  • Privacy
  • Terms

A meaningful career starts with a match

Get started
Apply on Mindrift
You'll apply on an external website
Think you're a good fit?See what the hiring team are looking for

View your fit

5 criteria for this job
Python development experience
8+ years of experience in Python development, especially with FastAPI.

View

JavaScript/TypeScript skills
Proven experience with JavaScript/TypeScript, particularly in React.

View

Testing experience
Experience writing functional and integration tests for software.

View

Containerization knowledge
Familiarity with Docker for creating and managing containerized applications.

View

Database proficiency
Experience with Postgres, Kafka, and Redis for data management.

View

Similar jobs

View all
Canva
Canva
Senior Python Software Engineer (Backend Workflows Team) - open to remote across ANZMelbourne, VIC · Full-time
This job is about joining a dynamic team at Canva that is redefining how the world experiences design through innovative Python development. The impact of this job is significant, as it directly contributes to enhancing the engineering experience and supporting Canva's AI strategy. The team thrives on collaboration, creativity, and a commitment to excellence, ensuring that every engineer can do their best work.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)QLD · Part-time
This job offers an exciting opportunity to connect with leading tech companies while working on project-based AI initiatives. You'll play a crucial role in evaluating AI coding agents by creating realistic tasks and environments that challenge their capabilities. The team values flexibility and encourages you to set your own schedule while ensuring high-quality outcomes.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)Australia · Part-time
This job offers an exciting opportunity to connect with leading tech companies and work on project-based AI initiatives. By creating realistic tasks and evaluation criteria, you will play a crucial role in assessing AI coding agents and their capabilities. The team values flexibility, allowing you to choose your own schedule while ensuring tasks meet acceptance criteria and deadlines.
Mindrift
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)VIC · Part-time
This job offers an exciting opportunity to connect with leading tech companies while working on project-based AI initiatives. By creating realistic developer environments and challenging tasks, you will play a crucial role in evaluating AI coding agents. The team values flexibility, allowing you to set your own schedule while ensuring tasks meet acceptance criteria.