Check location eligibility. Remote describes how the work is done; it does not necessarily mean applications are accepted worldwide. Check the location and residency requirements in this listing before applying.
About the role
Job Description: Machine Learning Engineer – AI Training & Model Evaluation
Role Type: Contractor
Location: Global (Fully Remote)
Time Commitment: 15+ Hours per Week
Schedule: Flexible
About the Role
We are seeking experienced Machine Learning Engineers with prior professional AI training or model evaluation experience to contribute to a project focused on improving advanced AI systems.
In this role, you will apply your machine learning expertise to develop challenging technical tasks, evaluate AI-generated outputs, identify model failures, and establish objective standards for technical accuracy. This opportunity is intended for professionals who have already completed substantial AI training, benchmarking, or model evaluation work.
Key Responsibilities
Create realistic machine learning engineering tasks that challenge advanced AI models.
Develop detailed task instructions, reproducible environments, reference solutions, and evaluation criteria.
Evaluate AI-generated code, experiments, analyses, and technical deliverables.
Identify technical issues involving data leakage, incorrect validation, unstable training, performance limitations, and reproducibility.
Assess model-generated solutions for correctness, reliability, and technical quality.
Provide actionable feedback to improve AI model performance and evaluation standards.
Required Skills and Qualifications
Prior paid professional experience in training, evaluating, benchmarking, or improving AI systems.
Strong practical proficiency in Python.
Hands-on experience with machine learning frameworks and tools such as PyTorch, JAX, TensorFlow, scikit-learn, XGBoost, LightGBM, or equivalent technologies.
Strong understanding of machine learning workflows, model validation, training methodologies, and performance evaluation.
Public Proof of Technical Expertise
Strong applicants should provide publicly verifiable evidence of their technical expertise, such as:
GitHub repositories or public machine learning implementations.
Kaggle, DrivenData, AIcrowd, or similar competition results.
Hugging Face models or datasets.
Research publications or technical reports.
Open-source machine learning contributions.
Public benchmarks, experiments, or technical projects.
Preferred Qualifications
Experience creating AI training tasks, reference answers, evaluation rubrics, graders, or benchmark problems.
Experience reviewing technical submissions from other AI trainers.
Kaggle Expert, Master, or Grandmaster status, or strong results in other technical competitions.
Publications or open-source contributions in machine learning.
Experience with long-horizon or agentic AI evaluations.
Availability to begin immediately.
Application Process
Submit your application, including details of your AI training experience and publicly verifiable technical contributions.
Complete an AI interview involving technical and integrity checks.
Undergo a Hiring Manager review.
If selected, complete project onboarding and begin your first assigned task.
Originally posted on Himalayas
