Skip to content

About

A curated list of awesome agentic machine learning projects, frameworks, and resources.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

32 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

Awesome Agentic ML Awesome

Agentic ML refers to autonomous AI systems that can plan, execute, and iterate on machine learning workflows with minimal human intervention—from data preprocessing to model training, evaluation, and deployment.

🤖 This resource list is maintained with the help of Claude by Anthropic.


Scope: What Counts as Agentic ML?

A resource belongs in this list when it demonstrates autonomous, closed-loop progress on machine learning workflows (not just general-purpose agent behavior).

Inclusion checklist (meet at least 3 of 4):

  • Goal-directed ML planning (selects or revises strategy for an ML objective)
  • Tool-using execution across ML lifecycle stages (data prep, feature engineering, model training, evaluation, experimentation)
  • Iterative feedback loop (uses metrics/errors/results to revise actions)
  • Empirical ML outcome (benchmark, competition result, ablation, or measurable improvement)

Exclude or deprioritize:

  • Generic web/desktop/coding agents without meaningful ML workflow contribution
  • Social-only announcements without a primary technical source (paper/repo/blog)
  • Stale resources unless there is a substantive new release/update in the recent window

Research Assistant Agent

This repository runs a weekly Research Assistant Agent via GitHub Actions to scout and triage potential additions.

  • Workflow: .github/workflows/weekly-resource-research.yml
  • Primary curation model: gpt-5.3-codex with xhigh effort
  • Default Grok scout model: grok-4-1-fast-reasoning (override with GROK_MODEL repository variable)
  • Signal sources: xAI Grok social scout + arXiv RSS scout + Codex curation pass with live web search validation

Behavior:

  • If high-confidence additions are found, the agent updates README.md and opens a draft PR with a supporting suggestion log.
  • If no high-confidence additions are found, the agent opens an issue log with the weekly scout outputs (instead of forcing changes).
  • The agent applies the inclusion checklist in this README and avoids generic non-ML-workflow agent news.

Contents


Frameworks & Platforms

End-to-end platforms and frameworks for building agentic ML systems.

Project Description Stars
AlphaEvolve Google DeepMind's evolutionary coding agent for scientific and algorithmic discovery using Gemini. -
AutoGluon Open-source AutoML toolkit by Amazon with foundational models and LLM agents. GitHub stars
EvoAgentX Open-source framework for building, evaluating, and evolving LLM-based agentic workflows. EMNLP 2025. GitHub stars
EvoMaster Foundational auto-research agent framework for self-evolving scientific agents, including ML-Master and MLE-bench Lite workflows. GitHub stars
Karpathy Agentic ML Engineer using Claude Code SDK and Google ADK. By K-Dense. GitHub stars
K-Dense Web Autonomous AI Scientist platform with dual-loop multi-agent system for research, coding, and ML. -
OpenEvolve Open-source implementation of Google DeepMind's AlphaEvolve for evolutionary code optimization. GitHub stars
TSFLab Self-improving workbench that equips coding agents for time-series forecasting research, benchmarking, and maintenance. GitHub stars

AutoML Agents

LLM-powered agents for automated machine learning pipelines.

Project Description Stars
Agent Laboratory Autonomous research framework with specialized agents for literature review, experimentation, and report writing. GitHub stars
AIBuildAI Hierarchical agent for automatically building AI models from task descriptions and training data. GitHub stars
AIDE AI-powered data science agent using tree search for solution exploration. GitHub stars
AIRA-dojo Meta's AI research agents using search policies (Greedy, MCTS, Evolutionary). GitHub stars
AiScientist File-as-Bus long-horizon ML research agent for paper reproduction and Kaggle-style MLE workflows. GitHub stars
AutoGluon Assistant Multi-agent system for end-to-end multimodal ML automation. Also known as MLZero. GitHub stars
AutoMind Adaptive agent with expert knowledge base from 455 Kaggle competitions and tree search. By ZJU NLP. GitHub stars
AutoML-Agent Multi-Agent LLM Framework for Full-Pipeline AutoML. GitHub stars
BobFlow Data science agent that iteratively builds and executes dataflow programs, with analyst inspection of the workflow. GitHub stars
Data Interpreter LLM agent for data science using hierarchical graph modeling and programmable node generation. Part of MetaGPT. ICLR 2025. GitHub stars
DS-Agent Automated data science agent using case-based reasoning from Kaggle. ICML 2024. GitHub stars
EvoDS Self-evolving data science agent that learns reusable skills and compresses long-horizon context. KDD 2026. GitHub stars
FM Agent Baidu's foundation model agent for ML engineering tasks. GitHub stars
FOREAGENT Predict-then-Verify ML agent using run-free preference prediction to reduce expensive execution loops. GitHub stars
InternAgent Unified framework for long-horizon scientific discovery, including ML algorithm design (InternAgent-1.5). The MLE-bench entry used DeepSeek-R1. GitHub stars
LADS (LightAutoDS) Multi-AutoML agentic system combining LLM code generation with AutoGluon, LightAutoML, and FEDOT. GitHub stars
MLEvolve Self-evolving multi-agent framework for ML algorithm discovery using progressive Monte Carlo graph search and retrospective memory. GitHub stars
MLE-STAR Google's ML engineering agent using web search and targeted code block refinement. Built with ADK. -
ML-Master AI-for-AI agent integrating exploration and reasoning with adaptive memory. By SJTU SAI. GitHub stars
MLExpert 35B mixture-of-experts model post-trained for long-horizon autonomous machine learning engineering. GitHub stars
OpenHands Open-source AI software development agent adaptable to ML tasks. GitHub stars
OpenRSI Open stack for recursive self-improvement in machine learning engineering, including OpenMLE and Frontis-MA1. GitHub stars
R&D-Agent Microsoft's research & development agent for ML tasks. GitHub stars
SELA Tree-Search Enhanced LLM Agents for AutoML using MCTS. Part of MetaGPT. GitHub stars

Research Papers

Academic papers on agentic ML, autonomous ML systems, and LLM-based ML agents.

Benchmarks & Evaluation

Papers introducing benchmarks and evaluation methodologies for agentic ML systems.

  • MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering (2024) - Paper | Code
    Benchmark by OpenAI with 75 Kaggle competitions for evaluating ML engineering agents.

  • MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline (2025) - Paper
    Automated pipeline transforming raw datasets into competition-style MLE challenges.

  • MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation (ICML 2024) - Paper
    Benchmark for evaluating LLM agents on ML research tasks including model training and debugging.

  • MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (2025) - Paper
    Benchmark with 201 research tasks from NeurIPS, ICLR, and ICML. Includes MLR-Judge for automated evaluation.

  • DataSciBench: An LLM Agent Benchmark for Data Science (2025) - Paper | Code Comprehensive benchmark with Task-Function-Code (TFC) framework for rigorous evaluation of LLMs on data science tasks.

  • LMR-BENCH: Can LLM Agents Reproduce NLP Research? (EMNLP 2025) - Paper Tasks LLM agents with reproducing functions from NLP research papers. Tests understanding of scientific methods and implementation.

  • DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation (2022) - Paper 1,000 data science problems from StackOverflow covering NumPy, Pandas, SciPy, Scikit-learn, Matplotlib, PyTorch, and TensorFlow.

  • MLE-Dojo: Interactive RL Environment for Machine Learning Engineering (2025) - Paper Transforms MLE-bench into a Gym-style RL environment with 200+ Kaggle competitions, enabling agent training via supervised fine-tuning and reinforcement learning.

  • Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training? (2026) - Paper | Code Benchmark for agents that autonomously design, implement, debug, and execute RL post-training pipelines.

  • Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agents (2026) - Paper | Data Diagnostic suites for silent target and objective misframing in data-science agents, built on DSBench and MLE-bench.

  • DSGym: A Holistic Framework for Evaluating and Training Data Science Agents (2026) - Paper | Code Modular execution framework and task suite for evaluating and training data science agents in self-contained environments.

  • FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics (2026) - Paper | Code Benchmark with 18 fundamental ML research tasks and process-level metrics for studying agent search dynamics.

  • Assembling Insights for Agentic Machine Learning Engineering Systems (2026) - Paper Introduces MLE-InsightBench, 160 Kaggle competitions across 12 domains, and MLE-InsightForge, which builds domain, competition, and improvement insights from source tasks that cannot leak the held-out target.

  • CausalDS: Benchmarking Causal Reasoning in Data-Science Agents (2026) - Paper | Code Synthetic causal scenes with tool-using data-science tasks across Pearl's ladder, including graded abstention when a target is not identified.

  • DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories (2026) - Paper | Code 48 tasks that ask agents to improve published baselines inside imperfect research repositories, with measurements of specification gaming.

  • DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments? (EMNLP 2026) - Paper | Code 275 desktop tasks spanning the data-science lifecycle, graded by deterministic checks of analysis, visuals, and model performance.

  • RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers? (2026) - Paper 100 NeurIPS 2025 papers with a pre-specified result, success criterion, and GPU-hour budget, split into run, retrain, and reimplement tiers.

  • Auto-Research Arena: Benchmarking and Advancing Auto-Research in Language Modeling (2026) - Paper | Code Multi-target benchmark for autonomous research on language-model pretraining, with nine targets across model quality, data, training, and inference efficiency. The paper reports that a simple sequential-search baseline outperforms several more elaborate harnesses.

  • AutoRSE: Evaluating LLM Agents as Recommendation Engineers for Self-Evolving Recommender Systems (2026) - Paper | Code Framework for evaluating agents that autonomously improve recommendation models on six industrial-grade tasks. The paper reports an average relative improvement of 16.1% on five of the six tasks, mostly from tuning, training strategy, and ensembling.

  • From AutoResearch to Continual AutoResearch: Do Agents Learn Across Tasks? (2026) - Paper Compares persistent and reset copies of the same agent on matched streams covering synthetic tasks, symbolic regression, forecasting, and tabular AutoML. The paper reports that persistent state improves research efficiency for all five frontier agents, without a consistent gain in final held-out solutions.

  • ReproBridge: Building Reproducibility into ML Research Artifacts (2026) - Paper | Project Agent workflow that saves a refined codebase, environment, launchers, and reproduction guide from an ML reproduction attempt. On 1,029 tasks from 36 papers, the paper reports mean reproduction success rising from 76.4% to 92.6%.

Autonomous Data Science Agents

LLM agents for end-to-end data science automation with case-based reasoning and knowledge retrieval.

  • DS-Agent: Automated Data Science by Empowering LLMs with Case-Based Reasoning (ICML 2024) - Paper | Code Two-stage framework (development + deployment) using case-based reasoning from Kaggle. GPT-4 achieves 100% success rate in development stage.

  • Agent Laboratory: Using LLM Agents as Research Assistants (2025) - Paper | Code End-to-end autonomous research framework with specialized agents (PhD, Postdoc, ML Engineer, Professor) for literature review, experimentation, and report writing. 84% cost reduction vs. prior methods.

  • DataMaster: Data-Centric Autonomous AI Research (2026) - Paper Data-agent framework using tree-structured search, shared data pools, and cumulative memory to optimize datasets through downstream feedback.

  • AdaPRA: Process Reward Agents with Adaptive Rubrics for Data Science (2026) - Paper | Code Persistent data-science harness whose process-reward agent checks the workspace against task-adaptive rubrics. The paper reports gains of 24.5 and 14.8 leaderboard-percentile points for an 80B Qwen model on DSPredict-Easy and DSPredict-Hard.

  • EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management (KDD 2026) - Paper | Code Learns reusable executable skills and when to compress long-horizon context. The paper reports a 28.9% average improvement over open-source data science agents on four benchmarks.

  • BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows (2026) - Paper | Code An agent iteratively builds and executes dataflow programs for data science tasks, while an analyst can inspect and correct the workflow. The paper reports accuracy gains of up to 18.6 points at just over one third the inference cost of a script-based ReAct agent.

Multi-Agent Systems

Frameworks using multiple specialized agents for end-to-end ML pipelines.

  • AIBuildAI: An AI Agent for Automatically Building AI Models (2026) - Paper | Code Hierarchical manager, designer, coder, and tuner agents for end-to-end AI model development on MLE-bench tasks.

  • AIBuildAI-2: A Knowledge-Enhanced Agent for Automatically Building AI Models (2026) - Paper Extends AIBuildAI with a hierarchical external knowledge system that loads task-relevant documents and writes completed runs back as new knowledge. The paper reports a 70.7% MLE-bench medal rate.

  • Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits (2026) - Paper | Code Alternates two coding agents on one MLE workspace, resetting context and capping experiments per session. The paper reports a 78.2% any-medal rate on the 75 MLE-bench tasks.

  • iML: Executable, Problem-Grounded, and Broadly Exploratory Code-Driven AutoML (2026) - Paper Multi-agent AutoML system that profiles the task, writes a blueprint, and generates modular pipelines with interface checks, execution, and iterative debugging. The paper reports a 90% valid-submission rate and a 45% medal rate on MLE-bench.

  • AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML (ICML 2025) - Paper | Code
    Multi-agent system with data, model, and operation agents for full-pipeline automation.

  • AutoAgents: Generating Specialized Agents for End-to-End Machine Learning Pipelines (2024) - Paper Dynamically composes specialized agents across data prep, training, and evaluation stages, with reported AutoML task speedups.

  • LightAutoDS-Tab: Multi-AutoML Agentic System for Tabular Data (2025) - Paper | Code
    Combines LLM-based code generation with multiple AutoML tools (AutoGluon, LightAutoML, FEDOT).

  • MLZero: A Multi-Agent System for End-to-end Machine Learning Automation (NeurIPS 2025) - Paper | Code
    Transforms raw multimodal data into ML solutions with zero human intervention.

  • SmartDS-Solver: Agentic AI for Vertical Domain Problem Solving in Data Science (ICLR 2026 Submission) - Paper Reasoning-centric system with SARTE algorithm for data science problem solving.

  • AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions (2024) - Paper Multi-agent framework for autonomous Kaggle competitions with 85% valid submission rate. Iterative development with code execution, debugging, and unit testing.

  • The AI Data Scientist (2025) - Paper Multi-subagent architecture with six specialized subagents (Data Cleaning, Hypothesis, Preprocessing, Feature Engineering, Model Training, Call-to-Action) for end-to-end workflows.

  • Data Interpreter: An LLM Agent for Data Science (ICLR 2025) - Paper | Code Hierarchical graph modeling with programmable node generation. Outperforms AutoGen on ML benchmarks.

  • MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines (ICML 2025) - Paper FSM-based framework that auto-generates multi-agent systems with state traceback for self-correction. Matches or exceeds human-designed systems.

  • Toward Autonomous Long-Horizon Engineering for ML Research (2026) - Paper | Code AiScientist coordinates specialist agents through durable workspace artifacts for paper reproduction and MLE competition workflows.

  • Agent Stack for ML Training Lifecycle: Generation, Validation, and Optimization (2026) - Paper LinkedIn stack with AutoTorch for generating and migrating training pipelines and AutoResearch for architecture and training-configuration search. Validation hooks repair candidates before full-scale training. The paper reports completed generation and training in 92 of 115 runs, and a primary-metric gain on 28 of 100 production pipelines.

  • Validation-Gated ML Pipeline Synthesis with Expert-Grounded Self-Evolving Skills (2026) - Paper EvoML uses orchestrator, executor, and curator agents to synthesize ML pipelines, advancing a phase only after structural, execution, and semantic checks, and writing recovered skills back into a library. The paper reports the best final model quality on all eight evaluation datasets.

Search & Planning Methods

Papers using tree search, MCTS, or structured planning for ML workflow optimization.

  • AI Research Agents for Machine Learning (2025) - Paper | Code
    Formalizes AI research agents as search policies with operators. Compares Greedy, MCTS, and Evolutionary strategies.

  • AIRA_2: Overcoming Bottlenecks in AI Research Agents (2026) - Paper Uses asynchronous multi-GPU workers, hidden consistent evaluation, and dynamic ReAct operators to scale MLE-bench-30 search.

  • AutoMind: Adaptive Knowledgeable Agent for Automated Data Science (2025) - Paper | Code
    Features curated expert knowledge base from 455 Kaggle competitions, agentic knowledgeable tree search, and self-adaptive coding strategy.

  • Can We Predict Before Executing Machine Learning Agents? (ACL 2026) - Paper | Code FOREAGENT uses a Predict-then-Verify loop to prune expensive ML execution, accelerating convergence while improving outcomes.

  • I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search (EACL 2026 Findings) - Paper | Code
    Introspective node expansion with hybrid LLM-estimated and actual performance rewards.

  • MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement (2025) - Paper | Blog
    Uses web search to retrieve models and targeted code block refinement via ablation studies.

  • ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning (2025) - Paper | Code Integrates exploration and reasoning with adaptive memory mechanism.

  • ML-Master 2.0: Toward Ultra-Long-Horizon Agentic Science (2026) - Paper | Code Hierarchical Cognitive Caching (HCC) for ultra-long-horizon ML engineering. Achieves 56.44% on MLE-bench via cognitive accumulation across 24-hour runs.

  • MARS: Modular Agent with Reflective Search for Automated AI Research (2026) - Paper CAIR agent using budget-aware MCTS, a design-decompose-implement pipeline, and comparative reflective memory. Official MLE-bench entries are MARS at 56.00% and MARS+ at 62.67% Any Medal (24h).

  • Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineering (2026) - Paper Hierarchical MLE agent in which an orchestrator keeps compact long-horizon state and sub-agents run solution attempts. The paper reports up to a 36.7% relative gain for a Qwen3-30B-Coder variant.

  • MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery (2026) - Paper | Code Progressive Monte Carlo graph search with retrospective memory for end-to-end ML algorithm discovery. Official MLE-bench result: 61.33% Any Medal in 12 hours (Feb 2026).

  • PiML: Automated Machine Learning Workflow Optimization using LLM Agents (AutoML 2025) - Paper Persistent iterative framework with adaptive memory and systematic debugging.

  • Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search (2026) - Paper | Code Introduces Gome, which maps diagnostic reasoning, success memory, and multi-trace execution into a directed optimization process for MLE agents.

  • RF-Agent: Automated Reward Function Design via Language Agent Tree Search (NeurIPS 2025) - Paper | Code Frames reward-function design as sequential decision-making with MCTS-guided language agents, showing gains across 17 low-level control tasks.

  • SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning (2024) - Paper | Code
    Leverages MCTS to expand the search space with insight pools.

  • ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond (2026) - Paper Organizes long-horizon research into recoverable executable states and allocates compute from validated progress. The paper reports 70.22% Any Medal on full MLE-bench in 24 hours.

  • MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model (2026) - Paper | Code Fine-tunes a model to predict a candidate program's terminal output, errors, and validation score without running it, then uses that prediction to scale test-time search. On 17 MLE-bench Lite tasks, the paper reports up to 22.1% higher observed mean performance than the same agent without the world model, with less real execution.

Domain-Specific Agentic ML

Agentic systems tailored for specific ML domains.

  • AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery (2025) - Paper | Blog Google DeepMind's evolutionary coding agent pairing Gemini LLMs with automated evaluators. Discovered novel matrix multiplication algorithms and optimizes data center operations.

  • AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific ML (2025) - Paper Specialized agents propose, critique, and refine SciML solutions.

  • AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research (NeurIPS 2025 Position) - Paper
    Position paper on AI automation for scientific discovery with multi-agent systems to simulate research societies.

  • ClimateAgent: Multi-Agent Orchestration for Complex Climate Data Science Workflows (TMLR) - Paper
    Multi-agent framework for end-to-end climate data analytics with dynamic API awareness and self-correction.

  • The AI Cosmologist: Agentic System for Automated Data Analysis (2025) - Paper
    Automates cosmological data analysis from idea generation to research dissemination.

  • TS-Agent: Structured Agentic Workflows for Financial Time-Series Modeling (2025) - Paper
    Modular framework for financial forecasting with structured knowledge banks.

  • InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery (2026) - Paper | Code Generation, verification, and evolution subsystems for extended discovery cycles, including autonomous design of methods for core machine learning problems.

  • A Multi-Agent Framework for Automated Machine Learning with Evidence-based Planning for Medical Image Analysis (2026) - Paper MedicalMLOmni plans model choice from published results and runs parallel hyperparameter search for single- and multi-modal medical tasks. The paper reports the best score on 19 of 20 benchmarks, with backbone-matched controls attributing the gains to the framework.

  • TSFLab: A Self-Improving Agent-Agnostic Workbench for Time Series Forecasting Research (2026) - Paper | Code Shared tools that let coding agents reproduce, benchmark, and develop forecasting methods, and keep those resources maintained. The paper reports 20-163% higher task completion and 11-66% fewer fresh tokens per completed task than the baselines.

LLM-Based ML Optimization

Using LLMs for specific ML optimization tasks.

  • Using Large Language Models for Hyperparameter Optimization (2023) - Paper Iterative HPO via LLM prompting. Matches or outperforms Bayesian optimization in limited-budget settings.

  • ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering (2025) - Paper Applies online reinforcement learning to train LLM agents for ML tasks with exploration-enriched fine-tuning and step-wise RL.

  • AceGRPO: Adaptive Curriculum Enhanced Group Relative Policy Optimization for Autonomous Machine Learning Engineering (2026) - Paper | Code Reinforcement learning method for MLE agents using evolving execution traces and learnability-guided adaptive sampling.

  • Synthetic Sandbox for Training Machine Learning Engineering Agents (2026) - Paper SandMLE generates micro-scale synthetic MLE environments to make on-policy trajectory-wise RL practical for ML engineering agents.

  • Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering (2026) - Paper | Code Post-trains a 35B meta-evolution agent on OpenMLE with Draft, Improve, Debug, and Crossover operators. On MLE-Bench Lite (12h, one 12 GB RTX 4090), the paper reports medal average rising from 39.39% to 60.61% with OpenMLE-Evo, and 71.21% with OpenMLE-Evo-Max.

  • MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering (2026) - Paper | Code Post-trains a 35B mixture-of-experts model with hierarchical agentic supervised fine-tuning and windowed interaction RL. On MLE-Bench Lite, the paper reports the base model's average medal rate rising from 37.9% to 72.7%.

  • MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development (2026) - Paper Trains agents to use diagnostic tools for data inspection, code verification, and experiment diagnosis. The paper reports in-domain success rising from 24.8% to 52.4% for Qwen3-8B and from 35.6% to 69.2% for Qwen3.5-35B-A3B.

  • Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents (2026) - Paper Optimizes reward per unit time for MLE actions whose durations vary. On a Qwen3.5-4B self-improvement loop, the paper reports higher fixed-budget reward than vanilla RL on MLE-Bench and NanoGPT.

  • From Intent to Improvement: Agentic Dataset Search for Better Model Training (2026) - Paper | Code AgenticDS turns a training-data request into a typed search plan, runs deterministic retrieval, and accepts a dataset only after provenance checks. On HuDS-Bench, the paper reports 25.7% higher NDCG@10 than the strongest dense-sparse hybrid, and the selected datasets give the best downstream accuracy in mathematics, code, and medicine.

  • Training LLM Agents to Automate LLM Post-Training with RSI-Dojo (2026) - Paper Converts research artifacts into 281 executable post-training tasks and fine-tunes agents on filtered training trajectories. The paper reports that RSI-Dojo-35B-A3B raises its base model by 16.87 PostTrainBench points, to 29.95 on PostTrainBench and 53.29 on PostTrainBench-Lite.

  • CAAFE: Context-Aware Automated Feature Engineering (NeurIPS 2024) - Paper LLM-driven automated feature engineering pipeline that generates and executes code for new features using dataset context.

  • LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers (TMLR 2026) - Paper | Code Combines evolutionary search with LLM reasoning to discover effective feature transformations. Outperforms CAAFE and other baselines on classification and regression benchmarks.

  • A Human-Centered Automated Machine Learning Agent with LLMs (2025) - Paper LLM-driven agent enabling natural language interaction throughout the entire ML workflow with adaptive hyperparameter optimization.

Surveys

Survey papers covering the agentic ML landscape.

  • Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey (ACM TOSEM, 2025) - Paper First SE-oriented, stage-wise review of LLM-based ML workflow automation covering data/feature engineering, model selection, HPO, and evaluation.

  • Large Language Model-based Data Science Agent: A Survey (2025) - Paper Comprehensive survey examining how LLM-agent systems automate end-to-end data science pipelines.

  • A Survey on Large Language Model-based Agents for Statistics and Data Science (The American Statistician, 2025) - Paper Overview of evolution, capabilities, and applications of LLM-based data agents for simplifying complex data tasks.

  • Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions (2025) - Paper Categorizes agentic systems for scientific discovery into autonomous and collaborative frameworks across chemistry, biology, and materials science.

  • From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery (2025) - Paper Domain-oriented review unifying process-oriented, autonomy-oriented, and mechanism-oriented perspectives on autonomous scientific discovery.

  • From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery (2025) - Paper Analyzes LLM progression from discrete task-oriented functions to sophisticated multi-stage agentic workflows across six stages of the scientific method.

  • A Comprehensive Survey of Self-Evolving AI Agents (2025) - Paper | Code Survey on agent evolution techniques that automatically enhance agent systems, bridging foundation models with continuous adaptability.

Foundation Models for ML

Pre-trained models that enable rapid ML development.

  • TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (ICLR 2023) - Paper | Code
    Prior-Data Fitted Network using in-context learning for instant tabular classification.

  • Unlocking the Full Potential of Data Science Requires Tabular Foundation Models, Agents, and Humans (NeurIPS 2025 Position) - Paper
    Position paper on collaborative systems integrating agents, tabular foundation models, and human experts for data science.


Datasets & Benchmarks

Benchmarks and datasets for evaluating agentic ML systems.

Benchmark Description Link
Agent^2 RL-Bench Benchmark for agents that autonomously engineer RL post-training loops for foundation models. Paper | GitHub
Ambig-DS Diagnostic suites for target and objective ambiguity in data-science agents. Paper | Data
Auto-Research Arena Nine-target benchmark for autonomous research on language-model pretraining. Paper | GitHub
AutoML-Agent Benchmark 18 diverse datasets across tabular, CV, NLP, time-series, and graph tasks. Paper
AutoRSE Six industrial recommendation tasks for agents that improve recommender models. Paper | GitHub
CausalDS Synthetic causal scenes for tool-using data-science agents, including abstention tasks. Paper | GitHub
Continual AutoResearch Matched task streams for testing whether ML research agents reuse experience across problems. Paper
DataSciBench Comprehensive data science benchmark with TFC framework for LLM evaluation. Paper | GitHub
DeltaML-Bench 48 repository tasks that ask agents to improve published ML baselines. Paper | GitHub
DS-1000 1,000 data science code generation problems from StackOverflow across 7 libraries. Paper
DSAgentBench 275 real-computer tasks covering end-to-end data-science workflows. Paper | GitHub
DSGym Modular task suite and execution environment for evaluating and training data science agents. Paper | GitHub
FML-bench 18 fundamental ML research tasks with process-level metrics for agent search dynamics. Paper | GitHub
GAIA General AI Assistants benchmark testing real-world reasoning and tool use. Paper
HuDS-Bench 200 dataset-search queries with graded judgments for training-data agents. Paper | GitHub
LMR-BENCH Benchmark tasking agents with reproducing functions from NLP research papers. Paper
MLE-bench Kaggle-based benchmark for ML engineering agents by OpenAI. 75 competitions. Paper | GitHub
MLE-Dojo Gym-style RL environment built on MLE-bench with 200+ Kaggle competitions for agent training. Paper
MLE-InsightBench 160 Kaggle competitions for studying insight reuse in agentic MLE systems. Paper
MLE-Smith Automated pipeline for generating competition-style MLE challenges from raw datasets. Paper
MLAgentBench Benchmark for LLM agents on ML experimentation tasks. Paper
MLR-Bench Open-ended ML research benchmark with 201 tasks from major ML conferences. Paper
RECLAIM 100 NeurIPS 2025 papers for reproducing a pre-specified machine learning claim. Paper
ReproBridge-Bench 1,029 reproduction tasks from 387 experiments across 36 machine learning papers. Paper | Project
RSI-Dojo 281 executable post-training environments for teaching agents to train models. Paper

MLE-bench Leaderboard

Top-performing agents on MLE-bench (75 Kaggle competitions, ICLR 2025 Oral). Scored by "Any Medal %" — percentage of competitions earning at least a bronze medal. Agents run on 36 vCPUs, 440GB RAM, and one 24GB A10 GPU. Official leaderboard submissions have been paused since April 24, 2026 while OpenAI develops an improved comparability process. Medal rates reported by later papers are not official leaderboard entries unless they appear in this table.

Rank Agent LLM All (%) Hours Date
1 Famou-Agent 2.0 Gemini-3-Pro-Preview 64.44 24 Feb 2026
2 AIBuildAI Claude-Opus-4.6 63.11 24 Mar 2026
3 CAIR MARS+ Gemini-3-Pro-Preview 62.67 24 Feb 2026
4 MLEvolve Gemini-3-Pro-Preview 61.33 12 Feb 2026
5 PiEvolve (Fractal AI) Gemini-3-Pro-Preview 61.33 24 Jan 2026
6 Famou-Agent 2.0 Gemini-2.5-Pro 59.56 24 Dec 2025
7 ML-Master 2.0 (SJTU) DeepSeek-V3.2-Speciale 56.44 24 Dec 2025
8 CAIR MARS Gemini-3-Pro-Preview 56.00 24 Jan 2026
9 PiEvolve (Fractal AI) Gemini-3-Pro-Preview 52.00 12 Jan 2026
10 Leeroo Gemini-3-Pro-Preview 50.67 24 Dec 2025
11 Thesis gpt-5-codex 48.44 24 Nov 2025
12 CAIR MLE-STAR-Pro-1.5 Gemini-2.5-Pro 44.00 24 Nov 2025
13 Famou-Agent Gemini-2.5-Pro 43.56 24 Oct 2025
14 Operand ensemble gpt-5 + multi-model 39.56 24 Oct 2025
15 CAIR MLE-STAR-Pro-1.0 Gemini-2.5-Pro 38.67 12 Nov 2025
16 InternAgent DeepSeek-R1 36.44 12 Sep 2025
17 R&D-Agent gpt-5 35.11 12 Sep 2025
18 Neo multi-agent Undisclosed 34.22 36 Jul 2025
19 AIRA-dojo (Meta) o3 31.60 24 May 2025
20 R&D-Agent o3 + GPT-4.1 30.22 24 Aug 2025

Top score improved from 16.9% (Oct 2024) to 64.4% (Feb 2026) — a ~3.8x improvement in 16 months. See MLE-bench README for the full leaderboard with per-difficulty breakdowns and grading reports.


Related Resources

Curated reading lists and paper collections on agentic ML.

Resource Description Stars
LLM4AutoML Curated list of papers on using LLMs for AutoML. GitHub stars
LLM-Based Data Science Agent Reading List Reading list of papers on LLM-based data science agents. GitHub stars
ai-agent-papers Biweekly-updated collection of AI agent research papers. GitHub stars
Awesome-Self-Evolving-Agents Survey and paper list on self-evolving AI agents bridging foundation models and lifelong systems. GitHub stars
Awesome-Agent-Papers Up-to-date survey on LLM agent methodology, applications, and challenges. GitHub stars

Contributing

Contributions are welcome! To add a project or paper, simply open an issue or submit a PR.

When proposing additions, include a short note on which inclusion criteria the item satisfies and link the strongest supporting evidence (paper/repo/benchmark/blog).


License

CC0

To the extent possible under law, the authors have waived all copyright and related rights to this work.

About

A curated list of awesome agentic machine learning projects, frameworks, and resources.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors