AI Scientist
How it works
The system acts as an agent (or a team of specialized agents) driven by an LLM. A typical pipeline: (1) idea generation — the model proposes new directions and checks novelty against the literature (e.g. via Semantic Scholar); (2) experimental iteration — the agent writes and runs code, collects metrics, generates plots; (3) paper write-up — the system produces a manuscript (e.g. in LaTeX) with citations; (4) automated peer review — a separate reviewer agent scores the paper and feeds back into the next iteration. Multi-agent systems (e.g. Google's AI co-scientist) split roles into Generation, Reflection, Ranking (Elo tournaments), Evolution, Proximity and Meta-review agents, using self-play-style debate and test-time compute scaling.
Problem solved
Scientific research — especially in ML — is a bottleneck limited by the number of skilled researchers and the time needed to iterate hypothesis→experiment→analysis→publication. The concept aims to scale and accelerate scientific discovery by automating this loop, lowering cost and increasing the number of ideas tested in parallel.
Components
LLM module proposing novel research directions and checking novelty against the literature.
Official
Agent that writes and executes code, collects metrics and generates visualizations.
Official
Generates a scientific manuscript (e.g. LaTeX) with automated citation of sources.
Official
A separate LLM agent scoring the work quality and providing feedback for iteration.
Official
Implementation
Systems may report false results or overstate originality, requiring human verification.
An agent executing arbitrary code may modify its environment or take unintended actions — sandboxing is required.
Publishing AI-generated work raises unresolved questions about authorship and reviewer-system overload.
Evolution
First widely publicized fully automated research pipeline generating complete ML papers for ~$15 per paper.
Multi-agent system built on Gemini 2.0 (Generation, Reflection, Ranking, Evolution, Proximity, Meta-review agents) for biomedical hypothesis generation; announced Feb 19, 2025.
A fully generated manuscript scored 6/7/6 at the ICBINB workshop (ICLR 2025) and was voluntarily withdrawn before publication; announced March 12, 2025.
Execution paradigm
Execution is sequential-iterative with a reviewer→generator feedback loop.
Tasks are routed to specialized agents (generation, experiment, write-up, review) depending on the research pipeline stage.
Parallelism
Many ideas/experiments can be explored in parallel, but a single research loop is largely sequential.
Hardware requirements
LLM inference driving the agents and the ML experiments being run require GPU accelerators.
The agent orchestration layer is hardware-independent; the GPU dependency comes from the underlying models and experiments.