AI for Science／代理基礎設施
Brain Researcher Raises First-Step Tool Selection Rate for Brain Imaging Agents from 23.3% to 93.6%
Brain Researcher constrains neuroimaging analysis agents with a knowledge graph, typed workflows, and review rules, while preserving hypotheses, evidence, and execution provenance. Tool routing improved substantially, but the rate of successfully grounding claims in verifiable evidence remained just 22.0%, and automated review also failed to catch a directional statistical error.
Brain Researcher shifts the unit of work for scientific agents from a single response to a “governed research trajectory.” The MIT-licensed v0.3.0 release includes a Python CLI, agent runtime, FastAPI orchestrator, Web UI, Neo4j knowledge graph interface, and 10 versioned MCP tool contracts. Researchers can predefine permitted analysis methods, required checks, and the scope of allowable claims. The system then links questions, tool selections, execution logs, evidence, and review conclusions into a traceable record.
In tests spanning seven models and 60 tasks, adding the full harness increased the share of runs whose first action selected the correct analysis path and tool from 23.3% to 93.6%. Capability coverage rose from 49.8% to 94.5%, while the proportion of outputs suitable for handoff to a downstream executor increased from 47.4% to 76.1%. A separate 50-question evidence test was less encouraging: the share of citations that were locatable and genuinely supported the associated claim rose only from 4.6% to 22.0%, showing that reliable routing does not guarantee reliable scientific conclusions.
The platform also performs multiverse analysis, running multiple reasonable analysis specifications in parallel to expose how sensitive conclusions are to estimators, confounding variables, and feature processing. For example, a schizophrenia connectome study ran 480 specifications. After correction, a directional result that had initially appeared broadly valid held only with the Pearson and Spearman estimators. Notably, the error was discovered through manual inspection: a fallback agent had ignored the sign of the effect, and the automated review layer failed to catch it.
Engineering teams should also note the boundaries of the public release. The repository provides local Docker Compose configuration, contracts, replay data, and some generators, but it does not include the complete knowledge graph, controlled-access data, the private benchmark corpus, raw audio, or some execution bindings. Kubernetes/Helm assets are also explicitly labeled experimental. The system is better viewed as a template for auditable research orchestration than as a finished product proven capable of autonomously producing new scientific discoveries.