代理框架與開發工具
PRAXIST Preserves Experimental Causality Across Generations With Evidence Graphs, Cuts MLE-bench Model Costs to About One-Twelfth of Baseline
PRAXIST has multiple research agents propose changes over successive generations, then records evaluation results, mechanisms, and lineage in an auditable evidence graph to avoid restarting exploration in every round. Its authors report 60 medals across 75 MLE-bench tasks at a model cost of $3,054, but the newly released system’s performance and costs have so far been measured primarily by its development team.
Sapient Intelligence has released PRAXIST Beta, positioning it not as a conventional chat-based research agent but as a “generational” R&D orchestrator designed for long-running operation. Users must first provide an executable project, a baseline, an evaluator, and clearly defined metrics. Multiple peer agents modify methods or implementations in parallel, with every candidate validated through the same evaluation path. A planning layer then carries effective mechanisms, failed hypotheses, and unresolved questions into the next generation.
The key design shift is turning experimental output from scattered logs into a typed evidence graph. Each result retains its parent lineage, code artifacts, evaluation evidence, and mechanistic claims, and is stored in persistent lanes such as incubator, frontier, and Gems. This allows the planner to recombine validated changes instead of merely selecting the highest-scoring checkpoint. The framework also provides Quality-Diversity configurations, a Deep Innovation Gate, resource-pressure scheduling, replay, recovery, and multi-objective Pareto ranking. PRAXIST manages the workflow and evidence lifecycle, while the project itself still defines the scientific hypotheses and what constitutes success.
The paper reports 60 medals across the full set of 75 MLE-bench tasks, including 49 gold medals. A Claude Code baseline using the same Claude Opus 4.8 model earned 55 medals, including 34 golds. Recorded model spending was $3,054 and $38,370, respectively. The team also demonstrates use cases in quantitative trading, SLAM, tokamak control, and rocket landing, but these open-ended cases use different task-specific metrics and cannot be treated as a single score for general research capability.
Continuous testing currently covers only Linux and CPython 3.11/3.12, and the system does not guarantee metric improvements. More importantly, the project is source-available rather than conventionally open source: organizations with at least $1 million in annual revenue must negotiate a commercial license, and externally published outputs must retain product attribution. Before adoption, users should independently rerun MLE-bench, audit evaluators for vulnerabilities that agents could exploit, and isolate long-running execution environments, credentials, and production systems.