MLOps／模型最佳化
Optuna 5.0 Changes Its Default Search Strategy, While Rustuna Tackles Framework Overhead in Large-Scale Trials
In its first major change to the default sampler since version 1.5, Optuna 5.0 adopts multivariate TPE and Constant Liar for single-objective search, while replacing NSGA-II with TPE for multi-objective search. Released alongside it, Rustuna reimplements a compatible core in Rust to target sampling and storage bottlenecks in workloads involving hundreds of thousands of fast evaluations.
Preferred Networks released Optuna 5.0 on September 7. The upgrade’s most immediate impact is not the addition of another sampler, but a change affecting existing studies that do not explicitly specify a sampler: for single-objective optimization, `TPESampler` now enables multivariate TPE by default, jointly estimating parameter distributions to capture interactions, and also enables Constant Liar to prevent parallel workers—which see only the same completed trials—from repeatedly proposing similar candidate points. For multi-objective optimization, TPE replaces NSGA-II. The team says the performance issues that previously hindered adoption have been improved, though full reproduction details have yet to be published separately.
Constrained optimization has also moved from sampler-specific interfaces into the core `Trial` API. Objective functions can write constraints through `trial.set_constraint()` and read them from `trial.constraints`; the existing `constraints_func` path is now being deprecated. The default parameter-importance evaluator has changed from f-ANOVA to PED-ANOVA, adding support for conditional search spaces and multi-objective studies. The stable `GPSampler` now also includes qLogEI, qLogCEI, qLogEHVI, and qLogCEHVI. It samples outcomes for running trials from the Gaussian process posterior, allowing batch suggestions to account for uncertainty instead of assigning a single fictitious value to incomplete points.
Released the same day, Rustuna provides a Rust implementation aimed at Python API compatibility, covering TPE, NSGA-II, CMA-ES, and random search, while allowing historical data that is no longer needed for the search to be discarded. The project’s maintainers claim speedups ranging from several times to hundreds of times when the objective function is very fast, but these are first-party results that vary by sampler and trial scale and should not be extrapolated to workloads whose costs are dominated by model training. Before upgrading, engineering teams should rerun representative studies with fixed seeds on both the old and new versions. Changes to default algorithms, feasibility determination, UTC timestamp normalization, and several removed APIs may alter results or data pipelines.