FLUX-mimic Uses a Video-Generation Backbone to Predict Robot Actions, Cutting Response Time to 101 Milliseconds on a Single RTX 5090

Black Forest Labs and mimic robotics have introduced FLUX-mimic, an attempt to directly convert the physical dynamics learned through video generation into robot control. Its foundation model, FLUX 3, was jointly trained on images, video, and audio from the outset. The team then added an action modality and strengthened its robotics capabilities with human-operation videos, demonstrations captured using wearable devices, teleoperation data, and real-world deployment data. BFL said that adding action prediction initially reduced video quality by as much as 10%, but performance returned to its original level after roughly 3,500 training steps. This suggests that generation and control can share a backbone without a lasting sacrifice in content-generation capability.

At inference time, the system does not need to generate a complete video before extracting actions from it. The FLUX 3 backbone predicts how the scene will evolve, while a compact action decoder reads its latent representations directly and generates a sequence of robot actions through a denoising process. The world model requires only a single forward pass. mimic also runs the sensor, model, and actuator pipelines concurrently, claiming that the backbone executes in under 80 milliseconds on a single RTX 5090, with an end-to-end response time of 101 milliseconds.

In a real-robot deformable-object sorting task, the preview model achieved a 95% completion rate without task-specific fine-tuning. The baselines listed by the team were π0.5 adapted using its data, at 55%, and a flow-matching policy extensively post-trained for the task, at 70%. Audi is also testing use cases including vehicle-door assembly and the handling of weather seals and cables.

The work is notable for incorporating large volumes of human video without action annotations into the robotics data pipeline, reducing dependence on expensive teleoperation demonstrations. However, the reported success rate, comparison setup, and descriptions of “deployment” all come from the collaborating teams. The weights, training data, number of trials, and cross-robot results have not yet been disclosed. The next questions are whether it can maintain reliability in independent evaluations, across different hardware, and on out-of-distribution tasks—and whether it provides a verifiable safe-stop mechanism when failures occur.