模型與代理研究
Photon-1 Learns to Operate Computers from Unlabeled Screen Videos, Replacing Action Labels with Latent-State Prediction
Induction Labs has introduced Photon-1, a 106B-A5B sparse MoE model that learns during pretraining solely by watching screen videos, without requiring mouse or keyboard action labels. The model first predicts the next latent screen state, then generates the actions needed to reach it, though current evidence of its performance comes primarily from the team’s internal evaluations.
Induction Labs has unveiled its “imagination model” architecture and its first experimental model, Photon-1, in an attempt to bypass a common data bottleneck for computer-use agents. Instead of first using an inverse dynamics model to infer mouse, keyboard, and other actions from videos, it learns directly from consecutive frames how states change. Its pretraining objective is autoregressive next-latent-token prediction. A differential visual encoder uses finite scalar quantization (FSQ) to compress each frame into 960 discrete tokens, or about 2.2 KB, with an emphasis on representing differences between adjacent frames.
According to the company’s [official technical overview](https://www.inductionlabs.com/news/scaling-video-pretraining), Photon-1 is a mixture-of-experts (MoE) model with 106B total parameters and 5B parameters activated per token. Its training data consists of 575 million frames extracted from roughly 2 million screen recordings—equivalent to 18 years of video at one frame per second—and 552B tokens. A single pretraining run with a 32K context window consumed approximately 30,000 H200 GPU-hours. After pretraining, the model was fine-tuned on fewer than 35,000 examples containing action trajectories, enabling it to first “imagine” the next desktop state and then output the corresponding actions. It was subsequently trained with online reinforcement learning using programmatically verified outcomes in virtual machines.
The team claims that Photon-1 outperforms Gemini 3.1 Flash-Lite on its internal computer-use tasks while using at least 30 times less pretraining FLOPs and costing roughly three times less at inference. The representations pretrained exclusively on computer-use videos can reportedly also be fine-tuned to play chess and simulated billiards. These comparisons have not been accompanied by a public benchmark, model weights, or independent reproduction, and the [Y Combinator launch page](https://www.ycombinator.com/companies/induction-labs) explicitly identifies the figures as team-reported results. The engineering questions most worth tracking are whether the compressed latent states can generalize across websites and operating systems, whether latent representations that become difficult to interpret after RL will hinder debugging, and whether action-label-free pretraining can preserve causal controllability when extended to a broader range of real-world videos.