Perceptron Mk1.5 Adds Video Object Tracking, Connecting Audio Understanding with Tool Calls

Perceptron released Mk1.5 on September 25, adding native audio understanding, video object tracking, and tool calling to its perception model. It is now available through the platform API. The model accepts text, images, video, and audio, and outputs text, points, boxes, polygons, and object tracks. It can be used for video analysis and embodied agents. [Official announcement](https://www.perceptron.inc/blog/introducing-perceptron-mk1-5)

The key technical change is that an object's spatial positions are linked into a time series. The SDK's `result.tracks` contains object labels, media indices, and timestamped observations; multiple images or videos are distinguished by `asset_idx`. For shelf inventory or sports footage, tracks can link the same target's positions at different times, allowing downstream code to process movement and event order. From an engineering perspective, this could reduce the work of adding a separate object association step after per-frame detection, but whether identity is maintained after occlusion should still be validated on real footage. [SDK documentation](https://pypi.org/project/perceptron/0.4.0/)

Audio support also has clear limits. The changelog specifies a maximum of 16,384 audio tokens per segment, or about 21.8 minutes, and these count toward the shared 36,864-token context alongside other inputs. Video audio tracks are not processed by default; you must explicitly enable `vision_config.enable_audio_in_video`. Uploading a video with sound alone does not mean the model will analyze the audio too. [Changelog](https://docs.perceptron.inc/perceptron-mk1.5/changelog)

Tool integration uses a multi-turn workflow in which the application executes a function and then returns the result. The model card specifies that a single request cannot declare tools and JSON Schema or regular expression output constraints at the same time. Agents that need a fixed-format result should therefore request a final answer separately after the tool loop ends. The application should also check `finish_reason` to avoid treating a response truncated by the length limit as complete. [Model card](https://docs.perceptron.inc/perceptron-mk1.5/models/perceptron-mk1.5)

On performance, Perceptron reports up to 4.7× faster end-to-end completion than Mk1. The tests used a single H100 and the median of three runs, covering chat, image question answering, and video workloads. This is vendor-reported testing and cannot be directly extrapolated to different resolutions, concurrency levels, or inference settings; it also does not establish real-time performance in a robot control loop. Teams should still measure network, tool execution, and tail latency from long videos. [Test conditions](https://www.perceptron.inc/blog/introducing-perceptron-mk1-5)

Deployment requires Python 3.10 or later and SDK 0.4.0 or later. Existing deployments should also note that Perceptron disabled three Isaac 0.1/0.2 model IDs on its API on the same day; calls to them will return 404. After migrating, teams should revalidate tracking quality, audio usage, and tool success rates rather than checking only that the API is reachable. [Package requirements](https://pypi.org/project/perceptron/0.4.0/), [migration notice](https://docs.perceptron.inc/perceptron-mk1.5/changelog)