代理工具與實驗基礎設施
AIBTest Launches MCP-Native A/B Testing Workflow, Enabling Agents to Verify Domains, Deploy SDKs, and Read Results
AIBTest breaks website A/B testing into four stages—domain verification, short-lived credentials, browser-side traffic allocation, and MCP reporting—with the goal of enabling AI agents not only to suggest experiments but also to operate them under constrained permissions. The first version emphasizes a credential-safe design and a narrowly scoped surface for DOM changes, but its reporting still focuses on descriptive metrics and does not claim statistical significance.
AIBTest is transforming the traditional human-operated A/B testing dashboard into MCP-native experimentation infrastructure that AI agents can use directly. According to its public pages, the workflow begins with verifying control of a domain through a DNS TXT record or HTTPS file. A local CLI then stores the private key and obtains a short-lived, scoped token. A browser SDK performs sticky A/B assignment, after which the agent retrieves exposure and conversion results through MCP.
The key technical feature of this design is not another experimentation dashboard, but the separation of the management plane, execution plane, and credential boundary. AIBTest’s discovery document provides an MCP URL, a versioned SDK URL, a Linux amd64 CLI download, and a skill URL. Its skill document instructs agents not to read or output the credential store and uses a local loopback proxy to manage the client assertion and remote access token. In other words, the model sees tools and reports but should never have access to private keys or bearer tokens.
For experiment execution, the first version limits DOM changes to `textContent`, `className`, or `style`. Traffic allocation can be adjusted from 1% to 100%, and an experiment can enter the `running` state only after the selector exists. The documentation also recommends using an anti-flicker pattern for experiments that change visible content, preventing visual jumps before and after the SDK loads. These constraints make the system more like a safe operational interface for agents than a remote scripting system capable of arbitrarily rewriting a front end.
One caveat is that AIBTest’s current public documentation shows reporting centered on descriptive metrics such as exposures, conversions, and conversion rate, and explicitly warns against claiming statistical significance based on conversion rate alone. It is better viewed as an early experimental closed loop for agentic web development: an agent can propose a hypothesis, create an experiment, deploy the SDK, observe results, and adjust traffic. However, inference thresholds, sample-size planning, and product decisions still require joint oversight from engineering and product teams.