Synthetic Clinical Data Can Still Be 79.44% Missing Despite Passing the Utility Threshold, While Two Refinements Widen the Source-Distribution Gap

Healthcare AI teams often use synthetic health records to avoid privacy and governance barriers, but the fact that a downstream agent can still complete its task does not mean the test data resembles a real clinical environment. On August 6, the Oracle Health and Life Sciences team released a utility-constrained data-refinement method and analyzed a care-gap benchmark built from Synthea patients, simulated electronic health record interactions, and a production downstream pipeline. In the original data, the patient–measure pair missingness rate reached 79.44%; only 12.75% of rows contained structured results suitable for reasoning, and 38.94% of patients had no actionable measures at all. The concentration of the first three tokens in descriptions was also 100%, indicating strong templating.

Rather than training another generative model, the researchers applied replayable transformations using fixed hashes and rules. Based on each measure’s original sparsity, they converted some `MISSING_DATA` entries into structured results, restored timestamps, due dates, and supporting facts, and then rewrote the descriptions. Refinement-A reduced missingness to 72.19%, increased actionable rows to 20.10%, and lowered the share of patients with zero actionable measures to 3.11%. Refinement-B preserved those figures while restoring the number of cases for which recommendations could be generated from 45/80 to 52/80. By comparison, although a control condition that merely increased data density reduced missingness to 59.40%, text concentration remained at 100%, showing that “more data” did not eliminate template bias.

The problem is that the improvements were not unidirectional. Both refinements produced more internally complete structures, yet increased the mean absolute gap relative to the operational reference population from 9.70 to 30.69. Refinement-B’s failure rate in another distribution-gap scenario test also rose from the 26.09% baseline to 45.45%. Engineering teams should therefore monitor missingness patterns, textual diversity, temporal consistency, population distributions, source fidelity, and task utility separately. The study still covers only a single care-gap pipeline, and downstream quality was judged mainly by evaluators rather than human-annotated gold standards, so it is not yet sufficient to show that the refined data can represent real-world health records.