
Somewhere in American hospital records, there is a pattern that should not exist.
Diagnoses of acute posthemorrhagic anaemia, a serious blood-loss condition that requires transfusion, have risen sharply at facilities that adopted AI billing tools. Blood transfusions have not. A condition is being recorded. The standard treatment for that condition is not being given. According to a Blue Cross Blue Shield Association analysis, the discrepancy is not a rounding error. It is a signature.
This is not a story about a medical error. No patient was misdiagnosed. No physician made a wrong call. What happened is more systemic and more troubling. An AI system trained to identify billable conditions found one. It coded it. The hospital billed for it. Nobody questioned whether the diagnosis reflected care that was actually delivered.
This is what AI looks like when there is no governance around it.
What the Bill Says About the Chart
The Blue Cross Blue Shield analysis examined what happened to hospital billing after AI coding tools arrived at scale. The numbers are not ambiguous. Inpatient spending attributable to AI coding practices reached an estimated $663 million. Outpatient spending tied to the same pattern reached $1.67 billion. One facility’s case complexity rating, the metric that determines how much a hospital can charge, rose 6.7 per cent in the year after adopting an AI billing tool. The average rise at comparable facilities in the same state was 0.9 per cent.
The practice is called upcoding: coding a patient as sicker, or their treatment as more complex, than the clinical record supports. It has existed in healthcare administration for decades. What AI has done is industrialise it. According to a federal data brief from the Office of the National Coordinator for Health Information Technology, 71 per cent of US hospitals were using predictive AI by 2024. AI use for billing specifically rose 25 percentage points in a single year, from 36 per cent of hospitals in 2023 to 61 per cent in 2024. The speed of that adoption has outrun every oversight mechanism that existed to check it.
The tool is not complicated. What was built around it is the problem. AI coding tools scan patient records and flag conditions that could legitimately be billed. In the right environment, with clinical oversight and audit processes, that is a useful capability. In the environment most hospitals actually built, which is one without meaningful governance, they become a revenue maximisation engine. The algorithm does what it was trained to do. Nobody verifies whether the conditions it codes for were actually treated. The bills go out.
The Insurer’s Algorithm Has a Different Objective
At the same time hospitals are using AI to add conditions to bills, health insurers are using AI to remove approvals from treatment requests.
Prior authorisation, the process by which insurers must approve procedures before they happen, has become a primary deployment zone for AI-driven decision-making. The American Medical Association surveyed physicians and found that 61 per cent reported health plan use of AI is increasing prior authorisation denials. A US Senate Permanent Subcommittee on Investigations report found that denial rates at UnitedHealthcare, CVS, and Humana’s Medicare Advantage plans rose as each insurer increased AI deployment in its review process.
The governance picture on the insurer side is no better than on the hospital side. A January 2026 study in Health Affairs by researchers at Stanford Health Care, drawing on a survey of 93 large health insurers, found that more than one-quarter of insurers do not document the accuracy of their AI models or test them for bias, around 40 per cent have no accountability practices in place for AI tools used in prior authorisation and claims decisions, and fewer than one-quarter even tell providers when AI was involved in a determination.
The result is a healthcare system in which AI is simultaneously inflating what hospitals charge and compressing what insurers approve. Patients sit between the two. The treatment they need may be denied before it is given and billed for a complication they were never treated for.
Arizona, Maryland, Nebraska, and Texas all passed legislation in 2025 requiring human oversight before AI can be used to deny a prior authorisation request, prohibiting it as the sole basis for medical necessity determinations. From 2026, the Centers for Medicare and Medicaid Services (CMS) will require payers to provide a specific reason for every AI-assisted denial and to publish aggregate approval data. That regulatory response confirms the scale of what is happening. Legislators do not write laws against things that are not happening.
Nobody Has Had to Answer for This
The question that neither the hospital nor the insurer has been required to answer is a straightforward one: who is responsible for what the algorithm decides?
A 2025 survey of 182 US hospital leaders by Black Book Research found that only 22 per cent are confident they could produce a complete AI audit trail within 30 days if asked. Only 29 per cent have implemented and enforced policies covering AI model inventory and accountability sign-offs. Forty-one per cent identified limited vendor documentation, the model cards and drift reports that explain how a system behaves over time, as their top barrier to audit readiness. The median share of IT and quality budgets allocated to AI governance is 4.2 per cent.
These are not numbers that describe an industry taking AI risk seriously. They describe an industry that deployed the technology and deferred the governance question for later.
The procurement happened fast. The governance never followed. Across billing departments and claims operations, AI has been handed consequential authority over patient finances and care access by organisations that did not build the structures that authority demands. The tools were procured. The governance was not.
The Wrong Diagnosis
Every time this gets written about as an AI problem, the real fix gets deferred.
If the algorithm is the villain, the solution is a better algorithm. A more accurate one. A less biased one. Another procurement cycle, another vendor, another pilot. That framing lets every decision-maker who signed the purchase order, approved the deployment, and chose not to build the oversight infrastructure step back from the frame. The machine did it. The machine was wrong.
In healthcare, the machine is doing exactly what it was built to do. It finds billable codes and it finds reasons to deny claims. It operates at the scale and speed that human reviewers cannot match. And it does all of this inside organisations that did not build the governance structures, the audit processes, the accountability frameworks, or the appeals mechanisms that consequential decisions at that scale require.
The United States is where this data exists. It is not where the problem stops.
That is not an AI failure. It is an organisational one. And unlike a broken algorithm, it cannot be fixed with a software update.








