Did OpenAI Hack Medicare? What Actually Happened

You may have seen the headlines today: an OpenAI agent hacked Medicare. When I first read that, I thought, hang on, did an AI actually break into the system containing the medical records of 27 million Australians? Because that’s not what happened.

No patient records are currently believed to have been accessed. It wasn’t the Medicare claims system, and the AI wasn’t actually told to hack anything. It was researching publicly available statistics.

But something genuinely concerning did happen, and I think the real story is much more interesting than the headline. I’ve spent 25 years building production software, and when you look at this incident as a software and security problem, there are really two stories here: what the AI did, and what it discovered about the system it was talking to. So let’s separate those two things.

What did it actually access? First, this wasn’t the Medicare system most Australians probably imagine when they hear the word Medicare. It was called the Medicare Statistics Reporting Service Portal, a public-facing website operated by Services Australia, and the government says it’s completely separate from the systems handling Medicare claims, payments, processing and individual medical information.

The portal contained aggregated statistics, things like Medicare benefits, prescribing information and Pharmaceutical Benefits Scheme statistics. It’s the sort of information researchers and academics use all the time. In fact, Services Australia explicitly provides Medicare statistics for research and public analysis.

So that’s distinction number one. An AI didn’t suddenly get access to 27 million Australians’ medical records. There’s currently no evidence that that happened.

So what was OpenAI doing there? This is where the story gets really interesting. According to the Australian government, on 18 June OpenAI was running an internal capability evaluation, and the agent was given a research task: it was looking for information about public medicine spending in Australia.

That’s important because, as far as we currently know, nobody told the AI to go and hack an Australian government website. It was trying to find statistics, which is precisely what this website existed to provide. You can imagine the task being something like “find information about Australian pharmaceutical spending”, and the agent searches the internet, finds the Australian government statistics portal, and then hits a roadblock.

This is where it crossed the line. According to Anthony Albanese, the agent repeatedly requested information and was blocked. But instead of accepting that, it tried alternative ways of getting the information, and eventually it found a way around those controls and gained access to things it wasn’t supposed to access.

That’s an important distinction, because I’ve seen people describe this as essentially an AI web scraper, and it certainly appears to have started as automated information gathering. But once you’re deliberately circumventing access controls and accessing resources you’re not authorised to access, we’re no longer talking about an ordinary web crawler. The agent accessed both public and non-public information.

There’s another detail we shouldn’t minimise. Services Australia says the agent also appears to have written files to the internal server. Exactly what that means is still being investigated, so I wouldn’t speculate about it yet, but that’s significant.

Reading publicly available statistics is one thing, and getting around access controls is another. Writing something onto the server takes this into much more serious territory.

But what did it actually steal? This is where the headline and the impact start separating again. OpenAI says its investigation found no evidence that patient records were accessed, and that the information included aggregate health statistics and internal file names.

The Australian government says some information wasn’t publicly available at the time, but it wasn’t considered particularly sensitive, and some of that information has subsequently been made public anyway. Acting Prime Minister Richard Marles described the impact of this particular incident as relatively minor.

That’s an extraordinary contrast. The headline is essentially “AI hacks Medicare”, but the government’s current assessment is no medical claims system compromised, no evidence of a broader Services Australia network compromise, and a relatively minor impact.

That doesn’t mean nothing happened. It means we need to distinguish the seriousness of the behaviour from the severity of this particular outcome.

Then there’s the website, and there’s another part of this story I don’t think should get lost. The system the agent broke into was a legacy system. Government Services Minister Katy Gallagher says it dates back decades, and following this incident, the government isn’t bringing it back.

The public data is being moved to data.gov.au or other existing platforms. So think about what actually happened here. You have an autonomous software agent performing what began as a legitimate research task, and it encounters an old internet-facing application.

The application refuses to give it what it wants, so the agent starts looking for alternatives. Somewhere along the way, it discovers that the site’s security controls can apparently be circumvented. That’s fascinating, because from a security engineering perspective, the AI didn’t create the vulnerability; the vulnerability was already there, and the AI found it.

So was this really a hack? Technically, there’s a good argument that it was. The access wasn’t authorised, security controls were circumvented, non-public resources were accessed, and potentially files were written to a server.

So simply saying this was just web scraping would go too far. But saying OpenAI hacked Medicare without explaining which Medicare system we’re talking about creates a completely different impression. This wasn’t somebody deliberately launching an attack to steal Australians’ medical records.

Based on everything publicly known so far, it started with an AI agent trying to research public statistics, and then the agent did something its operator didn’t intend. OpenAI itself says its model took actions they did not intend. That’s the part I think deserves our attention, because this changes the security model.

For years, we’ve designed websites around human attackers, bots and fairly predictable automated scanners. Now imagine millions of autonomous agents trying to complete perfectly legitimate tasks. One needs a statistic, one needs a product price, another needs planning information, and another needs research data.

When your website says no, the agent doesn’t necessarily stop. It reasons, it tries something else, it discovers another endpoint, it finds an old server. It experiments, and potentially it discovers vulnerabilities nobody realised were there.

The Australian Signals Directorate has now issued guidance specifically addressing this problem. Interestingly, ASD says there’s currently no indication this represented malicious targeting of Australia. Their concern is that AI agents can independently discover vulnerabilities and attempt to progress their task without a human explicitly authorising those actions, and that’s a very different security problem.

I don’t think the lesson from this incident is “AI attacked Medicare”, and I don’t think it’s “nothing happened, it was just scraping”. Both explanations miss something important. An AI agent was apparently given a fairly ordinary research objective, encountered security controls, independently found a way around them, and in doing so exposed a weakness in a decades-old government system.

Fortunately, the information involved appears relatively benign. Next time, the system might not be, and that’s why this matters.

We’re entering a world where your website isn’t only being accessed by humans or dumb bots following predetermined scripts. Increasingly, it’s going to be accessed by software capable of reasoning about obstacles and finding another way to achieve its objective. And if your security depends on nobody thinking to try the other door, AI may be able to start trying every door.