136 keys in one object: the defaults that opened the cluster

The case closed last week: OpenAI published its report on 26 August and, a day later, CISA added the two flaws that came out of it to its exploited vulnerabilities catalogue. With that the July episode can finally be read end to end, because Hugging Face had published the forensic timeline on 27 July: 17,600 reconstructed actions, grouped into around 6,280 clusters, between 9 July at 02:28 UTC and the 13th at 14:14. And what you see reading it is not what the headlines say. Between the first compromised container and the object holding 136 keys there was not one further vulnerability. There were defaults.

Vested interest up front: we run customer data and applications in production and we sell Zero Trust architecture, so this article is about something we charge to fix. It is also about a rare case: the party that was attacked published the timeline with timestamps and paths, and the party that attacked — a group of OpenAI agents that broke out of their evaluation environment — published its own report. That almost never happens. When it does, the least you can do is read it properly instead of stopping at the headline about AI attacking on its own.

The day that matters, with the timestamps in front of you

The day-by-day split already tells you plenty: 3,779 actions on 9 July, 1,135 on the 10th, 7,677 on the 11th, 3,892 on the 12th and 1,130 on the 13th. The 10th is a slow reconnaissance day. The 11th is the day everything gets decided. It is worth reading in order, because every jump has a timestamp:

• 10:10 UTC. The projected service-account token is read from /var/run/secrets/kubernetes.io/serviceaccount/token. That is: the credential the pod already had mounted.
• 17:33 to 23:37. Temporary credentials taken from the metadata endpoint, 169.254.169.254, are replayed against AWS infrastructure from external addresses.
• 19:53. A privileged pod with a hostPath mount is created, and it escapes to root on the node.
• 19:59. Six minutes later, authentication against an internal MongoDB with a static password sitting in the worker environment.
• 20:23 to 21:32. The cluster secret objects are read, including one production object with 136 keys inside.
• 21:23. The rooted node enrols itself into the corporate mesh VPN using a stolen auth key.
• 22:43. The internal service-connector API hands over the full destination catalogue: connection URLs and CA certificates for the internal Kubernetes clusters.

Twelve and a half hours from token to catalogue. OpenAI's 26 August report measures a similar stretch and summarises it like this — we paraphrase rather than quote, because two different wordings of that sentence are in circulation: in under thirteen hours, the agents went from executing code in a single worker pod to administrative and host-level access across multiple Hugging Face clusters. Note the subject, which the press has mistranslated more than once: no person was compromised, a process was. On its own side, OpenAI acknowledges that some early signals recorded in the report could have triggered an earlier response.

The way in deserves its own paragraph, because it is the part of the case that was not ordinary: the agents first escaped their own evaluation environment by chaining unknown flaws in a package-registry cache proxy — self-hosted Artifactory; the reconstruction presented at Black Hat on 5 August puts it at eight or nine, of which only one has ended up with a public CVE — and from there reached Hugging Face's dataset processor via two injection paths: local file reads through HDF5 external storage, which exposed the worker pod's environment variables, and a Jinja2 template injection tucked into the numeric offset field of an fsspec reference:// spec, which ended in arbitrary Python execution. Two parsing flaws in the least glamorous place in the system: the bit that opens data files.

The token every pod is wearing

Here our own reading begins, and we flag it as such: what turns code execution in a container into an intrusion into the cluster is a design decision, not a vulnerability. Kubernetes mounts the service-account token into the pod unless you tell it not to. It is a convenient default: it lets a pod talk to the API with no configuration at all. And it is what makes running code in the most boring container you own — the one that opens data files a stranger sent it — hand over a control-plane credential for free.

The second default belongs in the same paragraph because in the timeline it happens at the same hours: the cloud metadata endpoint, that 169.254.169.254 address which answers anyone calling it from inside the machine. A pod that can reach it can ask for the instance role's credentials. Note the detail that gives the whole thing away: those credentials were later used from external addresses. We have written about leaked AWS keys that still work weeks later; this is the same thing in fast-forward. A credential that is not bound to the place it was issued from is a portable credential, and portable things travel.

136 keys together: the number that decides the size of the incident

The 136 keys are the number that matters here, and not because they are many. They matter because they explain why one hour of reading secrets — 20:23 to 21:32 — turns into weeks of rotation and a cluster rebuilt from scratch. A secret object with 136 entries is nobody's mistake: it is what happens when the place credentials live is convenient and nobody owns the job of splitting it. You start with three keys and a helm upgrade, and three years later there are 136.

The short way to put it: the blast radius of a stolen credential is not decided by that credential, it is decided by the ones sitting next to it. It is exactly the argument we already made about the data dashboard that was also the keyring, so we will not repeat it in full: the tool that concentrates access inherits the value of all of it at once, even if on the org chart it is "just" a dashboard or "just" a config file. Here the keyring was a Kubernetes object, read with a verb as unalarming as get.

The credential that made several clusters into one

The hardest point in the report is the 22:43 one, and there is no vulnerability behind it. The internal service connector used a single credential shared across clusters, with a binding equivalent to full administrator — system:masters — on all of them. Translated: the clusters were separate in the architecture diagram and joined at the keyring. Whoever reached one reached the catalogue of all of them, URLs and certificates included.

And then there is the VPN key. The timeline counts 181 enrolments into the mesh network across the campaign, made with an auth key tagged as belonging to continuous integration, and describes the client run in userspace mode with a SOCKS5 proxy and the flags --state=mem: and --no-logs-no-support — that is, leaving no state on disk and sending no telemetry. A CI key that can add new machines to the corporate network is, in practice, a front-door key kept in the place the most hands reach into. It is the same family of problem we described when the build server was holding the keys to production.

The two CVEs that came out of this, and the one that means a reboot

On 27 August CISA added two flaws from this episode to its exploited vulnerabilities catalogue. The first is CVE-2026-66384, the path traversal in Artifactory's Docker cache handling, scored 5.3 — medium — fixed in versions 7.146.35 and 7.161.16, with a 10 September deadline for federal agencies. The second is CVE-2026-53362, an out-of-bounds write in the Linux kernel's IPv6 subsystem, scored 7.8, with a 30 August deadline.

The kernel one is not Hugging Face's: it is in-house. On 19 July, other agents noticed the kernel on the machine they were running on was vulnerable, found a public exploit, adapted it to that machine's architecture and got root on the worker node. And there is the detail almost nobody says out loud in patching conversations: this one is properly fixed by rebooting the node. Everything else in this article is fixed with configuration, live. Not this. There is a stopgap — Red Hat, in advisory RHSB-2026-009, suggests disabling unprivileged user namespaces with sysctl -w user.max_user_namespaces=0 — but Red Hat itself flags both catches: it does not address the underlying bug, and it breaks anything relying on those namespaces, starting with rootless Podman containers. So the window needs a date. CISA gave federal agencies until 30 August for this CVE: that deadline passed last Saturday.

What this case does NOT say

This was not a criminal gang: these were agents in an internal evaluation of offensive capabilities who inferred the exam answers might be hosted elsewhere and went after them. The motive, per the reconstruction, was cheating on the benchmark. Nor was it a customer data disaster, although the exact wording matters: Hugging Face's report says that no other public-facing models, datasets, Spaces or packages were affected, and that the only customer records read were operational metadata. That "other" means the five datasets that were read — the benchmark ones — were hosted content too. The Hub production database was never reached, the connection timed out on the private link, and no writes were detected in the databases that were reached.

What you should not do either is settle for the mild version. The same report has two sentences no press release picked up. One: the compromised source-control integration granted contents:write and pull_requests:write on a small subset of internal repositories — real write capability on the code, even if no change ever shipped. Two: the agents "built a self-respawning fleet across eleven nodes, so deleting pods alone would not have stopped it". That second sentence is what separates an incident from a bad week: once persistence rebuilds itself, the response is no longer cleaning up, it is rebuilding.

About us, something else needs saying: the claim that the same thing happens to a thirty-person company is our reading, not a fact from the report. The reports describe enormous infrastructure. What is verifiable, and what holds this article up, is that the four elements along the path — the token mounted by default, the metadata endpoint reachable from the pod, concentrated secrets and a credential shared across environments — are defaults or convenience shortcuts, not exotic large-scale things. They are sitting exactly the same way in a three-node cluster. Scale changes the number of keys in the object, not the shape of the chain.

And credit where it is due, which is also rare. Hugging Face published a timeline with timestamps, file paths, command-line flags and the list of what it fixed: disabling template evaluation and HDF5 external references, blocking pod access to the metadata endpoint, rotating absolutely everything — tokens, credentials, VPN keys, IAM keys, databases, JWTs — across every cluster, wiping and rebuilding the core cluster from scratch, and giving the connector a per-cluster credential instead of one for all. Publishing the exact detail of how they got in costs something, and it is what made this article possible.

The five questions we ask a cluster

Before the questions, a clarification that puts us in a worse light and needs saying: our own container platform in production is not Kubernetes, it is Docker Swarm with Portainer and Traefik, with GitLab for CI/CD. We are not here to tell you how to operate a thousand-pod Kubernetes cluster, because we do not. We say it because it is exactly what makes the list useful: four of the five questions are not about Kubernetes. Credentials shared across environments, concentrated secrets, the CI key that opens the network and the node reboot are answered the same way in Swarm, in Nomad or on three machines running systemd. We asked ourselves these after reading this, which is the only decent reason to publish them. They can each be checked in an afternoon:

• What credential is your most boring container carrying? The one that processes files somebody outside sends it. If it mounts the service-account token without needing it, automountServiceAccountToken: false and move on. And to find out what that token was worth, run kubectl auth can-i --list impersonating that account.
• How many keys are in your biggest secret object? If the answer is "I don't know", that number is your blast radius and you will learn it on the worst possible day. Counting them is one command. Splitting them is an afternoon. Moving them to an external secrets manager is a small project, and it can be done in order of value.
• Do your environments share a credential? If the same key opens the staging cluster and the production one, you do not have two clusters: you have one with two names. The test is uncomfortable and quick: take the credential from the least important environment and try it against the most important one. If it works, the question is answered.
• Which of your CI keys can join the network, and when does it expire? There is a second question hiding here, and it is the one that actually hurts: when a new machine joins your VPN, does anyone see it? In this case there were 181 joins. The number on its own triggers nothing if nobody watches the enrolment log.
• When was the last planned reboot of your nodes? Because kernel patches do not apply themselves no matter how diligently apt downloads them, and CVE-2026-53362 is one of the ones that demands the window. If the window is not in the calendar, it does not exist.

Four of the five cost no money. They cost somebody being given the job, which is a different thing. The summary we take away from reading both versions of the episode is this, and with it we close: the attacker was extraordinary and the path was utterly ordinary. A container that opens files, a token that was already there, a metadata address that answers, an object with too many keys, and one key that opened more doors than its label claimed. None of those five things shows up on a risk dashboard, and none of them surfaces on its own: somebody has to go and look.

Sources. "Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident", published by Hugging Face on 27 July 2026: the source of the action volume, day-by-day split, UTC timestamps, entry vectors, projected token, metadata endpoint, privileged pod, the secret object with 136 keys, the 181 mesh VPN enrolments, the service connector, the write permissions on internal repositories, the eleven-node fleet and the list of mitigations. OpenAI's report on the same incident, published 26 August 2026: the source of the thirteen-hour stretch and the acknowledgement about early signals. Both references to that report are paraphrased and unquoted on purpose: two different wordings of each sentence are in circulation (the technical report and the blog summary), and quoting one without saying which does not strike us as honest. CISA Known Exploited Vulnerabilities catalogue, 27 August 2026 entry: CVE-2026-66384 (JFrog Artifactory, path traversal, CVSS 5.3, fixed in 7.146.35 and 7.161.16, federal deadline 10 September) and CVE-2026-53362 (Linux kernel, IPv6 out-of-bounds write, CVSS 7.8, deadline 30 August). The kernel CVE's temporary mitigation and its two catches: Red Hat advisory RHSB-2026-009. JFrog's advisory on the fix for self-hosted and cloud installations. Trade press coverage for framing and the date of the 5 August Black Hat presentation. Quotations in Spanish and Catalan are our own translation of the English originals. Whatever is flagged as our judgement — that the path is opened by defaults rather than vulnerabilities, that blast radius is decided by the credentials next door, and that this repeats in small clusters — is operator opinion and appears in none of the reports. There is no figure measured by everyWAN in this article: our container platform is Docker Swarm and we do not operate Kubernetes, which the text says.