Assume all your environment variables have been decrypted

At 13:58 UTC on 30 October 2026, deployment platform Northflank moved its incident from "Investigating" to "Identified" and in the description field wrote this: "Assume all environment variables have been decrypted. Disable/rotate all keys." Two sentences. Neither one softens the other.

Start with the stopwatch, not the headline

Before checking whether you use Northflank, run the exercise that turns that sentence into a number of your own. Take one production secret, the least frightening one, and rotate it end to end: issue the new credential at the provider that mints it, deploy it, verify the service is still alive, and revoke the old one. With a stopwatch running.

Three figures come out. The first is the time: multiply it by the total number of live secrets and you know how long your own "rotate everything" takes. The second shows up when you try to multiply, and it is where nearly everybody gets stuck: the inventory. If nobody knows how many secrets exist or who issued them, then there is no rotation plan, there is a good intention and a free Thursday. The third is the blast radius: write down what broke when you changed that one key, and what else was quietly using it. The forgotten consumer is what turns an emergency rotation into an outage.

And the order matters more than it looks when you are in a hurry: rotating the credential on your platform does not invalidate it at the provider that issued it. Until you revoke it at source, the old key keeps working for whoever holds it. One honest exception is worth noting before you start: some services only accept one active credential at a time, and there the verification step falls inside a cutover window. Those are precisely the ones you do not want to discover during an incident.

Why that number matters this week

Northflank's clock is short. At 11:43 UTC cloud security firm Wiz alerted them; their engineers reviewed the alert one minute later. At 12:02, 12:50 and 13:05 they shipped network policy patches. By 13:40 they were publicly asking customers to rotate environment variables, and at 13:58 came the sentence in the headline. A little over two hours from an external alert to an unambiguous public instruction is a fast response, and saying "assume everything" instead of rationing the panic is more honest than the norm.

What came next explains why this piece starts with the stopwatch. At 15:12 they enabled secret rotation for Postgres and MongoDB and noted that multiple users are confirming that their AI keys have been used. Northflank confirmed nothing of its own that day: at 19:28 it was still saying "Our investigation is ongoing. We have no further confirmed findings to share at this time", and not until 23:38 did it publish that users had reported unauthorised use of AI API keys. The next day they enabled rotation for MySQL, Redis, Minio and RabbitMQ too, and switched on audit logs for every customer, free accounts included. As of publication they had still not disclosed how the access happened, how many customers were affected, or whether credentials other than AI keys were abused.

Put another way: for almost ten hours, the only actionable information a customer had was "assume all of it". If your response to that depends on an inventory that does not exist, the window goes on building it.

At least nine platforms in thirty-one days

Wiz's Black October report counts at least nine incidents disclosed during October 2026 at development and AI platforms. Among those named: Fireworks AI, Zeabur, Lightning AI, OpenCode, Railway — which published its preliminary report on 21 October — Warp, CommandCode and Northflank itself.

It is worth not stretching that figure further than it goes. Wiz says explicitly that it does not attribute the nine to a single actor and that it found no shared vulnerability or dependency connecting them; it treats this as a concentration of incidents. The common denominator it does describe is more modest and more useful: every public report describes targeted hunting for stored environment variables, favouring those that grant access to AI services. Nine platforms of the same kind appearing in the same month hunting for the same thing looks to us like target selection; that reading is ours, and Wiz stops at calling it a concentration.

The reason AI keys top the list is economic and needs no theory. Somebody still has to know what to do with a stolen database password: it wants network access, something valuable inside, and a buyer. An AI provider's API key is spent directly, because the consumption is billed to whoever issued it. Wiz's report, in the coverage we were able to consult, puts numbers on the phenomenon — they call it LLMJacking: one case of roughly 800,000 requests to Amazon Bedrock in 30 hours, around 90 % of them completed, and another of close to 1.7 million requests in 13 hours, more than a million of which succeeded. The combined cost the report estimates exceeds 100,000 dollars at list prices.

The system protecting you is the one that has to decrypt

Many people's first reaction on reading the news is to ask whether the platform encrypted the secrets. Northflank has not explained how they were decrypted, and we are not going to infer it from the verb it chose. Railway did document it, in its preliminary report of 21 October: the attacker pulled the secrets in encrypted form and could not open them at first; it then obtained continuous integration credentials and, with those, "an unauthorized query of encrypted environment variables, of which a subset were decrypted utilizing a combination of the compromised credentials".

There is the mechanism, in writing and signed by the provider. Any platform that runs your code has to be able to decrypt your environment variables at process start. Otherwise your application does not start. Encryption at rest protects against the disk being stolen; it does not protect you if somebody compromises the very system whose job is to decrypt.

Ask any provider whether they encrypt and they will say yes. The question nobody answers in writing is how many different places can decrypt your secrets today, and whether that list exists anywhere. In a mid-sized software company the answer usually includes the deployment provider, the CI system, the secrets manager, two or three SaaS tools with integrations, somebody's laptop, and a private repository with an example file that stopped being an example. It is the same question we raised when we wrote that the log does not show the agent, it shows whoever lent it the token, one floor down: there it was the agent using the credential, here it is the platform storing it.

Where this leaves us

It would be easy to close this with "which is why you need your own infrastructure", and it would not be true. We have secrets in places that run our code too. We use self-hosted n8n for internal automation, and it stores credentials the way anybody's does; what we do with it is treat it as a production service rather than a convenience tool: it is inventoried, it has an owner, and it has an update window. None of that prevents a provider incident. What it achieves is that on the day a sentence like Northflank's arrives, the list already exists.

The decision that is genuinely architectural, and the one we discuss with customers when we build infrastructure and cloud, turns out to be rather duller than picking a vendor: how many distinct platforms genuinely need to be able to decrypt your credentials, and whether each one earns its place there. Shortening that list pays off more than hardening every item on it, and it is the first thing we look at in a cybersecurity review.

If you run the stopwatch drill and the number that comes out makes you uncomfortable, or if you get stuck earlier, at the inventory, drop us a line. Half an afternoon of work and a figure of your own are worth more than any report about somebody else's incident.