GitLab's official recovery procedure starts with a DELETE

On 12 October GitLab shipped three versions at once — 19.4.2, 19.3.5 and 19.2.8 — over CVE-2026-97229, an integer overflow in the CI/CD regular expression compiler scored CVSS 10.0. The release note describes it as a flaw that "could have allowed an unauthenticated user to execute arbitrary code…". GitLab.com was already running the patched version and GitLab Dedicated customers do not need to take action; the server downstairs does.

On the same page, under "Important notes on upgrading", there is this line: "This patch will cause downtime during the upgrade as migrations must complete before GitLab can start". The heading above it reads "Single-node instances", which is how nearly every self-managed GitLab we come across in small and mid-sized companies is set up. Applying the critical patch there means stopping the service until the database migrations finish. 19.3.5 also ships migrations that run after the deployment.

The question that arrives after the restart

If your instance was on an affected version and let outsiders in, patching does not close the matter: somebody has to decide whether to treat it as compromised. And if they do, the work stops being an upgrade and becomes a rebuild with credential rotation.

Back in August we went through what a GitLab backup actually carries and why the secrets file is left out — cloning the repository is not a GitLab backup. The short version: gitlab-backup create does not store /etc/gitlab, and the reason the documentation gives is a good one — "Storing encrypted information in the same location as its key defeats the purpose of using encryption in the first place".

What we did not cover then is what happens the day you bring that backup up on another machine and the file is missing, or present but from a different installation. There is a documented procedure for that. It starts by deleting.

The warning at the top of the procedure

GitLab's troubleshooting guide spreads three warnings across two blocks before any command: "The steps in this section can potentially lead to data loss on the previously listed items", "Consider opening a Support Request if you're a Premium or Ultimate customer" and, under a heading of its own, "Be sure to create a full database backup before attempting any changes". They are worth repeating, because what follows is run against the database by hand and has no undo.

For CI/CD variables, the procedure is exactly this:

DELETE FROM ci_group_variables;
DELETE FROM ci_variables;

And for runner tokens, five statements across four tables, with the comments exactly as the documentation ships them:

-- Clear project tokens
UPDATE projects SET runners_token = null, runners_token_encrypted = null;
-- Clear group tokens
UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
-- Clear instance tokens
UPDATE application_settings SET runners_registration_token_encrypted = null;
-- Clear key used for JWT authentication
-- This may break the $CI_JWT_TOKEN job variable:
-- https://gitlab.com/gitlab-org/gitlab/-/issues/325965
UPDATE application_settings SET encrypted_ci_jwt_signing_key = null;
-- Clear runner tokens
UPDATE ci_runners SET token_encrypted = null;

The warning that comes with that block is the one to read twice: "The final UPDATE operation stops the runners from being able to pick up new jobs. You must register new runners".

Missing from that is the integrations and webhooks part, handled with a TRUNCATE across eight tables with cascade, and two-factor authentication, which the documentation settles by disabling it for every user so that each one sets it up again. Calling this restoring would be generous: what happens is that you empty what nobody can read any more so the service will start, which is the only thing you can do with an encrypted value whose key does not exist.

Nine inaccessible things, and none of them is the code

The same guide lists what becomes inaccessible when the key is lost. There are nine: CI/CD variables, Kubernetes and GCP integration, custom Pages domains, project error tracking, runner authentication, project mirroring, integrations, webhooks and deploy tokens.

Project mirroring is the one that breaks quietly: the mirror stops replicating and, if your secondary copy of the code depended on it, it freezes at the date of the incident without any alarm going off.

The code is not on the list. Repositories are in the backup and come back whole, which is why this gap survives for years without anyone noticing: the restore test ends when the familiar git log shows up and everyone relaxes. The machinery that pushes that code into production is not part of that test.

What it costs to write back what you deleted

The DELETE takes a second. The bill arrives afterwards, when every variable has to be put back, and for that you need to know what it held. If the container registry password, the cloud provider key or the ticketing system token lived only in that encrypted field, the only copy of that credential was inside the application you have just emptied.

In our experience that is where the hard conversation sits: the company password manager and the CI variables panel have been drifting apart for years with nobody comparing them.

The small print of the precondition

GitLab's note talks about an unauthenticated user; Rapid7's analysis, reported by Help Net Security, adds the full condition: the attacker must be able to commit to a branch in a public project "a malicious CI/CD configuration file .gitlab-ci.yml, and a repository path containing UTF-8 characters". The researchers reproduced code execution against the official GitLab CE 19.4.1 image, and the same input did not work against 19.4.2.

The difference does not change whether you have to patch — you do — and Rapid7 recommends doing it urgently, outside the normal patch cycles. It does change how many hours you have and what to look at first. If any user on your instance can create a public project and push a branch, the precondition is met by filling in a sign-up form. If it sits behind the VPN and only people on the payroll get in, you have more room, which is a different thing from being safe. At the time of Rapid7's analysis there was no confirmed in-the-wild exploitation.

The check that is not quick

There is a task that answers, against the live instance, whether everything encrypted can still be read:

sudo gitlab-rake gitlab:doctor:secrets VERBOSE=1

It walks every encrypted value in the database and verifies it decrypts with the current secrets file. It reports a line like Total: 1 row(s) affected, and with VERBOSE=1 it also tells you which rows and columns fail, with names like Group[10].runners_token. The documentation itself warns that "This can take a very long time, depending on the size of your database, as it checks all rows in all tables", so it is not something to fire off at eleven on a Tuesday morning without looking at the load first.

If that number is not zero, there are already values nobody can read, today, with the instance running and the dashboard green.

What you write off, decided in advance

The part that is almost never written down is the decision taken in advance: what you intend to recover, what you intend to rebuild and, above all, where the master copy of each thing lives. You make it in the cold, without hurry, and it takes five lines on half a page:

• CI/CD variables. Is the master copy in the password manager, or only in GitLab? If the honest answer is "we would have to go around asking", what is missing is a credential inventory.
• Runners. Re-registering them is cheap as long as somebody knows how many there are and on which machines. Does that list exist outside the very instance you have just lost?
• Deploy tokens. Who consumes them at the other end: a server, a cluster, the cloud provider. Nulling them without that list leaves services stopped, with nobody knowing why.
• Two-factor authentication. Disabling it for everyone is one line in the documentation and one person's morning spent announcing it and verifying identities.
• Mirroring, integrations and webhooks. Write down today which ones exist, because they are the last thing anyone misses and the first thing that quietly breaks a workflow.

Restoring and responding ask for different things

If the analysis concludes the server was compromised, everything that server could decrypt has to be treated as burned: variables, tokens, deploy keys. What the secrets file is good for at that point is reading what was there so it can be rotated wherever it is consumed, which is a different and considerably longer job.

The two jobs pull in opposite directions: restoring wants the key alive and incident response wants it dead. That is why the order gets decided beforehand, in the cold, the same way as when no patch exists and you have to govern the risk with what you have — your patching policy has no entry for "there is no patch". Here a patch does exist, and even so half the work falls outside what the patch solves.

What stays on the record

Monday's patch is urgent and goes in with a short outage. What does not fit in that window is finding out whether, on the day the backup has to come up on another machine, what you get back is a working GitLab or a GitLab with its history intact and its credentials empty. The check task takes as long as it takes, but it answers that today.

Sources (checked 14 Oct 2026): the versions, the date, the CVSS 10.0 score, the description of CVE-2026-97229, the downtime warning and the status of GitLab.com and Dedicated come from the critical patch release note for 19.4.2, 19.3.5 and 19.2.8. The full precondition, the reproduction against the 19.4.1 image, the absence of confirmed exploitation and the advice to upgrade urgently outside the normal patch cycles come from Rapid7's analysis as reported by Help Net Security. The preceding warnings, the SQL statements and the list of nine inaccessible items come from the backup troubleshooting guide; the reason configuration is kept out of the backup, from the backup documentation; and the gitlab:doctor:secrets task, its VERBOSE mode and the warning about how long it takes, from the check Rake tasks page. We have not reproduced the vulnerability: there is no original analysis of the flaw here, only public sources and the operational half. Ours are the reading that what does not come back is the machinery rather than the code, the order of the up-front decision, the five-line sheet and the observation about how the self-managed GitLab we come across is set up.