Back to Blog

On Proxmox, the jump to Ceph Tentacle bills you for what you built in 2020

On Proxmox, the jump to Ceph Tentacle bills you for what you built in 2020

The jump from Ceph Squid to Tentacle is the window to open for anyone who would rather not cross 31 October without new patches. The procedure is written down and it is clear. What is in no release note is the bill: the jump charges you for the cluster you already had. On OSDs created in 2020 the metadata space balloons, and what reaches the Proxmox panel is less total capacity. Two manager modules spent six years flagged as deprecated while still running on your cluster; now they actually disappear. And the filesystem step asks you, in writing, to go without a standby for a while. None of it is a bug: it is all documented. The problem is that it rarely makes it into the maintenance window.

The date, in two lines

Ceph's official calendar marks 31 October 2026 as the estimated end of life of the Squid branch (19). The word in brackets is theirs: the table reads "End of life (estimated)" for Squid and Tentacle, and plain "End of life" for Reef, which already closed on 20 March. The difference is not cosmetic, and we picked those dates apart back in September. Version 19.2.7, from 12 October, is the last published Squid; Tentacle (20) has been available since 18 November 2025 and its estimated date is 1 June 2027.

Two weeks ago we wrote that the last Squid patch was slow to reach the Proxmox repository. Today, at 17:05, we looked again at the public indexes on download.proxmox.com and the picture has half-changed: ceph-squid for trixie now serves 19.2.7-pve1… but only in the test component. In no-subscription the highest is still 19.2.6-pve4. If your cluster runs on the free repository and you are not planning to jump, you cross 31 October on the second-to-last patch of a branch that stops getting them. It is a small, bounded risk, but it deserves writing down rather than assuming.

The pair of versions that never existed

This is our favourite part of the Tentacle notes, because it changes how any compatibility matrix should be read. It states, literally, that upgrading to the latest minor release of the target series is the only supported and tested path from an older release, and gives the reason: a release cannot be tested against releases that did not exist when it shipped.

"Upgrades from Squid v19.2.6 to Tentacle releases earlier than v20.2.4 may fail: v20.2.4 introduced the key type changes, so Tentacle daemons older than v20.2.4 fail to start. The Managers upgrade, the rest may not, and the only way out is to redeploy the Managers with the previous image and start over."

The direction of the problem is surprising the first time: 19.2.6 is newer than several Tentacle releases — it shipped on 19 August 2026, and 20.2.3 on the 5th — so that particular pair could never have been tested. The "key type changes" have a name in the Proxmox wiki: Squid 19.2.6 and Tentacle 20.2.4 added health checks for keys using the aes encryption method affected by CVE-2025-30156. If that rings a bell, it is because we already covered the cephx key migration when those warnings showed up.

This is where it pays to look at the repository rather than assume it, so we measured. Proxmox's ceph-tentacle serves up to 20.2.4-pve5, so anyone following the wiki and letting apt take the highest version lands fine. Curiously, the 20.2.3 from the upstream example never reached that repository: there is 20.2.0, 20.2.1, 20.2.2 and then straight to 20.2.4. And the first three are still sitting there next to the good one, waiting for whoever has a pin, a held package, an internal mirror frozen months ago or an apt-cacher that remembers too much.

One detail ties two paragraphs above together and is worth not skipping: the ceiling of the free repository for trixie is 19.2.6-pve4, which is exactly the source version the Tentacle note warns about. Anyone on no-subscription who has touched nothing is, by definition, in the warned-about case.

The OSDs you created in 2020

The Proxmox wiki has a known-issues section with a single entry, and it is the one that will find you if your cluster has a few years on it: OSDs created under Ceph Octopus (v15, released early 2020) and before can see an increase in META usage after the upgrade to Ceph Tentacle up to version 20.2.4. The tracking ticket is 76256 and its title is blunt: bluefs: META space usage explodes on osds without sharding after upgrade to tentacle.

Watch that "up to version 20.2.4", because it reads the opposite way round: 20.2.4 is, as of today, the latest Tentacle published, and the Proxmox repository serves nothing above it. Translated: there is not yet a Tentacle in which this is fixed. The fix is merged into the development branch (v21) and the backport to Tentacle is still queued, which is what the ticket's "Backporting" status means.

The cause is on the same page: the RocksDB of those OSDs does not use sharding, which became the default from Ceph Pacific (v16) onwards. An OSD created in 2020 has crossed Octopus, Pacific, Quincy, Reef and Squid without anybody rebuilding it. Upgrades inherit the on-disk format that was already there.

And now the part that ends up in front of the board: the symptom arrives disguised as capacity. Verbatim, it "can present itself as a decrease of Total Space in the PVE Web UI since Ceph calculates the available space based on the fullest OSD". We already wrote about that mechanism when it had to be explained cold: in Ceph capacity is set by the fullest disk. Here it is in practice. Your cluster has not lost disks, has not lost data and is green; the oldest OSD has simply eaten a few gigabytes of metadata, and the capacity calculation takes that one as its reference. Verification is a single command, ceph osd df, looking at the META column.

The best thing about the ticket is that it brings the real output. Three identical OSDs on a cluster originally deployed on 15.2.13, three hosts, one disk per OSD. The META column reads 15 GiB on the first, 14 GiB on the second and 346 GiB on the third, which sits at 95.29% used with 33 GiB free while the other two run at 48% and 51%. And the kicker: on that same OSD, the sum of the files RocksDB actually holds on BlueFS is 8 GB. Three hundred-odd gigabytes occupied for eight gigabytes of content.

The temporary workaround is documented: manually compact the RocksDB with ceph daemon osd.# compact, with the warning that it is I/O intensive and must be run sequentially and during off-peak hours. But it is a patch that comes back: the person who opened the ticket describes playing whack-a-mole with compact and seeing the problem reappear on every OSD in the cluster several times. What holds are the other two exits, also in the wiki and read by almost nobody: enable sharding on the OSD — only possible while the OSD is down — or destroy and recreate the affected OSDs one at a time, which then come up with the new structure. Neither is "a longer window": it is a project with its own calendar.

Two modules that are gone

Tentacle removes two manager modules: mgr/restful and mgr/zabbix. The release note is explicit about why: they had been deprecated since 2020, have not been actively maintained in recent years, and started dragging vulnerabilities through their dependency chain (they cite CVE-2023-46136). As alternatives they suggest the dashboard module for what restful did, and prometheus — "the most widely adopted among the Ceph user community" — for what the Zabbix one did.

That mgr/zabbix was going away we already flagged as a blind spot while it was still a future warning: we wrote it in July, when this was a fifty-day plan. What has changed since then is the tense. A monitoring module that leaves makes the graph go flat, and a flat graph reads as "nothing new", which is precisely the opposite of what is happening. If your Zabbix reads Ceph status through it, the monitoring work goes before the jump. And since Zabbix 8.0 LTS lands in November, lining both up in the same plan is cheaper than doing them twice.

The step where they ask you to remove your own redundancy

If you use CephFS, read the whole procedure before opening the window. To upgrade the MDS daemons it asks you to disable allow_standby_replay, bring max_mds down to 1 — that last one is only needed if you run more than one MDS per filesystem — stop every standby MDS with systemctl stop ceph-mds.target, confirm only one is active, restart it and bring the others back. For that stretch your filesystem has a single MDS and zero spares.

That is the correct way to do it, and it still means that for a few minutes the redundancy you pay for is not there. On top of that comes the noout flag the procedure itself recommends setting at the start, and it is worth remembering exactly what it does: noout switches off re-replication while it is set. If a disk dies mid-jump, Ceph will not make the missing copy until you clear it. Both decisions are right, and both narrow the margin at the same time. Which is why the hour you open the window is not a scheduling detail.

If you are on Proxmox VE 8, this is not about you

We checked today, because this is the fact that changes the conversation with management. The distribution tree of Proxmox's ceph-tentacle repository contains a single suite: trixie. For bookworm — the base of Proxmox VE 8 — the URL returns 404: there is no path to ask for. And the ceph-squid tree for bookworm has not moved since 5 September: it serves 19.2.5 in no-subscription and 19.2.6 in test.

The uncomfortable conclusion: a cluster on Proxmox VE 8 is a major Proxmox upgrade away from running a supported Ceph again, not four days away from a decision. The order is not yours to choose either, and we wrote it in July, when this was a fifty-day plan: the official 8-to-9 guide puts it in bold, you must bring the cluster to Ceph 19.2 Squid before starting the Proxmox upgrade. The mandatory stop on the way is precisely the branch that loses support on Saturday. What we did not know back in July is that for bookworm the destination would not even exist.

Where you are and what it means

Situation What you can do this week Windows
PVE 9.1+ with Ceph 19.2.3-pve3 or newer You can jump. Follow the wiki and make sure you land on 20.2.4 or newer. One, plus a separate project if there are old OSDs to reshard or recreate.
PVE 9.0 Proxmox to 9.1 or newer first (pve-manager 9.1.4). The Ceph jump comes after. Two, separate.
PVE 8 (bookworm) with Squid Tentacle does not exist for bookworm. The 8 → 9 jump comes first; this is no longer a this-week project. Several, and a written plan.
PVE 8 with Reef (18.2.8) Reef has been out of support since 20 March. Reef → Squid, then Proxmox 8 → 9, then Tentacle. Three stops. Start by saying it out loud.

When NOT to jump this week

An end-of-support date is not a fire alarm. Nothing switches off on 31 October: what stops is new patches. That is a risk that grows slowly and can be written down, bounded and reviewed in a fortnight. Starting a branch upgrade with a cluster that is already limping, on a Friday, without having noted the previous META column, is a risk that goes off that same night. They are not comparable.

There is one nuance that is not in the Ceph wiki but in the Proxmox one, and it changes the criterion for a healthy cluster: the health checks added by 19.2.6 and 20.2.4 for insecure keys set the cluster to HEALTH_ERR and yet all services remain operational. In other words: you can be red for a known, documented, non-urgent reason. The manual's requirement — "the cluster must be healthy" — does not mean "green no matter what"; it means knowing exactly why you are the colour you are. If you do not know, that is today's work, not the jump.

The six checks before touching the repository

None of these commands change anything. They take ten minutes and can be run right now, while the jump is still unscheduled.

  • pveversion --verbose on every node. The requirement is pve-manager 9.1.4 or newer and Ceph packages at 19.2.3-pve3 or newer. The node left behind in the last update round is the one that will stall the project.
  • ceph -s and ceph versions. No PGs down, incomplete, undersized or backfilling, and no daemon lagging on a version it should not be on.
  • ceph osd df and save the output. That snapshot of the META column is the only way to know afterwards whether it grew or was always like that. Without the before picture, Monday's discussion has no data.
  • ceph mgr module ls. If zabbix or restful show up as enabled, you have monitoring work pending, and it goes before the jump.
  • ceph fs ls and, for each filesystem, note max_mds and whether allow_standby_replay is on. Those are the two values you will have to restore at the end, and nobody remembers them at two in the morning.
  • After changing the repository and before installing anything: apt update && apt-cache policy ceph. If the candidate is not 20.2.4 or newer, stop. That is where you catch pins, holds and internal mirrors with cobwebs on them.

Where this fits into what we do

We run distributed storage with Ceph on Proxmox in production, with the hardware and the maintenance windows of people who have to face the music the next morning. We do not sell Ceph licences — there are none — nor anybody's subscriptions: what we do is design the cluster, maintain it and be on the other end of the phone when the window goes sideways, which is what 24/7 support is for. If you have a cluster and you do not know what version its oldest OSDs were created under, write to us and we will look; that answer takes an afternoon and commits you to nothing.

And if you keep one sentence, make it the one behind ticket 76256: the format inside your disks was decided by the version you created them with, and it still is today.

Sources. Release and end-of-life dates (and the distinction between "End of life" and "End of life (estimated)") for Tentacle, Squid and Reef, the quote about the latest minor release of the target series and the 19.2.6 warning, removal of mgr/restful and mgr/zabbix with CVE-2023-46136, and the upgrade procedure: docs.ceph.com/en/latest/releases/ and the Tentacle release notes. Prerequisites (PVE 9.1, pve-manager 9.1.4, Ceph 19.2.3-pve3), MDS steps and the known META issue on Octopus-or-earlier OSDs: pve.proxmox.com/wiki/Ceph_Squid_to_Tentacle. The requirement to reach Squid before the 8 → 9 jump and the insecure-key warnings with HEALTH_ERR: pve.proxmox.com/wiki/Upgrade_from_8_to_9. Bug title, status, the ceph osd df output with the 346 GiB of META and the "whack-a-mole" account of compact, all from the ticket itself: tracker.ceph.com/issues/76256. Our own measurement on 27 October 2026 at 17:05 against the public indexes of download.proxmox.com (no-subscription and test components of ceph-squid and ceph-tentacle, bookworm and trixie suites): versions served, absence of the bookworm suite in ceph-tentacle (404) and index dates. Cover photo: "Backblaze 2.0 w/ 45 disks installed" by ChrisDag (Flickr, CC BY 2.0), cropped by us.

Tags: Ceph Proxmox Storage Patching Infrastructure

Subscribe to our newsletter

To receive IT stories, everyWAN news and exclusive subscriber offers, sign up to our mailing list

Minorisa de Sistemas Informaticos y Gestión S.L. © 2026
everyWAN
everyWAN