Tech Blog

everyWAN Blog

Technology, cybersecurity and IT trends that matter

Deep Analysis
Cybersecurity
IT Trends
Filter by:
Sala de cuadros eléctricos de una instalación real: una hilera de armarios de maniobra grises con un técnico midiendo en un interruptor
8 min read

The zone that went down in London was a power panel

On 20 October 2026, at 06:08 UTC, a "power event" inside a single zone — europe-west2-b, one of the three in Google Cloud's London region — took banks, parcel services and listing portals offline for nearly eight hours. No region went down: the power went down inside one zone. And "we are spread across two zones" is not an answer, because a zone name is a distribution label, not a place you can audit: AWS documents that us-east-1a is not the same physical location in two different accounts. We explain why the failure domain that costs the most is the one nobody draws, why Ceph's default hierarchy has had a bucket type called pdu for years, when NOT to raise your failure domain, and the one-row-per-replica map we fill in standing next to the rack.

Bahías de disco de una cabina montada en rack, con unidades etiquetadas y varios huecos vacíos
6 min read

The last Ceph Squid patch is not in your Proxmox repository

On 12 October 2026 Ceph shipped 19.2.7, the seventh backport in the Squid series, with the usual line: "we recommend that all deployments update to this release". Today we checked what the Proxmox repository actually serves, and the answer is 19.2.6-pve4, with an index dated 9 September. The branch dies on 31 October. Two different calendars, and the one that governs you is not the one on ceph.io.

Parte trasera de un armario de servidores con los latiguillos de red saliendo de cada nodo hacia los de al lado
10 min read

The switchless Ceph mesh has four variants. Two of them break on one cable

Three nodes wired directly to each other at 25 GbE, with no switch to buy, is one of the best decisions you can make on a small budget. Proxmox documents four ways to wire it and only two survive a single link going down: in the other two, its own wiki says, "the nodes now have a different view" and Ceph reports services as down that are perfectly alive. Since Proxmox VE 9 that mesh is configured from the web interface, with four fabric protocols. And the official documentation still labels that layer tech preview.

Bastidor de paneles de parcheo de fibra óptica con mazos de latiguillos naranjas
9 min read

43 CVEs in your array's fibre switch, and not one is critical

On 8 October 2026 the NVD published 43 Brocade Fabric OS vulnerabilities on the same day. None critical, the highest 8.7, and 38 of the 43 require credentials. Those three numbers are precisely why nobody will patch this. Inside: the attacker the batch imagines does not come from the internet —it is another switch in your own fabric—, why the 7.1 on CVE-2026-87659 misleads, the maintenance-window excuse dismantled with the vendor's own documentation, the two version branches that do not line up, and four ownership questions instead of a checklist.

Filas de bahías de disco SAS en servidores de almacenamiento montados en rack
13 min read

You set noout to upgrade Ceph and switched off re-replication

The Ceph upgrade procedure on Proxmox recommends, as its fourth step, that you run "ceph osd set noout": "optional, but recommended", says the wiki. Ceph's own OSD troubleshooting page says this is "more a thought exercise" than a suggestion that anyone "in the post-Luminous world" should run it. Both sentences are published today and both are defensible. What that flag switches off is not rebalancing: it is a "down" OSD ever reaching "out", which is precisely the step that makes CRUSH recompute and recreate the missing copy. And the flag does not tell the disk you rebooted apart from the disk that genuinely died. Inside: why mon_osd_down_out_subtree_limit already brakes the scenario that actually frightens you, the chain almost nobody spells out (degraded PGs are not flagged as clean, and what is not clean is not scrubbed, so checksum verification stops on exactly the data that just lost a copy), the arithmetic of the two warnings arriving at 10.5 and 12.25 days to a dashboard that has been amber since minute one, the table of what each flag really switches off, and the four commands that do the same job on one disk or one host without touching the rest of the cluster.

Bahías de disco de un servidor con bandejas medio extraídas y huecos vacíos
8 min read

Your redundancy assumes the spare arrives tomorrow

A drive dies on a Tuesday and nothing happens: that is what you paid the replica for. But what you bought is not immunity, it is a window of time in which to replace the part before the next failure — and that window was calculated assuming the spare arrives tomorrow. Western Digital told its earnings call it was "pretty much sold out for calendar year 2026" and Seagate said its nearline capacity was "fully allocated" for the same period, with cloud agreements now reaching into 2027 and 2028. We run the arithmetic on Backblaze's published failure rates: with the spare in 48 hours, the probability of a second failure before the repair is 0.18%; with the spare in 20 weeks, it is 11.5%. Same design, same drives, 66 times the risk from a variable that appears in no diagram. Where it hurts most (the node, not the disk), the retention cut nobody signs off, and the six questions we ask now before approving a design.

Nodos idénticos apilados en un rack, cada uno con su pantalla y su etiqueta
9 min read

Your three-node Ceph does not heal itself: it holds on

A dead disk in a three-node cluster fixes itself provided another disk remains on that same server: the CRUSH rule descends to another OSD on the same host. A dead HOST does not. With replica 3 and the failure domain set to "host" — what Proxmox ships by default — there is no fourth place to put the third copy, so placement groups sit at active+undersized+degraded until the node comes back. And the cluster keeps serving, which is exactly why nobody looks at it. Ceph's real timers (20 seconds, 10 minutes, 15 minutes), what you cannot touch while the degraded window lasts, why recovery can cost you corosync quorum, and why the number that matters is not how many nodes you have but how many distinct places.

Cabina de disco montada en rack, con dos filas de bandejas de disco y sus pilotos de actividad
10 min read

Moving to Proxmox and reusing your SAN: the snapshot is what does not travel

The array is paid for, so it stays: the cheapest-looking decision in the project and the one that changes your day-to-day the most. In Proxmox the snapshot is not taken by the hypervisor but by the storage plugin, and on a shared LUN with LVM-thick the documentation says «snapshots are not possible by default…». What fixes that has been labelled a technology preview since 9.0, and the vendor puts it in writing on its own roadmap. The seven real options and the decision tree we are willing to sign.

Varios servidores de 1U apilados en un rack, con cableado de red blanco recogido y pantallas LCD de estado en el frontal
13 min read

Ceph changes cipher: rebooting the virtual machine does not refresh the key

On 9 September Proxmox published the procedure to migrate cephx keys from aes to aes256k. The hard part is not cryptographic: it is knowing what counts as "refreshing a client". The documentation says it in six words — "A guest reboot is not enough" — and the failure does not show up when you run the command, but minutes or days later, when a ticket expires. What has changed since August, the kernel 7.0 gate, and the order we follow.

Filas de bandejas de disco SAS en el frontal de varios servidores de almacenamiento montados en rack
9 min read

Squid gained 42 days and Tentacle lost 170: a single commit moved Ceph's end of life

On 5 August, a commit in the Ceph documentation moved the two dates that govern the calendar of anyone running distributed storage in production: Squid went from 19 September to 31 October, and Tentacle from 18 November 2027 to 1 June 2027. The extension is visible in the chart; the 170-day cut to the version everyone is migrating to we have not seen discussed anywhere. What it means that the field is called target_eol, and why Tentacle lives 560 days where Squid lived 765.

Actualizar Ceph de Squid a Tentacle en Proxmox: la ventana en la que el clúster corre en dos versiones
9 min read

Ceph to Tentacle: during the upgrade your cluster runs two versions at once

Squid expires on 31 October and Tentacle has been in Proxmox's enterprise repository since July, so the maintenance window is no longer hypothetical. Inside it something happens that almost no plan accounts for: monitors, managers and OSDs restart separately, and the cluster spends hours — or days — split across two versions. Why "restart OSDs on one node at a time" decides your window, why calling noout "optional" is misleading, and the command most people mistake for the finish line.

Servidores de rack idénticos apilados en un carro metálico, pendientes de montar, delante de un rack a medio poblar
5 min read

Corosync adds 650 milliseconds per node: the clock your upgraded cluster is still carrying

The time a Proxmox cluster takes to re-form membership after losing a node is not fixed: it grows by 650 ms for every node you add, and on factory values a 29-node cluster reaches 45.21 seconds — exactly where the documentation itself asks you to fix it, before the 60-second watchdog starts rebooting healthy nodes. Proxmox VE 9.2 lowered it to 125 ms, but only when the cluster is created: clusters upgraded from 8 to 9 keep the old value.

Servidor de almacenamiento abierto en un rack con las bahías de discos a la vista y una unidad extraída de su carro
9 min read

In Ceph, capacity is not set by the cluster: it is set by the fullest disk

A single OSD above 95% stops writes across the whole cluster even while "ceph df" still shows dozens of free terabytes. The three default thresholds (0.85 / 0.90 / 0.95), why the mechanism that repairs switches off five points before the one that serves, and the capacity calculation with one node missing that almost nobody runs: with four nodes the ceiling is 71 points, and 67 if you want the rebuild to actually finish.

Chasis de servidor de almacenamiento extraído sobre sus guías en un pasillo de datacenter, con dos filas de discos de 3,5 pulgadas en bandejas metálicas y un destornillador apoyado en el borde
7 min read

Ceph patches four CVEs: the package closes three, the fourth is on you

On 19 August Ceph shipped Squid 19.2.6 and Tentacle 20.2.4 tagged [URGENT]. One of the four flaws is not fixed by installing anything: it forces you to rotate every CephX key in the cluster, ten manual steps, and it leaves the cluster in HEALTH_ERR in the meantime. What is inside each CVE, why anyone holding a "mon allow r" key could read your OSD LUKS passphrases, and what we found today in Proxmox's Ceph repository.

Un parquímetro con el indicador EXPIRED en rojo y un coche todavía aparcado detrás: la fecha ha pasado y no ha cambiado nada visible
9 min read

Proxmox VE 8 goes end of life in August: Debian will keep patching you, the hypervisor will not

Proxmox's official table says 2026-08 and does not give a day. What tends to fall outside the headline is that Debian 12's extended support runs to June 2028 through the usual channel, so apt will keep installing real patches on an unsupported node. What exactly freezes, how to check it, the order of the upgrade to 9.2, and why forcing it in August can be worse than being late.

Interior de un disco duro abierto: el rendimiento real de un OSD de Ceph no lo marca la ficha del fabricante sino el benchmark que midió el propio OSD
7 min read

Ceph does not perform like the datasheet: it performs like the benchmark the OSD ran at boot

Since Ceph Quincy the scheduler for BlueStore OSDs is mClock, and the work ceiling it shares out does not come from the vendor datasheet: it comes from a benchmark each OSD runs at boot. If that measurement is discarded, you are left with 315 IOPS for a spinning disk and 21,500 for a solid-state one, whatever you bought. How to check what your cluster believes, which parameters stopped having any effect, and when the drive really is the problem.

Parte trasera de un rack de nodos de almacenamiento: la mejora de Fast EC se enciende pool por pool, no con la actualización
7 min read

Fast EC ships switched off: the Ceph Tentacle flag that only turns once

Ceph Tentacle ships Fast EC, the erasure coding performance work people had been waiting years for. And it arrives switched off: you enable it pool by pool, with one command, and the monitor then refuses to clear it — rolling back means draining and recreating the pool. On top of that, the "at least double" in the headline was measured with a 16K stripe unit, which is precisely what an existing pool cannot have. The conditions the monitor enforces, the half of the improvement frozen on the day you created the pool, what happens to the cluster when you flip it, and why three-way replication is still faster.

Ceph Squid 19.2 llega a su fin de vida estimado el 19 de septiembre de 2026
7 min read

Your Ceph has a date: Squid runs out of patches on 19 September

Ceph's lifecycle table puts the estimated end of life of Squid (19.2) on 19 September 2026: fifty days from today. Its replacement, Tentacle (20.2), has been stable since November and Proxmox has marked it stable since 9.2. The problem is not the jump, it is that it is not a jump: it is a sequence of three maintenance windows whose order you do not get to choose, with two details almost nobody has looked at — the mgr/zabbix module is gone, and erasure coding optimisations do not switch themselves on.

Proxmox VE 8 llega a su fin de soporte en agosto de 2026
5 min read

Proxmox VE 8 runs out of patches in August: why we won't upgrade on the 30th

The lifecycle table in Proxmox's official documentation puts the end of life of the 8 branch in August 2026. Just over thirty days are left, and the typical reaction is to block out a weekend and jump to 9. Our stance is the opposite: there are six situations where upgrading this August is a worse idea than being late. The live-migration asymmetry that turns your rollback into a restore, why Ceph means two windows and not one, the containers with old systemd that will not start, and a realistic split of the thirty days left.

  • 1
  • 2

Subscribe to our newsletter

To receive IT stories, everyWAN news and exclusive subscriber offers, sign up to our mailing list

Minorisa de Sistemas Informaticos y Gestión S.L. © 2026
everyWAN
everyWAN