Back to Blog

Proxmox across a thousand sites is not one cluster: it is a thousand clusters

Proxmox across a thousand sites is not one cluster: it is a thousand clusters

5 milliseconds. That is the number that explains why a supermarket chain with more than 5,000 stores has advertised a job to pull VMware out of its sites and put Proxmox in, and why what that job asks for is not a cluster but a platform to "provision, manage, monitor and maintain the retail edge estate throughout its lifecycle". The number is in Proxmox's documentation and it governs the whole architecture of any migration spread across sites.

What the job ad asks for, and what the headline leaves out

The posting is published as Head of Software Development — reference 215705, based in Bengaluru — and in the body of the ad it describes itself as Head of Software Engineering: Edge Platform. It states the job bluntly: "modernising Tesco's edge virtualisation estate through the migration from VMware to Proxmox", delivering "a secure, reliable and cost-effective alternative virtualisation platform". That much you have read. The interesting part is the measure of success: "safe and predictable migration from VMware to Proxmox, successful validation of required store configurations, reduced legacy dependencies and effective operational handover".

The measure does not say "a working cluster". It says validation of required store configurations and operational handover. And the other deliverable is an "Edge Management Platform" to provision, manage, monitor and maintain the estate, in a context the ad itself describes as "managing software and infrastructure changes across thousands of locations". Whoever wrote that was not thinking about a hypervisor. They were thinking about the Tuesday when somebody has to update a thousand sites that share neither a network nor an electrician.

The commercial backdrop was already known and is worth quoting as what it is: an allegation in a lawsuit, not a proven fact. In July 2025 Tesco sued Broadcom, VMware International and Computacenter in the High Court of England and Wales, claiming over £100 million. It argues that it bought perpetual licences in January 2021 with support contracted through 2026, that after the acquisition Broadcom stopped selling standalone support for those licences, and that the offer it received amounted to a 175% increase. Hearings are expected in 2027. When a company claims a hundred million and at the same time advertises a job to pull the hypervisor out of its stores, the technical move and the legal move are the same move. It is worth not overshooting in the other direction either: VMware has told the press it does not consider Tesco a lost cause, and the core ending up on Hyper-V circulates as rumour, not as an announcement. What there is in writing is the job ad, and it is about the stores.

The 5 milliseconds that decide the architecture

Proxmox VE's cluster manager documentation puts it in writing: the cluster needs "latencies under 5 milliseconds (LAN performance) between all nodes to operate stably", and notes that higher latencies may work in small clusters but become unreliable as the node count grows. A few lines down it insists that corosync — the protocol that maintains membership and quorum — is sensitive to latency variation, and recommends giving it its own physical network interface, separate from everything else and from storage in particular. The documentation never uses the word WAN; it does not need to, because the number already rules it out.

Everything else follows from that. There is no "Proxmox cluster of a thousand stores". There are a thousand clusters, each with its own quorum, its own fencing, its own maintenance window and its own clock. That is the sentence to take into the board meeting: the migration does not swap one product for another, it swaps one thing you administer for N things you administer. And N is the number of stores, branches, sites, warehouses or clinics you have — not the number of virtual machines.

A site has room for two machines, rarely three. And two nodes have no majority: if one goes, the survivor loses quorum and can no longer write configuration. The official answer is the QDevice, which the documentation describes for "smaller 2-node clusters" as the way to provide a third vote. What decides the real behaviour is where that third vote lives: put it in the central data centre and you have just made the store's WAN link part of the store's quorum. And the store link is, by a distance, the piece that fails most often. We wrote about how the whole thing behaves when the failing piece is the one that votes when we explained why Proxmox HA does not prevent the outage, it shortens it.

With no array, your RPO is an interval in minutes

In the data centre you got used to shared storage: both nodes see the same disk and, if one dies, the other starts the machine with the data exactly where it was. A store has no array, and putting one in every site pays off neither in money nor in maintenance. What you are left with is Proxmox storage replication, and it is worth reading its whole page: it works on local ZFS only, the minimum interval is one minute and the maximum once a week, and the default for a replication job is every fifteen minutes.

And then comes this one: "High-Availability is allowed in combination with storage replication, but there may be some data loss between the last synced time and the time a node failed". Translated into board language: your recovery point objective at that site is the replication interval. If you never touched that default, you have decided — even if it was never written into any minutes — that a dead node can cost fifteen minutes of receipts, delivery notes or patient records. That decision is legitimate; taking it by omission is not. And it is exactly the gap still open on the shared-storage side, which we went through in what Proxmox is still missing to put a shared array underneath.

Why a supermarket chain sits down to write a platform

Proxmox has an official answer to the thousand-consoles problem, and it is a good one: Proxmox Datacenter Manager, stable since 1.0 on 4 December 2025 and documented today at 1.1.9. It aggregates clusters that do not talk to each other into a single view and allows live migration of guests between them, which until it arrived had no supported way of happening. For an estate of sites it is the missing piece.

And now the part that explains the job ad. Its own roadmap lists, as pending, things that are the daily bread of running an estate: "a console for remote resources (nodes and guests)", "bulk actions on virtual guests across the selection in the central guest list", "a backup-job overview with last-execution status", "firewall management" and "off-site replication copies of virtual guests for manual recovery". None of that is a flaw: it is a young product that publishes honestly what it does not do yet. But it means that today the Datacenter Manager gives you the view of the estate and not yet the operation of the estate. With five sites you cover the difference by hand. With five thousand, you hire somebody in Bengaluru to write the missing layer — and that is precisely what the ad says.

The arithmetic that flips sign at the edge

Proxmox VE subscriptions are paid per occupied physical socket, at published list prices: €120 a year for Community — which the vendor itself places outside production — €370 for Basic, €550 for Standard and €1,100 for Premium. VMware, since Broadcom's repackaging, is licensed per core with a documented floor of 16 cores per physical CPU.

In the data centre those two models look more alike than you would think: you have few, very large sockets with forty machines on top, and the per-machine cost comes out small either way. At the edge it inverts: you have many small sockets with few machines each. The 16-core floor punishes store-grade hardware specifically — an eight-core CPU is billed as sixteen — and the per-socket model stops diluting. Our own arithmetic, at list prices and before any discount: 300 sites with two single-socket nodes each is 600 sockets; at Basic, €222,000 a year; at Standard, €330,000. The saving against what you were paying is almost always real, but it is not automatic and, above all, the unit of account is the site.

One more figure that matters specifically at the edge, because the site opens at seven and the outage does not wait for office hours: on 2 September 2026 Proxmox announced the extension of its enterprise support to 24/7, live from 19 October for the Premium tier and planned for Standard during the last quarter of 2026. Until very recently, vendor support ran on Austrian business hours. If that was your argument for not looking at Proxmox, it expired on 19 October: a week ago.

The three-bucket inventory (do it before you pick a hypervisor)

This is the part you can do this week without installing anything and without asking for a quote. Take one representative site, list everything running inside it — machine by machine, service by service — and drop each item into one of these three buckets. You do it standing in front of the cabinet, with the list on paper and somebody who works there beside you, not from the console.

  1. Bucket A — must keep working with the link down. Tills and payment, scales and labelling, access control, local camera recording, delivery-note printing. Everything that, if it stops, sends people home. This is the only thing that justifies having a hypervisor on site at all.
  2. Bucket B — can degrade and catch up later. Replenishment, signage, reporting, stock sync, telemetry. If it queues and resends when the line comes back, it does not need local high availability: it needs a well-built queue and somebody watching the size of that queue.
  3. Bucket C — should not be there. The file server that stayed in the store because in 2014 there was no decent link, the database somebody put in "temporarily", the domain controller nobody dares touch. This bucket does not get migrated: it gets emptied. Every item you pull out of it is one fewer machine to patch across N sites, forever.

The reading rule: if bucket A does not fit in one hand, what you have at that site is not an edge, it is a small data centre with no staff. And if bucket C came out empty, look again: there is almost always something still running there only because one day there was no decent line and nobody revisited it afterwards. The size of bucket A decides whether you need two nodes or one machine with good backups is enough; the size of bucket C decides what the migration will actually cost.

What changes when you go from one cluster to three hundred

Piece In the data centre Across 300 sites
Quorum Majority among nodes, all on the same LAN Two nodes and a QDevice; you decide where the third vote lives and what happens when the link drops
Data Shared storage or Ceph; zero RPO Asynchronous ZFS replication; the RPO is the interval you configure
Updates One window, a team watching 300 windows; without wave rollout and automatic rollback it is perpetual manual work
Backups One job, one report somebody reads 300 reports nobody reads; you need an aggregated "what failed last night" view
Hands on site Somebody walks down to the room The most technical person on site is whoever opens the shop; everything goes through out-of-band remote access or it does not happen
Licensing Few large sockets, cost diluted across many machines Many small sockets; cost scales with the number of sites, not with the workload

What we would do, and when we would do none of it

We run Proxmox VE with Ceph in production in more than one room, we have moved companies from VMware to Proxmox, and we have also told people to stay where they were when the numbers did not add up. With that said, the order we would follow across an estate of sites is this one, and it is not the order people usually start with. First the three-bucket inventory and the emptying of bucket C, which is free and shrinks the problem. Second, write the replication interval into the minutes of a meeting, with a number and a signature, because it is the business decision hiding inside a dropdown. Third, decide where the third vote lives. And only then, the hypervisor.

And the opposite case: if you have three branches with a good link, do none of this. Centralise into your infrastructure and cloud, leave on site only the minimum that genuinely has to survive an outage — usually it fits in one machine, not a cluster — and spare yourself maintaining three islands. Edge virtualisation is not a medal; it is a cost you only pay when bucket A demands it. What does not work is the very common middle position: a site with two nodes, no third vote, no written replication interval and bucket C overflowing. That has every cost of high availability and none of its guarantees.

If you are in the middle of a VMware to Proxmox migration and the knot is the sites rather than the data centre, the useful conversation takes an afternoon and can be had before buying anything: the inventory of one representative site, the interval figure, and where the third vote goes. If you want us to have it with you, let us talk.

Sources. The news and the job-ad quotations: The Register's 26 October 2026 article on Tesco replacing VMware with Proxmox in its stores, and the posting itself on careers.tesco.com, published under the title Head of Software Development (reference 215705, Bengaluru) and describing itself in the body as Head of Software Engineering: Edge Platform, which is where "modernising Tesco's edge virtualisation estate through the migration from VMware to Proxmox", "a secure, reliable and cost-effective alternative virtualisation platform", the measure "safe and predictable migration from VMware to Proxmox, successful validation of required store configurations, reduced legacy dependencies and effective operational handover", the "Edge Management Platform to provision, manage, monitor and maintain the retail edge estate throughout its lifecycle" and "managing software and infrastructure changes across thousands of locations" all come from; the figure of more than 5,000 stores, the expectation of hearings in 2027, VMware's position (it "does not consider Tesco a lost cause") and the Hyper-V-for-the-core rumour — which the article itself labels as rumour — come from the same article. The lawsuit — filed on 15 July 2025 in the High Court of England and Wales against Broadcom, VMware International and Computacenter, claiming over £100 million, perpetual licences bought in January 2021 with support through 2026, and an alleged 175% increase — is drawn from public coverage of the case, and all of it is the claimant's allegation, not findings of fact. The figure of around 40,000 server workloads under migration and the trial window set between 1 November 2027 and 25 February 2028 come from the June 2026 coverage of the case; the 26 October article says only that hearings are expected in 2027. Proxmox VE: the 5-millisecond "LAN performance" latency figure, the recommendation of a dedicated physical network for corosync, and the QDevice as a third vote for 2-node clusters, from pve.proxmox.com/wiki/Cluster_Manager; local-ZFS-only support, the one-minute minimum interval, the fifteen-minute default and the sentence about data loss between the last sync and the node failure, from pve.proxmox.com/wiki/Storage_Replication. Proxmox Datacenter Manager: the official 1.0 stable announcement (forum.proxmox.com, 4 December 2025), which introduces live migration of guests between different clusters; documentation at version 1.1.9 (14 October 2026), whose guests page describes the migrate action as "migrate the guest to another node, within the same remote (cluster) or across remotes"; and the pending items quoted, from pdm.proxmox.com/docs/roadmap.html. Per-socket subscription prices (€120 / €370 / €550 / €1,100 a year) and Community's status as a non-production option, from proxmox.com's pricing page. The extension of support to 24/7: Proxmox press release of 2 September 2026, effective 19 October for Premium and planned for Standard in the last quarter of 2026. The 16-cores-per-physical-CPU floor on VMware Cloud Foundation and vSphere Foundation subscriptions is in Broadcom's own service description (VVF Service Product Description, ftpdocs.broadcom.com), which requires every core on the server to be licensed — including cores deactivated in the BIOS — with a minimum of 16 core licences per processor. The additional 72-cores-per-order minimum the channel announced in March 2025 was withdrawn weeks later and is not in force. The 600-socket and €222,000 figures are our own arithmetic on list prices, with no discounts or volume terms, and are there to show how the model scales, not as a quote. We have not migrated anybody's stores this week: this article reads public documents and cross-reads them against how an estate of sites is actually operated, and says so deliberately.

Tags: Virtualization Proxmox VMware Infrastructure Migration Networking

Subscribe to our newsletter

To receive IT stories, everyWAN news and exclusive subscriber offers, sign up to our mailing list

Minorisa de Sistemas Informaticos y Gestión S.L. © 2026
everyWAN
everyWAN