Migrating to Proxmox: the network is not held by the cluster, it is held by each node

In a VMware to Proxmox migration, the conversation always starts with the disks. How long they take, whether thin provisioning survives, what happens to the snapshots. Disks are the easy part: they either arrive or they do not, and you find out the same day. What nobody sees is something else, and it is the thing that bites six months later: in vSphere the virtual network is a datacentre object, and in Proxmox VE it is a file on each machine. You take the network design with you. You do not take the mechanism that kept it identical across every server.

Vested interest up front: we sell VMware to Proxmox migration and we run customer infrastructure and networks. We have also written about when not to migrate, so there is no enthusiasm on sale here. The topic is timely for a calendar reason: Proxmox's own lifecycle table puts Proxmox VE 8's end of support at August 2026 — the month, with no day — and that has filled the summer with people moving machines in a hurry, off VMware or from one branch to another. We already warned that apt will not tell you about that expiry date. Nobody tells you about today's one either.

The one file the cluster does not replicate

Start with the sentence that explains everything, in Proxmox's network configuration chapter: "the configuration can be done either via the GUI, or by manually editing the file /etc/network/interfaces, which contains the whole network configuration". The whole of it. For that node. And in case anyone thinks this is an oversight by a platform with no clustering, Proxmox VE does have a cluster filesystem: pmxcfs, "a database-driven file system for storing configuration files, replicated in real time to all cluster nodes using corosync". The documentation lists what lives there: corosync.conf, datacenter.cfg, the firewall, the high availability rules, the certificates, and every virtual machine's configuration under nodes/<NAME>/qemu-server/<VMID>.conf. /etc/network/interfaces is not on that list.

To measure the change it helps to remember where you are coming from. The vSphere Distributed Switch architecture documentation says that "the networking configuration that you create on vCenter Server (the management plane) is automatically pushed down to all host proxy switches (the data plane)". There is an object, it lives in one place, and that place makes sure every host looks like it. In Proxmox nobody inherits that place.

Here is an opinion of ours, flagged as such because it appears in no documentation: this is not an oversight, it is a sensible decision. A node's network is what lets you reach the node. A file that replicates itself in real time to every machine is, applied to networking, the most efficient way ever invented of locking yourself out of an entire cluster with one typo. Proxmox deliberately leaves it out, and even adds a safety net: "Proxmox VE does not write changes directly to /etc/network/interfaces. Instead, we write into a temporary file called /etc/network/interfaces.new", applied when you say so, via the Apply Configuration button, ifreload -a, or a reboot. The decision is right. The consequence has to be owned: whatever the software does not synchronise, you synchronise.

The bridge name is the contract

In vSphere, a VM's NIC points at a port group: an object, with its own identity, living in vCenter. In Proxmox, the NIC line in the VM configuration looks like this:

net0: virtio=BC:24:11:3A:7E:52,bridge=vmbr1,tag=30

vmbr1 is a string. There is no object behind it, no unique identifier, nothing guaranteeing that one node's vmbr1 is the same as its neighbour's. It resolves at boot time, against the file on whichever node it boots on. And look at how the responsibility splits: the reference does travel with the cluster — that line lives in /etc/pve, replicated in real time — and the thing being referenced does not. On names, the documentation is generous to the point of danger: "commonly vmbr[N], where 0 ≤ N ≤ 4094, but you can use any alphanumeric string that starts with a character and is at most 10 characters long". Ten characters to name the thing that decides whether a machine can see the network at all.

The expensive failure is not the obvious one. The obvious one — the bridge does not exist on the target node — shows up early and costs little. The expensive one is the other: the same name on both nodes, hanging off different things. On node A, vmbr1 sits on the bond of two 10 Gb NICs with an MTU of 9000. On node B, somebody once put it on a single 1 Gb NIC with an MTU of 1500, "temporarily". The VM migrates without complaint, boots without complaint, and from then on runs slower and drops large packets, and nobody connects the two events because the panel looks exactly the same.

The official requirements list, and what is missing from it

This is easier to see by looking at what Proxmox requires, in writing, for a live migration. The Requirements section of the virtual machines chapter has five bullets:

• The VM has no local resources that cannot be migrated. The documentation spells out which: passed-through PCI or USB devices block live migration, while local disks can be sent to the target just fine.
• The hosts are in the same Proxmox VE cluster.
• They have a working and reliable network connection between them.
• The target host has the same or higher versions of the Proxmox VE packages.
• The hosts have CPUs from the same vendor with similar capabilities.

Read the list again. The bridge is not on it. And it is not on it because, as far as the migration mechanism is concerned, it is not a requirement: it is an assumption. The documentation assumes that if you built a cluster, your nodes look alike. That is a reasonable assumption on day one, when the three servers are identical and the same person installed them on the same afternoon. It stops being reasonable the day a fourth node arrives from another vendor, with a different number of NICs, bought eighteen months later.

And there is one case where you are not the one checking that assumption: high availability. HA rules live in /etc/pve/ha/ — replicated across the whole cluster — and the HA manager decides which node a machine restarts on when its own disappears, at four in the morning, without asking. What decides where the VM goes belongs to the cluster. What decides whether it will have a network there does not. We have already written that HA does not prevent downtime, it shortens it; this is the same argument from another angle. Failover is worth nothing if the machine comes up somewhere its VLAN does not reach.

The MAC changes, and things hang off the MAC

The official Proxmox VE migration guide dispatches this in one line, under Post Migration: "update network settings. The name of the network adapter will most likely have changed". True, and an understatement, because the name is the effect and not the cause. The cause is in the virtual machines chapter: "Proxmox VE will generate for each NIC a random MAC address, so that your VM is addressable on Ethernet networks". The NIC does not move house: a new one gets manufactured.

DHCP reservations and hardware-bound licences already went on the list when we covered the rollback plan, so we will not repeat them. What that list did not say is what matters today: the address is not preserved, it is drawn again, so the thing that suffers most is port-based access control on the switch, 802.1X, where a new MAC does not produce a network error but an authentication refusal somewhere almost nobody watches during a migration.

The good news is that the MAC is a parameter, not a fate. The net[n] definition includes macaddr=<XX:XX:XX:XX:XX:XX>, and you set it with qm set like anything else. You decide it before the migration, with the list of machines that have something tied to a physical address in front of you, not afterwards by discovery. And in the guest, if it is Linux, the interface name changes because the device is new; the documented way to pin it is a systemd .link file with [Match] MACAddress= and [Link] Name=. Proxmox documents this for host NICs, but the mechanism is systemd's and it is the same inside a Linux guest.

The NIC that boots is not the NIC you want

Proxmox emulates vmxnet3, VMware's paravirtualised NIC, and that is a blessing on migration day: the machine boots with networking without touching the guest. The documentation is explicit about what it is for: "the vmxnet3 is another paravirtualized device, which should only be used when importing a VM from another hypervisor". It is there for the journey and nothing else. Of the good model it says: "the virtio model provides the best performance with very low CPU overhead".

The cost of leaving it "temporarily" has a detail people miss, and it is expensive in clusters with large frames. The mtu parameter definition reads: "force MTU of network device (VirtIO only). Setting to 1 or empty will use the bridge MTU". So while the NIC is still vmxnet3, the per-interface MTU is not yours: you inherit the bridge's. If you run a storage, backup or Ceph network on 9000-byte frames, that is precisely the lever you just gave up. Same story with multiqueue, which is also a VirtIO thing. So the conversion goes on the project calendar, with a date, not on the "someday" list.

With no bridge you are not left without a network: you are left with another one

This is the one that wastes the most time in a migration done in a hurry from the console or the API, and it is documented word for word: "if you do not specify a bridge, we create a kvm user (NATed) network device, which provides DHCP and DNS services". And it gives the addresses: 10.0.2.2 gateway, 10.0.2.3 DNS, 10.0.2.4 SMB server, with DHCP handing out addresses from 10.0.2.15.

That is why the symptom is not "the machine has no network". The symptom is "the machine has an IP, resolves names, reaches the internet and nobody can see it", which is the worst place to spend an hour at eleven at night. The practical rule fits in one sentence: if a freshly imported machine shows a 10.0.2.x address, do not go digging in the firewall or the switch; you are missing bridge=. The documentation adds that NAT mode "is much slower than the bridged mode, and should only be used for testing", and that it is not available in the web interface: CLI and API only. Which explains fairly well how it gets in.

SDN is the way out, and it has small print

Proxmox is not blind to any of this: that is what SDN is for, and since version 8.1 it ships installed by default. The sentence that matters is this one: "the Proxmox VE SDN configurations are located in /etc/pve/sdn, which is shared with all other cluster nodes through the Proxmox VE configuration file system". There is the missing piece. It also has pending changes and joint application — "new changes are not immediately applied but recorded as pending first", to roll them out "as single atomic one" — and it tracks what is deployed in .running-config and .version. It is the management plane you were missing.

And now the small print, which is where people who arrive looking for the vCenter they no longer have come unstuck. Look at what a VLAN zone asks for, which is the one you will use if you are coming from tagged port groups: "the local bridge or OVS switch, already configured on each node, that allows node-to-node connection". In other words: SDN does not remove the per-node file. It sits on top of it. What it does — and it is not nothing — is reduce what you must keep identical by hand to a single thing, the bridge underneath, and lift the whole layer above it, the VNets and the VLANs, into the cluster. You go from synchronising twenty lines per node to synchronising one. That is a huge improvement and it is not a magic wand.

The other half of the small print is published by Proxmox on the same page, under support status, and it is worth reading before you draw the architecture: SDN core "is fully supported", but "IPAM, including DHCP management for virtual guests, is in tech preview", and "complex routing via FRRouting and controller integration are in tech preview". Translated into decisions: the VNet and VLAN layer, go ahead. Your servers' DHCP, wherever it already works, not in a preview. And if the design heads towards QinQ, write down the arithmetic: double tagging needs four more bytes, "you must reduce the MTU to 1496 if your physical interface MTU is 1500".

Before this sounds like VMware was paradise

It was not. vSphere hosts also drift out of sync with the distributed switch; the vendor documents a state for exactly that, with its own support article and its message that the distributed switch configuration on some hosts differed from that of the vCenter Server. Anyone who has pulled a host out of maintenance with vCenter down knows the feeling. The difference is not that VMware never drifts: it is that in VMware there is something to compare against. There is an object stating how the world ought to be, so the drift has a name, shows up in the interface, and somebody sees it. In Proxmox there is no object. There are three files that ought to look alike. If they do not, nothing says so: one day a machine is simply slower on one node than on another.

And there is a case where none of this applies to you: if you have one or two nodes, a single flat vmbr0 bridge and no VLANs, this is not your problem, and do not build an SDN just to have one. There is nothing to buy here either: half of this article is solved with diff, and the Proxmox documentation even hands you the command for reviewing a change before applying it, diff -y /etc/network/interfaces /etc/network/interfaces.new. What costs money is not the tool: it is remembering.

What we do before signing off a cluster

• Write the bridge table before touching anything. Name (ten characters max, identical on every node), the physical link or bond it hangs off, whether it is VLAN aware, MTU, and which nodes it exists on. If a row has a gap, that gap is a future incident with the date still to be filled in.
• List which machines have something hanging off the MAC and pin it by hand with macaddr= on those, and only those. DHCP reservations, port authentication, hardware-bound licences, allow-lists.
• Go through tag= and trunks= machine by machine. In vSphere the VLAN lived in the port group, that is, in one place; here it lives in each VM's NIC line, that is, in three hundred. It is tedious and there is no honest shortcut.
• Put a date on the vmxnet3 to virtio change, in the project document and not in somebody's head. If 9000-byte frames are in play, that date comes sooner rather than later.
• Put a comparison of the network files across nodes into monitoring. It is a periodic diff that shouts when two nodes stop looking alike. It costs an afternoon and it replaces the object Proxmox does not have. What we already wrote about your infrastructure documentation applies here, and we will not repeat it.
• And the test that actually closes the matter: before signing off the cluster, live-migrate one machine from each VLAN to each node, and check it still talks afterwards. Not one machine: one per combination. It is the only way the assumption that is not in the requirements list stops being an assumption. It goes in the same part of the project as the rollback plan.

None of this is an argument against Proxmox. We run it in production with Ceph, across several datacentres, and we would not swap it. It is an argument against migrating a design without migrating the discipline that held it together. In vSphere, that discipline was supplied by a product that cost money. In Proxmox you supply it, and the good news is that it fits in a table and a scheduled diff. The bad news is that if nobody writes it down it does not exist, and by the day it shows, a month will have passed since somebody moved a cable on the wrong node.

Sources. Official Proxmox VE documentation, version 9.2.4 (chapters Network Configuration, Qemu/KVM Virtual Machines — networking, migration and the net[n] definition —, Proxmox Cluster File System (pmxcfs) and Software-Defined Network) and the official Migrate to Proxmox VE wiki. Proxmox VE 8 lifecycle: the table in Proxmox's official FAQ. vSphere Distributed Switch architecture and the host out-of-sync state: Broadcom documentation and knowledge base. Quotations in Spanish and Catalan are our own translation of the English originals. Whatever is flagged in the text as our judgement — that leaving networking outside the cluster filesystem is a deliberate and correct decision, and that the expensive failure is the same name hanging off different things — is operator opinion, is in no documentation, and is labelled as such. We measured nothing here: there is no performance figure of our own in this article because we have none we can publish.