Rich Gibbs

Moving Proxmox to VLAN 10: a Boot Scare and a Wrong Address

proxmox · networking · vlan · homelab · troubleshooting

A homelab diary entry on cutting a Proxmox host and two guest VMs over from a flat LAN to a routed VLAN on existing switch hardware: the physical cable move, a firmware PXE-boot surprise unrelated to the network change, a bottom-up verification sequence that closed an open question about internet reachability, and a false SSH alarm that turned out to be a misdirected connection to the wrong machine.

Today's job was the actual cutover: get the Proxmox host and its two guest VMs off the flat LAN and onto a routed VLAN, on switch hardware that was already sitting there. The planning happened in an earlier session. This one was about doing it.

Before touching the cable, I checked that neither VM was set to autostart, then shut both down cleanly. With both stopped, nothing was running that a cable move could interrupt — that's the whole safety argument, and it held. While I had the cable out I swapped in a better one and checked the link light on the new port before rebooting.

The reboot didn't go the way I expected: the host fell through to PXE network boot instead of booting off disk. Bad timing to look at, right after a network change, but it wasn't the network — firmware doesn't read the OS's network config, so this was a boot-order problem that happened to surface at the worst possible moment. Power-cycled, picked the disk by hand from the boot menu, moved on. Boot order in BIOS is still wrong and still needs fixing. Getting lucky once isn't the same as fixing it.

Verification went bottom-up, one dependency at a time: the host's own new address came up correctly, then the gateway answered, then the other network segment answered, then the old gateway answered too — with a TTL one notch lower than it would be on the same segment, which is exactly what you'd see for a packet that crossed one extra routing hop, not something I had to take on faith. Then a public internet address answered as the only path out, then DNS resolved. That chain specifically answers something I'd left open from the staging session: whether a host with no other route, only the new gateway, could actually reach the internet. It can.

For the VMs: pinged the first one's new address from the host, then started both by hand. The real proof wasn't the ping, though — it was that each VM's automation agent checked in over its messaging platform, which only happens if outbound internet and DNS both actually work. A public site loading on my phone over mobile data was the same kind of proof from the other direction: not a LAN shortcut, genuinely routed out and back through a tunnel service that had reconnected on its own. SSH across VLANs worked too. It asked for a fresh host key, which was expected since the address had changed — that's a reason not to be alarmed by the prompt, not proof by itself that the thing on the other end was who I expected.

Then a scare that wasn't one: SSH to one of the VMs looked broken. I'd actually connected to the other machine's new address, and the login I was using doesn't exist there. That other machine separately confirmed it hadn't touched any SSH or password settings — so whatever got "fixed" on that side was unrelated, just a stale key-based link sorting itself out. The underlying lesson: know the address map before you start debugging, or you'll spend your time debugging the wrong problem.

What to remember from this one:

  • Order changes by dependency, one at a time. Proxmox had to move before the wired WAN could, because it was sitting on the port the WAN needs — keeping that as a later, separate session means a failure there only has one possible cause.
  • Test bottom-up. Own address, then gateway, then the other segment, then the upstream router, then the internet, then DNS. Each step adds exactly one dependency, so the first failure tells you which layer broke.
  • Verify by function, not by reading config. A message from an agent or a page loading on a phone proved more than staring at a config file would have.
  • A surprise right after a change isn't automatically caused by that change. The PXE fallback looked like a network problem; it was firmware not reading the boot order. Read what the screen is actually telling you before blaming the thing you just touched.
  • Know the address map before you debug. The one real "failure" tonight was just a wrong address.

Still open at the end of this session: BIOS boot order on the box that fell through to PXE. A kernel-hang watch that's now actually relevant, since the offload workaround it depends on is live. A leftover pre-migration config file on both VMs that still needs removing. A dead node hanging around in a mesh network that should come out. VLAN-to-VLAN access between the management VLAN and VLAN 10 is open right now — deliberately, because that's what keeps management working while the ACL work is unfinished, not because leaving it open is fine long-term. Switch NTP and a reboot-durability test are still pending. The wired-WAN move itself is checked off with a pointer to a later session — not covering that here.

One more thing to flag for later: an unrelated change — a new recovery mechanism for blog publishing — landed in this same window. If something misbehaves soon after this, that's a second variable to rule out, not automatic proof the VLAN work caused it.