Sign in Subscribe
openwrt

Install OpenWRT on PVE

ロジャー

• 6 min read

I had built a 6-port router running OpenWRT.

Building budget OpenWRT router and upgrading to 2.5gbps network
PREFACE Recently I am upgrading my home network to 2.5gbps. The service provider (ISP) provides a optical modem for free (to be precise, “without extra charge”). There are not many choices in the market at the time of writing. A 2.5gbps router with one 2.5gbps WAN port
theMatrixDevロジャー

A few days ago, after a reboot, it did not load OpenWRT. Taken out the SSD and checked on another computer, the SSD could not be identified. I am not sure if its the SSD quality issue, or the high heat killed the SSD.

The bundled SSD. 128GB comes with only 2 chips.

I have purchased Kingston NV2 250GB SSD for replacement.

Preparation

Download PVE 8.1 ISO

ISO (PVE)
Proxmox

Download OpenWRT 23.05.2 x86_64 generic-squashfs-combined.img.gz

Index of /releases/23.05.2/targets/x86/64/

I have found a good tool which you can place many ISO image files onto a single USB thumb drive as boot disk, without needing to flash them one by one.

GitHub - ventoy/Ventoy: A new bootable USB solution.
A new bootable USB solution. Contribute to ventoy/Ventoy development by creating an account on GitHub.
GitHubventoy

Flash Ventoy with Etcher

balenaEtcher - Flash OS images to SD cards & USB drives
A cross-platform tool to flash OS images onto SD cards and USB drives safely and easily. Free and open source for makers around the world.
Flash OS images to SD cards & USB drives

Put PVE and OpenWRT ISO files onto Ventoy.

Install PVE

Plugin the USB thumb drive into the router computer. Using the router computer, install PVE and reboot. I used these info:

  • Management Interface: (Choose the port connecting the router with your PC)
  • Hostname: dev.thematrix (Anything you like)
  • IP Address: 192.168.50.2/24 (The static IP address of PVE)
  • Gateway: 192.168.50.1 (The static IP address of the OpenWRT to be installed)
  • DNS Server: 192.168.50.1

Configure PVE

Using your PC.

Manually assign an IP address. Example on Ubuntu:

SSH access PVE on the router computer:

ssh [email protected]

Remove enterprise sources, since we are using free unpaid version:

rm /etc/apt/sources.list.d/ceph.list
rm /etc/apt/sources.list.d/pve-enterprise.list

The router seller had provided me these to "prevent the router from dying" when using PVE:

nano /etc/default/grub

Replace:

GRUB_CMDLINE_LINUX_DEFAULT="quiet"

To:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_port_pm=off quiet splash intel_idle.max_cstate=2"

Apply changes:

update-grub
update-initramfs -u -k all
reboot

After reboot, access the router from browser. Go to http://192.168.50.2. Login with username root and your configured password.

I am going to use bridged network instead of passthrough, since I don't want to sacrifice one port for PVE management use. Bridged network speed is good enough to handle my 2.5gbps network.

All those enp*s0 are physical network interface.

Click on pve on the left, choose Create > Linux Bridge.

Add all the physical network interface to be bridged interface.

Beware of the numbers (order) and the management port. Click Apply Configuration.

Install OpenWRT

On PVE, click Create VM on the top right corner.

  1. General: Enter a name (e.g. openwrt). Check Start at boot.
  2. OS: Choose Do not use any media.
  3. System: Use default.
  4. Disk: Enter 1 for disk size. It will be deleted afterwards.
  5. CPU: Adjust sockets and cores accordingly. Choose host as type.
  6. Memory: Adjust accordingly.
  7. Network: Use default.
  8. Confirm: Uncheck Start after created. Finish.

Choose the newly created instance. Choose Hardware

  1. Remove CD/DVD Drive
  2. Detach Hard Disk
  3. Remove Unused Disk 0

Choose local (pve) on the left. Choose ISO Images. Click Upload

Upload the OpenWRT image file.

Copy the uploaded path.

Choose pve on the left. Choose Shell

Replace 100 (your instance ID) and path (uploaded OpenWRT image file) and run in Shell:

qm importdisk 101 /var/lib/vz/template/iso/openwrt-23.05.2-x86-64-generic-squashfs-combined.img local-lvm

Wait until it shows:

transferred 120.5 MiB of 120.5 MiB (100.00%)
Successfully imported disk as 'unused1:local-lvm:vm-101-disk-1'

Choose your instance on the left. Choose Hardware. There is a new Unused Disk.

Click Edit. Add.

Click Add > Network Device. Add all the bridged network interfaces.

Choose Options. Make SCSI0 to be the only option in Boot Order.

Click Start on the top right corner.

Click Console you will see the boot up progress.

Initialize OpenWRT

On the Console, run:

vi /etc/config/network

Modify the ipaddr on interface 'lan'. In my case, eth5 is my management interface.

Restart the network. Run:

/etc/init.d/network restart

Open http://192.168.50.1 on a web browser to continue OpenWRT configuration.

Bonus

OpenWRT and PVE died periodically. It occurred very frequently, once within an hour.

OpenWRT showed something like "read only system" and "IO error".

On PVE shell it also showed an error and I can no longer control PVE. I have to power cycle the router computer.

dmesg gave me some hints.

It seems to be a recent issue on PVE:

NVMe Issue: Unable to change power state from D3cold to D0, device inaccessible
Hi! I have an issue where one of my two NVMe Drives “disappears” some time after boot. I have two Samsung 990 PRO 4TB in a ZFS Mirror for my VM Storage, one of which seems to have this issue, the other one is perfectly fine. It doesn’t seem to be a temperature issue, like in other threads I…
Proxmox Support ForumYAGA

I suspect it is a compatibility issue, rather than a hardware one.

Swapping to an old spared used Kingston A2000 250GB, PVE and OpenWRT has been working well.