Bandwidth throttling your home lab and Rubrik replication/archiving
June 21, 2017
Posted by on
Having your own home lab is one of the best tools you can have to progress your knowledge and career in IT. But, having lots of VMs and hardware can equal a decent load on your home internet connection. This gets even worse when you start using replication and archiving technologies. So how can you stop your home lab from saturating your internet connection, interrupting the family movie night (yes this happened to me), and the plethora of streaming services used in a modern household?
You might first look at your router for the answer. If you can apply network throttling on your router then you are good to go, but like many households your router is probably lacking this sort of functionality. This doesn’t just apply to home labs. What if you implement a technology at work that uses the WAN for replication and archiving but doesn’t have built-in throttling? What if you can’t apply throttling at the physical network layer due to response times on requests, change control, lack of functionality, security restrictions or even political reasons? This is a use case I’ve come across a couple of times already at Rubrik and my solution is just as applicable to WAN traffic leaving a Rubrik cluster as it is your home lab.
So, what is the solution? Simple, create a dual NIC VM, install an open source firewall then apply traffic shaping to your lab/replication/archiving traffic using the VM as their new default gateway. I use FreeBSD based OpnSense in my environment, which you can download for free from:
Before I take you through the setup process lets first cover the network topology so you can see what I’m doing. If you are like me you want to keep a relatively simple home network so your user devices such as laptops/tablets/pcs can access the internet without relying on a VM in your home lab. You also want to be able to access any VM in your lab without having to change the IP on your device. The tricky part is how to do this when a firewall requires 2 NICs on different subnets for internal/external traffic? With a clever use of subnet masking, you can do just this.
With the above configuration, any user device and VM can communicate with each other so I don’t need to change the IP address on my laptop to access the lab. All traffic from end user devices goes straight out to the router and so isn’t reliant on the firewall VM and has no traffic shaping applied. All the lab VMs etc have the default gateway configured to use the green/internal firewall interface and because they are on the same subnet, despite having a different mask, they can route through the firewall. The firewall then applies traffic shaping to the VM traffic which it routes over the red/external interface to the physical router. If I want to bypass the firewall for any specific VM I can simply change the gateway to 192.168.1.1. Easy!
Now that we’ve clarified the network topology lets go through how to configure an OPNSense Firewall VM:
- Download the ISO from https://opnsense.org/ and unpack the compressed bz2 file (I use 7zip)
- Create a VM with the following spec:
- Boot the VM and attach the OPNsense ISO
- At the console login prompt use:
- Walk through the prompts to install the firewall to disk, otherwise, you are just running off the ISO and the config won’t persist through reboots:
- Wait 10 minutes or so for it to finish installing (reasonably slow given the small ISO size)
- Remove the ISO and reset the VM (reboot failed for me with a small error, but a reset did the trick)
- Login to the console using:
- Type “1” and press enter to assign the interfaces, type “n” to not set VLANs then “em0” to assign the first VMNIC as the WAN interface then “em1” to assign the second VMNIC as internal:
- Type “2” and press enter to configure the IP addressing on the firewall repeating the process for both the internal and external connections:
- Load the OPNSense interface using https://opnsenseinternalIP. The caveat being you need to use a host on the same subnet as configured on the internal interface (192.168.0.1 / 255.255.255.0). I’m using a VM in my home lab on IP 192.168.0.30 so it can communicate with 192.168.0.1:
- Run through the GUI wizard to check your network settings and change the root password
- Change a VM (or your Rubrik cluster) to use the new default gateway (192.168.0.1 in my environment) and test connectivity:
- Now we can apply traffic shaping to throttle the lab traffic by first configuring a pipe with the rate limit:
- Configure a rule for the LAN traffic and set the target to be the pipe you just configured:
- Above you can see the ability to be very granular in applying your throttling by limiting based on IPs and ports (hint; for Rubrik uses TCP 7785 for replication and 443 for archiving to AWS S3/Azure etc)
- To verify the rate limit has been applied go to the status page to see the real-time connections:
- Change the default gateway on all your home lab equipment and VMs to the new default gateway
- For a final test run a broadband speed checker on one of your VMs using the firewall as the default gateway. Here you can see my 10Mbps limit has been applied to my lab VM, but not my laptop:
Lab VM using the OPNsense firewall VM on 192.168.0.1 as the gateway:
Laptop using the physical router on 192.168.1.1 as the gateway:
And that’s it! If you followed these instructions you now have a powerful, free, firewall that you can use to shape the traffic ensuring it doesn’t saturate your WAN link. If you found this useful please like and share.