I understand. If you buy a used server then add some drives later you can have a great NAS IMHO. I upgraded from an X8DT6-F with 384GB of RAM and a pair of Xeon X5690’s right before things went sideways. The MoBo has the SAS controller already flashed to IT mode so it’s ready for ZFS. But it’s not exactly light on power and with a 2U chassis and a handful of used 8TB SAS drives you are looking at around 1200.

My current server is a X10DRH-C with dual Xeon E5-2683 v3’s with 128GB of RAM in a 4U chassis with 11 X 8TB SAS drives in a RaidZ3 configuration. Just the MoBo, chassis, cpu’s and 64GB of ram is running about 1150. The drives used are 110 each and before you think you should just get those, SAS drives require a SAS controller and you only get those in enterprise equipment.

But if you can pick up a little here and a little there you can have a nice system. But right now isn’t a great time to get in the game.

I’m not saying that.

What you need to do is decide now if the drive you will buy will be used for a RAID array. If it is a desktop drive won’t be in a RAID array on a NAS system. Many NAS’ will have random writes to the pool. Desktop drives aggressively park the heads, the load and unload of the heads wears them. In a NAS system they can actually wear out.

Over the last 15 years drives have become a bit more specialized. You already found out about surveillance drives not being a good fit for much other than surveillance/DVR. Desktop drives are fine for desktop loads and usage but outside of that or single drive usage they are not useful. NAS drives are meant for NAS usage in RAID arrays. Back when the WD green drives were available years ago you could convert them from a desktop drive to a NAS drive using a tool called wdidle (WD Idle) but that isn’t the case any more.

Using a NAS drive on its own will work in a pinch but if it has an error it won’t try to recover it like a Desktop drive would because it’s made with the idea that it will be in an array that will deal with the issue. Plus once you start loading it up you will have to wipe it to put it into an array unless you go for ZFS mirrors or RAID 1. If the NAS Appliances have some sort of special trickery that allows you to expand one disk at a time and add redundancy I’m completely unaware as I’ve never put much stock in them. I’ve been running FreeNAS/TrueNAS for over 10 years.

https://en.wikipedia.org/wiki/Standard_RAID_levels

If a NAS or Enterprise drive has an error it sends the information to the host to be logged so that the end user can have the information available.

So like an Unrecoverable Read Error (URE) pops up on a sector. A drive that is built for RAID use will just say, “Couldn’t read it” and moves on. A Consumer drive meant for a desktop will try and try and try and try to read that bad sector. In a NAS situation where another drive will be able to fill in the data the controller (hardware or software) will just deal with it by pulling the data from another drive and keep moving.

The drive may not be bad as a whole but it does mean that over time it is more likely that drive will have more errors.

NAS drives are not inherently more reliable, yes they can deal with a bit more vibration and such but it’s the firmware inside that is different. Enterprise drives are another step up again from NAS drives.

What are you using as a router? How are you you trying to setup a reverse proxy?

I will be honest, I would do it for free. Money is nice but helping people is more important to me than that. However my understanding is with OpnSense and using Caddy for reverse proxy with the ACME client for Let’s Encrypt certificates. Beyond that I am outside of my element.

It’s not just about the advanced functions. Many of the older more basic tools are single threaded which will potentially limit the performance. As you figure things out you will want to do more and you may find the current tools more of a limiting factor. But the choice is yours, I have 2 48port gigabit switches and WiFi SSID’s that connect to specific vLAN’s through tagging. I started with some dumb switches and added my 10Gbps backbone switch which I used as a dumb switch for years so I could connect my desktop and server over a faster connection.

In my equipment an untagged port is what a port is where the vLAN is stripped away. A tagged port has the vLAN tag passed to the device. If you can set multiple vlan tags on the same port that port becomes a trunked port. You may also be able to set a vlan as untagged on that port, if a device is plugged into that port it will by default be on the untagged vLAN. If the device is able to handle vLAN tagging it can live on the vLAN’s you set up as well.

It took me a bit to figure it all out and get it working. I spent about 20 hours configuring things before I started making the switch from a single net to multiple vLAN’s. I spent another 8 hours making the change and 5 or 6 more tweaking things.

No. I didn’t use any video’s to set mine up and wouldn’t even know where to find one that is up to date.

Dnsmasq isn’t being deprecated that I know of but when you begin doing more advanced functions some tools work better than others. The “New” rules are fully functional and I suspect the old ones will slowly be removed in later releases. They are revamping some stuff, ISC DHCP used to be the go to but that is being passed out by the creators. https://www.isc.org/kea/ and OPNsense is cleaning up things so it all works with the rest api and is higher performance.

Trunking is when a port can deal with all setup vLAN’s not sure about your switches as they are pretty basic looking. I’m using a pair of Dell PowerConnect 5548’s and a Quanta LB6M which are much more complex. But basically think of a vLAN as a branch of a tree and the Trunk is the base that connects to everything.

Ok, so first of all a TON of things have changed in OPNsense in the last couple updates so you may just want to pull everything you did. It looks like you are using the old firewall rules which while they are going to stick around they are trying to migrate people away from them.

You should be using KEA DHCP as that is the modern and latest and greatest DHCP server that is vLAN aware. Also UnboundDNS is a recursive DNS server and designed for modern networks, with the advantage of being able to use DNS Blocklists to block ads and other “junk.” Just remember that the blocklists live in RAM so if you don’t have much RAM available I wouldn’t recommend using them.

NOTE: If you are using Dnsmasq DNS&DHCP you will have to turn it off before you can enable KEA DHCP and UnboundDNS as it will tie up the ports needed and they will fail to start. You can copy everything over before making the change and if there is an issue you can switch back just by stopping the new ones and enabling the old one.

With that said one of the biggest things is that with a managed switch you will need to trunk the port that OPNsense is plugged in to for the LAN unless you are using multiple ports on your OpnSense install and then those ports will need to be properly tagged and you will need to trunk any ports that are linking switches together you will have to figure that out on your own but I suggest grabbing a copy of your switch’s manual (and if you use a chatbot upload that file to it for help.) Then you can use vLAN tagging for each port that you want to receive an IP address from a particular pool automatically. You can also trunk ports that you want to use for management but set the default the port will use for access so the device can get an ip via DHCP, this really only works well with Linux. If you are using windows you will need to just create firewall rules that allow your device to talk to the other vLAN’s instead.

When you create your firewall rules you have to understand that you can only preform one action per rule. If you want to allow your vLAN 1 network to talk to vLAN 10 that is one rule. To allow vLAN 10 to talk to the internet that is another rule. You can use floating rules to do the work on multiple vLAN’s but that should be limited. If you select more than one network interface a rule will becoming a floating rule and will process before other rules so if you create a rule to block something later on but have the same interface set on a floating rule the block will not work, it’s better to enable piece by piece than to blanket enable and then try to block. With 5 vLAN’s I have 5 floating rules and 34 regular rules plus 40 automatically generated rules (which handle things like allowing DHCP access and basic protections.)

Here is a firewall rule that allows my “Trusted” vLAN to access my “Camera” vLAN as an example. The Categories are not important but make finding what a rule deals with later on a lot easier, they are set under the firewalls - categories. You should also use good descriptions for this reason.

You will also have to explicitly allow access to services like DNS. This is how I am allowing my “Trusted” vLAN to access DNS services on my OPNsense.

This is how I allow my trusted network to access the internet. If you have multiple WAN’s you can choose a specific one or if you have failover configured you would likely select the “group” you created when you setup the failover.

If you need more help let me know. If you have been tinkering with a bunch of stuff you may want to start over, just backup your current configuration and reset everything to defaults. If you can’t figure it out you can reset to defaults and restore your configuration.

Give me a bit and I will be able to help you. I have a multi vlan setup with vLAN tagged switches and wifi ap’s.

Both OpnSense and PfSense have FreeBSD as a base. That means you really want to avoid realtek NIC’s.

I’m running OpnSense on a Supermicro X10SLL-F, with a Xeon E3-1226 v3, 16GB of RAM, and an Intel X520 10G NIC with the OS installed on a mirrored pair of 240GB Intel D3-S4510’s. I underprovioned the drives by about 50GB to lower write amplification and allow the drive controller a scratch pad space for garbage collection even if the drive fills up with logs and snapshots. This is simply done with:

zfs set quota=190G zroot

I have the following services (beyond the routing and firewall) running:

ACME Client - let’s encrypt certificates

Caddy - to expose my home services for access via reverse proxy

Chrony - network time server

CrowdSec - running all free rules

Intrusion Detection (also known as Suricata) - just under 193K rules

Kea DHCP for IPv4 providing IP addresses for 5 vLan’s

Monit

Ntopng with Redis as the database

NUT

Router Advertisements for IPv6 for 3 vLAN’s

SMART - for dive testing but it basically consumes no resources

UnboundDNS with abuse.ch, Steven Blacklist, and Hagezi multi Pro++ Blocklists.

Firewall has 79 rules, NAT has 8 Destination entries, IPv6 is provided by Route64 over Wireguard, remote management access is done via Wireguard tunnel. vLAN’s are Management, Trusted, Smarthome, Cameras, and a Guest network which is WiFi only. Smarthome and Cameras vLAN’s do not have internet access except for my HomeAssistant server.

I’m using a Quanta LB6M for my fiber backbone which is 10G with a LAG connection to a Dell PowerConnect 5548 and 5548P gigabit network stack.

I’m utilizing approximately 62% of RAM (which begins to have issues once you get over 80% used) and a load average that generally sits at .30 unless the GUI is open. Swap is 0

I have a 1Gbps fiber internet connection through a Callix 803g and a cellular backup via a Netgear LM1200 that is controlled by OpnSense when a 20% or greater packetloss is detected.

If you have a fiber connection or notice packetloss/slowdowns after 9 minutes you may need to set:

net.link.ether.inet.max_age = 540

This is because by default FreeBSD sets a 20 minute ARP maximum age while many ONT’s set a 9 minute maximum age.

If you plan to use Intrusion Detection the required RAM will balloon quickly and the UnboundDNS Blocklists live in RAM as well. You can easily get a similar setup to mine on Ebay for 200 or less, old server boards or workstation boards with IPMI are great to have if something acts up and the router is living in a closet without a monitor. The old server boards IMHO are also super stable and they are designed to run for years without being shut down.

I picked up enough used SAS drives to build an 11 drive RaidZ3 pool. 100 USD each shipped. DIF formatted so had to low level format to native 4k sectors, then run a full run of badblocks, and finally a long smart test to verify no errors. Had a couple bad drives that the seller replaced no questions asked when I provided the smart logs.

SAS controller and backplane opens up a lot of drives that SATA controllers can’t touch.

My old pool drives will be repurposed for a FrigateNVR storage point and a storage point for some other stuff as well as spares for other pools.