Inexpensive and power efficient refurb PCs for firewalls

This is a follow-up on my earlier post about the Lenovo M58p (Intel Core2 Duo E8400 @ 3GHz) that I'm using for my home firewall.  It clocks in at 38-40W idle and 50-60W under load (which is rare).

If you go to NewEgg's site and go into the Desktop Computers category, you can find all sorts of refurbished boxes for under $150.

So what sort of CPUs are there and how do they stack up in terms of TDP and idle TDP?

Well, according to various charts on the net:

Intel Core 2 Duo E6600 and E6750 should both draw less power then the E8400 that I'm using.  At a guess, that would put total power draw closer to 30W idle rather then 40W.  The Core2Duo line also performs better then the old PentiumD and Pentium4 chips when under load.

So Core2Duo is better then a Pentium D or Pentium 4 chip, and the slower E6600/E6750 C2D are better then the 3GHz E8400.

The i3/i5 series (Sandy Bridge or later) should offer lower power-draw then the older Core2Duo chips.  Power draw dropped again slightly with Ivy Bridge and Haswell.

That being said, the least expensive i3 refurb is $220-$250, while a Core2Duo unit can be found for $80 or less.  So I think until the i3 units start being retired in another 2 years, the C2D units are going to be the best choice.

Using aliases pfSense to create rules for protocols with multiple port ranges

File this one under "things I wish I had known sooner".  When setting up pfSense firewall rules on an interface, you'll run into protocols which have multiple ports that are not in a contiguous range.  One example of this is the common web server (HTTP) ports of 80, 443 and 8080-8081.

This leaves you with two options.

1. Setup multiple rules.  This is the best option because you only specify the exact ports that you want, with no extras thrown in.  The downside is that for some protocols, you will end up with multiple rules that have to be maintained.
2. Specify a rule with a broad port range.  Which is sort of okay if you are only allowing a handful of extra ports, but it is not ideal.

Enter the concept of aliases (under the Firewall -> Aliases menu) in the pfSense web UI.  Here you can create an alias which lists out all of the ports associated with a particular protocol.

After creating the alias, you then create or edit a rule and use that alias in any fields with a red background.  Such as the destination port field.

After clicking the "Save" button, rules that are using port aliases will show up in the rule list looking like:

Needless to say, that can make your life much easier when maintaining large lists of ports as long as all of the ports in question are using the same protocols.

Mail client ports (IMAP/POP3/SMTP) are also good candidates for an alias rule.  One caution is to never allow 25/tcp to egress your network, only your mail server in the DMZ should be allowed to contact other mail servers via port 25.  Every internal client should be forced to either use tcp/465 (SMTP/SSL) or tcp/587 (SMTP Submission) or route their SMTP traffic through your mail server.

pfSense rate limiting, egress filtering, opendns filtering for wifi hotspot

One of the experiments that I'm running with the new network is running an open / unsecured WiFi hotspot for the neighbors.

Some of the protections that I'm using:

• Uses OpenDNS servers with some categories of websites blocked.  I'm using the "OpenDNS Home" service which lets me pick and choose which categories are blocked by default.  In addition, the OpenDNS server will display a "blocked content" page for regular HTTP traffic where users can request an unblock.  Unfortunately, this feature does not work well for HTTPS (SSL) sites, but it still blocks the site.
• Access to other DNS servers is blocked, clients can only access the two OpenDNS server IP addresses.
• All rules on the interface are rate-limited to 3Mbps down and 1Mbps up.  This limits bandwidth abuse and slows down file sharing.
• Heavy egress filtering.  All outbound traffic is blocked by default except for the whitelisted ports/protocols.

Now, this is not 100% foolproof.  But I at least want to limit the possible damage and take at least some steps against abuse.  I'll probably use this setup at a company that I'm consulting for where they want to offer open WiFi in their waiting area.

One thing I would like to do is setup a "Captive Portal" on the interface which forces the user to enter their cell phone number and receive a voucher code via SMS that is good for 3 or 7 days.  I have to figure out how to do that with pfSense and see how it works in practice.

The other thing I plan on doing is setting up a similar SSID/VLAN, but with higher bandwidth limits, more ports and no OpenDNS filtering for authenticated guests.  That would probably be a 20Mbps down / 5Mbps up setup protected by WPA2/PSK.  Think along the lines of "neighbors" or "friends" who you want to allow use of the internet pipe, but do not want to allow onto your interior network.  This would also be a good setup to use in an office environment for BYODs that only need internet access (such as clients).

pfSense on Core2Duo E8400 refurbished PC

A rough power estimate for my little SFF (small form factor) refurbished PC that I'm using for a pfSense firewall:

• Intel Core2 Duo E8400 @ 3GHz
• 4GB RAM
• 120GB SSD
• Dual-port Intel PCIe x4 NIC

At idle, it consumes about 38W when the CPU throttles back to 1.8GHz.  That's pretty good for a PC that is not designed to be a low-power / fanless unit.  Under load, that goes up to about 60-65W.

38W @ $0.15/kWh = $50/yr

65W @ $0.15/kWh = $85/yr

So even if I got something that stayed below 15W, I'd only save $50-$60/yr.  A lot of the low power units are $300-$500, which would be a very long time before they'd pay off.

Note: You do need to enable PowerD under System -> Advanced -> Miscellaneous -> Power Savings in order to get the CPU to throttle down when idle.  I recommend "Hiadaptive" for an office firewall, but you might want to experiment.

The biggest CPU hog on my current pfSense setup is "ntopng".  While doing bi-directional gigabit testing, that ate up 10-12% of my CPU power.

pfSense Firewall CPU load estimate

According to the pfSense dashboard, I have:

Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
2 CPUs: 1 package(s) x 2 core(s)

When running a quick speed test, "top" shows about 5% system load at 60Mbps.  That gives a rough upper-end of around 1200Mbps (1.2Gbps) for switching speed.  At a guess, that might be closer to only 1Gbps performance under heavy traffic.

1Gbps of capacity is plenty for the moment where I have:

- 50/50 Mbps service from Verizon FIOS (seems to peak at 60/60)
- 802.11 b/g/n (11-54Mbps)
- 802.11ac (tops out at around 1Gbps)

But it may not be enough for connecting together multiple gigabit LAN segments.  So I will need to keep all high bandwidth traffic on the same VLAN so that the traffic gets handled by the switches without touching the pfSense firewall.

Update #1 (Jul 10 2015): Suricata cuts the performance of the WAN interface (in terms of CPU load per Mbps) by a factor of 5x-10x.  While I could probably route 1.2-1.5Gbps with this firewall, a 30Mbps load on the WAN, which is monitored by Suricata, resulted in 20% CPU load.  That puts my upper-bound for WAN traffic at only 150Mbps.

Firewall build (part 3) VLAN Security

Since I plan on using VLANs on the WiFi Access Points to separate guest vs friend vs trusted traffic, I need to make sure that I'm doing VLANs in a secure fashion and not leaving any large holes.

The primary recommendations from the listed sources are:

• Don't use VLAN 1 (the default VLAN) for anything
• Any ports that do VLAN trunking should use a dedicated VLAN ID
• Do explicit tagging of the native VLAN on trunking ports
• Set all end-user ports to non-trunking (a.k.a. "access ports"?)
• Disable unused ports and put them in a separate VLAN
• Disable Spanning Tree Protocol (STP) on end-user ports
• Use MD5 authentication for Virtual Trunking Protocol (VTP)
• Physically secure the switch and control access to the management functions

Reference Links:

1. Virtual LAN Security: weaknesses and countermeasures (SANS)
2. VLAN Hacking (InfoSec Institute)
3. VLAN Hopping (Wikipedia)

Firewall build (part 2) hardware needed for VPN duties

One thing to think about when sizing the hardware for a firewall is how much CPU power will be needed for OpenVPN (or IPSec / L2TP).  OpenVPN comes with a built in "speed" command which will benchmark your system and give you an idea of maximum possible bandwidth.

Just run "openssl speed" at the command line and look for the AES-128 and/or Blowfish results.  I prefer to look at the 1024 byte or 8192 byte columns in the output to figure out the upper range.  While Blowfish is good at the smaller block sizes, AES-128 catchs up and surpasses it with the larger block sizes.

Values at or above 100000k should indicate that the firewall has enough performance to drive an OpenVPN connection at close to gigabit speeds.  Or handle multiple OpenVPN connections at the same time, without completely saturating the CPU.

AMD Opteron 2210 HE @ 1.8GHz
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
blowfish cbc     68358.84k    74350.46k    75845.03k    76373.67k    76556.97k
aes-128 cbc      50477.29k    53816.28k    55093.08k   128709.63k   130465.79k

AMD Phenom II X4 810 @ 2.6GHz
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
blowfish cbc     94477.65k   101825.28k   103154.35k   103857.83k   104060.25k
aes-128 cbc      76376.65k    81608.09k    83915.50k   213516.45k   216016.95k

AMD Opteron 4180 @ 2.6GHz
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes

blowfish cbc     93781.75k   101154.41k   102983.68k   103730.52k   103923.71k

aes-128 cbc      76233.64k    81631.07k    83197.52k   213366.89k   215309.87

103720k = 98 MiB/s or ~980Mbps, which is pretty close to gigabit speeds

General guidelines/notes:

• I'm a firm believer in multi-core for servers and desktops.  So look for hardware that is at least dual-core when shopping.  An inexpensive quad-core would be even better and give a bit of headroom for monitoring tasks.
• For the AMD CPUs (Opteron / Athon64 / Phenom) made in 2007-2011, you'll want at least a 2.2GHz core.  For Intel Core2 CPUs or 1st/2nd generation i3/i5, try to get at least a 2.0GHz core.
• Intel Atom CPUs are underpowered, the 1.8GHz dual-core units are reported to top out at around 500Mbps for general routing and definitely can't handle gigabit speed OpenVPN.  But they are low power, so maybe that outweighs the performance issue.  A rule of thumb is that the Atom CPUs are about 1/3 to 1/2 as powerful as i3/i5 for the same clock speed.

Resource links:

• Firewall Hardware Sizing Guide [www.firewallhardware.it]
• Choose Your Hardware [www.pfsense.org]

Firewall build (part 1)

Part of moving to a new place is reevaluating your network.  On my current network, I have a fairly basic setup:

• One WiFi Access Point (WAP) running 802.11 b/g
• A Linux server acting as firewall / file share / backup storage
• A few laptops
• A few tablets/phones
• A few other PCs

When I set this all up a few years ago I kept it very simple.  The Linux server is the gateway device with routing / filtering / NAT and other features.  The WAP is part of the internal network running in WPS/PSK mode with a very long and randomly generated password.

After I move, I want to accomplish a few things:

• Use a refurb or low-power PC to run just the firewall / VPN
• Put the WiFi access point on a separate NIC
• Possibly run a DMZ
• Provide limited guest WiFi
• Evaluate pfSense instead of Linux+Shorewall

To do all that, I need a minimum of four network ports for perfect security or something with two ports if I use VLANs (not as safe, more difficult to configure and get right).

I've done some looking around and while a low-power 25-35W compact PC for the firewall would be nice, it would cost me around $600.  Maybe $400-$500 if I shop around.  There are also the really tiny units that will run monowall (m0n0wall), but those are also $200-$300 for something that will handle the faster WiFi / FIOS / cable modems.  Plus it can be difficult to find something with four network ports.

Firewalls don't need a lot of CPU power, but a dual/quad CPU Intel Atom isn't enough.  An i5/i7 would likely be complete overkill, even for 802.11ac / 802.11n or gigabit traffic.  The older Pentium / Celeron / Core Duo are probably a bit on the slow side.  The AMD Phenoms or Athlon64 chips are probably okay.

So what I've settled on is a refurbished PC that is at least a Core 2 Duo (2 cores) with 4GB of RAM, along with a refurbished NIC.  The pfSense distro only needs a handful of gigabytes to install, so any unit with at least 40GB of space will be plenty.  The base units can be picked up for as little as $50-$125 for the base computer, and add-in NIC cards are $10-$40 depending on what you use.  If the box dies, I get another and move the drive over.  If one of the NICs fry, I can pickup another NIC.  Power requirements will probably be around 80W to 120W.

For the smaller sized PCs, you might only have 1-2 expansion slots which means you'll need a multi-port NIC. The cost of the dual-port NICs is likely to be more then what you pay for the base PC.  I've seen dual-port refurbished NICs for as low as $50, but paying $100-$150 is more likely.  However, good NICs tend to work fine for close to a decade, and it can be moved from PC to PC.

Identifying bandwidth abusers in Linux

# /sbin/ip link

This Linux command will display information about your interfaces. When doing network analysis, the primary information that we're interested in is whether the interface is running in promiscuous mode. An adapter that is running in promiscuous mode can capture any packets that pass by on the wire, not just the ones destined for its MAC address. Here's an example of a ethernet adapter that is in promiscuous mode:

3: eth0: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:16:ff:ff:ff:25 brd ff:ff:ff:ff:ff:ff

If you're in a situation where there are multiple hosts on the WAN side and you want to monitor traffic for them, you'll need to use an interface in promiscuous mode. You'll also need to be connected to the same hub as those units, or connected to the same switch where your port is configured in monitoring mode.

# /sbin/ifconfig

Use this command to find out which interface is your WAN link and which interface is your LAN link (in the case of multi-homed systems).

/usr/bin/nload -t 10000 -u H -U H -i 1750 -o 1750 eth0

The nload utility is a console application that will graph the inbound and outbound activity on your network interface. You'll have to download and install this software yourself as it is not included in most distributions. If you have "rpmforge" configured on your CentOS5 installation, this is as simple as "yum install nload". Some key arguments are shown above. "-t 10000" sets the update time to every 10 seconds, "-i" and "-o" set the graph maximum height in kilobits per second (1750 works well for a 1.5Mbit T1).

/usr/sbin/nettop -d 5 -i eth0

This is another console utility that you can add to a Linux firewall. Nettop displays a tree-like listing of all activity on a particular interface, with the packets grouped by protocol and then port/service. This gives you a quick idea of what services are abusing your bandwidth.