* This benchmark will be re-done to include more details (gateway's memory & CPU utilization)
* There have been a typo which mistakenly puts the filtering device where it won't be. Will be corrected later
We will NOT deploy a filtering appliance afterall. We'll be using Kaspersky Antivirus to enforce Web Filtering policies (due February). We'll deploy a proxy only, however I'll continue with this benchmark.
Untangle is a gateway software package that installs to x86 hardware. It functions like any gateway appliance from any company like Cisco, BlueCoat, ...etc. Only this one is built on top of Open Source projects running on Linux, giving it the power of guaranteed stability and the flexibility to mesh everything into a single platform that does wonders.
I have deployed this on a subsidiary company that has 120 employees. The machine has been running for about 7 months so far without any problems whatsoever. The total cost was 110 KD ($330 roughly) for the hardware: 1.8GHz dual core CPU, 2GB RAM, 80GB HDD, built-in 10/100Mbps NIC + video card + audio, 2x 1Gbit Linksys NICs.
Untangle offers its basic package for free. This contains the following components:
- Web-Filter
- Spam Blocker
- Spyware Blocker
- Phish Blocker
- Virus Blocker
- Layer-7 Filtering / Protocol Control
- Intrusion Prevention
- Firewall
- Attack Blocker
- VPN
- Reporting
- Routing and Quality of Service
The main reason why Untangle was chosen because it has Layer-7 filtering. This allows you to block certain programs from accessing the network, like Yahoo Messenger, MSN Messenger, ICQ, IRC, Jabber, Peer-to-Peer software, online games, ...etc.
Only the components in bold above were activated, because the rest were useless to our company since there existed other appliances that are doing their job.
Prices apart, since Untangle allows us to demo the commercial components for 14 days for free, we have done the integration with the domain controller and setup test policies for various domain users and everything worked perfectly in less than 10 minutes!
Now it's time to benchmark it and see if it can handle the stress.
Benchmark Details
Host Machine
AMD Dual-core 2.4GHz, 4GB RAM, 2x Gbit NICs, 80GB HDD.Network Setup
EXT/WAN: 1x Gbit NIC is the external link which is connected to our LAN at work. The Internet link is 1Mbps.
INT/LAN: The 2nd NIC is a bridge to the internal network. I'm using a cross cable and hooking it directly to my laptop. (I've also tested it using a D-Link 5-port Gbit switch and got same results).
Caching: The Untangle box was NOT the DNS server (although it caches DNS requests).
As of this writing, Untangle does not offer a web-caching solution, though it's mentioned in their forums that one will emerge with following updates. Our tests didn't involve a web-caching scheme.
Deployment Options
Any appliance that we'll eventually go with will be either sitting between the firewall and the Internet router (bridged setup), or connected to the external switch like the firewalls are.1) Bridged setup: Requires 2 Ethernet ports/interfaces; an internal and an external interfaces.
No need to change the users' settings nor the firewall's gateway. The external port gets a public IP while the internal one is bridged and holds no IP. Whatever traffic is flowing towards the router (the firewalls' gateway) must flow through the appliance.
2) Switch setup: Requires 1 Ethernet port. It doesn't require the users to change any settings, however, it does require the network admin to change the firewall's default gateway to point at the proxy server. Obviously, the proxy's gateway would be the router like in (1).
Software
Siege is an open-source and free HTTP stress, benchmark and load balance testing tool.Siege was not used directly. The URLs fed to it were harvested using another nifty tool by the same author called sproxy.
Simulation Criteria
From our subsidiary company we got a max of 250 web transactions per minute. This translates to 4 requests per second.Our test a bit more stressful than these numbers: 100 browser requests per 5 seconds (random). This translates to 20 requests per second.
The links that were used for testing are:
http://www.eicar.org/download/eicar_com.zip
http://www.eicar.org/download/eicar_com.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://wwww.cnn.com
http://www.gooogle.com
http://www.kooora.com
http://www.sectorx.org
http://www.fox.com
http://www.joecartoon.com
http://www.killfrog.com
http://www.yahoo.com
http://www.nbk.com
http://www.cakephp.org
http://www.tucows.com
http://www.download.com
http://www.youtube.com
http://www.islamway.com
http://www.jazeeraairways.com
The links are a mix to test the virus blocker when enabled, small file downloads, sites with many images, and sites that should be blocked by the web-filter. I repeated the same virus file to increase the probability of siege using it.
Note: the file eicarcom2.zip is a zip file that contains another zip file. The 2nd zip file contains a virus. This is to test if the Virus Blocker detects multiple packing. (It does).
Siege was run with the following parameters:
for i in `seq 1 5`; do siege -i -b -c 100 -t1m; done-i, --internet: INTERNET, generates user simulation by randomly hitting the URLs read from the urls.txt file. This option is viable only with the urls.txt file.
-b, --benchmark: BENCHMARK, runs the test with NO DELAY for throughput benchmarking. By default each simulated user is invoked with at least a one second delay. This option removes that delay.
-c NUM, --concurrent=NUM: CONCURRENT, allows you to set the concurrent number of simulated users to num.
-t NUMm, --time=NUMm: TIME, allows you to run the test for a selected period of time. The format is "NUMm", where NUM is a time unit and the "m" modifier is either S, M, or H for seconds, minutes and hours.
This means that connections are established immediately without delay and we're simulating 100 browser requests on all URLs over the period of 1 minute, five times.
The test will be taken on 3 scenarios: Direct Connection, Without Antivirus, With Antivirus.
Results
Direct Connection: Connected directly without passing any filtering appliance.
| Resp Time | Trans Rate | Concurrency | Succ. Trans | Failed Trans | Failure Rate |
| 5.53 | 15 | 82 | 948 | 68 | 7.17% |
Without Antivirus: Connected through Untangle appliance without antivirus blocker.
| Resp Time | Trans Rate | Concurrency | Succ. Trans | Failed Trans | Failure Rate |
| 4.4 | 20 | 86 | 1165 | 83 | 7.12% |
With Antivirus: Connected through Untangle appliance using antivirus blocker.
| Resp Time | Trans Rate | Concurrency | Succ. Trans | Failed Trans | Failure Rate |
| 4.49 | 19.37 | 86.55 | 1170 | 99 | 8.46% |
1 comment:
I really like this post and others of yours. You know what I love to see also. Some graphs and pics along the post would make them alot more valuable.
Thanks and good luck :)
Post a Comment