The Security Skeptic

Considerations when comparing firewalls

09/13/2013 08:20:44 AM

Years ago, I cajoled Paul Robertson to allow me to incorporate his responses and mine from a firewall mailing list thread that asked, “how should I compare two firewalls?”

This is a reboot of a 23 June 2004 post. Some of these comparison criteria are less relevant in 2013 than 2004 but even in the era of next generation or web application firewalls, others are relevant still.

“I want to compare two firewalls… what aspects do we need to compare?”

A seemingly simple question, right? Download the spec sheets from the vendor, compare the hardware and security features each offers. Try to get an eval unit from each vendor, test drive the units, stress it a bit, get a feel for the the configuration console…

Taking this approach, it will be difficult to conclude anything other than, “this one handles well, the entertainment system is way cool, and the color matches my eyes”.

If you haven’t established proper selection criteria, based on your security architecture and requirements, you will probably end up with a serviceable firewall, but you may have to trade up to get what you need. Paul Robertson offered the following practical list of things you should consider when comparing firewalls:

  • How well do the boxes implement my proposed security policy?
  • Do they pass testing for implementing my security policy?
  • How do the boxes perform implementing my security policy?
  • What is my upgrade path should my performance requirements change?
  • How well can the devices be administered by multiple levels of people if my security policy defines and requires such?
  • Historically, how well has the vendor done?
  • What does it take to make them fall over? If you can’t make them fall over, you’re not testing hard enough.
  • How intuitive is my security policy when added to the systems?
  • What failover and backup issues must we address (test both.)?
  • Any license issues: how do they handle license failure, and how long does it take to recover?

To these I added:

  • What methods does the firewall provide to assist me in verifying that my security policy is enforced?
  • Are the log entries generated have sufficient detailed to help me lock onto a threat?
  • Does the firewall support all present and projected authentication methods my organization will use?
  • What routing, forwarding, and switching (VLAN) role is the firewall expected to play in my topology?

I also added related considerations if you intend to use an IPsec VPN for remote access:

  • What’s the origin/pedigree/reputation of the VPN client software?
  • Availability of non-Windows clients?
  • Interoperable with open source clients?
  • Client software reliability (track record) when installed on different Win OS, Wintel and non-Wintel hardware?
  • Suitability of client for use with other firewalls (if multi-organizational collaborative/B2B/B2C is something you must satisfy).
  • Client policy administration/enforcement method.
  • Client diagnostic and logging facilities.

I know these last criteria go beyond “just a firewall” and may be off-topic for some, so the last group can be disregarded if secure remote access will be handled by systems other than firewall. 

These combined lists give security administrators quite a bit to mull over. Factor these first, then decide between the fine Corinthian leather or Nauga hide interiors.

If you have any criteria to add, please post a comment. Warning: using the phrase “next generation” in your comment will cause your answer to be piped to dev/null.