 |
 Welcome to PC Flank's Advanced Leaktests Information Page
Throughout this page, more advanced information on each of the leaktests will be given, along with how each test was handled by each of the leading firewalls.
Firewall Performance against Leaktests
as of October 19, 2006
1) Atelier Web Firewall Tester (AWFT) homepage; direct download
| Technique |
Process memory injection, Launcher |
| Description: |
A suite of six tests combined into one product that uses different techniques to test the firewall's outbound strength. |
| Thoroughness: |
Hard |
Firewalls vs. AWFT
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
Total score: 6-4 (1,3,4,5 - passed; 2,6 - failed) |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
Total score: 3-7 (6th - passed, all others - failed) |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
Total score: 10-0 (all - passed) |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
Total score: 10-0 (all - passed) |
|
| |
Windows XP built-in firewall
|
|
Total score: 0-10 (all - failed) |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
Total score: 10-0 (all - passed) |
|
|
|
2.1) BreakOut #1 direct download if your default browser is IE; direct download if your default browser is Firefox
| Technique |
"SendMessage" Windows API |
| Description: |
The test instructs the default web browser to open up a specified URL address using the Windows functionality of inter-process communication through "SendMessage" handle.
|
| Thoroughness: |
Hard |
Firewalls vs. Breakout #1
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
failed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
2.2) BreakOut #2 direct download
| Technique |
Active Desktop modification |
| Description: |
The second Breakout leaktest creates a locally-placed HTML page pointing to a certain URL and sets this page as Active Desktop so when it is turned on the default browser accesses the link contained in the HTML file.
|
| Thoroughness: |
Hard |
Firewalls vs. Breakout #2
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
failed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
failed |
|
|
|
3) Comodo Parent Injection Leak Test (CPIL)
homepage; direct download
| Technique |
Process memory injection |
| Description: |
The test patches the instance of IE browser in memory and later using the modified executable attempts to access the test server location. Only few products succeed in passing the test. |
| Thoroughness: |
Hard |
Firewalls vs. CPIL
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro 6 (v. 6.5 build 737.000) |
|
passed |
|
|
|
4) Copycat direct download
| Technique |
Process memory injection
|
| Description: |
The test injects its code directly into the memory space of a trusted process on a PC, which could be Windows Explorer or the default web browser application, complicating the task for a firewall of detecting inappropriate activity. As distinguished from other process injection techniques, Copycat doesn't spawn parent process and tries to bypass the firewall directly on behalf of the hijacked application. |
| Thoroughness: |
Hard |
Firewalls vs. Copycat
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
5) DNStester homepage; direct download
| Technique |
Manipulation of DNS requests
|
| Description: |
Internet-enabled programs quite often use DNS requests to "map", or to obtain the IP addresses of target hosts with which they need to communicate. These DNS requests may be faked or manipulated with enabling rogue programs to transmit users' private information hidden within the seemingly innocuous request. Using the "DNS Client" service running on Windows XP or Win2K which is designed to speed up the process of finding the required remote host, DNStester sends inappropriate DNS commands to "svchost.exe" service in hopes that the firewall would fail to detect illegitimate activity. |
| Thoroughness: |
Hard |
Firewalls vs. DNStester
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
failed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
6) Firehole homepage; direct download
| Technique |
Launcher, DLL injection
|
| Description: |
The leaktest launches the default web browser and a small .dll file that is injected into the browser's memory area. The test aims to bypass firewalls that do not control which components are attached to the trusted program and what activity these embedded objects are performing. |
| Thoroughness: |
Medium |
Firewalls vs. Firehole
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
7) Ghost homepage; direct download
| Technique |
Launcher, PID manipulation
|
| Description: |
All active tasks on a computer are assigned special internal "tracking" numbers called PID (kernel's Process Identifier) by the Windows operating system. These tasks are identified on the basic of their PIDs. Firewalls, too, rely on this system of identification to ensure that firewall rules are enforced in relation to active applications. The Ghost leaktest works in such a way that its PID is constantly changing; it thus attempts to inundate the firewall with different PID numbers for the started process so that the firewall ceases to associate activity with the initiating application. This happens due to the test's ability to quickly close and reopen itself with a new PID. |
| Thoroughness: |
Medium |
Firewalls vs. Ghost
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
8) Jumper homepage; direct download
| Technique |
Registry tampering, DLL injection, Launcher
|
| Description: |
By tampering Windows registry database, the test appends Jumper's DLL to a set of other objects that are auto-started when Windows Explorer is relaunched (by modifying the "AppInit_DLLs" registry thread). It then terminates the original Windows Explorer instance forcing Windows to restore it, but this time Windows Explorer is opened up with the external DLL implanted in its memory. Subsequently, the rogue DLL modifies the browser's startpage registry entry in such a way that arbitrary data can be sent off by the browser past the firewall's outbound barriers. |
| Thoroughness: |
Hard |
Firewalls vs. Jumper
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
9) Leak Test homepage; direct download
| Technique |
Filename substitution
|
| Description: |
The leaktest tries to impersonate a trusted program on a computer by copying its name and location and later send data on its behalf. The test would defeat firewalls that rely solely on filename when identifying an application without looking further into signature verification. |
| Thoroughness: |
Easy |
Firewalls vs. Leak Test
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
passed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
10) MBTest direct download
| Technique |
Direct network interface access
|
| Description: |
The test creates a flood of erratic packets and sends them off to network adapter, bypassing standard TCI/IP stack monitored by a firewall. The test has a problem running on Windows XP machines with latest updates applied. |
| Thoroughness: |
Hard |
Firewalls vs. MBTest
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
N/A - test failed to start |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
N/A - test failed to start |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
N/A - test failed to start |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
N/A - test failed to start |
|
| |
Windows XP built-in firewall
|
|
N/A - test failed to start |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
N/A - test failed to start |
|
|
|
11) pcAudit homepage; direct download
| Technique |
DLL injection
|
| Description: |
The leaktest injects its dll into the trusted application (i.e. your default web browser) and on its behalf attempts to transmit data off the computer. |
| Thoroughness: |
Medium |
Firewalls vs. pcAudit
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
passed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
12) PCFlank Leaktest homepage; direct download
| Technique |
OLE automation
|
| Description: |
The test controls the browser's activity and dialog windows via the technique of OLE automation of application control. |
| Thoroughness: |
Hard |
Firewalls vs. PCFlank Leaktest
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
failed |
|
|
|
13) Surfer direct download
| Technique |
DDE inter-process protocol
|
| Description: |
The test aims to bypass the firewall's detection using the technique of DDE (Direct Data Exchange) control to manage the activity of a trusted application. |
| Thoroughness: |
Hard |
Firewalls vs. Surfer
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
failed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
14) Thermite direct download
| Technique |
Process memory injection
|
| Description: |
The test injects its code directly into the memory of another process, making a new thread of the parent process and transmitting data past the firewall on its behalf. |
| Thoroughness: |
Medium |
Firewalls vs. Thermite
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
15) TooLeaky homepage; direct download
| Technique |
Launcher
|
| Description: |
The leaktest launches the default web browser with command line parameters in a hidden window, attempting to bypass firewall protection by imitating a trusted program's activity. |
| Thoroughness: |
Easy |
Firewalls vs. TooLeaky
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
passed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
16) WallBreaker homepage; direct download
| Technique |
Launcher
|
| Description: |
The leaktest created by a well-known researcher and firewall analyst Guillaume Kaddouch who supports his own firewall testing website. WallBreaker attempts to obscure the sequence of program launches and obfuscate the originating application in the chain of program launch events, disorienting the firewall with a layered program calls commands so that the firewall loses control of who actually started the trusted program on a computer first. |
| Thoroughness: |
Medium |
Firewalls vs. WallBreaker
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
Total score: 0-4 (all - failed) |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
Total score: 0-4 (all - failed) |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
Total score: 4-0 (all - passed) |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
Total score: 4-0 (all - passed) |
|
| |
Windows XP built-in firewall
|
|
Total score: 0-4 (all - failed) |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
Total score: 4-0 (all - passed) |
|
|
|
17) YALTA homepage; direct download
| Technique |
Rules manipulation, direct network interface access |
| Description: |
The leaktest comprises a suite of two tests:
First and the basic of the tests attempts to manipulate firewall's existing rules allowing common network traffic through typical access ports such as UDP port 21 (ftp traffic) in order to make the firewall believe legitimate data is being transmitted.
Under the advanced test (not available for Windows XP-based systems), the test creates new network driver and through this attempts to transmit data, bypassing the standard TCI/IP stack monitored by the firewall. |
| Thoroughness: |
Medium |
Firewalls vs. YALTA
| |
Firewall |
|
Test Result |
|
| |
Sunbelt Kerio Personal Firewall (v. 4.3.268) |
|
passed |
|
| |
Norton Personal Firewall 2006 (v. 9.1.033) |
|
failed |
|
| |
Outpost Firewall Pro (v. 4.0.964.6926 (582))
|
|
passed |
|
| |
Tiny Desktop Firewall 2005 Pro (v. 6.5.126) |
|
passed |
|
| |
Windows XP built-in firewall
|
|
failed |
|
| |
Zone Alarm Pro (v. 6.5.737.000) |
|
passed |
|
|
|
|
|
