Dependent on Automated Web App Scanners? You may be missing vital clues!

Posted: August 9th, 2012
Filed under: IIS & HTTP

From our friends at Net-Square

A variety of automated web application scanning tools are available today, which can perform a vulnerability analysis of web applications really quickly and give you a list of vulnerabilities detected.  This certainly reduces a lot of time for your Infosec team, which is already loaded with security issues. They can now certify and approve deployment of applications to production after getting a clean report. But, how good are these automated tools? Can we really rely on them?

Three security experts, Adam Doupe, Marco Cova, and Giovanni Vigna of the University of California, Santa Barbara, put the best of automated and semi-automated scanners to test and have concluded that “while certain kinds of vulnerabilities are well-established and seem to work reliably, there are whole classes of vulnerabilities that are not well-understood and cannot be detected by the state-of-the-art scanners”.

Their study, aptly titled “Why Johnny can’t pentest”, demonstrates how very well regarded automated scanners miss as many as 60% of the findings. And this is really not surprising. Even the best of scanners are limited in application “coverage”. What you don’t see, you don’t report. Net-Square has a history in building some of the best automated scanners in the past, and we are well aware of problems with automated scanners. The second fundamental problem is that automated scanners can never perform vulnerability chaining. Critical findings discovered by Net-Square analysts sometimes take more than two or three bugs to exploit.

The other problem is an operational one; customers and firms undertaking application testing are so focused on reducing the number of false positives that they weaken the actual premise of testing. When I talked to a client about Net-Square’s automated scanner – NS-Webscan, the first question was about its rate of false positives. But wait, aren’t we supposed to care about what it finds in the first place? Many firms performing application testing use automated scanners to generate the reports and have their analysts to filter out the false positives. How did we find this? While interviewing candidates from these firms.

Automated scanners aren’t entirely useless. They are best utilized in reducing the load for manual testing. Obvious vulnerabilities get detected right away. Not all customers can engage a team of sharp penetration testers throughout the year. Automated scanners, like NS-Webscan, provide an intermediate solution for Infosec Management to certify rollout of minor releases and changes between two cycles of manual penetration testing.  However, being entirely dependent upon automated testing is like an ostrich sticking its head into the sand.

We use NS-Webscan to initially check to see how vulnerable the application is. If we find many issues in the first round of testing then our analysts know that they have a long road ahead on that particular application test, with many interesting vulnerabilities to be discovered. The bottom line is, no amount of automation can match the skill and cunning of a hacker’s brain.

No Comments »