Measuring The Effectiveness Of Application Penetration Tests

Quality is an interesting field for many people. The more that gets published, the more it appears that companies that focus on meeting quality standards that are excellent but not necessarily demanded by their customer have no effect on the value of the company as measured by the stock price. Of course, the same dynamic appears to hold when it comes to the penetration testing of applications.

In fact, as recent research has shown, it is likely that the frequency of testing done by firms that have software applications is driven by customer expectations and then compliance.

The effectiveness of pen testing ideals

So while a majority of firms institute scheduled penetration testing to assuage customer concerns or complaints that they know will happen over the course of the year, in some countries, such as the US, compliance to laws that create financial penalties for companies that have software breaches compel firms to test irrespective of customer perceptions.

As it has become apparent to many that there will always be more security flaws in software that supports or underpins the software applications that you are testing than is published, your firm’s vulnerability and indeed your own job security can be measured by the scope of the testing that you do.

Here are some penetration testing considerations that should factor into your company’s test efforts:

ROI:

It is said that ROI is king of analytical rewards because it can show the intrinsic value of testing when it is ordinarily thought of as a cost centre. Whatever you think of it, do utilize it to justify adding scope and frequency to your test efforts.

The way that ROI works is fairly straightforward. You simply measure all the work time and resources that are required to accomplish something using your old process. You then compare it to what it takes to get things done using your new process.

For application penetration testing, ROI can help unmask efficiencies and preserve those processes or choices if they are deemed universally applicable.

Scope creep:

While ROI can tell you how much of the scope that you added is worth it, it does not take the place of making good decisions about the scope of your testing to begin with. Validating your test plan before you start to measure ROI is one way that you can ensure that you will have only what you need.

Typically speaking, validation is done by committee in larger corporations. The idea is that the more experts you have creating and reviewing the plan, the better your results will be.

That is not to say that the process is perfect everywhere- one large company set up a large testing effort that was validated by committee and deemed successful. The next time there was a need to test, there was no committee. The developer simply looked at what was necessary and then used a subset of the old test cases plus a few new tests. When seeking to validate the cases with a veteran of the old test effort, the developer realized that the veteran had already reviewed and approved the necessary test cases during the last test effort. Unfortunately, this time, the veteran forgot that they had signed off after weeks of review work. Their ‘fresh’ approach to approving test cases caused them to reject most of the cases they had previously okayed with an entire committee. The net result was that they were shown their own signatures and agreed to allow the test cases.

Reverse penetration

Reverse penetration testing involves creating conditions that allow your website or software to be attacked by third parties. One of the most common approaches is to visit sites that have known hackers lurking and bait them into coming back to your site. If they can cause damage, you will know what you need to work on.

Because you will need to monitor your site closely when you are using this approach, reverse penetration testing is considered similar to honeypotting.

With regards to measurement, you will want to use metrics that relate to both how often your site was attacked and the degree of success that each attack had. You can also analyze the type of attacker because you are the one who initiated the plan- and so know the source of your attacker in most cases.

Reverse penetration testing is popular with some firms because you are literally getting people to do your strategic infrastructure testing to you without compensation.

Recidivism

Although development is rather shy about talking about recidivism, it is a fact of life in some firms because it is not being tracked well enough for management to do something effective about it. Therefore, to be effective, it is a good idea to analyze your internal systems as well as finished software code in order to setup a framework that can point out recurring problems.

While you are looking for areas to include, you can talk to development directors that are routinely charged with lowering recidivism among their workers. They will likely have the insight that you need. One example, a fairly common complaint among developers, is that someone muffed the build by checking in code that overwrote what was working. This can be caused by your configuration management software allowing this type of thing. It can also be a function of bad habits.

Another effective approach to recidivism is to be systemic about analyzing developer activity during a project. The more relevant the data that you can obtain, the better the chance that you will create an effective measurement mechanism that saves you time and money.

Overall, solidifying your metrics gathering during a project and after it is complete can make a difference in how well your penetration testing efforts end up being perceived.

After all, it isn’t easy operating in an environment where development directors know that there are security flaws in the software that supports your applications- giving rise to a group of hackers that can likely undermine your efforts whenever they like. So, to be successful, using ROI and addressing concerns like scope creep and recidivism are all important components.

View current available Pen Testing positions.