Skip to main content

The Hawthorne Effect of Penetration Testing

August 08, 2014

Back in the 1920s, there was a study done to test room illumination and its effects on the productivity of workers. What the testers expected to see was the level of illumination at which the work was optimal. What came out of the research, however, was something quite interesting. No matter what level of illumination, productivity of the workers increased. After the study, worker output began to decline again. Why was that? Because the subjects knew when the study was taking place, and therefore knew individuals would be watching them. They didn't know the specifics of the study, but they knew their level of work effort was being monitored. Years later this came to be known as the Hawthorne effect.

You may be asking yourself, what does this have to do with present day penetration testing? Let’s discuss the similarities. When you hire someone to do a penetration test, there are probably several people who are told about it. If onsite testing is to be performed, then a certain level of access needs to be granted and the consultants will need a work station. In other words, news will likely get out that a penetration test is being performed, alerting those individuals who are defending your organization (if they were not already notified). Even if you ask people not to tell anyone, they might anyway because they don't want to see their friends in other departments “fail”. This can do a couple of things, either the penetration testers are immediately “the enemy” or every single blip that no one cared about in the past is now the pen testers’ “fault.” I've been on plenty of engagements where we are still in the initial throws of a morning meeting, laptops still in bags and a network admin pokes their head in to say “Hey we noticed something, is it you guys?” This type of behavior may be a result of the Hawthorne effect. Many people in the office know penetration testing is occurring, so they are “on guard” for everything, even things that simply don't exist, because they believe their job is being “tested”. Much like the workers mentioned in the example earlier, they believe they are being monitored. Although this is awesome for the organization as the level of network monitoring increases during the testing, I would be willing to bet that a week after the pen testers leave, that same level is not adhered to.

When we engage a client, we always recommend that they not tell anyone about the upcoming pen test. We often hear, “Well, we have to tell .” My answer is, “No, you don't.” The only people that truly need to understand that a penetration test is occurring are the individuals who asked for it, and the person signing the statement of work (if they differ). Though you may feel the need to provide notice as a professional courtesy, there is no true “need to know” there.

Before you completely write me off, let me explain a bit more. We don't look at a penetration test as something working against a system or set of systems. We treat it as a readiness campaign for your organization. Our methodology is to test in two phases. The first portion, usually a day or two, is done as passively as possible. This is beneficial because most of this phase of testing is not caught unless the organization is extremely mature. What’s even better is that we can use this phase to help tune your defenses to catch the techniques we use, many of which your adversaries are using maliciously. The second portion of the testing is to identify vulnerabilities for all systems in scope. Since this usually employs automated tools, every bell and whistle in your arsenal should be going off. However, this provides no true test of the organization because an attacker will usually follow the first approach, but never the second. If your organization can't catch the first part of the attack, then catching the second means nothing.

A true pen test includes not only the systems involved, but all of your technologies, people and processes. It's about if your configurations are holding up, if your SIEM is alerting, and if your defenses are working symbiotically and as planned so that the individuals monitoring have actionable alerts. You can leverage a penetration test to provide information across a very wide spectrum, and to learn more than just if your systems are vulnerable to the latest exploit. I believe this provides an unbiased view of the readiness of your organization and is a better use of testing time and your budget. By not telling anyone about it, you've just increased the value-add of your penetration test.

Related Blogs

November 13, 2017

PCI Compliance Every Day – Requirement 10

When people think of PCI business as usual (BAU) they do not typically see the requirements in section 10 as having much of anything to do with BAU. H...

See Details

October 19, 2017

PCI Compliance Every Day – Requirement 11

The most widely known requirements in PCI DSS 3.2 section 11 with a timing implication are the quarterly external and internal vulnerability scans (11...

See Details

August 20, 2015

Four Thoughts for SIEM Success

Security information and event management (SIEM) is a unique security tool in its ability to rapidly identify threats to an organization. Automaticall...

See Details

How Can We Help?

Let us know what you need, and we will have an Optiv professional contact you shortly.


Privacy Policy

Related Insights

March 29, 2017

Attack and Penetration Services

Learn how our experts work to expose weakness to validate your security program.

See Details

November 21, 2014

Strategy and Tactics: Penetration Testing in the Security Program

In the war of information security, the eldritch horror of knowing resides in the bowels of the vulnerability scanning report. Before, you might have ...

See Details

April 22, 2016

Co-Managed SIEM

Move beyond alerts to improve risk awareness with co-managed SIEM.

See Details

Stay in the Know

For all the latest cybersecurity and Optiv news, subscribe to our blog and connect with us on Social.

Subscribe

Join our Email List

We take your privacy seriously and promise never to share your email with anyone.

Stay Connected

Find cyber security Events in your area.