PCI Meetup #8: When is a PCI Penetration Test Good Enough?

In the latest PCI Meetup, the sole agenda was again the PCI penetration test, and pentest quality/standards in general. In my last post, I pointed out a few existing standards and some that are currently in development. The general consensus in the meetup is that existing standards are insufficient, and that any hope of creating a baseline for pentest quality rests on future efforts to draft an industry standard.

For a moment, let us hypothesize that a robust pentest standard has been created, and the PCI council is willing to incorporate it into the PCI DSS. What else do we need? With any standard, we will still need parameters specific to PCI for it to be useful. With the wrong parameters, the very best standard won't achieve our goal of determining whether or not cardholder data is reasonably safe, and we can't assume the pentester will assume the correct parameters. In addition to requiring a penetration testing standard, the PCI DSS should also detail how the test should be executed, in order to maximize effectiveness and ensure the test represents real-world scenarios as closely as possible.

• Scope: The PCI scope defines an island, or islands within the enterprise where cardholder data is (allegedly) protected by the PCI DSS requirements, and is segregated from the rest of the network. The penetration test should not be limited to the PCI scope. In fact, one of the goals of the penetration test should be to test the effectiveness of cardholder DMZs from the outside, and determine whether weaknesses outside the cardholder network can be used to penetrate it.
• Goals: Obtaining cardholder data should be the primary goal. Testing the effectiveness of segregation and attempting to penetrate the cardholder network defenses should also be a primary goal.
• Level/Strength of the Test: It is unreasonable to expect the basic, annual PCI penetration test to include custom, unreleased 0-day exploits. It just isn't feasible, given the cost and the number of pentesting companies/individuals capable of providing this level of testing. The penetration test should represent the skill of a fairly technical attacker with commonly available tools (e.g. Metasploit). Ideally, the penetration testing standard would use a tiered approach, similar to the OWASP Web App Standard mentioned in my last post. The PCI council could then choose the minimum tier, or baseline for merchants to be tested against.
• Attack vectors: Currently, PCI requires the penetration test to include internal, external and webapp tests. This is not sufficient. In the future, social engineering must be a requirement, and for good reason as this example shows: An administrator recently gave out the root password for rootkit.com, allegedly to a teenager. If a teenager could similarly penetrate your cardholder environment with a spoofed email or phone call, the value of the $4.5 million you spent on PCI compliance goes to $0. Instantly.

I look forward to your responses, opinions and the next meetup!