Welcome to the Club: Advice to First-Time Pentesters

This is the first post in a series offering advice to InfoSec newcomers. Not being the most colorful crayon in the box, I'll just call it "Welcome to the Club", and will tag all posts in the series accordingly.

Hop in. I'll be right behind you.

Occasionally, folks starting out in InfoSec will ask me for advice. I try to give it without sending them screaming toward a different, less-punishing career, like working in a Siberian diamond mine.

An acquaintance recently contacted me via LinkedIn to ask for advice on his first paid pentest gig, and this is what I told him.

As you progress from pentest to pentest, your skill and ability to find flaws, use tools, etc will increase, so I'm not going to give you any technical advice at this point. On the first gig, it is more important to ensure there will be a second gig than to try to cover every technical avenue possible. It would also be ideal for your first gig to also be the client's first pentest - then as your skills increase, their ability to implement your findings (in theory) and security posture should increase as well.

The best way to have a good first pentest is to focus on good communication with the client. This skill is important for consultants of any kind, but more so in any situation where there is the potential to cause harm in the course of doing the job they are paying you for. Relationship building is also important. Don't think about any gig as just one job. Think of it as the potential to start a relationship where you could potentially establish yourself as their go-to for any security work.

Yeah, could you stop scanning? It isn't going well for us.

Come up with a good plan, share it with the customer, and stick to it. If something changes, e.g. you find issues going deeper than you expected and you need to change the plan, notify them before going down any "rabbit holes". Make them aware that pentesting - even just scanning - is a potentially disruptive activity, but that you'll do your best to minimize the risks to their network. Make sure they know how to contact you, and that you can stop scanning/pentest activities relatively quickly if there are any issues.

Manage the client's expectations well, and they should be happy. Happy clients spread your services via word-of-mouth and rehire you. Positive word-of-mouth and reoccurring gigs build a solid business. Never stop learning and trying new things on pentests, and the technical side will improve as you gather experience.

There is also a ton of advice posted by the "Pentest Lessons" Twitter account.

Defining Trust

The other day I joined a Twitter discussion between Rafal Los, Wim Remes and several others over "trust". It struck us that we needed a clear definition of Trust, and that it would take more than 140 characters.

Rafal quickly put together a post, Trust - Making an intelligent, defensible trust valuation, and the debate continued. As I felt myself and Rafal were on the same page, and that some of the commenters weren't quite getting it, I was inspired to contribute a post of my own. I'm a believer in gaining understanding through examples, so I've put together a few scenarios in this post to try to drive the point home. I'd love to hear what you think. Comment here, on Rafal's post, or hit us up on Twitter.

The Question

Is trust binary? Is it a yes/no decision? All or nothing? Are there levels of trust? Go get a burbon, beer or chamomile, and we'll explore this question a bit. I'd urge you to think about this before I muddy the waters. We're not just talking about Trust as it relates to users, information security or IT vendors. There is no reason the answer to this question can't apply to social relationships and other situations.

Trust Fall, by SkinnyAndy

How do we define Trust?

There is an opportunity for trust to come into play any time we lack control over a product, a person's actions, an environment, or situation. I believe trust to be heuristic, requiring many rules that result in various levels. We see evidence of these levels in the simplest of examples: you may trust code you wrote more than that of your vendor's software; you probably trust your own network more than a partner's. I think some good examples and/or scenarios are necessary effectively define what it means to have different levels of trust. 

What should these "trust levels" be? I believe they can be formal or informal, but ultimately, they are the result of rules you use to determine "how much" you choose to trust someone or something. The ones I've come up with are completely arbitrary, and off the top of my head. One could define only two levels, or go up to ten or more. I think four is sufficient for the scenarios I present here. Yes, I realize there are actually five levels listed in the scale below. Note the zero level is not a level of trust, but the absence of it.

Sawaba's Amazing Non-Binary Trust Scale

4 - Full Trust

3 - High Trust

2 - Moderate Trust

1 - Low (initial trust; trust out of necessity or desperation)

0 - Distrust, i.e. no trust

We also need to understand how levels of trust are affected. This list is not all-inclusive, and is geared toward measuring IT products and services, to support the scenarios and examples I'll use later.

Enhancers	Detractors
Meets promises and expectations	Caught lying
Time without incident or detractors	Missed deadlines or promises
Consistancy	Mishandled or ignored vulnerabilities
Stability	Slow response to addressing issues
Quick to address issues	Inaccurate quotes
Ability to test and/or validate product	Breaches or other security incidents
Transparency	Surprise costs

Scenario 1

Purchasing a software product from a vendor. Let us assume this is a licensed, closed source software product that will install and run on servers/workstations on the local network. Though the customer in this example does not have access to the source code, they can test behavior, performance, capture network traffic, examine logs/output, etcetera.

Trust Level 0 - Haven't dealt with vendor yet. Unaware of reputation.

Trust Level 1 - Initial conversations and demo went well. "Gut check" says things are good so far.

Trust Level 2 - Checked vendor's reputation and tested product. Due diligence processes/procedures have been carried out and yielded positive results. Most people/companies are ready to do business at this "moderate" level of trust, though they may refrain from initially signing long-term contracts. Many consider this a "trial period".

Trust Level 3 - After a year or more, the vendor has "earned" a higher level of trust by consistently meeting expectations over a significant period of time. Most vendor/product relationships need not go past this level, at least by my arbitrary scale. I prefer to reserve the highest level of trust for more extreme situations where human safety and life and death are concerns. Recall, in this scenario, we don't have full control. We can't see source code, so there is always a chance a disgruntled programmer could insert a back door, for example. Perhaps over a very long period of time (10 years or more?) the level of trust could rise even higher.

Scenario 2

Using a piece of open source software.

With the services of an experienced, knowledgeable programmer trained to spot serious security vulnerabilities, stability issues, and performance concerns, a high level of trust can easily be achieved. Spend enough time reviewing and testing (especially when patches, or upgrades are released!), and it is reasonable to consider that full trust in the product could be attained.

I believe you can make the argument that, with 100% control and ability to verify/validate, we have zero need for trust in this case.

Scenario 3

A cloud service, say a Saas sales product, for example.

You can build trust based on

• interactions with the company
• reputation
• a limited ability to test
• time without incident

However, in this scenario, it is reasonable to believe that the level of trust may not pass the moderate level, due to the lack of transparency and control inherent in the model. Consider:

• We can't see or review the source code
• We can't see or review most of the operating environment
• We may not know if incidents occur
• We don't know for sure who has access to our data
• They may say they encrypt our data, but we have no way of validating whether they do it correctly
• Even if they are audited, and compliant with regulations designed to give assurance, we cannot put full trust in the auditors, especially with a history of varying quality and efficacy in audit practices and the auditors themselves
• We have to take the vendor's word on the majority of items that present a risk to our data 

As a result, we might take measures to compensate for the lack of trust. To use an example, if we decide to use Dropbox, perhaps we independently encrypt all files before allowing Dropbox to sync them to compensate for a lack of trust. This is a real-world example that resulted after reports came out that many Dropbox employees had access to customer files. This was not previously clearly stated to customers, and resulted in a drop in the level of trust. These reports became a detractor.

Conclusion

There is an opportunity to trust an individual, company or product when either parties lacks control to some extent. When levels of control vary, so do levels of trust. It is therefore, not an "all or nothing" model, though both extremes (0% control and 100% control) can be experienced, and can reasonably occur.

Global Payments Breach

Update:

Welcome InfoSec Daily Podcast listeners! I'm going to address a few items related to this story that were discussed on last night's show.

• To the best of my knowledge, participation in VISA's Service Provider Registry is required for all service providers potentially storing VISA cardholder data. Based on my experience, this is primarily a way to track service providers and a marketing tool. Even though Global has been booted off the list, they can still continue to do business, process VISA cards, and sign up new merchants. If anyone has more direct experience or corrections, please comment below.
• PCI is applied different to service providers like processors. However, it is in the opposite direction from what you were thinking. Service providers actually have more requirements to comply with than a merchant would. They do the full PCI DSS plus a few additional requirements that apply only to service providers. They also have to perform level 1 compliance (full Report on Compliance annually, third party annual audit required) with much fewer annual transactions than a merchant would. I think where this misunderstanding came from is the fact that, traditionally, issuers haven't needed to be PCI compliant. That's changed in recent years.
• YES, the requirement not to store track data applies equally to processors as it does to merchants. Issuers (financial institutions that actually brand and send out credit cards) are the only ones with a good chance of getting an exception for storing track data, as they are the original source for producing/creating that data.

Here's the original post:

It isn’t so much the size of this breach that is significant, but the fact that one of the largest global payment processors got popped. Visa has allowed them to continue processing credit cards, but dropped them off their service provider registry (which is a BIG deal). The breach only affects North American merchants and cardholders. To give you an idea of how bad a breach at a large credit card processor can be, if a month’s worth of the transactions they handle were exposed, it is entirely possible that over 90% of all cardholders in the US would need new credit/debit cards.

This doesn’t happen often. I only know of two other cases where a processor was hit by a breach. CardSystems Services, as a business, was literally destroyed by their breach. VISA and AMEX revoked processing rights, forcing CardSystems to have to shut down operations and sell off assets almost overnight. Heartland Payment Systems is the most recent case, and the second largest breach ever at 130 million. They were also stripped from the registry, but managed to recover, regain PCI compliance, and get back onto the registry within a year.

Global Payments had a public conference call at 8AM this morning that I didn’t have time to listen to, but has resulted in an explosion of news stories on the breach.

The worst thing I've been able to determine from the details so far, is that it seems Global Payments was storing Track Data. The PCI DSS explicitly forbids storing track data (requirement 3.2.1), and PCI considers the storage of sensitive data to be one of the most serious PCI violations. CardSystems was effectively shut down for a lesser violation, though their breach was much larger.

It will be interesting to see if any of the details of the breach are released. These details are essential for the rest of the industry to learn from Global's mistakes. I'd like to see:

• The attack vectors used, and the level of sophistication necessary to breach Global.
• How long the attackers had access to systems
• If track data really was stored, and what Global's excuse for such a violation is
• Why the breach was limited to only 1.5 million accounts in North America. A large processor like Global might process 1.5 million transactions in just a few days. Why weren't more accounts stolen? Why only North America? Perhaps some effective segmentation was in place? That would be good news the PCI Council would be happy to point out.
• And of course, we'll hopefully eventually find out who the perps were, and their level of hacking expertise

Time will tell.

PCI Meetup #8: When is a PCI Penetration Test Good Enough?

In the latest PCI Meetup, the sole agenda was again the PCI penetration test, and pentest quality/standards in general. In my last post, I pointed out a few existing standards and some that are currently in development. The general consensus in the meetup is that existing standards are insufficient, and that any hope of creating a baseline for pentest quality rests on future efforts to draft an industry standard.

For a moment, let us hypothesize that a robust pentest standard has been created, and the PCI council is willing to incorporate it into the PCI DSS. What else do we need? With any standard, we will still need parameters specific to PCI for it to be useful. With the wrong parameters, the very best standard won't achieve our goal of determining whether or not cardholder data is reasonably safe, and we can't assume the pentester will assume the correct parameters. In addition to requiring a penetration testing standard, the PCI DSS should also detail how the test should be executed, in order to maximize effectiveness and ensure the test represents real-world scenarios as closely as possible.

• Scope: The PCI scope defines an island, or islands within the enterprise where cardholder data is (allegedly) protected by the PCI DSS requirements, and is segregated from the rest of the network. The penetration test should not be limited to the PCI scope. In fact, one of the goals of the penetration test should be to test the effectiveness of cardholder DMZs from the outside, and determine whether weaknesses outside the cardholder network can be used to penetrate it.
• Goals: Obtaining cardholder data should be the primary goal. Testing the effectiveness of segregation and attempting to penetrate the cardholder network defenses should also be a primary goal.
• Level/Strength of the Test: It is unreasonable to expect the basic, annual PCI penetration test to include custom, unreleased 0-day exploits. It just isn't feasible, given the cost and the number of pentesting companies/individuals capable of providing this level of testing. The penetration test should represent the skill of a fairly technical attacker with commonly available tools (e.g. Metasploit). Ideally, the penetration testing standard would use a tiered approach, similar to the OWASP Web App Standard mentioned in my last post. The PCI council could then choose the minimum tier, or baseline for merchants to be tested against.
• Attack vectors: Currently, PCI requires the penetration test to include internal, external and webapp tests. This is not sufficient. In the future, social engineering must be a requirement, and for good reason as this example shows: An administrator recently gave out the root password for rootkit.com, allegedly to a teenager. If a teenager could similarly penetrate your cardholder environment with a spoofed email or phone call, the value of the $4.5 million you spent on PCI compliance goes to $0. Instantly.

I look forward to your responses, opinions and the next meetup!

PCI Meetup #7: Penetration Test Quality

We had an interesting group for the last PCI Meetup . The majority of individuals participating in the conversation were professional penetration testers. Netragard's PCI Compliance challenge sparked some good discussion on Twitter, and ended up driving the agenda for the seventh PCI Meetup in the same direction. We spent the majority of the Meetup discussing the need for a penetration test standard, especially in the PCI DSS.

Illustrating the Need

Netragard uses an analogy, pointing out that the quality of some penetration tests is akin to testing "the effectiveness of a bullet-proof vest with a squirt gun". I'm sure they're quite skilled, as no security services company would issue such a challenge unless they are:

1. Currently occupying a place of honor here, or

2. Very good at what they do.

That got me thinking though - if company A's pentest is equivalent to a squirt gun, how do I know how Netragard, or any other pentesting company for that matter, stack up against their competition? To continue the analogy, are company B's efforts equivalent to a pressure washer? A 9mm handgun? A rocket launcher? Let us examine the problem from a different point-of-view with two hypothetical proposals.

	Company A	Company B
Work Description:	Int/Ext Pentest	Int/Ext Pentest
Hours:	40	40
Price:	$8000	$8000

In this example, it appears that both companies are offering to perform the same work for the same price, with the same level of effort. Company A, however, is handing off a glorified vulnerability scan as a penetration test, whereas Company B is fuzzing and coding custom exploits as they perform what most in the industry would consider to be an A-rate penetration test. How can a customer tell the difference? Those of us in the business might suggest requesting a list of pentester credentials from each company, or a high-level synopsis of their internal pentesting methodology. Both of these are great, but they get us no closer to being able to measure Company A or Company B's ability to perform a quality penetration test.

This line of thinking brought me to two conclusions. First, we need some sort of standard, or set of standards to measure penetration test quality against, and second, that PCI should require a minimum standard for the required annual penetration test. Currently, what passes for a penetration test in PCI is entirely up to the QSA. The QSA could let Company A's pentest pass as sufficient, or they could go to the other extreme and require something that far exceeds the intent of the requirement. Without a standard, anything could conceivably be passed off as a penetration test - thus Netragard's desire to challenge the status quo.

Consider also that not all PCI requirements are equal. The PCI requirement for an annual penetration test is just one of over 200 requirements spread across 12 categories, sure, but is not on the same level of importance as say, ensuring network diagrams are up-to-date. It is, in fact, the only PCI requirement that can actually test whether or not the other requirements are likely to succeed in preventing a potential breach! The standard of quality this single requirement is held to stands between determining the effectiveness of compliance work performed, and having no clue.

The PCI DSS was likely designed so that the annual penetration test would act as a built-in test for effectiveness. If some sort of standard for quality can be assured, then PCI in general could become significantly more effective.

Standards

I realized that I am far from the first person to recognize the need for penetration testing standards. After a bit of research, I realized there was more out there than I realized.

• ISECOM's OSSTMM manual is a well-known set of security testing guidelines.
• OWASP maintains a guide for testing web-based applications and a multi-level security standard.
• There appears to be an ANSI standard now for penetration testing specifically within the financial services industry (I have not yet forked out $100 to check it out though - let me know if you have, and what your impressions are)
• The NBISE is gathering information to put together a penetration testing standard
• ...and just last night, I listened to Chris Nickerson discussing this very problem on the InfoSec Daily Podcast, and it seems he is collaborating an industry solution as well (bookmark and check back later)

I think all of these have potential to help weed out the "fakers" in the industry, but should one of these be included as part of PCI's annual penetration test requirement, or does the PCI data security standard need its own custom penetration test standard?

Conclusion

I've gone as far in the Meetups as to suggest that a good quality pentest could replace the entire PCI DSS. If the pentester gets their hands on cardholder data, you fail. If they don't, you're compliant. Is that not the bottom line? The pentest becomes, at least, a partial measure of the likelihood that a breach may occur at a given company. This has many benefits, like protecting businesses who are adequately protecting cardholder data from wasting millions unnecessarily. The major problem is that everything rests on the quality of that single annual penetration test.

If there is enough interest, we may discuss this again in the eighth PCI Meetup, which will take place this Thursday, February 10th, 2011 at 8PM EST. If you want to join the discussion, drop me a note here or on Twitter with your SkypeID, and I'll make sure to include you in the discussion!