PCI DSS and Penetration Testing

At times it can be frustrating when the PCI DSS lacks clarity on certain requirements, though it is generally very prescriptive in nature. It creates problems for both the QSA and the client who can and will disagree with interpretation. This is especially true when it comes to penetration testing. The requirement states:

11.3 Perform penetration testing at least once a year and after any significant infrastructure or application upgrade or modification (such as an operating system upgrade, a sub-network added to the environment, or a web server added to the environment). These penetration tests must include the following - network layer and application layer.

Now the first concern is that other requirements of the PCI DSS, such as 11.2 for vulnerability scanning, clearly distinguish where it should be performed (internal and external networks). Now generally, penetration tests are done externally. But the standard, when not specific, is supposed to have an understood "... as it applies to PCI data" in each requirement. So perhaps the penetration test should be on the PCI network. But from a risk perspective, wouldn't it be best to determine how might someone break in from the internal corporate network to the PCI network? As a QSA, I think any of them have to be acceptable.

Another concern lies in what a penetration test is. Other than the layers noted, it isn't really clear. Even there, I think it would be best to clarify the routing network versus the network operating system vs the operating system itself. All of these are distinct types of assets to attack. And as for the application layer, there isn't the same level of distinguishing between general applications and web applications, which are handled very differently within a pentest. Beyond these immediate symantics, there are many more nuances to what a pentest is that simply isn't defined or even referenced to some other body like OWASP for web application security.

But what bothers me the most about this requirement, is the lack of dictating who can perform a penetration test. PCI has done a great job establishing the ASV program to certify who can do a vulnerability scan. You'd think, at a minimum, that only ASV's could do them given that penetration tests are sort of the PhD to the associates degree of vulnerability scanning. But to the previous point, there are a lot of people who can do scans. Very few are actually good at penetration testing. So the real goal should be to make an even higher standard and program for pentests.

With all that said, I am still a big fan of the PCI DSS. It really does outline a pretty darn good formula for a good, effective security program. And with each rewrite, the DSS does tend to improve and provide clarity. For example, when requirement 6.6 was added for web application review or web application firewall, they did clearly state that an organization that specializes in those types of assessments should be used. Could they at least say that for the penetration tests?