Highlights from FedRAMP®’s new Penetration Test Guidance

The new FedRAMP® Penetration Test Guidance focuses on standardizing the testing methodologies used by C3PAOs with a list of mandatory attack vectors for all authorized systems.

Introducing the new guidance

In an effort to stay on top of the evolving threats being faced by the cloud community, the FedRAMP PMO released Version 3.0 of their Penetration Test Guidance, dated June 30, 2022. 3PAOs and CSPs should begin using the updated pen test guide for pen tests beginning shortly after June 5, 2022, which is the date it was posted. At a high level, this guidance introduces a number of changes when compared to its predecessor. The most notable include:

• Updated attack models originating from the MITRE ATT&CK knowledge base (Section 2.2)
• Detailed phishing campaign guidance (Section 3.1.1)
• Eliminating the use of development environments for penetration testing (Section 3.3.3)
• External testing will require relaxing all protection mechanisms “Poor Separation Measures and Defense In Depth” (Section 3.2.2)
• Corporate to CSP Management testing has been deprecated
• Absent attack vectors will result in high SAR findings (Section 3.0)

The sections below have been compiled to provide some additional insights on these changes and outline some general thoughts on the Penetration Test Guidance.

Threat model updates (Section 2.1)

In the new version of the Penetration Testing Guidance, threat modeling has been described in a more concise manner, rooted in the primary attack surfaces for cloud services. The models presented roll up to three primary headings:

1. Internet-based attacks from an untrusted actor
2. CSP Corporate-based attacks from both trusted and untrusted actors
3. Internal threats from both trusted and untrusted

While the formatting has been streamlined, the core content surrounding threat models existed in the prior version of the guidance as well.

Attack model updates (Section 2.2)

One of the most significant changes to the penetration testing approach is found in the Attack Model section. In the previous version of the guidance, the PMO focused the initial sections on describing the Rules of Engagement content and outlining Threat Models but didn’t spend much time speaking to the penetration test approach. These new attack models are highly prescriptive and may materially change the reporting structure and operational theory behind cloud penetration tests. This should be brought to the attention of the CSP if the CSP has not reviewed the updated Penetration Testing Guidance.

It's been commonly said that penetration testing is equal parts an art and science. Each tester has developed their own testing approach that is founded in industry-recognized methodologies. The most common methodology adheres to the standard five phases of penetration testing: Reconnaissance, Scanning, Gaining Access, Maintaining Access, and Analysis/Reporting. In providing the specific attack models in Section 2.2, FedRAMP is promoting consistency in penetration test approaches for the program. Instead of relying on the expertise and unique testing methods of individual 3PAO firms, the PMO has formed a list of penetration testing activities that should be attempted as part of an assessment.

The FedRAMP PMO has elected to use MITRE’s ATT&CK knowledge base as the source material for this update. While not stated in the guidance itself, it will likely be beneficial for 3PAOs to integrate these models into their Rules of Engagement and Penetration Test Report templates, so they are easily able to demonstrate compliance when packages undergo review by the PMO. Not all attack models will be relevant to every cloud system, but stakeholders should keep in mind that the PMO and JAB have historically required that systems defend any models that are omitted from testing.

Attack Vector 1: External to Corporate (i.e., Phishing)

The updated guidance provides significantly more clarity on the phishing campaign when compared to its predecessor. Some of the main takeaways include:

• Sampling of personnel is acceptable as long as it is documented in the ROE and approved by the AO prior to test execution.
• The attacker will be allowed to circumvent all perimeter security mechanisms for the phishing exercise.
• 3PAOs are permitted to leverage in-house phishing programs that are owned/operated by the CSP, as long as they comply with the guidance.
• Landing pages should identify that the email was a phish. In our experience, a detailed landing page may be considered “retraining” for victims, which is able to serve as remediation evidence for CSPs.
• Testers have the option of soliciting the user to run an untrusted script instead of harvesting credentials.
• 3PAOs will coordinate with the CSP to customize the phishing pretext based on threats at the time of testing. Testers are not permitted to use a “one size fits all” methodology (Section 5.0).

Attack Vector 2: External to CSP Target System

While most of the narrative within the External to CSP Target System section is used to provide enhanced definitions of threat terms, the final subsection relays prescriptive guidance on external system testing. Highlights include:

• All internet-facing endpoints are subject to non-credentialed pen testing. This not only includes the customer-facing application/API endpoints, but also administrative endpoints and management system interfaces. Specifically called out endpoints include out-of-band interfaces, VPNs, site-to-site connection points, and external service data ingress points.
• A “less than ideal scenario” has been prescribed, which requires that all “passive or active blocking security devices, such as web application firewalls and software-based security controls, will be bypassed to facilitate testing”.
• “3PAOs are also required to elevate risk ratings higher for compromise scenarios originating from public access.” The PMO historically identified that many times a CSP/3PAO may desire to downgrade this type of risk. The PMO does not want this to happen anymore.

Attack Vector 3: Tenant to CSP Management System

Similar to the previous version of the penetration testing guidance, the 3PAO will be issued a sample tenant account within the CSP’s information system, which will be used to attempt escalation to the cloud management plane. It is required that the highest customer privilege level be assigned to the 3PAO to generate the most reliable results.

The most noteworthy clarification provided within this vector is that, “while cloud providers may prefer to evaluate a tenant within the development/test environments, these are rarely identical to the production deployment, and will not be used as a valid representation for the FedRAMP penetration test vectors. A CSP’s production environment should be sufficiently resilient to sustain a FedRAMP penetration test.”

Attack Vector 4: Tenant-to-Tenant (Section 3.1.4)

Of all the test vectors contained within the updated guidance, the Tenant-to-Tenant has undergone the least amount of change. The requirement remains that CSPs will provide the 3PAOs with administrative credentials to two tenants. The 3PAO will then attempt to use the access of one tenant to compromise the other.

Attack Vector 5: Mobile Application to Target System (Section 3.1.5)

Like the Tenant-to-Tenant vector, the guidelines around testing Mobile Applications remain greatly unchanged. The primary clarification is that this attack vector is able to be marked as out-of-scope if mobile applications are not part of the CSP’s cloud service offering.

Attack Vector 6: Client-side Application and/or Agents to Target System (Section 3.1.6)

FedRAMP introduces a new attack vector in the updated guidance to address client-side software that is deployed in tandem with the cloud service offering. If client-side software is required to use the offering, CSPs must include it in the authorization boundary and submit it for 3PAO pen testing. If the software is not required to use the offering, CSPs may elect to exclude it from the boundary.

Specific testing methods for both Mobile Application and Client-side software vectors have not been described in the penetration test guidance, but testers should reference and leverage the MITRE ATT&CK framework identified as part of the updated Attack Models to structure their tests.

The Rules of Engagement and Penetration Test Report

The PMO has provided additional direction on what 3PAOs should include in both their Rules of Engagement (ROE) and Penetration Test Report documents. A notable addition to the ROE from the previous version of the guidance is the requirement to include the following statement:

Appropriate personnel such as the CIO, CISO, and ISSO are informed of any critical high-impact vulnerabilities as soon as they are discovered.

While this was commonly done previously, the PMO has standardized this approach to ensure that any critical findings are rapidly reported and addressed.

Guidance on creating the Penetration Test Report also received some updates. The PMO confirmed that they do not intend to release a template for this report but have listed six topics that the report will be required to cover (detailed in Section 6 of the guidance).

The updated Penetration Test Guidance provides some much-needed clarification on the attack vectors and methodologies. Many of the new requirements will take time to implement, but this is a solid step forward in unifying the industry’s testing methodology for cloud systems.