Kubernetes Vulnerability: What You Can and Should Do to Protect Your Enterprise

VULNERABILITY SUMMARY

This week, news was released regarding a critical security Common Vulnerability and Exposure (CVE) associated with the Kubernetes container software (CVE-2018-1002105). While this is only a reported vulnerability at this stage (and no actual exploits have been reported to date), organizations that have Kubernetes deployed within their environment(s) are strongly advised to treat this matter with high priority.

The nature of the vulnerability contained within CVE-2018-1002105 is related to privilege escalation. Organizations running versions of Kubernetes containing the vulnerability may be susceptible to critical risks associated with execution and detection of unauthorized transactions committed via the Kubernetes Application Program Interface (API).

Depending on the organization’s architecture, this vulnerability is the equivalent of leaving the front door open to the enterprise’s entire cloud environment. The security risks are made even more challenging in that a compromise might not be detectable as it obfuscates the detection capabilities.

The goal of this blog is to provide general steps toward understanding your status with respect to this vulnerability and taking the appropriate defensive measures given your environment(s). There are concrete actions organizations can take to protect themselves from potential threats, such as downloading patches that have already been released, watching for patches that are yet to be released, and modifying architectural configuration for short-term risk mitigation.

The approach should follow a process similar to the following:

• Confirm whether your organization is affected by the Kubernetes vulnerability given your deployment version
• Evaluate risk factors to determine actual impacts of the vulnerability on your organization’s environment(s)
• Weigh the tradeoffs that various patch and/or remediation approaches have on your environment to determine the appropriate path to follow

Please note, this blog aims to provide a general understanding of what risks organizations may face as a result of the Kubernetes CVE-2018-1002105, and high-level approaches that could be considered for addressing the vulnerability. It does not constitute a formal assessment of risks and remediations for your organization.

HOW TO DETERMINE IF YOUR SYSTEM IS VULNERABLE

First, confirm whether your organization’s Kubernetes deployment is affected by the vulnerability. Make sure to check which versions of the Kubernetes platform your organization is running and whether they match those identified within the affected list. If you are running off of an as-a-service model, check with your provider to confirm whether they have performed the proper remediation / mitigation steps on their services.

Versions of Kubernetes affected by this vulnerability include:

• Kubernetes v1.0.x-1.9.x
• Kubernetes v1.10.0-1.10.10 (fixed in v1.10.11)
• Kubernetes v1.11.0-1.11.4 (fixed in v1.11.5)
• Kubernetes v1.12.0-1.12.2 (fixed in v1.12.3)

The vulnerability has been fixed in the following distributions (visit the GitHub link for actionable steps):

• v1.10.11
• v1.11.5
• v1.12.3
• v1.13.0-rc.1

The following cloud service providers have released public notices that the software behind their Kubernetes as-a-service offerings have been patched; customers on these platforms are advised to take quick action in downloading applicable patches:

• Google Cloud Platform (GCP)
• Microsoft Azure
• Amazon Web Services

If you find that you’re running a version of Kubernetes affected by the vulnerability, then your organization is susceptible to an attack and should begin to consider the following risk factors.

RISK FACTORS

The Kubernetes CVE-2018-1002105 does not pose the same risk levels to all organizations running the platform. There are several factors that play into such determinations. The risk factors presented here do not represent a “comprehensive” list, but they do represent some of the key variables you should consider when understanding how such a vulnerability may impact your environment(s). They include the following:

Think of these variables as levers, and as these levers move one way vs. another, your organization’s risk may increase or decrease accordingly.

Data Types and Volume

Data types and their volume are probably the foremost thing to be concerned about when evaluating your risk exposure to a vulnerability such as the recent Kubernetes CVE-2018-1002105. You should be asking yourself how important the data within your application(s) are to

• Your organization (e.g., firm Intellectual Property)
• To your customers / end users (e.g., payment data, personally identifiable data, etc.)
• To the public (e.g., national defense secrets)

In some cases, such as firm intellectual property, it is up to your organization to assign the value at risk for preserving the confidentiality, integrity, and/or available of that data against a vulnerability. For other data types (such as personally identifiable information, protected health information, and payment card or financial data), the risks and consequences of failing to protect data are not entirely up to you, but also involve input by additional parties (such as regulatory agencies).

The more sensitive and/or valuable data types and volumes your organization possesses, the greater your risks are for not patching the Kubernetes CVE-2018-1002105.

Data Flows

The next step is to consider how data flows in relation to your organization’s Kubernetes-based environment(s). Specifically, key data flows include users and system accounts that access the Kubernetes API services, as well as services the Kubernetes and containers are connected to.

On one end of the spectrum, you may have a Kubernetes deployment with an internet-exposed API, and several connections between its containers and additional services and/or applications. In this case, your organization may be at higher risk than having a Kubernetes deployment with private-facing API services and limited service connections.

The more open your network access is to your Kubernetes API service (as well other services connected to your Kubernetes and/or containers), the greater your risks.

User Populations

Another key variable is your user population as it pertains to the Kubernetes deployment. Your organization should consider what types of user populations it has, which of those have access to the Kubernetes services, and the level of permissions those user populations require (i.e., Role-based Access Control, or RBAC).

The Kubernetes CVE-2018-1002105 GitHub page provides a breakdown of the user types that can potentially access the API services. They range from anonymous users to privileged admin accounts. Your organization should find out which of these account types currently exist for your Kubernetes deployment. If your user population for the API is already limited to a handful of privileged accounts, authorized to run exec/attach/portforward transactions (with all other API accounts removed and/or permissions disabled), then the level of risk may be lowered. Compare this to a Kubernetes deployment that has business cases dependent upon anonymous accounts, authenticated-non-privileged accounts, as well as privileged accounts, which may have a higher level of associated risk.

The larger your user population and RBAC requirements related to the Kubernetes services, the greater your risks.

REMEDIATION AND MITIGATION APPROACHES

To be clear, the optimal remediation decision for the Kubernetes CVE-2018-1002105 is to update to a version containing the security patch. That said, running version updates may not be an immediate option for some organizations that require longer lead times to perform proper change management and testing. For those organizations, there may be shorter-term options to help mitigate risks until the time that the Kubernetes CVE-2018-1002105 patch can be promoted to production.

Determine Responsibility

The first step is to determine who exactly is responsible for taking the remediation steps. This will depend on the nature of your Kubernetes deployment. There are two main categories here:

• Kubernetes As-a-Service – In this case, your organization will have no control over patching, as this falls onto the service provider under their shared responsibility, or
• Managed Kubernetes on Virtual Machines – In this case, your organization assumes full responsible for taking remediation / mitigation steps

If you find that your organization owns responsibility for the Kubernetes CVE-2018-1002105 remediation / mitigation, then you may consider the following options.

Run Updates to Patch the Kubernetes Vulnerability

The most straightforward approach for remediating the Kubernetes CVE-2018-1002105 is to run an update to a version identified as containing the security patch. These versions can be found on the GitHub repo page for the vulnerability.

Modify Configurations to the Kubernetes API

As mentioned, if running an immediate update is not a viable option for your organization’s deployment, the Kubernetes GitHub page has provided a number of API configuration changes, which may help to reduce risk exposure associated with the vulnerability. These include:

• Suspending the Kubernetes aggregated API servers (i.e., shutting down API services)
• Disabling capabilities that allow anonymous requests to be passed by the API services
• Removing all anonymous access to all Kubernetes aggregated API services
• Removing all access from any users not authorized to have privileged access on Kubernetes, and/or
• Removing production exec/attach/portforward permissions from any user who should not be able to have full access to Kubernetes

This mitigation helps to address data flow and user population-related risks. These API configuration changes modify user access permissions (which makes sense given the nature of the vulnerability). They equate to either completely turning off API services or removing accounts and/or permissions for all accounts (except those that should have elevated access).

The Kubernetes GitHub page provides specific details on how to enforce these configurations.

Note: These configuration changes may impact additional services by disrupting transactions dependent on use of the Kubernetes API services. Your organization should first determine if and where these dependencies may exist prior to committing changes. Additionally, these changes should only be considered for short-term risk mitigation until the time that the patch can implemented.

Remove the Kubernetes API Exposure from Public Exposure

This mitigation is related to data flow risks. It consists of changing routing and firewall rules to pull your Kubernetes API servers back from public exposure (if it is not already). As with making API access configuration changes, your organization should first understand its requirements around the Kubernetes API services, identify where external dependencies may exist, and determine whether such network access changes will impact functionalities.

Again, this option does not remediate the vulnerability; rather, it is a risk mitigation until proper patching can be performed. Your organization will still have risks associated with privilege escalation originating from inside the network.

Password, Secret Rotation, and Access Certification Exercise

One of the complexities of the Kubernetes CVE-2018-1002105 is that it allows unauthorized transactions to go undetected. As a result, organizations should follow the assumption that their application environment has already been compromised. An immediate exercise should be taken to enforce password resets, key rotations, as well as access certification steps. This process will help to identify, remove, and/or thwart accounts that may have been created or modified from executing malicious activities against your environment.