Exposure of vSphere's CPI and CSI credentials in Rancher

Impact

A vulnerability has been identified in the way that Rancher stores vSphere's CPI (Cloud Provider Interface) and CSI (Container Storage Interface) credentials used to deploy clusters through the vSphere cloud provider. This issue leads to the vSphere CPI and CSI passwords being stored in a plaintext object inside Rancher. This vulnerability is only applicable to users that deploy clusters in vSphere environments.

The exposed passwords were accessible in the following objects:

• Can be accessed by users that are cluster members of the provisioned clusters:
  • When provisioning a new cluster with the vSphere cloud provider through Rancher's UI (user interface), Cluster Templates and Terraform on the object provisioning.cattle.io in spec.rkeConfig.chartValues.rancher-vsphere-cpi and spec.rkeConfig.chartValues.rancher-vsphere-csi.
  • On the object rke.cattle.io.rkecontrolplane in spec.chartValues.rancher-vsphere-cpi and spec.chartValues.rancher-vsphere-csi.
• Can be accessed by users with privileged access to the clusters' infrastructure (host OS):
  • Inside the plan files in the provisioned downstream clusters' filesystems.

Note: if you believe that the vSphere credentials might have been accessed by unauthorized users, it's highly recommended to change them, after updating Rancher to a patched version.

Please consult the associated MITRE ATT&CK - Technique - Credential Access for further information about this category of attack.

Patches

Patched versions include Rancher releases 2.8.9 and 2.9.3.

After updating your environment to one of the patched Rancher's versions, it's mandatory to execute this script that provides an automated way to mitigate any vulnerable leftover vSphere clusters' credentials within Rancher's local cluster. This script doesn't need to be executed in case you are installing a fresh and new environment.

The script will fetch all objects in Rancher's local cluster, loops through them, if the affected vSphere charts are present, then it extracts the username and password parameters into a secret in the fleet-default namespace for both with the appropriate annotation to synchronize them to the downstream clusters. Finally, it updates the cluster's to reference those secrets rather than existing plaintext values.