An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where sensitive fields, like passwords, API keys and Rancher's service account token (used to provision clusters), were stored in plaintext directly on Kubernetes objects like Clusters, for example cluster.management.cattle.io. Anyone with read access to those objects in the Kubernetes API could retrieve the plaintext version of those sensitive data.
The exposed credentials are visible in Rancher to authenticated Cluster Owners, Cluster Members, Project Owners, Project Members and User Base on the endpoints:
/v1/management.cattle.io.catalogs/v1/management.cattle.io.cluster/v1/management.cattle.io.clustertemplates/v1/management.cattle.io.notifiers/v1/project.cattle.io.sourcecodeproviderconfig/k8s/clusters/local/apis/management.cattle.io/v3/catalogs/k8s/clusters/local/apis/management.cattle.io/v3/clusters/k8s/clusters/local/apis/management.cattle.io/v3/clustertemplates/k8s/clusters/local/apis/management.cattle.io/v3/notifiers/k8s/clusters/local/apis/project.cattle.io/v3/sourcecodeproviderconfigsSensitive fields are now stripped from Clusters and other objects and moved to a Secret before the object is stored. The Secret is retrieved when the credential is needed. For objects that existed before this security fix, a one-time migration happens on startup.
Important:
serviceAccountToken allows any standard user to escalate its privileges to cluster administrator in Rancher.The fields that have been addressed by this security fix are:
Notifier.SMTPConfig.PasswordNotifier.WechatConfig.SecretNotifier.DingtalkConfig.SecretCatalog.Spec.PasswordSourceCodeProviderConfig.GithubPipelineConfig.ClientSecretSourceCodeProviderConfig.GitlabPipelineConfig.ClientSecretSourceCodeProviderConfig.BitbucketCloudPipelineConfig.ClientSecretSourceCodeProviderConfig.BitbucketServerPipelineConfig.PrivateKeyCluster.Spec.RancherKubernetesEngineConfig.BackupConfig.S3BackupConfig.SecretKeyCluster.Spec.RancherKubernetesEngineConfig.PrivateRegistries.PasswordCluster.Spec.RancherKubernetesEngineConfig.Network.WeaveNetworkProvider.PasswordCluster.Spec.RancherKubernetesEngineConfig.CloudProvider.VsphereCloudProvider.Global.PasswordCluster.Spec.RancherKubernetesEngineConfig.CloudProvider.VsphereCloudProvider.VirtualCenter.PasswordCluster.Spec.RancherKubernetesEngineConfig.CloudProvider.OpenstackCloudProvider.Global.PasswordCluster.Spec.RancherKubernetesEngineConfig.CloudProvider.AzureCloudProvider.AADClientSecretCluster.Spec.RancherKubernetesEngineConfig.CloudProvider.AzureCloudProvider.AADClientCertPasswordCluster.Status.ServiceAccountTokenClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.PrivateRegistries.PasswordClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.Network.WeaveNetworkProvider.PasswordClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.CloudProvider.VsphereCloudProvider.Global.PasswordClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.CloudProvider.VsphereCloudProvider.VirtualCenter.PasswordClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.CloudProvider.OpenstackCloudProvider.Global.PasswordClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.CloudProvider.AzureCloudProvider.AADClientSecretClusterTemplate.Spec.ClusterConfig.RancherKubernetesEngineConfig.CloudProvider.AzureCloudProvider.AADClientCertPasswordPatched versions include releases 2.5.16, 2.6.7 and later versions.
After upgrading to a patched version, it is important to check for the SecretsMigrated condition on Clusters, ClusterTemplates, and Catalogs to confirm when secrets have been fully migrated off of those objects and the objects scoped within them (Notifiers and SourceCodeProviderConfigs).
Limit access in Rancher to trusted users. There is not a direct mitigation besides upgrading to the patched Rancher versions.
Important:
serviceAccountToken. This rotation is not done by the version upgrade. Please see the helper script below.The script available in rancherlabs/support-tools/rotate-tokens repository can be used as a helper to rotate the service account token (used to provision clusters). The script requires a valid Rancher API token, kubectl access to the local cluster and the jq command.
We would like to recognize and appreciate Florian Struck (from Continum AG) and Marco Stuurman (from Shock Media B.V.) for the responsible disclosure of this security issue.
If you have any questions or comments about this advisory:
{
"cwe_ids": [
"CWE-312"
],
"github_reviewed": true,
"github_reviewed_at": "2022-09-23T18:11:28Z",
"nvd_published_at": "2022-09-07T09:15:00Z",
"severity": "CRITICAL"
}