Project cloud-provider-vsphere has been deleted.

[saintdle]

What happened?

Booting up cluster, unable to get CSI pods running which has taken cluster offline, when troubleshooting, unable to pull the image, testing with docker pull, get the following

❯ docker pull gcr.io/cloud-provider-vsphere/csi/release/driver:v3.3.0
Error response from daemon: Head "https://gcr.io/v2/cloud-provider-vsphere/csi/release/driver/manifests/v3.3.0": denied: Project cloud-provider-vsphere has been deleted.

What did you expect to happen?

Image to be available

How can we reproduce it (as minimally and precisely as possible)?

docker pull gcr.io/cloud-provider-vsphere/csi/release/driver:v3.3.0

Anything else we need to know (please consider providing level 4 or above logs of CPI)?

No response

Kubernetes version

❯ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.7
Kubecolor Version: v0.4.0

Cloud provider or hardware configuration

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Kernel (e.g. uname -a)

Install tools

Container runtime (CRI) and and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

Others

[hyorch]

Images have been migrated to another registry.
https://github.com/kubernetes/cloud-provider-vsphere#warning-kubernetes-image-registry-migration-for-cloud-provider-vsphere

[addyess]

@hyorch thanks for pointing at the note about the new registry. I don't see the older releases (1.27.0 for example) nor do I quite understand where the various syncer, driver, or manager for the vsphere-storage drivers

effectively -- where did these images go?

gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.18.0
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.19.0
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.2.1
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.20.0
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.21.1
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.22.3
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.23.1
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.24.2
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.25.0
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.26.0
gcr.io/cloud-provider-vsphere/cpi/release/manager:v1.27.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.5.3
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.5.4
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.6.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.6.1
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.6.2
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.6.3
gcr.io/cloud-provider-vsphere/csi/release/driver:v2.7.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.0.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.0.2
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.0.3
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.1.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.1.1
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.1.2
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.2.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.3.0
gcr.io/cloud-provider-vsphere/csi/release/driver:v3.3.1
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.5.3
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.5.4
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.6.0
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.6.1
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.6.2
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.6.3
gcr.io/cloud-provider-vsphere/csi/release/syncer:v2.7.0
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.0.0
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.0.2
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.0.3
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.1.0
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.1.1
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.1.2
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.2.0
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.3.0
gcr.io/cloud-provider-vsphere/csi/release/syncer:v3.3.1