Ensure that the container is running the patched image

There are three copies of the container image present in the Kubernetes environment during deployment or patching.

The first image copy is created on a local docker instance during image load operation. To check this copy, perform the following:

Run:
$ docker load -i images/pdk8soptr-20.4.tar.gz
Sample output:
```
Loaded image: msdp-operator:20.4
```
Taking the image name from step 1, run:
$ docker image ls | grep msdp-operator
Sample output:
```
msdp-operator 20.4 353d2bd50105 2 days ago 480 MB
```
Taking the value from step 2, run:
$ docker inspect 353d2bd50105 | jq .[].Id
```
"sha256:353d2bd50105cbc3c61540e10cf32a152432d5173bb6318b8e"
```

The second copy is created in the following respective registry's:

(AKS-specific): Azure Container Registry (ACR)
(EKS-specific): Amazon Elastic Container Registry (ECR)

To check this copy, perform the following:

Keep the image name and version same as original, run:
(AKS-specific): $ docker image tag msdp-operator:20.4 testregistry.azurecr.io/msdp-operator:20.4
(EKS-specific): $ docker image tag msdp-operator:20.4 046777922665.dkr.ecr.us-east-2.amazonaws.com/nbuxeksdeploy.aws.io/msdp-operator:20.4

Run:

$ docker image ls | grep msdp-operator

Sample output:

(AKS-specific):

msdp-operator  20.4 353d2bd50105 2 days ago 480 MB
 tregistry.azurecr.io/msdp-operator 20.4 353d2bd50105 2 days ago 480 MB

(EKS-specific):

msdp-operator  20.4 353d2bd50105 2 days ago 480 MB
msdp-operator            20.4 046777922665.dkr.ecr.us-east-2.
amazonaws.com/nbuxeksdeploy.aws.io/msdp-operator
20.4 353d2bd50105 2 days ago 480 MB /msdp-operator 20.4 
353d2bd50105 2 days ago 480 MB

To push the image to the registry, run:
(AKS-specific): $ docker push testregistry.azurecr.io/msdp-operator
(EKS-specific): $ docker push testregistry.<account id>.dkr.ecr.<region>.amazonaws.com/<registry>:<tag>.io/msdp-operator
The push refers to a repository [testregistry.azurecr.io/msdp-operator]
- (AKS-specific): [testregistry.azurecr.io/msdp-operator]
- (EKS-specific): [046777922665.dkr.ecr.us-east-2.amazonaws.com/nbuxeksdeploy.aws.io/msdp-operator]
0a504041c925: Layer already exists
```
20.4: digest: 
sha256:d294f260813599562eb5ace9e0acd91d61b7dbc53c3 size:
 2622 
```

To verify local image digest after the push operation, run:

$ docker inspect 353d2bd50105 | jq .[].RepoDigests

Sample output:

(AKS-specific):

[
   "testregistry.azurecr.io/msdp-operator@sha256:
d294f260813599562eb5ace9e0acd91d61b7dbc53c3"
 ]

(EKS-specific):

[
   "testregistry.<account id>.dkr.ecr.<region>.amazonaws.com/<registry>:<tag>.io/
msdp-operator@sha256: d294f260813599562eb5ace9e0acd91d61b7dbc53c3"
 ]

To verify image presence in the registry, run:

(AKS-specific): $ az acr repository list --name testregistry

(EKS-specific): $ aws ecr describe-repositories --repository-names "veritas/main_test1"

Sample output:

(AKS-specific):

 [
   "msdp-operator",
 ]

(EKS-specific):

 "repositories": [
        {
            "repositoryArn": "arn:aws:ecr:us-east-2:046777922665:

repository/veritas/main_test1",
            "registryId": "046777922665",
            "repositoryName": "veritas/main_test1",
            "repositoryUri": "046777922665.dkr.ecr.us-east-2.

amazonaws.com/veritas/main_test1",
            "createdAt": "2022-04-13T07:27:52+00:00",
            "imageTagMutability": "MUTABLE",
            "imageScanningConfiguration": {
                "scanOnPush": false
            },
            "encryptionConfiguration": {
                "encryptionType": "AES256"
            }
        }
    ]

To verify image digest in registry, run:

(AKS-specific): $ az acr repository show -n testregistry --image msdp-operator:20.4

(EKS-specific): $ aws ecr describe-images --registry-id 046777922665 --repository-name "veritas/main_test1" --image-ids imageTag=latestTest5

Sample output:

(AKS-specific):

{
   "changeableAttributes": {
     "deleteEnabled": true,
     "listEnabled": true,
     "readEnabled": true,
     "writeEnabled": true
   },
   "createdTime": "2022-02-01T13:43:26.6809388Z",
   "digest": "sha256:d294f260813599562eb5ace9e0acd91d61b7dbc53c3",
   "lastUpdateTime": "2022-02-01T13:43:26.6809388Z",
   "name": "20.4",
   "signed": false
 }

(EKS-specific):

"imageDetails": [
        {
            "registryId": "046777922665",
            "repositoryName": "veritas/main_test1",
            "imageDigest": 
"sha256:d0095074286a50c6bca3daeddbaf264cf4006a92fa3a074daa4739cc995b36f8",
            "imageTags": [
                "latestTest5"
            ],
            "imageSizeInBytes": 38995046,
            "imagePushedAt": "2022-04-13T15:56:07+00:00",
            "imageManifestMediaType": "application/vnd.docker.
distribution.manifest.v2+json",
            "artifactMediaType": "application/vnd.docker.container.image.v1+json"
        }
    ]

The third copy is located on a Kubernetes node running the container after it is pulled from the registry. To check this copy, perform the following:

Run;

$ kubectl get nodes -o wide

(AKS-specific):

NAME           STATUS  VERSION   INTERNAL-IP   OS-IMAGE             
aks-agentpool-7601-vmss000 Ready v1.21.7 10.240.0.4 Ubuntu 20.44.6 LTS

(EKS-specific):

NAME           STATUS  VERSION   INTERNAL-IP   OS-IMAGE             
eks-agentpool-7601-vmss000 Ready v1.21.7 10.240.0.4 Ubuntu 20.44.6 LTS

Use kubectl debug to run a container on the node:
(AKS-specific): $ kubectl debug node/aks-nodepool1-7601-vmss000-it --image=mcr.microsoft.com/aks/fundamental/base-ubuntu:v0.0.11 root@aks-agentpool-7601-vmss000:/#
(EKS-specific): $ kubectl debug node/eks-nodepool1-7601-vmss000-it --image=mcr.microsoft.com/eks/fundamental/base-ubuntu:v0.0.11 root@eks-agentpool-7601-vmss000:/#
You can interact with the node session from the privileged container:
chroot /host

Verify the presence of the image:

/usr/local/bin/crictl image | grep msdp

Sample output:

(AKS-specific):

testregistry.azurecr.io/msdp-operator  20.4  353d2bd50105c  182MB

(EKS-specific):

<account id>.dkr.ecr.<region>.amazonaws.com/msdp-operator  20.4  353d2bd50105c  182MB

Verify the image ID on the Kubernetes node, run:
/usr/local/bin/crictl inspecti 353d2bd50105c | jq .[].id
Sample output
```
"sha256:353d2bd50105cbc3c61540e10cf32a152432d5173bb6318b8e"
null
```

Verify the image digest on the Kubernetes node, run:

/usr/local/bin/crictl inspecti 353d2bd50105c | jq .[].repoDigests

Sample output

(AKS-specific):

[
  "testregistry.azurecr.io/msdp-operator@sha256:
d294f260813599562eb5ace9e0acd91d61b7dbc53c3"
]
null

(EKS-specific):

[
  "<account id>.dkr.ecr.<region>.amazonaws.com/msdp-operator@sha256:
d294f260813599562eb5ace9e0acd91d61b7dbc53c3"
]
null

How to make sure that you are running the correct image

Use the steps given above to identify image ID and Digest and compare with values obtained from the registry and the Kubernetes node running the container.

Note:

MSDP Scaleout images (uss-engine, uss-mds, uss-controller, msdp-operator) use IfNotPresent imagePullPolicy. A unique image tag is required in order for a Kubernetes node to pull an updated image.

Ensure that the container is running the patched image

Feedback

Feedback