Managing the Encryption Crawler
Use the crcontrol command to manage the Encryption Crawler. The following table describes the options you can use to manage how the Encryption Crawler functions.
Table: crcontrol command options
Option | Description |
|---|---|
--encconverton | To enable and start the Encryption Crawler process, use --encconverton [num]. The num variable is optional and indicates the number for the partition index (starting from 1). The parameter enables the Encryption Crawler for the specified MSDP partition. If num is not specified, it is enabled for all MSDP partitions. The num variable is not supported on a BYO setup when the For example, [root@vrawebsrv4663 ~]# /usr/openv/pdde/pdcr/bin/crcontrol --encconverton Encryption conversion turned on for all partitions |
--encconvertoff | To disable and stop the Encryption Crawler process, use --encconvertoff [num]. The num variable is optional and indicates the number for the partition index (starting from 1). The parameter enables the Encryption Crawler for the specified MSDP partition. If num is not specified, it disabled for all MSDP partitions. The num variable is not supported on a BYO setup when the |
--encconvertlevel | To switch between Graceful mode and Aggressive mode, use --encconvertlevel level. The level is required.
|
--encconvertstate | To determine the mode of the Encryption Crawler process and the progress, use --encconvertstate [verbose]. Optionally, you can specify a verbose level (0-2) for this option.
The verbose parameter is not supported on a BYO setup when the For example, /usr/openv/pdde/pdcr/bin/crcontrol --encconvertstate 2 |
Once the Encryption Crawler is turned on, you can monitor the status, mode, and progress with the crcontrol --encconvertstate command.
Table: Encryption Crawler monitor
Item | Description |
|---|---|
Status | Shows if the Encryption Crawler is ON, OFF, or Finished. |
Level | Shows in which level and mode the Encryption Crawler is. The value is in the format mode (level), for example Graceful (1). |
Busy | Shows if the Encryption Crawler is busy or not. |
Max Group ID | The maximum container group ID to process when the Encryption Crawler is turned on. It's the data boundary and doesn't change once Encryption Crawler is turned on. |
Current Group ID | Currently processing this group ID. |
Current Container ID | Currently processing this container ID. |
Containers Estimated | The estimated number of data containers in the MSDP pool that the Encryption Crawler must process. It's a statistic information and there may be inaccuracy for performance reasons. Once the Encryption Crawler is turned on, the value is not updated. |
Containers Scanned | The number of data containers the Encryption Crawler must process. |
Containers Converted | The number of containers encrypted by the Encryption Crawler process. |
Containers Skipped | The number of data containers that the Encryption Crawler skipped. The reasons vary and are described in About the skipped data containers. If there are skipped data containers, you can check the Encryption Crawler log or the history log for the details. The encryption_reporting tool may help report and encrypt the individual containers after the Encryption Crawler process finishes. Details about this encryption_reporting tool are available. See Encrypting the data. |
Data Size Scanned | The aggregated data size of the scanned data containers for Containers Scanned. |
Data Size Converted | The aggregated data size of the converted data containers for Containers Converted. |
Progress | The proportion of the total estimated data containers that the Encryption Crawler has scanned. Progress = Containers Scanned / Containers Estimated |
Conversion Ratio | The proportion of the scanned data size which the Encryption Crawler has converted. Conversion Ratio = Data Size Converted / Data Size Scanned |
Mount Points Information | The status of each mount point. If a verbose value of 1 is specified for the --encconvertstate option, the details of the unfinished mount points are printed. If a verbose value of 2 is specified for --encconvertstate option, the details of all the mount points are printed regardless of completion state. |
The Progress line in the log can be used to extrapolate how long the Encryption Crawler is expected to take. For example, if 3.3% of the pool is completed in 24 hours, the process may take about 30 days to finish.
Note:
The Encryption Crawler processes the data containers in reverse order from new to old.
It's possible to back up new data after encryption is enforced but before the Encryption Crawler is turned on. If that happens, the Conversion Ratio could be less than 99% for the new data containers at the beginning. While the process is running, the value of Conversion Ratio can become higher with the fact that the older data containers can potentially have more unencrypted data. In this case, the Conversion Ratio, Containers Converted, and Containers Estimated can help estimate the speed for these data containers.
Monitoring the change of Conversion Ratio can give some indication for the proportion of the unencrypted data while the Encryption Crawler is active.
Note:
During the encryption process, the progress survives in the case of MSDP restart.
The reasons the Encryption Crawler skips some data containers as reported by Containers Skipped include:
If a data container is to be expired but not yet deleted, it is skipped.
If a data container has a possible data integrity issue, it is skipped. The Encryption Crawler conveys the container to the CRC check process to identify and possibly fix the container.
If Instant Access or Universal Share is configured, and if some shares are not checkpointed before the Encryption Crawler process, the shares may hold some data containers with exclusive permission. Those data containers are skipped. Cohesity recommends that you create checkpoints for all the shares of Instant Access or Universal Share before turning on the Encryption Crawler process. By doing so, VpFS releases the exclusive permission of those data containers for spoold and the Encryption Crawler to process them.
Appliances starting with the release of 3.1.2 can have empty data containers the VpFS root share
vpfs0reserves, even if Instant Access or Universal Share is configured. This situation can also occur on a BYO setup where Instant Access or Universal Share is configured. Normally, VpFS does not release the exclusive permission of those data containers. Those data containers are skipped. You can ignore these skipped containers.Here how to check if the skipped data containers are empty and if the VpFS root share
vpfs0owns them. You can check the other VpFS owned data containers in the similar way.You can find the skipped data containers that are identified as owned by VpFS in the Encryption Crawler log by looking for the following:
n152-h21:/home/maintenance # grep VpFS /msdp/data/dp1/pdvol/log/spoold/enccrawler.log
February 04 05:13:14 WARNING [139931343951616]: -1: __getDcidListFromOneGroup: 1 containers owned by VpFS in group 7 were skipped. min DC ID 7168, max DC ID 7168
Check if the VpFS root share
vpfs0owns the data containers.
n152-h21:/home/maintenance # cat /msdp/data/dp1/4pdvol/7/.shareid vpfs0 106627568
The data containers that the VpFS root share
vpfs0owns, are empty.n152-h21:/home/maintenance # ls -Al /msdp/data/dp1/4pdvol/7 total 24 -rw-r--r-- 1 root root 64 Feb 1 02:40 7168.bhd -rw-r--r-- 1 root root 0 Feb 1 02:40 7168.bin -rw------- 1 root root 12 Feb 1 02:40 .dcidboundary -rw-r----- 1 root root 15 Feb 1 02:40 .shareid drwxr-xr-x 3 root root 96 Feb 4 15:37 var n152-h21:/home/maintenance # /usr/openv/pdde/pdcr/bin/dcscan 7168 Path = /msdp/data/dp1/4pdvol/7/7168.[bhd, bin] *** Header for container 7168 *** version : 1 flags : 0x4000(DC_ENTRY_SHA256) data file last position : 0 header file last position : 64 source id : 0 retention : 0 file size : 0 delete space : 0 active records : 0 total records : 0 deleted records : 0 crc32 : 0x1d74009d