Command usage example outputs

When encryption is not enforced or the rolling data conversion is not finished, the crcontrol command denies Encryption Crawler related operations. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/crcontrol --encconvertstate
CRControlEncConvertInfoGet failed : operation not supported
Please double check the server encryption settings

Check the data format of a data container before the Encryption Crawler process. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|head -n 15
Path = /MSDP/data/3/3080.[bhd, bin]
*** Header for container 3080 ***
version                    : 1
flags                      : 0xe000(DC_ENTRY_FULL|DC_ENTRY_SHA256|DC_ENTRY_BINHEADER)
data file last position    : 67001810
header file last position  : 55252
source id                  : 2505958
retention                  : 0
file size                  : 67001810
delete space               : 0
active records             : 511
total records              : 511
deleted records            : 0
crc32                      : 0x4fd80a49

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|tail -n 15
type of record : SO
version        : 4
flags          : 0x2
backup session : 1670238781
fptype         : 3
size           : 131118
record crc     : 4164163489
data crc       : 1313121942
ctime          : 1642086781
offset         : 66870692
digest         : 7f7fd0c5d8fc64d9a7e25c7c079af86613b40d9feff9d316cdfc09c1eafb1690
KMS Enc        : NO
SO crc         : 85135236
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|wc
    511    5621   38325
[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|tail -n 5
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|grep -i -e "AES" -e "Encrypted"

Check the data format of a data container after the Encryption Crawler process. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|head -n 15
Path = /MSDP/data/3/3080.[bhd, bin]
*** Header for container 3080 ***
version                    : 1
flags                      : 0xe000(DC_ENTRY_FULL|DC_ENTRY_SHA256|DC_ENTRY_BINHEADER)
data file last position    : 67009986
header file last position  : 55252
source id                  : 2505958
retention                  : 0
file size                  : 67009986
delete space               : 0
active records             : 511
total records              : 511
deleted records            : 0
crc32                      : 0x54380a69

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|tail -n 15
type of record : SO
version        : 4
flags          : 0x2
backup session : 1670238781
fptype         : 3
size           : 131134
record crc     : 4210300849
data crc       : 1992124019
ctime          : 1642086781
offset         : 66878852
digest         : 7f7fd0c5d8fc64d9a7e25c7c079af86613b40d9feff9d316cdfc09c1eafb1690
KMS Enc        : NO
SO crc         : 85331847
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|wc
    511    8176   59276

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|tail -n 5
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|grep -i -e "AES" -e "Encrypted"
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-is-encrypted 3080
1 of 1: unencrypted 0: container 3080: size 67009986

Using dcscan --so-is-encrypted to check if a container or a list of containers are encrypted.

The status message unencrypted 0 indicate it's encrypted already, and unencrypted 1 indicates it's unencrypted and needs to be encrypted. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-is-encrypted 3080
1 of 1: unencrypted 1: container 3080: size 67001810

Using the reporting tool to report on the unencrypted MSDP data.

Veritas recommends using the reporting tool encryption_reporting to report the unencrypted data in the MSDP pool.

Note:

The encryption reporting tool is not supported on LinuxS or Flex WORM setups.

Table:

OS and Python requirements	Details
Python requirements for encryption_reporting on Linux installations.	NetBackup Red Hat installations come with Python and there are no extra steps for getting Python running.
Python requirements for encryption_reporting on Windows BYO installations.	NetBackup 10.0 and newer versions require you to install Python 3.6.8-3.9.6. Currently, no additional software packages are required to be installed. Installing Python 3.6.8-3.9.6 Download the Python Windows Installer for Python 3.6.8-3.9.6 from: https://www.python.org/downloads/ Run the installer and select Add Python 3.X.X to PATH. Run python encryption_reporting.py command by navigating to the directory containing the file (`..\Veritas\pdde`).

OS and Python requirements

Details

Python requirements for encryption_reporting on Linux installations.

NetBackup Red Hat installations come with Python and there are no extra steps for getting Python running.

Python requirements for encryption_reporting on Windows BYO installations.

NetBackup 10.0 and newer versions require you to install Python 3.6.8-3.9.6. Currently, no additional software packages are required to be installed.

Installing Python 3.6.8-3.9.6

Download the Python Windows Installer for Python 3.6.8-3.9.6 from:
https://www.python.org/downloads/
Run the installer and select Add Python 3.X.X to PATH.
Run python encryption_reporting.py command by navigating to the directory containing the file (..\Veritas\pdde).

By default, the reporting tool creates a thread pool of two threads. The tool uses these threads to search for unencrypted data or to encrypt the unencrypted data. A thread is used to process one MSDP mount point to completion. Upon completing the processing of a mount point, the thread is returned to the thread pool. The thread is then used to process any additional mount point that is queued up for processing.

The number of threads is equal to the number of mountpoints that can be processed concurrently. You can increase or decrease the thread pool's thread count by specifying the -n option. The minimum thread count is 1 and the maximum is 20.

The reporting tool is I/O intensive. Increasing the thread count up to the total number of MSDP mountpoints usually means better performance for the reporting tool. It also means more load on the system which can affect performance of backup, restore, deduplication, and replication jobs. No performance gains are observed for using more threads than there are mountpoints.

When using the reporting tool to search for the unencrypted data, each thread invokes one instance of dcscan. Each dcscan instance uses roughly N * 160 MB of memory. In this equation, N is the number of MSDP mountpoints on the server. If there are a total of 12 MSDP mountpoints, each dcscan instance uses about 1.8 GB of memory. If there are four threads running in the reporting tool, the reporting tool and the dcscan processes consume more than 7 GB of memory.

On a Windows BYO, the default path to dcscan is C:\Program Files\Veritas\pdde. If you have dcscan installed somewhere else, you must use the -d or --dcscan_dir option to specify the correct location.

The encryption_reporting does not account for data encrypted with the Encryption Crawler. If you have previously run the Encryption Crawler to encrypt data, you must clear the metadata files with the -c option if they exist. Then re-run encryption_reporting to get up-to-date information.

In certain circumstances, data may be reported as Encrypted needs KMS convert. This means that the data is encrypted, but not with KMS. If you see this message, use the crawler commands ./crcontrol - encconvertreset and ./crcontrol - encconverton to encrypt the rest of the data with KMS.

Veritas does not recommend that you run the reporting tool while the Encryption Crawler process is active.

Common command lines usage

./encryption_reporting -h
Display the help output for the command.
./encryption_reporting -n 4
Reports the amount of unencrypted and encrypted data once the script completes scanning. Use the -n option to define the number of threads in the thread pool. The default number of threads is 2.
./encryption_reporting -r
This command reports the amount of unencrypted data from the metadata files that were generated during a previous scan. It doesn't perform a scan.
./encryption_reporting -e -n 4
Uses the metadata files to submit data container encryption commands through crcontrol. Use the -n option to define the number of threads use in the thread pool. The default number of threads is 2.
./encryption_reporting -c
Delete the metadata files that are created during the scan. Be aware this command deletes all metadata files the previous scan generated.
./encryption_reporting
Runs the script to determine the amount of encrypted and unencrypted data on the media server.
This command generates metadata files for each container directory in the MSDP log directory under a directory called unencrypted_metadata.
The script reads in a configfilepath from /etc/pdregistry.cfg and parses out the path to read in the mount points from fstab.cfg. It reads in all mount points in fstab.cfg.
To determine the amount of encrypted and unencrypted data, look for a line similar to the one shown, bold added for emphasis:
2021-01-28 17:46:05,555 - root - CRITICAL - unencrypted bytes 58.53GB, encrypted bytes 14.46GB

Command usage example outputs

Feedback

Feedback