Deduplication backup process to the Media Server Deduplication Pool (MSDP)
The deduplication backup process to the Media Server Deduplication Pool (MSDP) is as follows:
The client bpbkar sends data to the NetBackup backup tape manager - the bptm process
pdvfs (using bptm as a proxy) connects to the NetBackup Deduplication Manager (spad) to record metadata (image records) in the spadb mini-catalog and connects to the NetBackup Deduplication Engine (spoold) to store the image data in the
.bhd/.binfiles in the data directory (dedup_path\data)spoold writes
tlogsto the.tlogfiles in the queue (dedupe_path\queue) directory and to the processed directory. The tlog data from the queue directory is processed into the crdb later when the next content router queue processing job runs. Beginning with NetBackup 7.6,.tlogfiles do not contain additions to the database.
The functional overview is as follows:
In this scenario, the client backs up data directly to the media server and the media server deduplicates the data before it stores it locally. Ensure that this is on the correct media server which is not always the same as the MSDP storage server (due to load balancing).
For deduplication-specific logging, enable the following on the media server:
Verbose 5 bptm logging:
Create a log directory named
bptmin/usr/openv/netbackup/logs(Windows:install_path\NetBackup\logs)Set the bptm log verbosity to 5 in the NetBackup Administration Console. To do this, click on for the media server. If you use UNIX/Linux, set the bptm log verbosity to 5 in the
/usr/openv/netbackup/bp.conffile by appending the following line:BPTM_VERBOSE = 5
Edit the
pd.confconfiguration file that is located at:Windows:
install_path\NetBackup\bin\ost-plugins\pd.confUNIX/Linux:
/usr/openv/lib/ost-plugins/pd.confand uncomment and/or modify the following line:
LOGLEVEL = 10
Note:
You can also modify DEBUGLOG in the
pd.conffile to specify a path to which to log; however, we recommend leaving the DEBUGLOG entry commented out. The logging information (PDVFS debug logging) then logs to the bptm and bpdm logs.
Enable verbose spad/spoold logging (optional).
Edit the
dedup_path\etc\puredisk\spa.cfganddedup_path\etc\puredisk\contentrouter.cfgfiles so that the following line:Logging=long,thread is changed to Logging=full,thread
Ensure that you are on the correct media server and restart the MSDP storage server services.
Caution:
If you enable verbose logging, it can impact the performance on MSDP.
Reproduce the backup failure.
Within the NetBackup Administration Console, click on , open the job details and click the tab. It displays the media server host name that ran the backup and the bptm process ID number (PID).
Find a line similar to bptm(pid=value); this value is the bptm PID to locate in the bptm log.
Extract the bptm PID found in step 3 from the bptm log on the media server. This step only gathers the single-line entries; review the raw logs to see the multi-line log entries. In the following examples, 3144 is the bptm PID:
Windows command line:
findstr "\[3144." 092611.log > bptmpid3144.txt
UNIX/Linux command line:
grep "\[3144\]" log.092611 > bptmpid3144.txt
Gather the spoold session logs that cover the dates from when the backup started and when it failed from the following logs:
Windows:
dedup_path\log\spoold\mediasvr_IP_or_hostname\bptm\Receive\MMDDYY.log dedup_path\log\spoold\mediasvr_IP_or_hostname\bptm\Store\MMDDYY.log
UNIX/Linux:
dedup_path/log/spoold/mediasvr_IP_or_hostname/bptm/Receive/MMDDYY.log dedup_path/log/spoold/mediasvr_IP_or_hostname/bptm/Store/MMDDYY.log