Best practices: NetBackup deduplication
Veritas recommends that you use the NetBackup stream handler where it is available for backup data streams. For data streams without the stream handler, NetBackup uses the default fixed-length deduplication (FLD) of 128 KB to segment the data streams for fingerprint calculation.
If the deduplication rates are poor with the default FLD, consider the following options:
Try variable length deduplication (VLD). VLD can substantially improve the deduplication ratio for some database workload backups such as DB2, for which the stream handler is not available. A customer reported that the deduplication ratio improved from 0% to over 80% after switching to VLD.
More information about how VLD works and the resource overhead is available:
Use AdvancedDisk. If VLD fails to increase the deduplication ratio, consider AdvancedDisk. Internal test results showed that backup to AdvancedDisk can outperform a 0% deduplication rate for backups by up to 25%.
For VLD, the default window size for scanning is 32KB - 128KB. We recommend that you use the default window size first. If you see improved deduplication ratios but want to fine-tune the window size to improve performance, you can use data from the job detail report from bpdbjobs command for analysis. The following is a sample bpdbjobs command syntax to extract detail DB2 job information:
bpdbjobs -report -all_columns -jobid db2jobid | tr "," "\n"
The following output shows that out of the 99.5% deduplication ratio, 96.3% comes from scanning the 104K-128K segment size. By giving up 3.6% (1.2 + 1.3 + 1.1) of the deduplication ratio, the scanning window size can be substantially reduced. This reduction can result in a substantial CPU saving because fingerprinting is a CPU-intensive operation.
VLD enabled\ SO Count=1417216\ 32K~56K:1.2%\ 56K~80K:1.3%\ 80K~104K:1.1%\ 104K~128K:96.3%) for (full qualified media server hostname ): scanned: 178118722 KB\ CR sent: 931507 KB\ CR sent over FC: 0 KB\ dedup: 99.5%\ cache hits: 179 (0.0%)\ rebased: 367 (0.0%)\ where dedup space saving:98.9%\ compression space saving:0.5%
For more NetBackup deduplication best practices, see the MSDP deployment best practices section in the NetBackup Deduplication Guide.