Memory considerations
Beside faster backup, one of the core advantages of MSDP is storage space saving. The amount of storage space saving is determined by the deduplication ratio of the backup images, and the deduplication ratio is affected by the fingerprint lookup cache hit ratio. By default, MSDP is allowed to use up to 50% of the system RAM for fingerprint caching, the cache size for fingerprint is specified by the parameter, MaxCacheSize, defined in the MSDP configuration file, contentrouter.cfg. For older NetBackup releases, 8.2 and lower, the default was set at 75%. Beginning with NetBackup 8.3, the default is changed to 50%.
When the system is new, few fingerprints exist to consume the allocated cache, only a very small amount of the 50% RAM will be used, the system may show lots of free memory in the beginning. However, as more data is backed up to the system, more and more unique fingerprints will be created and more RAM will be consumed to cache these fingerprints. With SHA-2 algorithm, each fingerprint takes 48 bytes of RAM and can't be paged out. The fingerprint cache size and the average segment size determine how many fingerprints would be stored in memory for fingerprint lookup after a client-side cache lookup miss and may impact the deduplication rate. Big backup images, multi-streaming backup images and database multi-channel backup images rely more on the global fingerprint cache for better deduplication since the client-side fingerprint cache is too small to hold the corresponding fingerprints. We recommend configure at least 1GB of RAM for each TB of deduplication storage pool on the MSDP server to achieve a good deduplication rate.
Limiting the MaxCacheSize is necessary to prevent MSDP FP fingerprint caching mechanism from over consuming the RAM and creating the potential memory starvation as more unique fingerprints are created and cached. Note that beside FP cache, the system RAM is also needed for other software components running on the server, including OS, File System and other NetBackup processes, such as bptm, bpdm, and so on.
In addition, memory is also needed to maintain the following MSDP objects:
Cloud tier fingerprint cache
Data stores
Task related spooler memory
Memory manager
Restore prefetch pools
Compaction operations
Reference database operations
Request queues