Best practices: Disk pool configuration - setting concurrent jobs and maximum I/O streams
Setting Maximum Concurrent Jobs and Maximum I/O Streams are an important means of throttling workloads where both AdvancedDisk or an MSDP disk pool is the target.
The correlation between Maximum Concurrent Jobs set on the storage unit, and Maximum I/O Streams set on the disk pool is significant. The Maximum I/O Streams setting on the disk pool, whether MSDP or AdvancedDisk, sets the maximum number of streams that can run concurrently on that disk pool, regardless of the Maximum Concurrent Jobs setting on the storage unit, even if multiple storage units exist referencing the same disk pool.
In the case that there are multiple storage units that reference the same disk pool, the total number of concurrent jobs that is configured to run is equal to the total of the Maximum Concurrent Jobs that is set across each storage unit referencing that specific disk pool. Whatever that total Maximum Concurrent Jobs number is, the maximum jobs that can actually be run is a number that is less than or equal to the Maximum I/O Streams set on the disk pool.
For MSDP pools, it is recommended that the Maximum Concurrent Jobs sum should not be greater than 90% of the Maximum I/O Streams set on the disk pool, or by subtracting 10 from the total, whichever is greater.
For example, a single MSDP pool with Maximum I/O Streams set to 200 should have a total Maximum Concurrent Jobs value not greater than 180. If more than one storage unit references that MSDP pool, the sum across all those storage units should be 180 or less. Meaning that in this example, a single storage unit referencing that disk pool would have Maximum Concurrent Jobs set to 180. If there were two storage units that referenced that same MSDP pool, then that total Maximum Concurrent Jobs could be 100 set on one and 80 set to the other, or any combination totaling no more than 180.
In the case of AdvancedDisk disk pools, we recommend staying with the 90% recommendation, as that still reserves some streams for duplication.
For example, in the case of an AdvancedDisk pool with Maximum I/O Streams set to 30, the Maximum Concurrent Jobs should be set to no more than 27.
That said, to minimize complexity, we recommend that the use of multiple storage units referencing the same disk pool is limited, and ideally avoided.
The 10% of the Maximum I/O Streams that is reserved is specifically used for secondary operations, like the use of A.I.R. with MSDP, or duplication activities which are supported with both AdvancedDisk and MSDP.
One of the most common backup performance issues is caused by a mismatch between the Maximum I/O Streams and Maximum Concurrent Jobs. This configuration error often results in throughput bottlenecks that can prevent workloads from completing within their allotted backup windows.
When the Maximum I/O Streams is set on the disk pool, the default is set to 2 per volume. It is recommended that you set this value to 10 I/O streams per independent LUN. For example, if you have 6 LUNs per disk pool, set the value to 10 and the total Maximum I/O streams would be 60 (6 x 10). However, a disk pool created for MSDP works differently. The entire disk pool is considered to be one volume regardless of the number of LUNs included. Therefore, for a 6 LUN MSDP disk pool, the value should be set to 60. .
The recommended way to determine the ideal setting for Maximum I/O Streams is to start low (proportional to the number of volumes in the disk pool), monitor compute, I/O, and network bandwidth, and increase to the point that jobs are not waiting in queue for more than a minute before becoming active, and while ensuring that compute, I/O, and network bandwidth does not become depleted. Remember when the Maximum I/O Streams setting is changed on a disk pool, that Maximum Concurrent Jobs should be updated based upon the aforementioned guidance. Try to keep the aggregate media server hardware resource usage under 75%.
It is not recommended that Maximum I/O Streams ever be set to unlimited.
More information is available in the following technical article:
NetBackup - Configuring the Max Jobs and Max IO Streams settings