How to control system variables for consistent testing conditions
For reliable performance evaluation, eliminate as many unpredictable variables as possible to create a consistent backup environment. Only a consistent environment can produce reliable and reproducible performance measurements. Note that it is essential to ensure that the performance of the backup environment is reproducible. Without it, you won't know if the performance difference is due to tuning or just the run-to-run variation.
This topic explains some of the variables to consider as they relate to the NetBackup server, the network, the NetBackup client, or the data itself.
Table: System variables to control for testing
Variables | Considerations for controlling |
|---|---|
Server variables | Eliminate all other NetBackup activity from your environment when you measure the performance of a particular NetBackup operation. Areas to consider include the automatic scheduling of backup jobs by the NetBackup scheduler, and background tasks that wake up periodically, such as CRQP (Content Router Queue Processing), which by default wakes up twice every day at 00:20 AM and 12:20 PM. CRQP, compaction, and CRC check may negatively impact the measured performance When policies are created, they are usually set up to allow the NetBackup scheduler to initiate the backups. The NetBackup scheduler initiates backups according to the following: traditional NetBackup frequency-based scheduling, or on certain days of the week, month, or other time interval. This process is called calendar-based scheduling. As part of the backup policy, the indicates when the NetBackup scheduler can start backups using either frequency-based or calendar-based scheduling. When you perform backups to test performance, this scheduling might kick in and interfere with the performance test. The NetBackup scheduler may initiate backups unexpectedly, especially if the operations you intend to measure run for an extended period of time. See Running a performance test without interference from other jobs. |
Network variables | Network performance is key to optimum performance with NetBackup. Ideally, you should use a separate network for testing, to prevent unrelated network activity from skewing the results. In many cases, a separate network is not available. If not, ensure that non-NetBackup activity is kept to a minimum during the test. If possible, schedule the test when backups are not active. Even occasional or sudden increases of network activity may be enough to skew the test results. If you share the network with production backups occurring for other systems, you must account for this activity during the test. Another network variable is host name resolution. NetBackup depends heavily upon a timely resolution of host names to operate correctly. If you have any delays in host name resolution, try to eliminate that delay. An example of such a delay is a reverse name lookup to identify a server name from an incoming connection from an IP address. You can use the HOSTS (Windows) or /etc/hosts (Linux/UNIX) file for host name resolution on systems in your test environment. |
Client variables | Make sure that the client system is relatively quiescent during performance testing. A lot of activity, especially disk-intensive activity such as Windows virus scanning, can limit the data transfer rate and skew the test results. Do not allow another NetBackup server, such as a production server, to access the client during the test. NetBackup may attempt to back up the same client to two different servers at the same time. The results can be severely affected for a performance test that is in progress. Different file systems have different performance characteristics. It may not be valid to compare data throughput on Linux/UNIX VxFS or Windows FAT file systems to Linux/UNIX NFS or Windows NTFS systems. In addition, different OS releases may introduce changes that can also affect system performance. For such a comparison, factor the difference between the OS releases and the file systems into your performance tests and into any conclusions. |
Data variables | Monitoring the data you back up improves the repeatability of performance testing. If possible, move the data you use for testing to its own drive or logical partition (not a mirrored drive). Defragment the drive before you begin performance testing. For testing restores, start with an empty disk drive or a recently defragmented disk drive with ample empty space. For testing backups to tape, always start each test with an empty piece of media, as follows:
When you test restores, always restore from the same backup image on the media to achieve consistent results between tests. A large set of data generates a more reliable, reproducible test than a small set of data. Startup and shutdown overhead within the NetBackup operation will probably skew a performance test with a small data set. These variables are difficult to keep consistent between test runs and are therefore likely to produce inconsistent test results. A large set of data minimizes the effect of startup and shutdown times. Design the data set to represent the makeup of the data in the intended production environment. If the data set in the production environment contains many small files on file servers, the data set for the tests should also contain many small files. A representative data set can more accurately predict the NetBackup performance that can be expected in a production environment. The type of data can help reveal bottlenecks in the system. Files that contain non-compressible (random) data increase the amount of I/O against the storage media and affect the backup performance, particularly when the data has low deduplication ratio. As long as the other components of the data transfer path can keep up, you may find the storage media is the bottleneck. On the other hand, files containing highly-compressible data can be processed at higher rates when hardware compression is enabled. The result may be a higher overall throughput and may expose the network as the bottleneck. Many values in NetBackup provide data amounts in kilobytes and rates in kilobytes per second. For greater accuracy, divide by 1024 rather than rounding off to 1000 when you convert from kilobytes to megabytes or kilobytes per second to megabytes per second. |