Backup jobs become unresponsive and consume significant space on the temporary staging location.
NetBackup Cloud object store data protection feature uses the ListObjects S3 API to iterate over the list of objects to further read and back up the objects in a bucket. The ListObjects S3 API returns up to 1000 objects per request in lexicographical order, based on their key names and the NextContinuationToken. This NextContinuationToken is used for pagination. For example, for a ListObjects S3 API call, to get the next set of 1000 objects and a new NextContinuationToken is used to get the subsequent page.
For certain Cloud object store providers, like Hitachi, the NextContinuationToken does not work correctly if the object names contain certain special characters, potentially hinders backup performance.
This behavior disrupts the cos_sqllite database that NetBackup uses in the temporary staging area. This database stores the object list for a backup job that is in progress. Because of this disruption, the cos_sqllite database drastically grows in size, filling up the disk space in the temporary staging area. This leads the NetBackup jobs to slow down and eventually fail.
Workaround:
Reconfigure the NextContinuationToken in the
ListObjects S3API calls to return the proper value for each batch.Cancel the existing backup job and retry backup.