Q-Cloud Tuning Considerations
StorNext’s capability to push data may exceed the bandwidth capability of your network for Q-Cloud. This may consume your entire WAN bandwidth and can cause packet lost leading to disconnections between StorNext and Q-Cloud destinations.
Review the following sections and consider the tuning suggestions.
- By default, Storage Manager policies retain 10 inactive versions of files, in addition to the current (active) version. As this will consume up to 11 times the storage of the active version, most users will want to save 0 inactive versions.
- By default, Storage Manager policies retain files in the database, and on media, for up to seven years. To get cost savings quickly after deleting unwanted files, check the Clean Database When File Removed checkbox in the policy.
- To free up space, delete files and execute the command
fsclean
to clean the database and Q-Cloud. - Q-Cloud Vault is intended for the coldest archive data: it offers a very low per-TB storage cost, but carries charges for data out, retrieve, and deletion (or change) of data in less than 90 days. Use Q-Cloud Vault for unchanging files that will very rarely be accessed. Time to first byte is up to 5 hours with Q-Cloud Vault, and retrieves must be manually throttled to control retrieve fees. For additional information, see Q-Cloud Retrieval.
- Q-Cloud modification to policy class settings may be advisable if many files change frequently such that, in addition to lowering Maximum Inactive Versions, to have the Storage Manager policy Maximum File Age (Hours) set sufficiently high to minimize storing files that are likely to change soon. Also, for Q-Cloud Vault, writing many versions of files may also imply many deletions of old files, which may lead to deletion or change fees. Of course, higher Maximum File Age (Hours) settings comes with risks if something happens to the file before it is stored.
- Q-Cloud Archive is intended for data that changes more frequently than Q-Cloud Vault: it has a higher per-TB storage cost, but does not have retrieve fees or early deletion fees.
You can manage the TCP/IP traffic by tuning TCP buffers. For example, where we have seen server side disconnections, reducing the size of tcp_rmem
or tcp_wmem
has improved connection reliability. Quantum does not have any specific settings that are known to work for all situations as it is very dependent on your networking environment.
Using network interface bonding can improve both network bandwidth and reliability. If you have more than one network interface on the host to Q-Cloud destinations, you may want to consider using NIC bonding.
The current default concurrent streams set for each Q-cloud device is 48 concurrent streams which may be too high for your WAN. If this is too high, you can lower this stream count. The optimal stream count depends on your network bandwidth and reliability.
You may want to consider isolating Q-Cloud traffic where you can configure QOS for the overall WAN bandwidth allowed to be consumed. StorNext does not contain any configurable parameters to manage the total bandwidth it consumes.
A valid time on your StorNext system that is within 15 minutes of Q-Cloud is required. If not, the request will fail with the RequestTimeTooSkewed
error code. Quantum recommends you configure NTP on your system.
If the network is unreliable, and the system is experiencing a large amount of connectivity or client/server connection resets, Storage Manager may take the iopath to the Q-Cloud destination off-line. When this happens you will have to manually execute the command fschstate
to enable the path on‑line.
To configure Storage Manager to perform continuous retries, you can set the following parameters in the /usr/adic/TSM/config/fs_sysparm_override
file:
FS_THRESHOLD_INC_NUM
: Amount to increment each time a drive device is used with errors encountered.FS_THRESHOLD_DEC_NUM
: Amount to decrement each time a drive device is used without errors.FS_DRIVE_ERR_THRESHOLD
: Threshold value, which when equaled or surpassed, drives will be taken off-line.
The default values are:
FS_THRESHOLD_INC_NUM=5;
FS_THRESHOLD_DEC_NUM=1;
FS_DRIVE_ERR_THRESHOLD=20;
When the FS_DRIVE_ERR_THRESHOLD
is met, Storage Manager will take the path off-line. To configure Storage Manager to never take a path off-line, you can set FS_THRESHOLD_INC_NUM
to 0
.
Note: These are Global parameters that affect all Storage Manager devices, not just Q-Cloud paths.
The Compression or Encryption features add additional load on a metadata controller’s resources. Quantum recommends the Stream Count for Q-Cloud storage destinations be configured for two to four streams. Increasing the Stream Count with Encryption and Compression increases the CPU load on the metadata controller. For additional information on Compression and Encryption, see Client Side Compression and Encryption.