Known Issues

If a file is being copied to the StorNext file system using Windows Explorer and Windows Explorer crashes before it finishes copying all the data, the file might contain data blocks from old, deleted files. This problem occurs because Windows Explorer sets EOF to the size of the file before it writes the data to the file. This leaves a gap of uninitialized data in the file.

Note: This problem can also occur with other programs that set EOF beyond the end of data.

This problem does not occur if Windows Explorer encounters an error while writing the file; Windows Explorer will delete the partially written file.

Workaround:

To prevent this problem from occurring on StorNext, you can use the StorNext "client configuration" application's advanced mount option "Restrict Pre-allocation API" on Window systems and the "protect_alloc=yes" mount option on Linux systems. This option will set the unwritten parts of the file to zero. When this option is set, non-root users are unable to use the preallocation ioctl. This option also implies sparse=yes.

For more information on this option, see the man page mount_cvfs(8). The sparse option will introduce some overhead when using Windows Explorer. Before setting the protect_alloc option, see the sparse option in mount_cvfs(8) for a description of how it changes StorNext behavior.

StorNext 5.4.0.x incorrectly allowed the Unix ID Mapping type to be set to none when the Security Model is set to acl. As a result, file systems fail to start when the Unix ID Mapping type is set to none when the Security Model is set to acl.

Beginning with StorNext 6, the FSM does not start when this invalid combination of settings is used.

Workaround:

To prevent this issue, set the Unix ID Mapping to either winbind or algorithmic for any file system where the Security Model is set to acl. You can make the adjustment before or after upgrading.

A StorNext NAS client cannot rename a file if the file has the read-only attribute set. This problem only affects StorNext NAS clients.

Workaround

A StorNext NAS client must remove the read-only attribute before it can rename the file.

If you set the parameter audit=true in the file system configuration file, the result causes all I/O activity, including reads, to be recorded in the mdarchive. This can greatly increase the number of updates applied to the mdarchive which, in turn, increases the amount of mdarchive compaction activity by the FSM.

There is one instance where this increased compaction activity caused the mdarchive to grow beyond the capacity of the HA shared file system.

Workaround

To work around this issue, Quantum recommends you increase the parameter metadataArchiveCache from the default 2 GB to at least 4 GB, but preferably larger when setting audit to true in the configuration file.

If you access StorNext file systems from Apple Xsan clients, then you might encounter I/O error messages in the system log that do not contain details about real I/O errors detected on the Xsan client.

Workaround

If you encounter the errors on an Xsan client, contact Apple.

File retrieves from media to disk can be suboptimal for fast tape drives like the Oracle STK T10K drives. This scenario can occur when the retrieve event is initiated on a host that is different from the host running the mover process, which requires the use of synchronous direct I/O.

Workaround:

To work around this issue and achieve optimal performance for both file stores and retrieves with the T10K drives, increase the default I/O size used by the mover process and make the mover process use asynchronous buffered I/O when the use of synchronous direct I/O is not required, using the following steps:

Note: This workaround might also help improve the performance of the faster LTO drives by updating the FS_LTO_BLOCK_FACTOR system parameter.

Note: Changes to FS_xxx_BLOCK_FACTOR only affects tapes formatted after the change.

1. Change the FS_T10K_BLOCK_FACTOR system parameter from 8 to 32 by adding the following entry to
   /usr/adic/TSM/config/fs_sysparm_override:

   FS_T10K_BLOCK_FACTOR=32;

   Note: The T10K default I/O block size is 512 KB or 8 * 64 KB. With the block factor changed to 32, the new T10K I/O block size will be 2 MB or 32 * 64 KB. Presently, the FS_T10K_BLOCK_FACTOR system parameter must not be set to a value that exceeds 32.

2. Restart Storage Manager to ensure the change in Step 1 goes into effect:

   # tsmstop
   # tsmstart

3. Verify the FS_T10K_BLOCK_FACTOR system parameter contains the new value:

   # showsysparm FS_T10K_BLOCK_FACTOR
   FS_T10K_BLOCK_FACTOR=32
4. Save the current copies of your /etc/fstab on the MDCs and the DDM clients.

5. Modify /etc/fstab on the MDCs and the DDM clients to use the auto_dma_write_length and auto_dma_read_length mount options as follows:

   snfs1 /stornext/snfs1 cvfs
   rw,auto_dma_write_length=16m,auto_dma_read_length=16m 0 0
6. Unmount and re-mount your file systems.

7. Use new T10K media to store a copy of the file from the disk.

   Note: Step 7 is very important; when the new copy is made to the new tapes, the new tapes are labeled with a 2 MB block size, which is used for subsequent writes or reads to and from the media. Tapes on which fsformat was run before the change will use the block factor in use at that time. This change will not impact those tapes.

Executing the command snbackup -s while a full or partial backup is running might result in a message that
/usr/adic/TSM/internal/locks/backup.lf is in an invalid format.

This is due to the snbackup -s process reading the backup.lf status file while the backup process is updating it.

Workaround:

Ignore the message; to clear-up the process, re-execute the command snbackup -s (provided that the backup is not writing to the backup.lf status file while snbackup -s is trying to read it again).

When copying files between media using the CLI command fsmedcopy, the file is not re-segmented to match the segment size of the destination media. Rather, the original segments are copied to the target media type and the distribution of segments across destination media will, therefore, be the same as the distribution on the source media.

Note: This behavior might cause file data segment distribution to be sub-optimal on the destination media.

Workaround:

Currently, a workaround does not exist for this known issue.

Your DDMs might experience a timeout if you try to connect to the database. The issue is identified by an error log in /usr/adic/TSM/logs/tac which contains the text:

Workaround:

1. Increase the database connection timeout value by adding the following line to /usr/adic/mysql/my.cnf under the section labeled [mysqld] connect-timeout=240.
2. Cycle the Storage Manager in order to pick up the updated timeout value.

If you have the IBM APFO driver installed and configured, then when you perform an fsmedread operation of a partial tape block from a full tape block, the operation can fail with errno=12.

Note: This issue affects all IBM APFO versions 3.0.19 and earlier, and has an impact primarily on disaster recovery procedures.

Workaround:

To correct this, perform an fsmedread operation without the IBM APFO driver.

Using autofs to mount a StorNext file system on an MDC is not supported when the same file system also has a native mount point.

For example, if the StorNext file system snfs1 is mounted as /stornext/snfs1, then the MDC should not also have an autofs configuration that mounts it on the MDC in another location such as /space/snfs1. Doing so, causes the fsCheckAffinities and fsCheckTsmFilesystemConfig health checks to fail and generate RAS tickets.

Additionally, this might cause the StorNext GUI to fail unexpectedly for certain operations.

Workaround:

There is currently no workaround for this issue. If you experience this issue, contact Quantum Technical Support.

There is a known issue where stripe group expansion using the StorNext GUI can fail and requires manual intervention to restore file system operation.

Stripe group expansion allows an additional disk (LUN) to be added to an existing stripe group, growing the file system user data space.

The other way to add space to a file system is to create a new stripe group and add this to the file system.

When the GUI executes stripe group expansion, it stops the file system, modifies the configuration file and runs cvupdatefs. If cvupdatefs takes more than 5 minutes to complete, the GUI kills the cvupdatefs utility and reports an error. At this point the file system does not start because the configuration file does not match the current state of the metadata.

Workaround:

To address this issue, you can do one of two things:

• Optimally, you would not attempt the stripe group expansion at all. Instead add a new stripe group to the file system.
• If stripe group expansion is deemed necessary, use the cvupdatefs CLI directly instead of through the GUI. This method does not have any time limitations.

In the case that the StorNext GUI attempt was made and hit the 5 minute timeout, file system operations can be resumed by performing the following:

1. Restore the previous version of the configuration file. This can be found in the following directory:

   /usr/cvfs/data/<fs>/config_history
2. Run the cvfsck utility to verify and potentially correct the metadata.
3. Start the file system.

You cannot use the StorNext GUI to configure or modify a policy class with the GOOGLES3 media type. The process fails with an error similar to the following:

Workaround:

To workaround the issue, Quantum recommends you use the GOOGLE media type rather than the GOOGLES3 media type, not only because the GOOGLES3 media type provides very limited functionality compared to the GOOGLE media type, but also because you cannot use the StorNext GUI to configure or modify a policy class with the GOOGLES3 media type. However, if you have to use the GOOGLES3 media type, then use the CLI fsaddclass command to configure a policy class and the fsmodclass command to modify an existing policy class.

The status and capacity monitors for file systems, libraries, storage disks, and tape drives does not display data on the StorNext GUI home page. This issue is caused as a result of missing font RPM packages on your system.

Workaround:

To workaround the issue, Quantum recommends you do the following:

1. Install the java-1.8.0-openjdk package on your system:

   yum install java-1.8.0-openjdk

2. Restart the Apache Tomcat service:

   systemctl restart stornext_web

If you have an open StorNext user interface browser session, then the browser page might flash due to an ICEfaces error. This occurs when the browser page displays partial data rows, empty data, or all the data.

Workaround:

To workaround the issue, Quantum recommends you do the following:

1. Log out of your StorNext user interface browser session(s).

2. Close all browser tabs and windows.

3. Open a new browser window and access the StorNext user interface (see Access the StorNext User Interface (new UI introduced in StorNext 7.0.1) or Access the StorNext Software GUI).

The Safari browser becomes unresponsive when you attempt to configure an Email server using the StorNext GUI.

Workaround:

To workaround this issue, perform the following procedure:

1. Shut down the Safari browser window(s).
2. Restart the Safari browser, and then retry the operation.
3. Uncheck the Verify SMTP Server Connectivity box, and then retry the operation.
4. Set Authentication to NONE, and then retry the operation.
5. Disable the Safari User names and passwords AutoFill under Safari > Preferences > AutoFill, and then retry operation.

After an appliance firmware upgrade, you might be unable to use previously functioning tape devices because the lin_tape device driver was automatically unloaded during the upgrade.

Workaround:

To workaround this issue, rebuild the lin_tape device driver as shown in the following example:

Quantum installs a self signed certificate (valid for 365 days) in /usr/cvfs/config/certs/ only when installing a snfs-common RPM and if a preexisting certificate file does not exist.

If you use monitoring software (for example, Zabbix), an expired self signed certificate is flagged after it expires.

Workaround:

To workaround this issue, do the following to update an expired self signed certificate:

1. Stop CVFS on the affected client:

   # service cvfs stop

2. Update the certificate:

   # cd /usr/cvfs/config/certs

   # mv server.crt server.crt.orig

   # mv server.key server.key.orig

   # openssl req -x509 -newkey rsa:2048 -keyout server.key -out server.crt -days 365 -subj "/C=US/ST=California/L=SanJose/O=Quantum/OU=Demo/CN=`hostname`" -nodes

3. Start CVFS on the affected client.

   # service cvfs start

The rpm database might corrupt if an rpm operation or a query is unexpectedly interrupted by a signal, reboot, or a system crash on an Xcellis XWD environment in particular, but could also impact a StorNext MDC in a customer-supplied environment and anywhere that the Appliance Controller software or the Unified User Interface (UUI) software runs.

Querying the database might result in errors and report that an installed package is not installed. For example:

These query errors could result in the inability to start StorNext or other Quantum services.

On rare occasions, you might see corruption of the rpm database after a reboot of the system while an rpm activity occured. StorNext, Appliance Controller, and UUI invoke periodic rpm queries that could be running when a reboot occurs that could lead to corruption. The rpm database corruption is more likely to be seen in the event of an ungraceful reboot resulting from a kernel panic, power outage, or SMITH reset, or upon sending the kill signal to a running rpm command which can happen via systemd as part of a normal shutdown/reboot sequence.

Workaround:

Run the following command to repair the rpm database:

After you rebuild the database, run a query to verify the database is repaired:

You might see the following error messages when the Rocky 8 kernel first boots:

Workaround:

You can ignore the error messages.

Note: Your system might require an updated BIOS from the manufacturer to fix and eliminate the warnings.

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

The operating system conversion portion of the upgrade to StorNext 7.2.0 might fail if non-UTF8 files or directories exist in the / directory.

Workaround:

Before you initiate the upgrade on your system, create and execute the python script below on each of the nodes in the system.

1. Create a file labeled, checkNonUtf.py in the /tmp directory with the following contents:

   #!/usr/bin/python3
   import os
   for path in os.listdir('/'):
           try:
                   path.encode('utf-8')
           except UnicodeEncodeError:
                   # print path, or attempt to rename file
                   print('Non-UTF8 file or directory found on "/"')
                   exit(1)
   print('No non-UTF8 files or directories found on "/"')

2. Run the following command on each of the nodes in the system to execute the script:

   python3 checkNonUtf.py

3. If the script reports Non-UTF8 file or directory found on /, then move these files or directories out of the / directory or rename them to be UTF8 named files.

   Note: If these files/directories are required on the system, move/rename these back after the upgrade.

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

After the upgrade to StorNext 7.2.0, attempts to use the email notification option to the sntier command fail due to the /bin/mail application not being installed on the system.

Workaround:

If your require this feature, you can download and install the Rocky 8 version of the mailx RPM on the Xcellis nodes.

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

Unprivileged users are not allowed to log in during the upgrade process until the platform upgrade to StorNext 7.2.0 completes.

Workaround:

Before you initiate the upgrade on your system, do the following:

1. Run the following commands as a root user on the node:

   cp /usr/lib/tmpfiles.d/systemd-nologin.conf \

   /etc/tmpfiles.d/

2. After the upgrade is complete, remove the file in /etc/tmpfiles.d/:

   rm /etc/tmpfiles.d/systemd-nologin.conf

Note: Note: The upgrade to StorNext 7.2.0 might require up to three hours to complete on an Xcellis node. After the initial part of the upgrade which performs the operating system conversion, you might want to connect to the node from an SSN terminal as user stornext and monitor the upgrade logs for the remaining portion of the platform upgrade. Performing this workaround before the upgrade is initiated allows this.

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

If your environment includes an Xcellis Workflow Extender that originally had StorNext version 6.1.1 or earlier installed, the upgrade will fail.

Workaround:

Quantum recommends you contact Quantum Support (see Contacting Quantum Support) for a software patch before you upgrade to StorNext version 7.2.0.

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

On environments with a large number of LUNs visible on the Xcellis node, the node might run out of memory after the upgrade to StorNext 7.2.0.

Workaround:

Before you initiate the upgrade on your system, set the nr_requests value in /usr/cvfs/config/deviceparams to a value of 256. The value should persist after the upgrade is complete.

Example

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

The upgrade to StorNext 7.2.0 might fail in some cases due to a lack of free space available in the /var file system to complete the operating system conversion portion of the upgrade.

Workaround:

Before you initiate the upgrade, verify the /var file system contains at least 16 GB of free space on each node.

If your system is running StorNext version 7.2.0 (or later) and Appliance Controller version 4.1.0, you might receive the following operation failure error when you run the qtmcontroller iscsiadm command, and your card link goes down:

Workaround:

To workaround this issue, run the following Appliance Controller command:

Caution: Before you upgrade to StorNext version 7.2.0, Quantum strongly recommends you review this known issue and applicable workaround.

Upgrades on systems originating from StorNext version 6.0.6 or earlier might fail in the pre-upgrade check when upgrading to StorNext version 7.2.0 due to the presence the obsolete rpms openssl098e and tg3 being installed. The following message might appear in the /var/log/DXi/upgrades/Node1.preupgradechk.out or /var/log/DXi/upgrades/Node2.preupgradechk.out log file if these rpms are present:

Note: This issue only affects upgrades to StorNext version 7.2.0 and is fixed in StorNext version 7.2.4. This issue does not impact StorNext version 7.2.2 as that release is only for fresh installations on Xcellis Workflow Director Gen 3 and Xcellis Workflow Extender Gen 3 systems.

Workaround:

To workaround this issue, run the following commands to manually remove the openssl098e and tg3 rpms, if they are installed prior to upgrading to StorNext version 7.2.0:

The CLIENTS page of the UUI displays duplicate clients.

Workaround:

To workaround this issue, do the following.

1. On the APIGW page of the UUI, click Deregister Endpoint for the endpoint of the duplicate client.

2. Restart the Unified Connector on the client that contains the duplicate entries.

If your system is running StorNext 7.1.1 or earlier, the label N/A is displayed in the Reserved column when you display strip group information in the UUI.

Workaround:

There is currently no workaround for this issue. If you experience this issue, contact Quantum Technical Support.

When you upgrade the UUI software, the upgrade process might appear to be complete but the Quantum Unified UI Software Upgrade page does not update due to an update-service request that continuously fails with a 404 status code.

Workaround:

To workaround this issue, execute the following command as root on the node where the UUI is running:

When you navigate to the Metrics > Performance and Metrics > Utilization UUI page, you might experience a long delay for the page to display properly.

Workaround:

To workaround this issue, refresh your browser page.

If you navigate to the Metrics > Utilization page, and attempt to download a Historical Usage graph, the image file for the graph might not display properly.

Workaround:

Do the following to workaround this issue:

1. Click the Print chart icon in the UUI.

2. Save the document as a PDF.

When you navigate to the Metrics > Performance and Metrics > Utilization UUI page, you might receive an error when you view a widget in full screen mode and attempt to print or export a data chart.

Example

Workaround:

To workaround this issue, close your browser window and then reopen a new browser window.

If you navigate to the Metrics > Performance page, and attempt to save any data graph as a PNG file or a JPEG file, the image for the graph contains a black background with black text, which results in an illegible data graph.

Workaround:

To workaround this issue, download the data graph in a different format, such as a PDF document, or an SVG vector image.

When you upgrade the UUI software, the upgrade process might appear to be complete but the UUI login page displays an error, preventing you from accessing and logging in to the UUI.

Workaround:

To workaround this issue, execute the following command:

During a fresh installation of the StorNext 7.2 software on the secondary node of an Xcellis system, the UUI Unified Connector client registration is missing after StorNext is converted to an HA pair. On the top navigation menu of the UUI, click CLIENTS to verify the issue; you might notice a client registration for both the primary node and the secondary node.

Workaround:

If you are missing the client registration for the secondary node, Quantum recommends you perform either of the following options to workaround this issue.

Workaround Option 1

As the root user, SSH to the secondary node and execute the following command:

Workaround Option 2

Reboot the secondary node.

In the UUI (Clients > Software Downloads), the Software Downloads dialog displays a DDM client binary file that is not supported for Rocky 9, Alma 9, and Oracle 9 clients in this release of StorNext.

Example

Workaround:

To workaround this issue, ignore the DDM client binary file that appears in the UUI and do not attempt to download the binary file. The DDM client is not supported for Rocky 9, Alma 9, and Oracle 9 clients in this release of StorNext.

In the UUI (Metrics > Dashboards), the error below might appear in your widget when you perform the following steps:

1. On the top navigation menu, click Metrics, and then click Dashboards on the left navigation menu.

2. Create a new dashboard.

3. Add a widget for a file system to the dashboard you created in Step 2.

4. Stop and start the file system.

5. On the top navigation menu, click Metrics, and then click Dashboards on the left navigation menu.

Example of error

Workaround:

To workaround this issue, remove the existing widget from dashboard and add a new widget.