StorNext GUI Appliances: Upgrade StorNext Software/System Firmware

For systems with a StorNext GUI, the Firmware Upgrade page allows you to perform a firmware upgrade for the server. Depending on the version of firmware being applied, the upgrade includes updates to the server hardware firmware (if applicable), server OS (if applicable), and StorNext software.

This applies to Xcellis Workflow Director, Xcellis Foundation, aiWARE for Xcellis Embedded, Artico, Pro Foundation, M660, and M440 systems.

Note: The following StorNext appliances Can NOT be upgraded to StorNext 7.0.1 and later:

All aiWARE for Xcellis models (including Embedded (DAE), Standard, and High Performance), Artico (R520), Xcellis Workflow Extender (R520), Pro Foundation, M660, and M440 systems (which function as StorNext metadata controllers).

STOP! If you are upgrading an Xcellis Workflow Extender or G300 gateway system, use the StorNext Gateway Appliances: Upgrade StorNext Software/System Firmware process instead.

STOP! If you are upgrading an Xcellis Workflow Extender or Xcellis Workflow Director to StorNext version 7.2.0 or later, review the known issues in the StorNext Installation, Upgrade, HA, and Other Known Issues section for suggested workarounds for known upgrade issues before you initiate the upgrade. Also verify the items below:

Important Considerations

During the initial start-up of StorNext, after an upgrade to a new version of StorNext software, the Storage Manager database must be updated. This process is completed as part of the initial start-up of the StorNext software. StorNext file systems are accessible while the database schema is being updated, but Storage Manager functionality (including stores and retrieves) are offline.

Do NOT interrupt StorNext services while the database is being updated. Interrupting the database schema update might result in an inconsistent database, and might require assistance from Quantum Support to repair or restore the database.

Note: When you upgrade from StorNext 6.x to StorNext 7.x, the update takes approximately one hour for every hundred million entries in the filecomp tables.

  1. Log in to the server node:
    1. Open an SSH connection to the appropriate server and use the IP address assigned to the node on the Management or LAN Client network, or use the Service Port IP address.

      2. Note: The service port is no longer a dedicated service port. This port might be configured by the user for use as a metadata or client port with a different IP configuration. If this port has been reconfigured by the user, then command line access over a network connection requires an appropriate IP address for your configured system on that network.

      • Node 1 : <service port IP address>, the default service port IP address is 10.17.21.1
      • Node 2 : <service port IP address> (dual-server node, or supported "HA-ready" single-server node StorNext appliances), the default service port IP address is 10.17.21.2

    2. Initiate an ssh session to the system using Terminal or PuTTY:
      1. Log in to the command line using the following credentials:
        • User name: stornext
        • Password: <StorNext user account password>

          Note: password is the default password for the stornext user account. If the password has been changed, use the current password. The first time you log in, you are prompted to change the password to a different one.

      2. Enter sudo rootsh to gain root user access.
      3. Enter the password for the stornext user account again.

  2. Open the Service Menu:

    1. Launch the Service Menu. Enter:

      2. sh /opt/DXi/scripts/service.sh -admin

      The Service Menu displays.

  3. Navigate to Hardware Configuration > Setup IPMI > Reboot IPMI.

  4. At the prompt to reboot the IPMI controller, press y (yes).

  5. The BMC/IPMI will reboot.

    6. For single-server node systems, the procedure is complete.

  6. For dual-server node systems, reboot the BMC on the other server node using the steps you just completed on this server node.

This applies to Xcellis Workflow Director (Gen 2 and R630), Xcellis Workflow Extender (Gen 2 and R630), Xcellis Foundation, and Artico (R630) systems.

Upgrade time estimates

Component Upgrade Time Estimate
(approximate minimum)
Server node

150 to 180 minutes*
* Note: Server node reboot times can vary widely, depending on the size of the SAN in your environment, whether or not server node firmware drivers or the operating system need to be upgraded, and the specific StorNext Release installed prior to beginning the upgrade.

Note: The upgrade might require 40 minutes or more during the Leapp process.

This applies to and systems.

Upgrade time estimates

Component Upgrade Time Estimate
(approximate minimum)
Server node

30 to 90 minutes*
* Note: Server node reboot times can vary widely, depending on the size of the SAN in your environment, whether or not server node firmware drivers or the operating system need to be upgraded, and the specific StorNext Release installed prior to beginning the upgrade.

This applies to and systems.

Upgrade time estimates

Component Upgrade Time Estimate
(approximate minimum)
Server node

30 to 90 minutes*
* Note: Server node reboot times can vary widely, depending on the size of the SAN in your environment, whether or not server node firmware drivers or the operating system need to be upgraded, and the specific StorNext Release installed prior to beginning the upgrade.
Component Upgrade Time Estimate
(approximate minimum)
Server node(s)

30 to 90 minutes per node*

Metadata Array

This applies to M660, M440, and M330 Metadata Appliances.

20 minutes***
* Note: Server node reboot times can vary widely, depending on the size of the SAN in your environment, whether or not server node firmware drivers or the operating system need to be upgraded, and the specific StorNext Release installed prior to beginning the upgrade.

This section includes examples of internal upgrades testing. Applying a firmware upgrade from StorNext 6.1.1 to 6.2 at Quantum, one of our dual-node Xcellis Workflow Director Gen 3 systems took 4 hours to complete the upgrade.

Note: This data reflects the duration of each of these items, per server node. This will take twice as much time for these items on dual-server node systems, since these have to be applied on both servers, sequentially.

This applies to and systems.

Example software upgrade time required:

Software Component
(in sequential order)		 Average Test Duration
(in minutes)
VerifyRPMs
RemoveRPMs
CreateRepo
QuantumBase
ExtraRPMs
StorNext
StorNextClient
StorNextGatewayAgent
OSUpdate
∞SANtricity 00:27
AdditionalRPMs
ConfigPatches
RebuildInitramfs
Post-Install
FirmwareBundle/Reboot(s)
PHPUpdate
‡NASUpdate
Total ~
∞Checks if the metadata array firmware needs to be upgraded, and upgrades it, if needed.
‡Checks if the Appliance Controller needs to be upgraded and upgrades it if, needed or possible.

This applies to and systems.

Software Component
(in sequential order)		 Average Test Duration
(in minutes)
VerifyRPMs
RemoveRPMs
CreateRepo
QuantumBase
ExtraRPMs
StorNext
StorNextClient
StorNextGatewayAgent
OSUpdate
∞SANtricity 00:27
AdditionalRPMs
ConfigPatches
RebuildInitramfs
Post-Install
FirmwareBundle/Reboot(s)
PHPUpdate
‡NASUpdate
Total ~
∞Checks if the metadata array firmware needs to be upgraded, and upgrades it, if needed.
‡Checks if the Appliance Controller needs to be upgraded and upgrades it if, needed or possible.

The following applies to M660, M440, and M330 Metadata Appliances.

Metadata array operations and the StorNext file system are unavailable while metadata array controller firmware is installed, so metadata access will be unavailable during a reboot of the server.

Table 1: System Upgrades to the following StorNext releases require Downtime for Metadata Controller Array Firmware updates.
StorNext Release* Controller Array
Firmware Version**

6.0 up until 6.4.1

08.20.09
5.4.0.4 08.20.09
5.4.0.3 08.20.09
5.4.0.2 08.20.09
5.4.0.1 08.20.09
5.3.2.1 08.20.09
5.3.2 08.20.09
5.3.1 08.20.09

5.3.0***

 08.20.09
5.2.2 08.20.09
5.2.1 08.10.13
5.2.0.2 08.10.13
5.2.0.1 08.10.13
5.2 08.10.13
5.1.1 08.10.13
5.1 07.84.46
5.0.1 07.84.46
5.0 07.84.46
4.7.1 07.84.46
4.7.0.1 07.84.46
4.7.0 07.84.46
4.6.1 07.80.55
4.6 07.80.55
4.3.3 07.80.55
4.3.2 07.80.55
4.3.1 07.80.55
4.3.0 07.80.55
4.2.2.0.1 07.75.17
* Note: Some StorNext releases are not supported upgrade paths. See Upgrade Firmware: Before You Begin.
**Note: If your array firmware has already been upgraded to the current version, it will not be upgraded/replaced. However, if an older version of array firmware was in place before the upgrade, it will upgrade as part of the system firmware upgrade.

*** StorNext 5.3 was the last release that upgraded Netapp firmware. The code is still included in StorNext releases after 5.3, but the array firmware upgrade process is not invoked in later releases.

Example:

If you were upgrading the StorNext firmware from StorNext Release 5.2.1 to StorNext 5 Release 5.2.2 on an M660, the array firmware would be upgraded from 08.10.13 to 08.20.09. This is considered a downtime upgrade.

System firmware updates for the BIOS, BMC, and peripheral cards are upgraded as part of the system updates. The system applies updates to these if needed on the next server boot. Applying the firmware updates may require two or three extra reboots depending on the firmware versions that are currently installed. The firmware updates can take up to 30-40 minutes to apply, plus the time it takes to reboot the system.

Deferring these updates will still apply the StorNext software update, but will take less time since it skips the firmware updates. You can still apply the firmware updates at a later time.

Contact Quantum Support for help setting up firmware deferrals. Reference Defer Appliance Firmware Updates (access requires CSWeb login).

Note: Always upgrade the Appliance Controller software before you upgrade StorNext, especially for StorNext NAS-clustered systems. See How to Upgrade on the Appliance Controller Documentation Center.

Caution: A system running StorNext 6.0.6 or later with a clustered NAS environment should not run NAS/Appliance Controller 1.3.0 or earlier. Running these configurations may produce unpredictable results.

To find the current Release Notes for your system, locate your system type on the Identify My StorNext Appliance, and Access Training and Documentation page and expand that section. You will find links to the Release Notes for your specific appliance.

Before beginning an upgrade, do the following:

  1. * Check that the date and time (NTP) on both server nodes to make sure the time is synced.

    2. Note: If you run the validation process before you click Activate, you have a greater assurance that the system is in a proper state for a successful upgrade. If desired, upload the .fw (firmware) files and validate them prior to a planned maintenance window for the system's upgrade. You can upload and validate files at any time before you click Activate.

    Once a system is upgraded to StorNext 6.2.0 or later, the Validate button on the StorNext GUI Firmware Upgrade page performs a much more extensive upgrade validation check than in previous releases. The complete set of pre-upgrade checks that used to run only when you clicked Activate, are now run. These checks include a checksum and version verification against the upgrade files, and also include a complete check of RPM dependencies as well as potential conflicts on the system which could cause an upgrade to fail. There are also more extensive version checks for StorNext and other packages.

    Beginning with StorNext 7.2.0, the pre-upgrade check validation process also checks for RPM files that cannot be upgraded. Upon validation failure, an Admin Alert containing a list of RPMs found that are not expected on the node(s), is generated. You must manually remove these RPMs from the node(s) before you activate the upgrade. If the system you are upgrading includes the lin_tape RPMs, you must manually remove the RPMs before you start the upgrade process, and then install the RedHat 8 version of the RPMs after the upgrade process is complete.

    If you click the "Validate" button on the StorNext GUI Firmware Upgrade page, this will verify the uploaded .fw files against a checksum for those files. The validate also performs some basic version checking to make sure the uploaded files are valid for the running system.

    Errors reported from the validation check are reported back to the user in a context sensitive pop-up window explaining what failed in the validation check as well as an Admin Alert is generated with similar detailed information about the failure.

  2. Collect a DSET and StorNext snapshot from both servers (for dual-node [HA] systems only) before starting the upgrade.
  3. Check the /etc/hosts files to verify that no manual modifications have been made, and that the entries are correct. If there are, make a backup of those files to refer to after the upgrade is complete.

    4. * For appliances except Xcellis Workflow Extender and G300 systems.

Not all StorNext releases upgrade directly to another StorNext release. As a result, an upgrade to the current version of StorNext may require multiple, incremental upgrades, depending on the version of StorNext currently installed. Consider the following:

  • For information about supported upgrade paths for StorNext 7.x, consult the StorNext 7 Compatibility Guide in the Compatibility Guides section of the PDFs page of the StorNext 7 Documentation Center. If your system is running an earlier StorNext release, consult an earlier version of the StorNext Compatibility Guide that applies to the release your system is running.
  • For information about supported upgrade paths and dependencies for StorNext 6.x, consult the StorNext 6 Compatibility Guide in the Compatibility Guides section of the PDFs page of the StorNext 6 Documentation Center. If your system is running an earlier StorNext release, consult an earlier version of the StorNext Compatibility Guide that applies to the release your system is running.
  • For information about supported upgrade paths and dependencies for StorNext 5.x, consult the StorNext 5 Compatibility Guide in the Compatibility Guides section of the StorNext 5 Documentation Center. If your system is running an earlier StorNext release, consult an earlier version of the StorNext Compatibility Guide that applies to the release your system is running.
  • If your system is running a StorNext release prior to the supported upgrade releases for a given StorNext release, consult an earlier version of the StorNext Compatibility Guide that applies to your specific upgrade.

Within your StorNext network/cluster, StorNext appliances operating as StorNext MDCs (or customer-supplied MDCs) must be running the same StorNext release or higher than StorNext clients. In other words, StorNext clients cannot be running a later StorNext release than a StorNext MDC in the network/cluster.

Note: The following appliances operate asStorNext MDCs: Xcellis Workflow Director, Xcellis Foundation, aiWARE for Xcellis Embedded, Artico, Pro Foundation, M660, and M440 systems.

Note: The following StorNext releases are supported on the system. These tables represent only which StorNext releases are supported on the system, but not compatibility. See also StorNext compatibilityand StorNext appliance (MDC) vs. client dependencies.

StorNext 7 Releases

 

System

Release
7.0.1 7.0.2 7.0.3 7.1                

Xcellis Workflow Director Gen 3

✓ ✓ ✓ ✓                

Xcellis Workflow Extender Gen 3

✓ ✓ ✓ ✓                
StorNext 6 Releases

 

Release

System

 6.0b 6.0.1b

6.0.1.1a, b

 6.0.5b 6.0.5.1b 6.0.6b 6.0.6.1b 6.1.0c 6.1.1c 6.2.0 , 6.2.1, 6.3.0 and 6.3.1d 6.3.1.1d 6.4

Xcellis Workflow Director (Gen 2)

X X X X X X X ✓ ✓ X ✓ ✓

Xcellis Workflow Extender (Gen 2)

X X X X X X X ✓ ✓ X ✓ ✓

bStorNext 6.0.6.1 replaced previous 6.x StorNext releases, and firmware upgrade files will only be provided for StorNext 6.0.6.1 or later.

cStorNext 6.1.1 replaces StorNext 6.1.0, and includes all fixes issues and enhancements completed for StorNext 6.1.0, including the Spectre and Meltdown vulnerabilities addressed in that release. StorNext 6.1.0 firmware upgrade files are no longer available.

dStorNext 6.3.1.1 supersedes and replaces 6.2.0, 6.2.1, 6.3.0, and 6.3.1, and includes all fixes issues and enhancements completed for those releases. Firmware upgrade files for these releases are no longer available.

  • 6.0.5 or later – Upgrade the system firmware to StorNext 5.3.0 or later prior to upgrading to 6.0.5 or later.

  • 6.0.1.1 – upgrade the system firmware to StorNext 6.0.1 prior to upgrading to 6.0.1.1.

StorNext 4 Releases

 

Release

System

 4.3.2 4.3.3 4.7 4.7.0.1 4.7.1 4.7.2 4.7.3

The Firmware Upgrade option in theStorNext GUI allows you to perform a firmware upgrade on systems with a StorNext GUI (listed above). Depending on the version being applied, the firmware upgrade includes updates to the firmware running on the appliance servers (if applicable), the system OS (if applicable) and StorNext software.

Caution: Use the procedure listed here to upgrade your appliance.

For dual-server node systems, Quantum recommends you ensure node 2 is operating as primary before you begin the upgrade, so that the upgrade upgrades server node 1, first, before applying the upgrade to server node 2. Fail over the system if server node 2 is not operating as primary.

Initiate a graceful server failover. While the instructions linked here are located on the Xcellis Workflow Director Documentation Center, the process applies to all Quantum dual-server node systems.

Before you upgrade firmware, see Before You Begin.

Note: Firmware files are large - over 12 GB (or larger) for releases where both files are required (typical), so plan time to download the file(s) required for the upgrade. The /scratch file system requires at least 20 GB of free space available before you perform the upgrade.

In order to receive a firmware file, you will need one of the following:

  • The System Serial Number

    • System serial numbers and StorNext software serial numbers are alphanumeric. The table below provides examples of serial numbers.

    Type of Serial Number Example(s) of Serial Number(s)

    System Serial Number

    AV1607CKH12345

    CX1306CKB12345

    EW1450CKD12345

    StorNext Software Serial Number

    SN12345

    You can find the System Serial Number on the right-hand side of the StorNext GUI, as shown here:

  • The StorNext Serial Number.

    • You can find the StorNext Serial Number in the license.dat file. The license.dat file can be found on the server node at /usr/cvfs/config/license.dat for UNIX systems, and c:\Program Files\StorNext\config\license.dat for Windows systems. Open the license.dat file and locate the Serial Number.

    Example:

    # Serial Number: Q8675309

  • The cvfsid string for each server node.

    • Example:

    ECF4BCEECC0E linux 0 xcellis13

If you need firmware upgrade files to upgrade your system:

  1. Fill out the following form, here:

    2. StorNext 7 Upgrade

    The form looks like:

    (click to enlarge)

  2. Click the box to agree and confirm that you have read and accepted the EULA, Privacy Policy and Terms of Use.
  3. Click Submit.
  4. After Quantum receives your information, you will receive an email from "Quantum Software Downloads" to the email address you entered on the form linked above.

    5. The email will look similar to:

    (click to enlarge)

  5. Click the "here" link in the email to access the "mySoftware" page containing the Appliance Upgrade file(s) available for your system.

    6. The page will look similar to:

    (click to enlarge)

  6. Click the Download button next to the firmware file(s) for the version of StorNext you wish to update to.

    7. Note: Most StorNext versions include both a "(1 of 2)" and a "(2 of 2)" file. Download both files.

Upgrade StorNext:

Note: For information about estimating how long the upgrade might take for your given upgrade, see How long is this going to take?.

Note: In the StorNextGUI, DO NOT deactivate the "Upgrade in Progress" state or start up services manually on either server node during the upgrade.

Caution: If StorNext services are manually started while the metadata array firmware is being upgraded, the MDC nodes will likely SMITH.

Note: For StorNext 7 and later, the StorNext Unified UI runs on server node 2 only. For Releases prior to 7.1, StorNext Connect also runs on server node 2 only, so those interfaces will be unavailable during the upgrade.

Obtain your StorNext upgrade file(s). Then do the following to upgrade StorNext on your system:

  1. Log into the StorNext GUI.
  2. Choose Tools > Firmware Upgrade.
  3. Select Auto Upload to upload the file immediately after you select it.
  4. Click Choose File and navigate to the directory containing the firmware file(s). Firmware files are identifiable by the .fw extension.
  5. Navigate to the location on your system containing the firmware file(s), and select one of the files. On Windows systems, click Open. While you cannot select both files in the dialog box, you can select either file at this time.
    • There are typically two .fw files required for updating firmware. The filenames are similar to QTM-SNA-upd-6.2.0.OS7-22241-80564E.1of2.fw and QTM-SNA-upd-6.2.0.OS7-22241-80564E.2of2.fw. For these type of two-part upgrades, upload both files. For some releases, only a single firmware file is provided. Select the single file for the upgrade. Example: QTM-StorNext-upd-5.3.2_64242A-64242A.OS7.1of1.md5.
    • Uploading the firmware upgrade files in a network with low latency should only take a matter of minutes. High network latency in your environment can slow the upload of firmware files onto the system.
    • StorNext licenses for the system remain on the system and do not need to be re-applied after the firmware upgrade is complete.
    • The green status indicator at the top of the page indicates upload progress, not the upgrade progress. To monitor upgrade progress, check the logs available under the Reports menu.
  6. If you did not select the Auto Upload option, select the file and click Upload. If you selected Auto Upload, the file is immediately uploaded to the system. Wait for the first file to upload and display in the Uploaded Files list.
  7. If there is more than one upgrade file, click Choose File again, and select the other firmware upgrade file. On Windows systems, click Open.
  8. If you want to validate the file, click Validate. After the message that says "Firmware upgrade file is valid.", click OK. Repeat for the other file, if desired.
    • Files are automatically validated after you click Apply in Step 5. Once Apply is clicked, the upgrade process begins, and you cannot postpone the upgrade until a later time.

      • Note: If you run the validation process before you click Activate, you have a greater assurance that the system is in a proper state for a successful upgrade. If desired, upload the .fw (firmware) files and validate them prior to a planned maintenance window for the system's upgrade. You can upload and validate files at any time before you click Activate.

      Once a system is upgraded to StorNext 6.2.0 or later, the Validate button on the StorNext GUI Firmware Upgrade page performs a much more extensive upgrade validation check than in previous releases. The complete set of pre-upgrade checks that used to run only when you clicked Activate, are now run. These checks include a checksum and version verification against the upgrade files, and also include a complete check of RPM dependencies as well as potential conflicts on the system which could cause an upgrade to fail. There are also more extensive version checks for StorNext and other packages.

      Beginning with StorNext 7.2.0, the pre-upgrade check validation process also checks for RPM files that cannot be upgraded. Upon validation failure, an Admin Alert containing a list of RPMs found that are not expected on the node(s), is generated. You must manually remove these RPMs from the node(s) before you activate the upgrade. If the system you are upgrading includes the lin_tape RPMs, you must manually remove the RPMs before you start the upgrade process, and then install the RedHat 8 version of the RPMs after the upgrade process is complete.

      If you click the "Validate" button on the StorNext GUI Firmware Upgrade page, this will verify the uploaded .fw files against a checksum for those files. The validate also performs some basic version checking to make sure the uploaded files are valid for the running system.

      Errors reported from the validation check are reported back to the user in a context sensitive pop-up window explaining what failed in the validation check as well as an Admin Alert is generated with similar detailed information about the failure.

  9. Select either of the uploaded files click Apply. Both firmware files are applied to the system during the upgrade. For single file firmware upgrades, select the file and click Apply. Click Yes to confirm the installation.

    10. Note: For StorNext appliances, after you click Apply, there are some instances where the firmware upgrade will fail to complete. See the 71644 Known Issue in the Release Notes for your appliance for the workaround if the firmware upgrade fails on your system. This issue was fixed in StorNext 6.1.0.

  10. Wait for a few minutes. If you see a message stating that both nodes of the system will be rebooted during the upgrade process, click Yes to continue with the upgrade.
    • While it is true for dual-server node systems (that support HA fail-over) the server nodes will reboot, but DO NOT be alarmed. The server nodes reboot sequentially, meaning that StorNext operations continue during each server upgrade. The server node not being upgraded operates as the primary node during upgrade of the other server node. Once the first server upgrade is complete, 1.) the running node will fail over so the other server node will take begin operating as primary, 2.) the server node previously operating as primary will be taken offline and upgraded, and 3.) when the final server upgrade is complete, that server node will reboot and will come back on-line when the reboot is complete.
    • For single-server node upgrades, StorNext operations WILL BE interrupted during the upgrade, since there is no other server node to fail over to during the upgrade.

While the system firmware upgrade is being applied, you will not something that indicates upgrade progress. There are, however, a few different things that indicate the system is being upgraded:

Note: For firmware upgrades prior to StorNext 5 Release 5.4.0, this section does not apply. See GUI indications that a firmware upgrade is in process - before StorNext 5.4.0.

  1. Log into the StorNext GUI.
  2. Choose Tools > Firmware Upgrade.
  3. Select Auto Upload to upload the file immediately after you select it.
  4. Click Choose File and navigate to the directory containing the firmware file(s). Firmware files are identifiable by the .fw extension.
  5. Navigate to the location on your system containing the firmware file(s), and select one of the files. On Windows systems, click Open. While you cannot select both files in the dialog box, you can select either file at this time.
    • There are typically two .fw files required for updating firmware. The filenames are similar to QTM-SNA-upd-6.2.0.OS7-22241-80564E.1of2.fw and QTM-SNA-upd-6.2.0.OS7-22241-80564E.2of2.fw. For these type of two-part upgrades, upload both files. For some releases, only a single firmware file is provided. Select the single file for the upgrade. Example: QTM-StorNext-upd-5.3.2_64242A-64242A.OS7.1of1.md5.
    • Uploading the firmware upgrade files in a network with low latency should only take a matter of minutes. High network latency in your environment can slow the upload of firmware files onto the system.
    • StorNext licenses for the system remain on the system and do not need to be re-applied after the firmware upgrade is complete.
    • The green status indicator at the top of the page indicates upload progress, not the upgrade progress. To monitor upgrade progress, check the logs available under the Reports menu.
  6. If you did not select the Auto Upload option, select the file and click Upload. If you selected Auto Upload, the file is immediately uploaded to the system. Wait for the first file to upload and display in the Uploaded Files list.
  7. If there is more than one upgrade file, click Choose File again, and select the other firmware upgrade file. On Windows systems, click Open.
  8. If you want to validate the file, click Validate. After the message that says "Firmware upgrade file is valid.", click OK. Repeat for the other file, if desired.
    • Files are automatically validated after you click Apply in Step 5. Once Apply is clicked, the upgrade process begins, and you cannot postpone the upgrade until a later time.

      • Note: If you run the validation process before you click Activate, you have a greater assurance that the system is in a proper state for a successful upgrade. If desired, upload the .fw (firmware) files and validate them prior to a planned maintenance window for the system's upgrade. You can upload and validate files at any time before you click Activate.

      Once a system is upgraded to StorNext 6.2.0 or later, the Validate button on the StorNext GUI Firmware Upgrade page performs a much more extensive upgrade validation check than in previous releases. The complete set of pre-upgrade checks that used to run only when you clicked Activate, are now run. These checks include a checksum and version verification against the upgrade files, and also include a complete check of RPM dependencies as well as potential conflicts on the system which could cause an upgrade to fail. There are also more extensive version checks for StorNext and other packages.

      Beginning with StorNext 7.2.0, the pre-upgrade check validation process also checks for RPM files that cannot be upgraded. Upon validation failure, an Admin Alert containing a list of RPMs found that are not expected on the node(s), is generated. You must manually remove these RPMs from the node(s) before you activate the upgrade. If the system you are upgrading includes the lin_tape RPMs, you must manually remove the RPMs before you start the upgrade process, and then install the RedHat 8 version of the RPMs after the upgrade process is complete.

      If you click the "Validate" button on the StorNext GUI Firmware Upgrade page, this will verify the uploaded .fw files against a checksum for those files. The validate also performs some basic version checking to make sure the uploaded files are valid for the running system.

      Errors reported from the validation check are reported back to the user in a context sensitive pop-up window explaining what failed in the validation check as well as an Admin Alert is generated with similar detailed information about the failure.

  9. Select either of the uploaded files click Apply. Both firmware files are applied to the system during the upgrade. For single file firmware upgrades, select the file and click Apply. Click Yes to confirm the installation.

    10. Note: For StorNext appliances, after you click Apply, there are some instances where the firmware upgrade will fail to complete. See the 71644 Known Issue in the Release Notes for your appliance for the workaround if the firmware upgrade fails on your system. This issue was fixed in StorNext 6.1.0.

  10. Wait for a few minutes. If you see a message stating that both nodes of the system will be rebooted during the upgrade process, click Yes to continue with the upgrade.
    • While it is true for dual-server node systems (that support HA fail-over) the server nodes will reboot, but DO NOT be alarmed. The server nodes reboot sequentially, meaning that StorNext operations continue during each server upgrade. The server node not being upgraded operates as the primary node during upgrade of the other server node. Once the first server upgrade is complete, 1.) the running node will fail over so the other server node will take begin operating as primary, 2.) the server node previously operating as primary will be taken offline and upgraded, and 3.) when the final server upgrade is complete, that server node will reboot and will come back on-line when the reboot is complete.
    • For single-server node upgrades, StorNext operations WILL BE interrupted during the upgrade, since there is no other server node to fail over to during the upgrade.

Note: For firmware upgrades for StorNext 5.4.0 and later, this section does not apply. See Monitoring upgrade progress - the StorNext "Firmware Upgrade Status" page.

For firmware upgrades supporting StorNext 5 releases prior to 5.4.0:

  • The StorNext GUI will display a messsage similar to the following when the upgrade begins:

    • Firmware upgrade in progress. Login session will be terminated when the upgrade process has completed on the other node. Log in to the other node to continue monitoring the upgrade process. The upgrade is complete when full GUI functionality has been restored.

  • The normally-green Secondary node icon changes to orange, indicating that the process is currently upgrading node 2.
  • After a while, the StorNext GUI on node 1 will cease to function.
  • Once the upgrade is complete on node 2, StorNext operations will fail over to node 2, and you can login to the node 2 GUI and monitor the upgrade progress.

If you SSH and login to the command line of the node of a server node that is currently operating as primary while the server node currently acting as secondary is being upgraded, you will either attempt to login and the session will terminate immediately after, or will see a note similar to the following, indicating the system firmware upgrade is in process:

*********************** WARNING ***************************

*** An upgrade is currently in progress on this system! ***

*** Please do not change any settings or software until ***

*** the upgrade completes. ***

*********************** WARNING ***************************

If you attempt to SSH into the server node that is currently upgrading its firmware, the session will not connect to the system.

You can monitor the upgrade process using the following log files:

Log File Description

/var/log/DXi/upgrades/NodeX.upgradeoutput.log

 Upgrade output log for the main part of the upgrade (not including NetApp firmware updates).

/var/log/DXi/upgrades/NodeX.upgrade_progress

 Upgrade Progress Log for the main part of the upgrade (not including NetApp firmware updates).

/var/log/DXi/baseos.log

 Final upgrade progress at the end of the final boot of the second node (shows the NetApp firmware update has started).

/var/log/DXi/UpgradeArrays.log

 Contains the NetApp firmware upgrade progress. The firmware update command will not respond for 30-40 minutes so there is no specific change in progress noted until the controllers reboot.

/var/log/messages

 Contains other boot messages like the application of firmware bundle updates and hardware detect script execution, both of which could take several minutes.

Each of these log files will continue to log upgrade status progress detail until the StorNext GUI is once again fully-accessible at the end of the upgrade.

This ONLY applies to Artico (R520), Pro Foundation, M660, M440, and M330 systems.

Note: SNFS services are stopped prior to updating the controller firmware, so the StorNext GUI does not show the array(s) as available while the controller firmware is updated.

The end of the NodeX.upgradeoutput.logfile includes the following statement that indicates the array firmware update has started:

New NetApp FW needs to be activated. Bringing down the cluster.

The message below is added to the log after the array controllers reboot:

Successfully activated new NetApp FW on the array controllers.

These are the only messages provided on the upgrade progress for NetApp controllers.

The array controllers will reboot after the firmware update is complete (about 20-30 minutes after the update begins). Messages about the SCSI driver and multi-path driver devices are removed and added are displayed during the upgrade.

These occurrences are normal since the controllers drop off the SAS BUS and are added again during the update. LUN access is unavailable during the controller reboot.

Note: SNFS services will start up automatically after the upgrade is completed and all LUNs will be accessible on the metadata array.

Note: For the sake of this example, node 1 is the server node currently operating as primary, and node 2 is the server node currently operating as secondary.

  1. Node 2 is upgraded.
  2. After node 2 upgrade/reboot completes, node 1 fails over operations to node 2.
  3. Node 1 is upgraded.
  4. After node 1 is upgraded/rebooted, node 2 will fail-over operations back to node 1 (for StorNext Connect or systems licensed and configured as a NAS gateway). If your system doesn't fail-over the way you'd like, see Initiate a Graceful Server Fail-over.

After you upgrade StorNext, complete the relevant steps in Important! After You Upgrade.

After the firmware upgrade is complete, review the Known Issues in the Release Notes for your particular appliance (see Current Release Notes - Supported Appliances), and perform workarounds as needed.

In addition, review the following post-firmware upgrade items to ensure you perform the workarounds needed for your system, depending on the features you have installed:

After StorNext upgrades on the system Flexsync may not be functional, and you won't see the Flexsync servers listed on the SERVICES page of the StorNext UI. The reason for this is that the Flexsync servers do not register with the StorNextAPI gateway until the Flexsync is restarted on both server nodes.

Open an SSH connection to the server and use the IP address used to access the server on the Management (public) or LAN Client (private) network:

  1. Log in to the command line using the following credentials:
    • User name: stornext
    • Password: <StorNext user account password>

      Note: password is the default password for the stornext user account. If the password has been changed, use the current password. The first time you log in, you are prompted to change the password to a different one.

  2. Enter sudo rootsh to gain root user access.
  3. Enter the password for the stornext user account again.

Note: If the password for the stornext user account was changed after installation, use the current password.

  1. Verify the operating status of the service before restarting, enter:
    2. service flexsyncadmind status
  2. You should see output like the following (text in red indicates that the service has failed to register with the StorNext API gateway:

  3. Enter the following command to restart flexsync services on this server node:
    4. service flexsyncadmind restart

    The service will restart.

  4. Verify the operating status of the service. Enter:
    5. service flexsyncadmind status

    You should see output like the following (text in green indicates that the service is running. Notice there are no error messages in this panel:

  5. Repeat all steps above on server node 2.
  6. Open the StorNext UI. In the top navigation, click SERVICES.

    7. You will see both Flexsync servers listed, and a green/healthy status:

If your system is running as a FlexSync host and you upgrade StorNext to 6.3.1.1, it also installs the latest version of FlexSync. If there are other systems configured as FlexSync hosts, FlexSync will not function on those other systems until they are upgraded to the FlexSync version installed with the StorNext upgrade.

Error message displayed:

Host Unreachable, Upgrade Required

The same FlexSynch version must be installed on ALL FlexSync hosts (see Can different versions of FlexSync be installed or upgraded within a configuration? on the FlexSync Documentation Center).

Workaround:

Upgrade FlexSync on ALL FlexSync hosts. See Upgrade FlexSync in the FlexSync Documentation Center.

The following applies to Xcellis Workflow Director Gen 3, Xcellis Foundation, aiWARE for XcellisEmbedded, Artico, Pro Foundation, M660, M440 systems.

After a successful upgrade from StorNext 5, move the files from the /usr/adic/database/metadumps directory to an external location, or delete the files, since they are no longer used by StorNext and might be consuming a large amount of disk space.

Note: Beginning with StorNext 6, file system metadata is saved to the /usr/adic/database/mdarchives directory.

As of StorNext 6, there are several features that either require you to enable the Metadata Archive feature, or benefit when you enable this option. These include, but are not limited to the following:

The following applies to Xcellis Workflow Director Gen 3, Xcellis Foundation, aiWARE for XcellisEmbedded, Artico, Pro Foundation, M660, M440 systems.

Note: Applies only to systems where the /etc/multipath.conf file has been edited to include custom configuration settings for your storage.

For StorNext 6.0.6 and later, during the system firmware upgrade, the current /etc/multipath.conf file is no longer overwritten, and a backup file is no longer created. Because of this, you will no longer need to edit the /etc/multipath.conf file to add your custom multipath settings after the upgrade (if applicable).

For StorNext 5.3.1 to 6.0.5.x, the /etc/multipath.conf file was overwritten with a new /etc/multipath.conf file when the system firmware upgrade was applied to the system. For StorNext releases 5.3.1 through 6.0.5.x, copy previously-applied multipath settings from the /etc/multipath.conf.bak file into the /etc/multipath.conf file using a command-line editor like vi, after every firmware upgrade. Once you upgrade to 6.0.6 or later, which no longer overwrites the file, your custom multipath settings will remain in the file and will no longer have to be manually updated from the backup file..

After a firmware upgrade, you might be unable to use previously functioning tape devices with StorNext because the lin_tape device driver was automatically unloaded during the upgrade.

If you encounter this issue do the following:

  1. Login to the command-line of the server.
  2. Rebuild the lin_tape driver as shown in the following example:

> rpm -e lin_taped

> rpm -e lin_tape

> rpmbuild --rebuild /root/lin_tape-1.76.06-1.src.rpm

> rpm -ivh /root/rpmbuild/RPMS/x86_64/lin_tape-1.76.06-1.x86_64.rpm

> rpm -ivh /root/lin_taped-1.76.0-rhel6.x86_64.rpm

This applies to Xcellis Workflow Director, Xcellis Foundation, aiWARE for Xcellis Embedded, and Artico (R630) systems and Artico (R520), Pro Foundation, M660, M440, and M330 systems.

(missing or bad snippet)

Use this procedure to perform a fail-over of server operations from a server node operating as primary to a server node operating as secondary without losing client access to metadata. This can also be used to reverse which server is primary and secondary, if the system previously failed over.

Access the command line of the node currently operating as primary, and initiate the graceful fail-over to the server node currently acting as secondary:

  1. Open an SSH connection to the appropriate server and use the IP address assigned to the node on the Management or LAN Client network, or use the Service Port IP address.

    2. Note: The service port is no longer a dedicated service port. This port might be configured by the user for use as a metadata or client port with a different IP configuration. If this port has been reconfigured by the user, then command line access over a network connection requires an appropriate IP address for your configured system on that network.

    • Node 1 : <service port IP address>, the default service port IP address is 10.17.21.1
    • Node 2 : <service port IP address> (dual-server node, or supported "HA-ready" single-server node StorNext appliances), the default service port IP address is 10.17.21.2

  2. Initiate an ssh session to the system using Terminal or PuTTY:
    1. Log in to the command line using the following credentials:
      • User name: stornext
      • Password: <StorNext user account password>

        Note: password is the default password for the stornext user account. If the password has been changed, use the current password. The first time you log in, you are prompted to change the password to a different one.

    2. Enter sudo rootsh to gain root user access.
    3. Enter the password for the stornext user account again.
  1. Confirm that the server node you are connected to is operating as the primary. Enter:

    2. snhamgr -m status

  2. Verify the output is as follows (in green):

    3. LocalMode=default

    LocalStatus=primary

    RemoteMode=default

    RemoteStatus=running

    This indicates the "Local" node you are connected to is operating as primary ("LocalStatus=primary"), and that the other node is operating as secondary ("RemoteStatus=running"). If you do not see this status, close this PuTTY/terminal session and SSH in to the other server node.

  3. On the node operating as the primary, initiate an HA fail-over to the node operating as the secondary.

    4. Note: If you have NFS-HA enabled, click here for the instructions to stop NAS prior to using this command. Enter:

    Wait until the secondary node becomes the primary. (Time may vary.)

  4. Verify the node has failed over. Enter:

    5. snhamgr -m status

  5. Verify the output is (bold text used for clarification):

    6. LocalMode=default

    LocalStatus=stopped

    RemoteMode=primary

    RemoteStatus=running

    This indicates the "Local" node you are connected to is operating as secondary ("LocalStatus=stopped"), and that the other node is operating as secondary ("RemoteMode=primary").

    If this is not what is shown, wait and enter the snhamgr -m status command again.

    You can leave the SSH connection to this server node running if you will access this server again soon.

To verify the node previously operating as secondary is now operating as primary:

  1. Open an SSH connection to the appropriate server and use the IP address assigned to the node on the Management or LAN Client network, or use the Service Port IP address.

    2. Note: The service port is no longer a dedicated service port. This port might be configured by the user for use as a metadata or client port with a different IP configuration. If this port has been reconfigured by the user, then command line access over a network connection requires an appropriate IP address for your configured system on that network.

    • Node 1 : <service port IP address>, the default service port IP address is 10.17.21.1
    • Node 2 : <service port IP address> (dual-server node, or supported "HA-ready" single-server node StorNext appliances), the default service port IP address is 10.17.21.2

  2. Initiate an ssh session to the system using Terminal or PuTTY:
    1. Log in to the command line using the following credentials:
      • User name: stornext
      • Password: <StorNext user account password>

        Note: password is the default password for the stornext user account. If the password has been changed, use the current password. The first time you log in, you are prompted to change the password to a different one.

    2. Enter sudo rootsh to gain root user access.
    3. Enter the password for the stornext user account again.
  1. Confirm that the server node you are connected to is operating as the primary. Enter:

    2. snhamgr -m status

  2. Verify the output is (bold text used for clarification):

    3. :default:primary:default:stopped:

    This indicates the node you are connected to is "primary", and that the node previously operating as primary is still "stopped".

  1. Open an SSH connection to the appropriate server and use the IP address assigned to the node on the Management or LAN Client network, or use the Service Port IP address.

    2. Note: The service port is no longer a dedicated service port. This port might be configured by the user for use as a metadata or client port with a different IP configuration. If this port has been reconfigured by the user, then command line access over a network connection requires an appropriate IP address for your configured system on that network.

    • Node 1 : <service port IP address>, the default service port IP address is 10.17.21.1
    • Node 2 : <service port IP address> (dual-server node, or supported "HA-ready" single-server node StorNext appliances), the default service port IP address is 10.17.21.2

  2. Initiate an ssh session to the system using Terminal or PuTTY:
    1. Log in to the command line using the following credentials:
      • User name: stornext
      • Password: <StorNext user account password>

        Note: password is the default password for the stornext user account. If the password has been changed, use the current password. The first time you log in, you are prompted to change the password to a different one.

    2. Enter sudo rootsh to gain root user access.
    3. Enter the password for the stornext user account again.
  1. Confirm that the server node you are connected to is operating as the primary. Enter:

    2. snhamgr -m status

  2. Verify the output is (bold text used for clarification):

    3. :default:stopped:default:primary:

    This indicates the node you are connected to is operating as secondary, that its services are "stopped", and that the other node is now operating as "primary".

  3. Restart StorNext services on the secondary node. Enter:
  4. Confirm that the node is operating as the secondary. Enter:

    5. snhamgr -m status

  5. Verify the output is (bold text used for clarification):

    6. :default:running:default:primary:

Quantum recommends to clear the browser and cookie cache after upgrading to a StorNext release before logging into the GUI on the upgraded system.

The steps used to clear the cache depend on the browser:

  1. Click the configuration icon and the Settings option.
  2. Click the Show advanced settings option.
  3. In the Privacy section, click Clear browsing data.
  4. Select the beginning of time option in the Obliterate the following items from menu.
  5. Select the Clear browsing history, Empty the cache, and Delete cookies and other... options.
  6. Click Clear browsing data.
  7. Close the Settings tab.
  1. Go to Tools > Internet Options.
  2. On the General tab, look in the Browsing History section and click Delete.
  3. Select the Temporary Internet Files and Cookies options.
  4. Click Delete.
  5. Click OK to close the Internet Options dialog.
  1. Click Tools > Clear Recent History. (If this option is not visible, right-click the top of the browser window and select the Menu Bar option).
  2. In the Time Range to Clear menu, select Everything.
  3. Click Details, and select the Browsing & Download History, Cookies, and Cache options.
  4. Click Clear Now.

If you upgrade your appliance from StorNext version 7.1 (or later), Quantum recommends you perform the following to verify the Unified Connector software running on the appliance(s) that you upgraded is also upgraded:

  1. Log in to the Unified UI.

  2. On the top navigation menu, click CLIENTS.

  3. Verify the version listed under the UC Version column matches the version listed under the StorNext Version column.

    Note: This only applies to the nodes that you upgraded. The clients that are not appliances (or appliances that were just discovered by the API gateway) can have a different version of the Unified Connector software.

The firmware upgrade process is complete.