Quantum| HOWTO: Restart failed Stornext firmware upgrades on Metadata Appliances |
Note: please scroll down and read additional comments from SUS:
There are times when Stornext firmware upgrades need to be restarted when the upgrade process fails.
There are various ways to restart the upgrade Ive found.
1. Simplest way is to do a reboot of the node.
2. Next is to manually tell the upgrade process to restart. In this case, verify if upgrade files still exist in /scratch/upgrade
ls -l /scratch/upgrade
cp -p /scratch/upgrade/Upgrade.M660R /etc/rc.d/init.d/Upgrade
chkconfig Upgrade on
reboot
3. You can also check if upgrade failed because of version mismatch:
Nov 12 09:47:55: Firmware Image checksum is OK.
Nov 12 09:48:21: ******************* VersionCheck.sh ********************* Using '/opt/DXi/base_version'
'/opt/DXi/base_version' line: 0: Major=5 '/opt/DXi/base_version' line: 1: Minor=3 '/opt/DXi/base_version' line: 2: Patch=1 '/opt/DXi/base_version' line: 3: LCR= '/opt/DXi/base_version' line: 4: Port= '/opt/DXi/base_version' line: 5: Build=16362 '/opt/DXi/base_version' line: 6: Version=5.3.1-16362 '/opt/DXi/base_version' line: 7: Platform=DXiSNA Using '/opt/DXi/version_info'
'/opt/DXi/version_info' line: 0: Major=5 '/opt/DXi/version_info' line: 1: Minor=2 '/opt/DXi/version_info' line: 2: Patch=1 '/opt/DXi/version_info' line: 3: LCR= '/opt/DXi/version_info' line: 4: Port= '/opt/DXi/version_info' line: 5: Build=15662 '/opt/DXi/version_info' line: 6: Version=5.2.1-15662 '/opt/DXi/version_info' line: 7: Platform=M660
Result: NotAllowed
Nov 12 09:48:21: Upgrade denied.
Nov 12 09:48:21: Image Version mismatch, aborting Upgrade.
Nov 12 09:48:21: The firmware image file has failed verification.
Solution: Restart the upgrade:
Modify /scratch/upgrades/attempts, set to 0
remove /usr/adic/HAMS/shared/Platform/upgrade_failed
"chkconfig Upgrade on"
reboot
====
4. On XCellis, restart process is different as its based on CentOS 7 Operating System.
XCellis upgrade:
cp /scratch/upgrade/Upgrade.BHM /opt/DXi/systemd/startup/Upgrade
cp /scratch/upgrade/quantum-upgrade.service /usr/lib/systemd/system/
systemctl set-default quantum-upgrade.target
systemctl enable quantum-upgrade.service
systemctl reboot
4. Finally, there is one last way to upgrade which involves just upgraing Stornext application.
You may have to do this on each node.
Pls remember to involve SES before implementing procedure below as you want to make sure
all the appliance platform .rpms are upgraded as well.
You can verify by doing "rpm -qa" on both nodes to see if all packages are at the same level.
cd /tmp/stornext/
./install.stornext
Upgrade
Mamoon –
I wanted to send you a note regarding issues encountered at Watchtower with a failed upgrade from 5.4.0.1. to 5.4.0.4 on an Xcellis system. The steps that were executed were the following as highlighted in the document:
https://qwikipedia.brainkeeper.net/index.php?action=workspace.page&ZDPOFMWE=0ZZskdaU
– cp /scratch/upgrade/Upgrade.BHM /opt/DXi/systemd/startup/Upgrade
– cp /scratch/upgrade/quantum-upgrade.service /usr/lib/systemd/system/
– systemctl set-default quantum-upgrade.target
– systemctl enable quantum-upgrade.service
– systemctl reboot
While the steps were correct for CentOS7, the instructions don’t completely convey that there could be inherent problems that these steps won’t fix. Basically, this sequence of steps was executed a couple of times with the upgrade never being done. It took some debugging to determine that when the upgrade script was executed after the reboot, it would simply waltz through the script putting out a message that the upgrade had been started and then exit. In reality, nothing was done. I found this out by looking for any semblance of the upgrade script running and it wasn’t. We had to go back a few layers in the process and finally execute the script Install_New_Image to get the upgrade to complete. It seems to me that the instructions should alert the reader to the fact that there might be extra debugging steps they may need to take to complete a failed upgrade.
Also, it would be nice to inform the user that the systemctl steps should be looked at and put back if they haven’t been changed.
Thanks. --sc
This page was generated by the BrainKeeper Enterprise Wiki, © 2018 |