General Operating Guidelines and Limitations

Solaris hosts may need to rescan disk devices after StorNext labels have been applied.

In particular, when a StorNext label is put on a LUN less than 1TB in size, Solaris hosts will not be able to use that LUN until they have done a device rescan. A device rescan is accomplished with a boot flag:

reboot -- -r

This issue will be addressed in a future StorNext release.

In the meantime, work around this issue by rescanning devices using the boot flag reboot -- -r

If the labeling operation was performed on a Solaris host, that host does not need to do the rescan. However, some intermediate versions of the Solaris 10 Kernel Jumbo Patch break the necessary functionality to support this; please be sure you have applied the latest Solaris 10 Kernel Jumbo Patch before labeling any StorNext LUNs.

Current versions of the Linux DM Multipath driver assign a default value of 1000 for rr_min_io which is too high for most configurations having multiple active paths to storage. Using a smaller value such as 32 will typically result in significantly improved performance. Refer to the RedHat or SuSE documentation provided with your version of Linux for details on how to apply this setting.

Note: Experimentation may be required to determine the optimal value.

StorNext File System does not support the Linux sendfile() system call.

This issue causes Apache web servers to deliver blank pages when content resides on StorNext file systems.

This issue also affects Samba servers running on Linux.

The workaround is to disable sendfile usage by adding the following entry into the Apache configuration file httpd.conf:

EnableSendfile off

The workaround for Samba servers is to add the following line into the configuration file:

sendfile=no

Many versions of Linux run a cron script nightly to build a database used by the slocate command. If StorNext file systems are mounted, they are traversed by this cron job which can have a dramatic impact on the performance of other applications currently using these file systems. Perform the following steps (based on Linux version) to prevent the cron script from traversing StorNext file systems.

RedHat Enterprise Linux 4, 5, and 6

Add “cvfs” to the list of file system types to be skipped. This is usually done by modifying the “PRUNEFS” line in the /etc/updatedb.conf file to read:

PRUNEFS=”cvfs sysfs selinuxfs usbdevfs devpts NFS nfs nfs4 afs sfs proc smbfs cifs autofs auto iso9660 udf”

SUSE Linux Enterprise Server 10, and 11

The optional “findutils-locate” package is used to build the slocate database. The default behavior is to disable building the database. If enabled, to prevent cvfs file systems from being scanned, add “cvfs” to the list of file system types to be skipped. This is usually done by modifying the “UPDATEDB_PRUNEFS” line in the /etc/sysconfig/locate file to read:

UPDATEDB_PRUNEFS=”cvfs”

StorNext users migrating their metadata controllers from Apple Xsan to Linux should be aware of the following upgrade considerations:

If the file system is running Xsan 2.1.1 or earlier, it should be a simple upgrade: just replace the MDC.

If the file system is running Xsan 2.2 or later with “NamedStreams No” (which is the default for Xsan 2.2,) it should also be a simple upgrade: just replace the MDC.

If the file system is running Xsan 2.2 or later with “NamedStreams Yes,” you must completely remake (reformat) the file system. For obvious reasons, you should do a complete backup before migrating.

Due to the way Linux handles errors, the appearance of SCSI “No Sense” messages in system logs can indicate possible data corruption on disk devices.

This affects StorNext users on Red Hat 4, Red Hat 5, Red Hat 6, SuSe 10 and SuSe 11.

This issue is not caused by StorNext, and is described in detail in StorNext Product Alert 20.

For additional information, see Red Hat 5 CR 468088 and SuSE 10 CR 10440734121.

Software firewalls such as “iptables” on Linux and Windows Firewall can interfere with the proper functioning of StorNext and result in unexpected errors unless specifically configured for use with StorNext.

Quantum strongly recommends that all software firewalls be disabled on systems used as StorNext clients and servers. If required, StorNext can be configured for use with hardware firewalls.

For more information, refer to the fsports man-page or help file and the section Ports Used By StorNext.

Changing the haFsType parameter in a file system configuration file to one of the HA types, and then (re)starting its FSM enables HA-specific features that change the functionality of StorNext.

When the HaShared or HaManaged types are configured, other changes must be completed by successfully running the cnvt2ha.sh script, which is indicated by the creation of the


/usr/adic/install/.snsm_ha_configured touch file ($SNSM_HA_CONFIGURED environment variable). No conversion is done or necessary for SNFS only (HaUnmanaged) configurations.

If the conversion is not successfully completed, the HaManaged FSMs will not start, and the HaShared FSM will cause an HA Reset when it is stopped.

To remedy this situation, edit every FSM configuration file to set its haFsType parameter to HaUnmonitored, then run the following commands to avoid the HA Reset in this special case only:

touch /usr/cvfs/install/.vip_down_hint

service cvfs stop

Subtree Check Option in NFS No Longer Supported

Although supported in previous StorNext releases, the subtree_check option (which controls NFS checks on a file handle being within an exported subdirectory of a file system) is no longer supported as of StorNext 4.0.

SuSe Linux distributions automatically associate the FQDN of the local machine with the address 127.0.0.2 in the /etc/hosts file. There is no benefit from doing this when the machine is connected to a network that can resolve its name to an IP address.

However, the existence of this entry can sometimes cause a failure of configuration synchronization within and between the server computers in an HA configuration. For this reason, the 127.0.0.2 entry should be deleted from the /etc/hosts file.

cpuspeed, an external Linux service on recent Intel processors, is not correctly tuned to allow StorNext to take advantage of processor speed. Suse systems may also be impacted, as may AMD processors with similar capabilities.

On processors with a variable clockspeed (turboboost), the cpuspeed service on Redhat controls the actual running speed of the processors based on system load.

A workload such as a heavily used FSM and probably Storage Manager does not register as something which needs a faster cpu. Turning off the cpuspeed service has been shown to double metadata performance on affected hardware.

Looking at the reported CPU clock speed by doing
cat /proc/ cpuinfo while the system is under load displays if a system is impacted by this issue.

During testing a Quantum PX502 library running Red Hat 6.1 did not finish power-on diagnostics. When the same test was run on a PX502 library running either Red Hat 5.X or SuSE 10 / 11, power-on diagnostics completed and the system initialized without any issues.

The workaround for this issue is to disconnect the SAN from the library running Red Hat 6.1. If the library powers on while the SAN is disconnected from the library controller, the library finishes its power-on diagnostics and performs an audit of the library. Subsequently reconnecting the Red Hat 6.1 server to the SAN (library ready) causes the library to perform a new physical audit of the library.

Note: Testing was performed on the Red Hat 6.1 system which did not have StorNext loaded or running.

If you create a symbolic (soft) link in Linux to a directory on a StorNext file system, the link cannot be used by Windows. Windows also cannot process a symbolic link which contains a path to a file in another file system.

When a StorNext file system is mounted to a drive letter or a directory, configure the Windows backup utility to NOT include the StorNext file system.

Virtual Hard Disk (VHD) files are not supported on a StorNext file system.

In StorNext releases prior to 3.5, the StorNext Windows client attempted to keep the UNIX uid, gid and mode bits synchronized with similar fields in the Windows security descriptor. However, these Windows and UNIX fields were often not synchronized correctly due to mapping and other problems. One consequence of this problem was that changing the owner in Windows incorrectly changed the UNIX uid and file permissions and propagated these errors into sub-directories.

In release 3.5, the StorNext Windows client sets the UNIX uid, gid and mode bits only when Windows creates a file. The StorNext Windows client will no longer change the Unix uid, gid or mode bits when a Windows user changes the Windows security descriptor or Read-Only file attribute.

If you change the UNIX mode bits and the file is accessible from Windows, you must change the Windows security descriptor (if Windows Security is configured On) or Read-Only file attribute to ensure the change is reflected on both Windows and UNIX.

StorNext upgrades on Vista machines can fail in the middle of installation. This problem is caused by the way Windows Vista handles software upgrades. A related error is described in Microsoft article 263253 (see http://support.microsoft.com/kb/263253).

To work around this issue, follow these steps:

If you are using the StorNext client software with the Windows Operating System turn off the Recycle Bin in the StorNext file systems mapped on the Windows machine.

You must disable the Recycle Bin for the drive on which a StorNext file system is mounted. Also, each occurrence of file system remapping (unmounting/mounting) will require disabling the Recycle Bin. For example, if you mount a file system on E: (and disable the Recycle Bin for that drive) and then remap the file system to F:, you must then disable the Recycle Bin on the F: drive.

As of release 3.5, StorNext supports mounting file systems to a directory. For Windows Server 2003 and Windows XP you must disable the Recycle Bin for the root drive letter of the directory-mounted file system. (For example: For C:\MOUNT\File_System you would disable the Recycle Bin for the C: drive.)

For Windows Server 2003 or Windows XP:

(Disabling the Recycle Bin, Continued)

For Windows Server 2008, Windows Server 2012, Windows Vista, Windows 7 and Windows 8 systems, you must disable the Recycle Bin on C: and the File system name:

Before attempting to upgrade from a previous StorNext release, make sure you have free space on the file system. If the file system is nearly full when you begin the upgrade, serious errors may occur or the upgrade could fail. Best practice is to maintain an area on the file system which is not used for data or system files, but is reserved as an empty buffer to ensure that upgrades and other operations complete successfully.

StorNext's configuration of the tape drives within an ACSLS library can get out of sync with the ACSLS server's configuration of the tape drives. If this occurs, then when StorNext sends a request(s) to the ACSLS server for a specific drive it could be sending the wrong location to the drive.

The problem occurs if location modification occurs with the tape drives in the ACSLS library such that what StorNext thinks are the correct ACSLS locations no longer match what the ACSLS server knows about.

This can happen by one of the following library maintenance type activities:

Workaround:

Contact Quantum Technical Support.

There is currently no automated way to update StorNext to re-sync itself with the tape drive changes done within the library. It requires knowledge of what library scenarios were done in order to make the correct changes within StorNext. The necessary changes can be done to update StorNext's library/tape drive configuration to match what ACSLS knows of, but this is a manual process, and requires detailed knowledge of what needs to be updated in order to accomplish the procedure.

StorNext does not support hot-swapping tape drives. When replacing or adding new tape drives you must first stop StorNext before installing the new drive.

Tools outside of StorNext that issue a st command to rewind tapes may result in data loss.

Workaround(s):

It may not be possible to determine this ahead of time.

Gathering hardware info is imperative.

They get created on the next reboot.

Stopping TSM could take 10-15 minutes if very busy.

Not optimal to stop/start TSM all the time.

The StorNext Cluster-Wide Central Control file (nss_cctl.xml) is used to enforce the cluster-wide security control on StorNext nodes (client nodes, fsm nodes, and nodes running cvadmin). This file resides under /usr/cvfs/config on an nss coordinator server.

Currently the nss coordinator server capable of parsing this xml file must be on the Linux platform.

Quantum recommends disabling sleep mode on the Xsan client to prevent the client from going into a sleep state while files are open in the StorNext file system or data is in the file system cache.

Note: If the client is in a sleep state while a file is open, then it will prevent other clients from opening the same file.

It is not possible to delete data within a StorNext policy relation point from an Xsan client via the Finder. Rather, data must be deleted using the shell.

Disks with existing non-StorNext labels may not show up in the StorNext GUI in order to protect non-StorNext disks from being accidentally overwritten. If you need to label a disk that is not visible in the StorNext GUI, use the cvlabel command to label the disk or use cvlabel -U to remove the existing label from the disks. (Refer to the cvlabel man pages for instructions on labeling and unlabeling drives.)

Caution: Modifying the label on an active non-StorNext disk can make the disk unusable. Proceed with caution.

On HA systems only:

The /usr/cvfs/config/ha_peer file supports some essential HA features by providing an address for HA administrative communications between the MDCs in an HA Cluster. If CVFS is started without this file having correct information, the probability of an HA Reset increases. To correct this condition, restore the ha_peer file to the IP address of the peer MDC, and restart StorNext by running the following command: service cvfs restart

Note: The peer will be Primary after running this command.

If the ha_peer file is removed for any length of time while StorNext is running, the snhamgr(1) HA Manager subsystem could stop functioning, which impacts the GUI HA Manage status page and the starting and stopping of CVFS, as well as any command line use of snhamgr itself. If this occurs, restore the ha_peer file to the IP address of the peer MDC, and then restart the HA Manager service by running the following command: service snhamgr restart

On HA systems only:

You may receive the following incorrect error message when scanning for a secondary MDC from the StorNext Convert to HA page:

WARN com.quantum.qutosgui.jsf.ha.HaMBean - doScanHost: Secondary system cannot be same as the primary system.

This message is generated if /usr/adic/util/cnvt2ha.sh fails for any reason (for example, if the file system exists on the secondary, if a shared file system can't mount, etc). Upon secondary conversion failures, StorNext resets the ha_peer file to 255.255.255.255 on the secondary. Since the conversion fails, the primary ha_peer file is not updated and faulty comparison logic causes the erroneous error message (255.255.255.255 == 255.255.255.255).

The workaround consists of two steps:

On HA systems only:

When a non-managed file system is converted to a managed file system in an HA pair, it is possible for the FSMPM on the secondary MDC to continue to operate this FSM as non-managed, which incorrectly allows the FSM to start on the secondary MDC.

Restarting the CVFS service corrects the problem. Quantum recommends taking the following steps as a temporary workaround after converting any non-managed file systems to managed file systems:

Use caution when configuring the netmask for the HA Virtual Interface (VIP).

The VIP is an alias IP address that is associated with a real interface. For example, if the VIP is based on eth0, eth0:ha will be created as the VIP.

The netmask you associate with the VIP should generally be the same as that of the base interface, but in no case should it be more specific. For example, if the netmask on eth0 is 255.255.224.0 (a /19), then configuring the VIP netmask as anything more than a /19, such as a /24 (255.255.255.0) would be incorrect.

Using the same /19 mask on both eth0 and eth0:ha is the correct approach.

Note: The above applies only when the IP address of the VIP falls into the subnet defined by the base interface’s IP address and mask.

Quotas can only be enabled or disabled by modifying the Quotas parameter of the file system configuration file. The CLI snquota -L -Ffile-system command informs you whether the file system has quotas enabled.

Note: For a given file system, you can enable quotas in the StorNext GUI within the Advanced Parameters > Features tab. Click Quotas to toggle the quotas function. For additional information on how to manage quotas, see Manage Quotas.

If you run multiple fsretrieve commands simultaneously to find files (for example, find -type -f | xargs fsretrieve), you might receive error messages because doing this taxes system resources.

Instead, use the recursive retrieve command. When you use this command the files under a directory are retrieved in batches, and more sorting is done to put files in tape order for increased performance. Run recursive retrieve by entering
% fsretrieve -R .

As of StorNext 5, stripe group expansion, also known as bandwidth expansion, is not supported. The functionality works but has been deprecated and should not be used.

For example, if you create a file system that has two different-sized disks in a userdata only stripe group and then attempt to add a new disk to that stripe group and expand it, the expansion will fail.

In some cases the physical IP address must be included in the dpserver file in addition to the interface name. Note these conditions:

When there is one IP address associated with a NIC interface, the interface name alone is a sufficient identifier

If there are multiple IP addresses associated with a NIC interface, one IP address is required in addition to the interface name

On HA systems, the physical IP address is required if virtual IP is configured for the NIC interface. For additional information, see StorNext LAN Clients in HA Environments.

By design, replication or deduplication must be completed before data files can be truncated if these files are associated with both a replication/dedup policy and a Storage Manager policy. Even if the Storage Manager policy is configured with the “Truncate Immediately” option, the truncation may not occur at store time unless the file has been replicated or deduplicated.

Note the following recommendations and limitations for using DXi as a virtual tape library for StorNext:

Recommended library emulation: “ADIC Scalar i2000”

Recommended tape drive emulation: “IBM LTO-x” or “HP LTO-x”

DDM (Distributed Data Mover): This feature is currently not supported due to lack of full SCSI3 support in DXi.

When a file system with two affinities is to be managed by the Storage Manager, the GUI forces those affinities to be named tier1 and tier2. This will cause an issue if a site has an existing unmanaged file system with two affinities with different names and wants to change that file system to be managed. There is a process for converting a file system so it can be managed but it is non-trivial and time consuming. Please contact Quantum Support if this is desired.

Note: The restriction is in the StorNext GUI because of a current system limitation where affinity names must match between one managed file system and another. If a site was upgraded from a pre-4.0 version to post-4.0, the affinity names get passed along during the upgrade. For example, if prior to StorNext 4.0 the affinity names were aff1 and aff2, the GUI would restrict any new file systems to have those affinity names as opposed to tier1 and tier2.

StorNext does not currently support converting from a managed file system to an unmanaged file system.

For more information about JournalSize, refer to FSBlockSize, Metadata Disk Size, and JournalSize Settings.

StorNext does not currently support running two truncation policies for different policy classes at the same time.

Workaround:

To avoid errors, do not run two truncation policies for different classes at the same time.

Caution: As the File System Manager (FSM) supports over 1000 clients (with more than 1000 file requests per client), the resource limits of your MDC may be exhausted with additional load from other processes. Exceeding the file descriptor limit will cause errors to your system. Quantum recommends you not run additional applications on the MDC.