Offline File Manager (for Apple macOS)

If you are using a StorNext HA environment, then you must use a StorNext HA virtual IP (VIP) address on your system for the Offline File Manager application to work properly.

See How to Add or Change the System VIP Address.

For proper operation of the Finder when using the Offline File Manager (for Apple macOS) application, it is important that you exclude Apple double files from truncation. This is the default behavior for StorNext 6; however, Quantum recommends you verify the configuration.

You must also exclude the configuration file used by the Offline File Manager (for Apple macOS) application from truncation, or it will not function.

The configuration file /usr/adic/TSM/config/excludes.truncate must include:

The Finder also maintains metadata content in ._DS_Store; storing this file triggers work for Storage Manager, which is not needed. You can control this by using /usr/adic/TSM/config/excludes.store, which must include:

On Systems Running StorNext 6.3.0 or Later

The named streams feature (see Add a File System) allows Offline File Manager to reject Finder (macOS) or Explorer (Microsoft Windows) from retrieving a truncated file.

If you enable named streams on your file system, then you do not have to configure your system to disable retrieves. The Offline File Manager performs operations to disable the client using the application.

Before you perform any browsing operation on your client, verify the Offline File Manager is running to prevent unintentional retrieves in Finder (macOS) or Explorer (Microsoft Windows).

Note: The addition of extended attributes to control a client’s ability to retrieve files allows the MDC to have retrieves denied. In StorNext 6.3 (or later), if retrieves are denied on the MDC, then some other operations may not complete. The most common need for an MDC to have retrieves disabled is when the MDC is used as a NAS server. If it is not a NAS server, then you can enable retrieves . For example:

Note: If you were previously using an nss_cctl.xml file only to deny retrieves for clients, you can remove the file. If you do not use named streams or a central control file, you will be able to open a truncated file.

On Systems Running StorNext 6.0.x to 6.2.x

To disable on-demand retrieve operations, you must configure the Central Control feature of StorNext. Central Control allows you to restrict the access that different machines are granted to the file systems in a cluster. Currently there is no GUI option for this, so you must log into your MDC.

1. Log on to an SSH client, and connect to your system.

   Central Control is managed from the StorNext nameservers by an XML configuration file in the directory /usr/cvfs/config/nss_cctl.xml (see the nss_cctl(4) command in the StorNext Man Pages Reference Guide for additional information). These nameservers must also be on StorNext 6.x if they are not the same system as the MDC.

   For initial configuration, the command nss_cctl_template is available. For example, nss_cctl_template>/usr/cvfs/config/nss_cctl.xml.

2. At the prompt, enter the following command:

   nss_cctl_template>/usr/cvfs/config/nss_cctl.xml

   The nss_cctl.xml script uses the snprobe command to walk the current cluster and output a template file based on the cluster it finds. This tool generates a permissive configuration file with all capabilities enabled for all visible clients.

3. Edit the configuration properties in the nss_cctl.xml file.

   The configuration file contains a securityControl section for each file system, and a separate section at the end for the fake file system #SNFS_ALL#. When looking for the controls for a file system, it invokes a command that queries for a file system specific record and then falls back to the one for #SNFS_ALL#.

   Within each file system there are one or more controlEntries. Each control entry specifies the addresses of one or more clients, and the controls to be applied to them.

   Tips for Configuring Client Retrieval and Denial

   • If there are a mix of clients, some that wish to deny retrieves and others that do not, create separate control entries within the file system’s security control tags.
   • For DLC clients, add the client’s hostname or IP address to the client section with the desired controls.
   • For NAS, controls can only be set for the NAS server. This is due to the NAS server’s IP being used for all operations. Clients connected via NAS cannot be controlled on a per client basis.
   • For HA, use the VIP in the client section.

   For example:

   <?xml version="1.0" ?>
   <snfsControl xmlns="https://www.quantum.com/snfs/cctl/v1.0">
     <securityControl fileSystem="snfs1">
       <controlEntry>
         <client>
           <address value="fruit.company.com"/>
           <address value="banana.company.com"/>
           <address value="mdc-1.company.com"/>
           <address value="mdc-2.company.com"/>
         </client>
         <controls>
           <mountReadOnly value="false"/>
           <mountDlanClient value="true"/>
           <takeOwnership value="true"/>
           <exec value="true"/>
           <suid value="true"/>
           <denyRetrieves value="true"/>
           <globalSuperUser value="true"/>
         </controls>
       </controlEntry>
       <controlEntry>
         <client>
           <address value="orange.company.com"/>
           <address value="apricot.company.com"/>
         </client>
         <controls>
           <mountReadOnly value="false"/>
           <mountDlanClient value="true"/>
           <takeOwnership value="true"/>
           <exec value="true"/>
           <suid value="true"/>
           <denyRetrieves value="false"/>
           <globalSuperUser value="true”/>
         </controls>
       </controlEntry>
     </securityControl>
     <securityControl fileSystem="#SNFS_ALL#">
       <controlEntry>
         <client>
           <address value="0.0.0.0/0"/>
         </client>
         <controls>
           <mountReadOnly value="false"/>
           <mountDlanClient value="true"/>
           <takeOwnership value="true"/>
           <exec value="true"/>
           <suid value="true"/>
           <snfsAdmin value="true"/>
           <snfsAdminConnect value="true"/>
           <denyRetrieves value="false"/>
           <globalSuperUser value="true"/>
         </controls>
       </controlEntry>
     </securityControl>
   </snfsControl>

   Central Control does not allow access to hosts that are not listed. However, it does support a netmask when specifying hosts, so a subnet can be covered by one record if desired.

   Note: If you add new clients, you must add them to the securityControl control configuration.

4. Edit the denyRetrieves control entry, within the nss_cctl.xml file, by setting it to true.

   <denyRetrieves value="true"/>

   The denyRetrieves control prevents a client from triggering on-demand file retrieves. Set this to true to prevent the client from triggering retrieves. This control functionality runs on the MDC, so older clients will have the control enforced after you have configured the file nss_cctl.xml.

   The client controls relate to SAN and LAN clients. SMB and NFS client control is routed through those client's NAS server and cannot be individually controlled. This is particularly important when configuring denyRetrieves for Offline File Manager.

   • SAN and DLC clients: The addresses of the clients themselves.
   • NAS clients via a NAS server on an HA pair: The VIP of the HA pair.
   • NAS clients via a NAS server on an MDC: The address of the MDC.
   • NAS clients via a NAS server on a gateway: The address of gateway.

   Note: If you set denyRetrieves to true for a NAS server, then this disables retrieves for that server and all NAS clients connecting using that server.

5. After you have configured the file, add it to the nameserver nodes.

   The nameserver processes will pick up an updated configuration on service restart, or on sending a SIGHUP to the fsmpm process. The updated configuration is only seen by clients at mount time, and by the FSM when a client connects.

   You can perform this by finding the pid and using kill to send the signal. To send a signal to the fsmpm, follow the commands below:

   For example:

   [root@node ~]# ps -ef | grep fsmpm

   root 9772 1 0 Dec11 ? 00:00:00 /bin/bash /usr/cvfs/lib/LogToFile fsmpm /usr/cvfs/debug

   root 9777 9772 0 Dec11 ? 00:00:00 /bin/bash /usr/cvfs/lib/LogToFile fsmpm /usr/cvfs/debug

   root 9778 1 0 Dec11 ? 00:00:00 fsmpm-watcher

   root 9779 9778 0 Dec11 ? 00:05:03 /usr/cvfs/bin/fsmpm -- -- /usr/cvfs/debug/fsmpm-sync

   root 23056 22583 0 08:00 pts/4 00:00:00 grep --color=auto fsmpm

   [root@node ~]# kill -HUP 9779

The Offline File Manager application requires information about the location of the MDC and the web services configuration to operate. This information is placed in a JSON configuration file in the root of the file system, or in the case of NAS access, in the root of each NAS share.

Below is the man page help output for the snwebsetup command:

snwebsetup [-h] [-u USER] [-p PASSWORD] -s SERVER dirs [dirs ...] [-g GATEWAY] [-a STOREACTIONS]

Setup remote rest access to filesystem or share

arguments:
  dirs                  directories to setup
  -h, --help            show this help message and exit
  -u USER, --user USER
                        web services user
  -p PASSWORD, --password PASSWORD
                        web services password
  -s SERVER, --server SERVER
                        web services URI
  -g GATEWAY, --gateway GATEWAY
                        Add api gateway entry
  -a STOREACTIONS, --storeactions STOREACTIONS
                        Enable Offline File Manager store actions

At the command prompt, enter one of the following commands:

• For the HTTPS protocol:

  snwebsetup -u <username> -p <password> -s https://mdc.company.com /stornext/<SNFS> -g <URL of the registered API gateway> -a enabled

• For the HTTP protocol:

  snwebsetup -u <username> -p <password> -s http://mdc.company.com:81 /stornext/<SNFS> -g <URL of the registered API gateway> -a enabled

The server option must specify the web address and protocol of the web services, so it expects a URL (for example, https://mdc.company.com or http://mdc.company.com:81).

The user and password fields must match an account that has been set up with access to web services. The application uses them to access the services.

Note: When you use the service over NAS, the online/offline state of a file is determined by file attributes which require StorNext 6.2 (or later).

On systems running StorNext 6.3.0 or later:

To use the icon overlay functionality, your file system must meet the following requirements:

• You must enable Named Streams on the file system.
• The file system must have access for the client to open them.
• The icons are not available as a NAS client.

If the requirements are not met, the icons only display the state of the file (online/offline).

Note: Due to a Microsoft Windows limitation, if you have other applications that use icon overlays, then your Offline File Manager icon overlays might not appear.

You can use the StorNext GUI or the CLI to enable named streams; do one of the following to enable named streams:

Use the StorNext GUI

1. On the menu, click Configuration, and then click File Systems.
2. Select a file system, and then click Edit....
3. Click Configuration Parameters.
4. Click Features.
5. Click Named Streams.
6. Click Apply. You are prompted to confirm the configuration change.
7. Click Yes.

Use the CLI

1. Edit your stornext.cfgx file:

   <snfs:globals>
       <snfs:namedStreams msgId="namedStreams">true</snfs:namedStreams>
       <snfs:storageManager msgId="storageManager">true</snfs:storageManager>
       <snfs:bufferCacheSize msgId="bufferCacheSize">1073741824</snfs:bufferCacheSize>
   </snfs:globals>
   

2. Stop the file system:

   [node@node] /usr/cvfs/bin/cvadmin -e 'stop snfs1'

3. Run cvupdatefs:

   [node@node] /usr/cvfs/bin/cvupdatefs
   Checked Build disabled - default.
   StorNext File Systems on host snt375492-mdc1:
   
   1) *snfs2 
   2)  snfs1
   
   The asterisk (*) denotes the file system is active.
   
   Choose a file system by number (1-3) or zero (0) to exit -> 2
   
   Buf_init: L1 size 1024 MB, L2 size 0 MB
   
   The following changes have been detected in the configuration
   Please review these changes carefully.
   
   Stripe Group Name          Stripe Status               MetaData     Journal
   =================          =============               ========     =======
   sg0                        No Change                   No Change    No Change
   
                                                 
   *** Named streams will be enabled in the file system. ***
   
   *Warning*: This will modify the file system "snfs1"
   
   Are you sure you want to continue? [(y)es/(N)o] y
   

4. Start the file system:

   [node@node] /usr/cvfs/bin/cvadmin -e 'start snfs1'

When browsing content, either using Finder or the standard open file dialog boxes in an application, the Finder plug-in is activated for any StorNext content that has the JSON configuration file in its root, and for NAS content that has the configuration file in the root directory of the system.

macOS only supports a single finder extension active on a directory at once. If you have another extension installed and managing a directory, Quantum recommends you disable it, because only one of the extensions is used. Apple does not provide a mechanism to select one over the other.

Files that do not have a secondary copy have no overlay icon. The preview icon of a file may still be presented; however, this is not retrieved from the file content when the file is offline. It is retrieved from a copy previously cached by Quick Look.

Beginning with StorNext 6.3.0, you can use a richer set of icon overlays to describe the metadata of a file.

Note: Offline File Manager actions might not function on files that contain illegal NTFS characters. Quantum recommends you use the /usr/cvfs/lib/snfsnamescanner feature to locate and rename illegal names in your StorNext file system. See the SNFSNAMESCANNER(8) command in the StorNext Man Pages Reference Guide for details.

The table below illustrates the icons and their descriptions.

Icon	Description
	This file is online and has multiple tiers.
	This file is online and has object-store media backing.
	This file is online and has a tape library backing.
	This file is online and has an sdisk backing.
	This file is online.
	This file is offline and has multiple tiers.
	This file is offline and has object-store media backing.
	This file is offline and has a tape library backing.
	This file is offline and has an sdisk backing.
	This file is offline.

Note: For requirements, see Configure a File System To Enable the Offline File Manager Application above. If you do not meet the requirements, only the green check icon overlay and the red check icon overlay are displayed.

The image below illustrates how some files are truncated and other files are available on primary storage. You can see that some files have multiple copies while other files only have a single copy on a tape library.

Even with the Offline File Manager (for Apple macOS) installed and running, a truncated file cannot be directly read; it must be retrieved first. To do this, select a file and bring up the context menu for it (right click). The Finder menu includes an offline actions menu, which allows you to truncate, store to secondary, or retrieve content from secondary.

Caution: Do not perform an Offline File action on more than 10,000 selected files or directories at a time.

The files appear in a working state until the retrieve operation is complete, then the icon overlay changes to represent the current state.

If the file system was configured without placing the web services user name and password into the configuration, the first time one of these items is selected, a pop-up window asks you for the account information. After you enter the account information, the credentials are stored on the Keychain and normally are not requested again.

After an action completes, a notification and a button appears which you can click to view the log.

If the notification disappears or you need to view a log, click Open Log in the drop-down menu.

You can click the Enable recursive retrieves menu option to enable recursive retrieves. By default, Offline File Manager allows an action on only the first level down from a directory. When you enable recursive retrieves, you can perform an action on the entire directory tree.

Caution: Use this option with caution as you might unintentionally retrieve or truncate millions of files.

Beginning with Offline File Manager (for Apple macOS) 2.1.0, you can use file system pooling to move files and folders between defined pools of NVMe, SSD, and HDD within your file system.

To use the feature, your system must meet the following requirements:

• Use an API gateway; your API gateway allows Offline File Manager to locate a pooling host, assuming you configure pooling on a client or MDC, and also register the client or MDC with your API gateway.

• Enable file system pooling on a client or MDC.

  • Register your client or MDC with your API gateway; to register your components with your API gateway, see API Gateway Registration and Configuration.

• You must initiate a Find tiering service operation at least one time on your macOS client.

  Initiate a Find Tiering Service Operation

  Do the following to initiate a Find tiering service operation on your macOS client.

  1. On the menu bar, click Offline File Manager, and then click Find tiering service.

     

     • If the pooling service is successfully enabled, then you can perform file system pooling operations. The Tiering is enabled dialog appears.

       

     • If you did not previously enter your API gateway user credentials, then you are prompted to enter the credentials in the API Gateway Login dialog.

       

  2. Click OK.

There are two interface options that allow you to perform a file system pooling operation.

Use the Action Window (Extended File Data) Dialog

You can use the Action Window (Extended File Data) dialog to obtain extended file information and perform a pooling operation. Since Offline File Manager allows you to perform a pooling operation on multiple files, the Action Window (Extended File Data) displays the information for the first file; however, the pooling operation is applied to all the selected files.

Note: File system pooling is determined by Offline File Manager's ability to locate a pooling host. If a pooling host is not located, then the pooling operation functions and buttons are disabled.

Note: If a file is truncated, then the pooling operation functions and buttons are also disabled since the file is not on primary storage and pooling is not always configured to retrieve files.

Considerations for a Directory

When viewing the extended file attributes of a directory, the Primary Tiering Location value that is displayed in the Action Window (Extended File Attributes) dialog indicates the storage pool location where new files in the directory are placed, and not the storage pool location of files and directories within the directory and below it. For example, the image below illustrates the directory labeled testdirectory is configured with a storage pool location of slow (any new files created in testdirectory are placed in the slow storage pool location). However, the two files (fast.txt and slow.txt) within testdirectory reside in a specified storage pool location. The fast.txt file resides in the fast storage pool location, and the slow.txt file resides in the slow storage pool location.

Do the following to display the Action Window (Extended File Data) dialog and perform a pooling operation:

1. In Finder, navigate to your file system and then select multiple files.

2. Right-click the multiple files, click Offline Actions, and then click Action Window.

   

3. The Action Window (Extended File Data) dialog appears.

   

   The table below provides a description of the fields in the Action Window (Extended File Data) dialog.

   Field	Description
   Primary Tiering	Displays the status of the file system pooling process on the selected file(s): Enabled: File system pooling is enabled on the file(s). Disabled: File system pooling is disabled on the file(s).
   Primary Tiering Location	Displays the storage pool location on the selected file(s).
   Location Store Time Class Policy Truncate Time	The Location, Store Time, Class Policy, and Truncate Time, fields are snippets of the fsfileinfo command that describe a file’s Storage Manager policy.
   Filepath	Displays the directory path of the selected file(s).

4. Click a file system pooling operation:

   • Promote

   • Demote

   • Move

   • Remove

   • Truncate

   Note: For more information about a file system pooling operation (for example, Move, Promote, Demote, and so on), see StorNext File System Pooling.

5. The Destination dialog appears.

   

6. Click a pool name in the Destination field, and then click OK. A notification displays the beginning and end of a successful pooling operation.

Use the Primary Tiering Dialog

You can use the Primary Tiering dialog to perform a path based or a pattern based pooling operation.

Do the following to display the Primary Tiering dialog and perform a pooling operation:

1. On the menu bar, click Offline File Manager, and then click Perform tiering operation.

   

   The Primary Tiering dialog appears.

   

2. In the Primary Tiering dialog, complete the following:

   Option	Description
   Action	Click a file system pooling operation: Demote Move Promote Remove
   Mount	Click a file system pooling mount point. Note: Use the text box as an extension of the Mount point select. For example, if you select \Volumes\snfs\ as the mount point and you input movies\10-11-2020\ as the path, then the pooling operation is performed on Volumes\snfs\movies\10-11-2020\. Note: This is also a recursive action due to the final slash. Note: The Pattern type uses the same logic with separate syntax rules. It is important that you to understand a path and a pattern are handled differently. For example, if your input is \Volume\snfs\my_policy\file1, then the Path operation type treats your input as a single file. If you click Pattern for your Type, then the pooling operation treats your input as a search within the directory its entire sub-directory (\Volumes\snfs\my_policy) for the pattern file1. If there are multiple directories in the sub-directory, then all directories are checked for files that match the pattern. These files, if needed, are moved. For example: \Volumes\snfs\my_policy\file1 \Volumes\snfs\my_policy\test\file1 \Volumes\snfs\my_policy\first\last\one\two\file1
   Type	Click a file system pooling operation type: Path Pattern Note: For more information about a pooling pattern rule, see StorNext File System Pooling.

3. Click OK to continue the pooling process, or click Cancel to return to the previous dialog. A destination dialog appears where you can select the pool the operation is applied to.

   

4. Click a pool name in the Destination field, and then click OK. A notification displays the beginning and end of a successful pooling operation.

Troubleshoot Offline File Manager File System Pooling

• If a truncated file can be opened on the Offline File Manager client and the system meets any of the requirements below.

  Requirement A

  Appliance Controller (NAS) is installed on the primary MDC or standalone MDC and the client is accessing a file system through NFS/SMB.

  Note: The configuration is not supported to work with Offline File Manager.

  The primary/standalone MDC cannot have retrieves denied. This is due to Web Service requests to retrieve data that are blocked. To resolve this issue, you must have Appliance Controller on the secondary MDC or a StorNext gateway client. This allow the Offline File Manager client to be blocked by the nss_cctl.xml file as described above, by adding the secondary MDC or the StorNext gateway client IP address to a control element that has denyRetrieves set to true.

  Note: If you need to use the central control file (/usr/cvfs/config/nss_cctl.xml) to resolve this issue, see the Central Control section above (below the heading On Systems Running StorNext 6.0.x to 6.2.x).

  Requirement B

  If you do not enable Named Streams, then Offline File Manager cannot configure the system to deny retrieves. To configure the system, you must create a central control nss_cctl.xml file and then add all the IP addresses of your clients to the list where the control element has denyRetrieves set to true.

• Error message indicates a possible issue with credentials.

  If you receive an error message about credentials, then verify Keychain Access contains the correct information. The user information for pooling is the same user information used when you configure your API gateway.

  Note: The Keychain Access might be different than the Web Services user credentials.

  Do the following to edit or remove the Keychain Access.

  1. Open Keychain Access.

     

  2. Search for the API gateway host name, called apigateway.credentials.

  3. Right-click the Internet Password type that matches:

     • For File System Pooling, it is an API gateway host name.

     • For Web Services, it is the StorNext user interface IP address.

       Caution: Do NOT delete any entry that is not an Internet Password type.

  4. You can edit the entry or delete the entry.

  5. Restart the Offline File Manager application.

• Offline File Manager does not open after you double click the application icon.

  Verify you have Python 3.7.9 installed on your system.

• An incorrect pooling location appears on your Destination menu.

  When you perform multiple pooling operations on the same file from the Action Window (Extended File Data) dialog, an incorrect Current Location appears due to a caching bug within macOS. If you close the Action Window (Extended File Data) dialog and then open it, the correct location should appear but the cache is not always invalidated from this process. The best methods to address this issue are to (a) navigate away from the file and (b) allow additional time between processes.

Do the following if the icon overlays do not appear.

1. Open the Terminal application.

   

2. At the terminal prompt, execute the following command to restart Finder:

   killall Finder

   For example:

   

This error occurs when snwebsetup has not been ran on the appliance to configure the web services for the Offline File Manager/Commandline Tools. If this is not the case, this issue can also occur as a DNS issue.

Do the following to ping your appliance using its hostname.

1. Open the Terminal application.

   

2. At the terminal prompt, enter the following command:

   ping <hostname>

3. After a few seconds, stop the ping operation; press Ctrl+C. If that fails (100% packet loss), enter the following command:

   flushcache; sudo killall -HUP mDNSResponder

4. At the terminal prompt, execute the following command to restart Finder:

   killall Finder

   For example:

   

This issue prevents the Offline File Manager from writing to a named stream. This occurs when you do not have write permissions on the file system.

Depending on the security model you configured, do one of the following to grant write permissions.

Method 1

To determine if this issue occurs on your system, attempt to add (or edit) a file in the file system. If you are required to authenticate, the system requests to escalate the permissions.

Method 2

1. Open the Terminal application.

   

2. At the terminal prompt, enter the following command:

   sudo xattr -p com.quantum.retrieves /path/to/stornext

   If the output is enabled, you have not denied retrieves.

3. If you want to disable retrieves, enter the following command:

   sudo xattr -w com.quantum.retrieves disabled /path/to/stornext