Linear Tape File System (LTFS) Media Format

StorNext for LTFS includes support for the following features:

• Policy-class-copy steering by media format type for LTO tape media
• Automatic media selection and formatting
• Distributed data mover support
• File versioning on LTFS media
• File recovery from LTFS media
• LTFS media defragmentation
• Tape-to-tape copy from ANTF to LTFS, and from LTFS to ANTF

• LTO7 media

• Import of native LTFS media into StorNext
• Export/import of StorNext managed LTFS media

Features NOT supported in StorNext for LTFS:

• Segmented files
• LTFS volume spanning
• StorNext backup to LTFS

Quantum StorNext can write data to tape using two different formats. The first format is a Quantum proprietary format referred to as ANTF and the second format is an open standard format from IBM called LTFS.

ANTF

The ANTF was created by Quantum for the most efficient usage and performance of reading/writing data from/to tape. It is the Quantum default and most heavily used tape format. It performs well, especially when used with high-duty-cycle tape workflows. The data for the files are self-contained within a cluster on tape.

• You can read tapes outside of StorNext utilizing the Quantum StorNext utilities.
• You can export ANTF tapes from one StorNext system and imported in another StorNext system.

Pros	Cons
Utilizes clusters to write smaller files as a group to tape. High performance, low metadata overhead. Fast reads/writes of data from/to tape using clustering of file data. Efficient utilization of space on tape (file data and metadata). Can be used as interchange between StorNext systems.	Not an open format. Cannot mount up a tape as a filesystem and read/write data from/to the tape. Cannot be used outside of StorNext without offline StorNext utilities.

When should I use ANTF?

• You should use ANTF for tape backup/tiering.
• In a more dynamic environment that is typical of the majority of StorNext customers, ANTF StorNext is the most efficient format and performs significantly better than the LTFS format for both reading/writing data and avoids wearing out tapes earlier than expected.

LTFS

LTFS is an open format from IBM. Using free utilities from various sources, you can mount a tape written in the LTFS format and can read it like a typical file system. With LTFS there are two partitions that are created on a tape.

• The first partition contains the metadata about the files on tape.
• The second partition contains the actual data along with copies of the first partition's metadata.

Pros

Cons

Open format. Can be read/written as a filesystem using certain utilities. LTFS does not require any external metadata tracking for operations.

Slower to read and write data from tape due to having to read the first partition information and then reading the data from second partition. LTFS can wear out tapes much earlier than expected, for example, in tape scenarios where there is a high-duty-tape workflow. The wear counters are incremented significantly due to having to read/write the data from/to the first partition quite often at the beginning of the tape many times. LTFS requires a larger memory footprint due to the entire first partition index information having to be read into memory. LTFS consumes more space for the metadata due to the fixed first partition and copies of the first partition written several times to the second partition.

When should I use LTFS?

• You should use LTFS to take your tapes out of StorNext and utilize in a non-StorNext system. For example, as an interchange format between foreign devices and StorNext.
• You should use LTFS for systems where the primary purpose is to archive data. For example, tapes remain in drives and are written to until they are filled.

Quantum recommends that you use ANTF tapes for typical operations and high-duty-cycle tape workflows.

Due to tapes being worn out early when you use LTFS, if LTFS is your preferred format, Quantum recommends that you use ANTF for typical reading and writing and then use the StorNext LTFS Export feature to create copies of the data on LTFS tapes. See Exporting and Importing LTFS Media.

Access of StorNext-generated LTFS media outside of StorNext (for example, by an Open LTFS package,) should be considered read-only.

Be aware that a StorNext-generated LTFS medium may be rendered unusable within StorNext if it is modified outside of StorNext. If the medium is required for data retrieval of truncated files within the StorNext file system, access to those files may not be possible if the medium is modified outside of StorNext.

Exporting and importing LTFS media allows you to move data out of one system and into another.

Use either the Export Media or Export Copy of Media action in the StorNext GUI to export storage media that contain file data from the Tertiary Manager system. After the export, the media can be physically removed, either by using Media Manager commands or by using the Library Operator Interface (LOI) in the StorNext GUI. After media are exported from one Tertiary Manager system, they may be imported into another Tertiary Manager system using the Import Media action.

Note: The Tertiary Manager system does not support LTFS segmented files.

When media is imported, it is marked as Write Protected to avoid key collisions in a Tertiary Manager database.

To write to imported media, execute the fsmedcopy command to move the contents from the imported medium to another medium. The imported medium reverts to a blank status after the contents are copied.

For additional information, see the commands fsexport(1) and fsimport(1) in the StorNext Man Pages Reference Guide.

Now that two different media formats (ANTF/LTFS) are supported for LTO media in StorNext, you should keep in mind some points about configuration and usage to make the best choices for your StorNext environment.

In the Steering tab in the Storage Manager Policies configuration page, in the Media Format list, you can select either ANTF or LTFS formats for the LTO (tape) Media Type.

Note: This only applies to tape formats; there are no Media Format selections available for either Object Storage or Storage Disks.

LTFS appears in the Configuration > Storage Manager Policies > Storage Manager tab (in the Steering column), and in the View page (in the Media Format column).

The LTFS media format also appears in the Reports > Media page (in the Media Format column).

Additional checks have been included in the StorNext Linux-based installation to ensure that some additional RPM packages required to support the LTFS media format are installed prior to installing StorNext. For additional LTFS requirements, see the StorNext Compatibility Guide, the StorNext Installation Guide, and the StorNext Upgrade Guide.

Following is a list of these items to consider:

• StorNext Storage Manager LTFS support is based on LTFS Format Specification v 2.0.0. The LTFS Format Specification does not support backwards compatibility with earlier versions of the specification.

• The LTFS Format Specifications v 2.0.0 and v 2.2.1 are compatible with each other. If a tape is formatted with version v 2.0.0 and is written to by StorNext, the tape format will be upgraded to v 2.2.1. Similarly, a tape formatted with version v 2.2.1 will be downgraded to a v 2.0.0 if it is written to by a release prior to StorNext 5.4.0.

• Neither LTFS Format v 2.0.0 nor v 2.2.1 provides a means for maintaining file ownership and permissions when storing files to LTFS media. Ownership and permissions are managed at LTFS file system mount time using options passed to the Open LTFS package commands. When media formatted with LTFS v 2.0.0 or v 2.2.1 is used within StorNext Storage manager, access is restricted to the root user and the tape is not accessible outside of the StorNext Storage Manager processes. When the media is mounted outside of StorNext Storage Manager, care should be taken to ensure that proper access permissions are used when the LTFS volume is mounted.
• Storage Manager does not support segmented files for LTFS. This includes files being stored as well as files being copied from tape to tape (fsfilecopy/fsmedcopy). To manage this, the configuration parameter DEF_MED_SPC_LTO_LTFS in the file /usr/adic/TSM/config/fs_sysparm_override can be used to specify the maximum file size that can be stored to LTFS media.
• StorNext supports a maximum tape block size of 1MB for LTFS. To manage this, the configuration parameter FS_LTO_LTFS_BLOCK_FACTOR in the file /usr/adic/TSM/config/fs_sysparm_override can be used to calculate the LTFS tape block size and limit it to 1MB for LTFS formatted media.
• LTFS allocates at least one tape block for each file, and no two files share the same tape block. Therefore, every file consumes an amount of space on tape that is a multiple of the tape block size. As a consequence, very small files do not make efficient use of tape capacity, especially when large tape block sizes are configured.
• StorNext Backup (_adic_backup policy class) is not supported for LTFS.
• Because LTFS is a file system on a tape, additional overhead is incurred when Storage Manager processes have to mount and unmount the LTFS file system. Because of this, ANTF will outperform LTFS in most situations. Therefore, StorNext users SHOULD NOT configure LTFS for the primary copy used for file retrieval (for example, copy 1).

• To minimize this additional overhead, you can configure the following system parameters to process high file counts and high byte counts for store and retrieve requests (see the /usr/adic/TSM/config/fs_sysparm.README file for more information on the parameters):

  Parameter	Configuration Setting
  MAX_FILES_PER_CLUSTER	In the /usr/adic/TSM/config/fs_sysparm_override file; keep the default of 3000.
  FS_CLUSTER_LIMIT_LTO	In the /usr/adic/TSM/config/fs_sysparm_override file; increase to: 1,000,000,000,000 (1 TB) for LTO-L5 media. 2,000,000,000,000 (2 TB) for LTO-L6 media. 6,000,000,000,000 (6 TB) for LTO-L7 media. 9,000,000,000,000 (9 TB) for LTO-M8 media. 12,000,000,000,000 (12 TB) for LTO-L8 media.
  FS_CLUSTER_LIMIT_LTO	Set to the lowest recommended LTO generation value in environments where multiple LTO generations are being used with LTFS.

• The amount of metadata stored in the LTFS index partition grows as files are added to an LTFS volume. This metadata is always loaded into the fs_fmover process memory when the LTFS volume is mounted, and continues to grow as files are added to the LTFS volume. This means that memory requirements for the fs_fmover process are larger when the source and/or destination media format type is LTFS as compared to when the source and destination media format types are both ANTF.

• The LTFS standard prohibits the use of the following characters in directory and file names:

  • : (colon)
  • / (slash)

  WARNING: Attempting to store files to LTFS that have prohibited characters in directory or file names in the path will fail and cause an admin alert to be generated.

  These files will then be removed from the store policy candidate list by having the FS_NO_STORE flag set in the file attributes. To successfully store these files, they must be renamed to no longer contain any prohibited characters in any of the directory or file names in the path. After the files have been renamed, you must clear the FS_NO_STORE attribute flag using the fschfiat -sp command or the Tools > File and Directory Actions > Modify File Attributes GUI page so that the policy can place the files back on the store candidate list. Keep in mind that all future file modifications will be ignored and no additional copies will be made of these files, regardless of media type or format, until the FS_NO_STORE attribute flag has been cleared.

  WARNING: Quantum recommends that you avoid using certain characters for interoperability between operating systems. Do not use the following characters in either directory or file names if the files are to be stored to LTFS media:

  • * (asterisk)
  • ? (question mark)
  • < (left angle quote)
  • > (right angle quote)
  • " (quotation mark)
  • | (vertical bar)
  • \ (backslash)

  While not prohibited on Linux or macOS, you cannot use these characters on Windows because of additional restrictions for characters used in directory and file names.

• When you use LTFS as your storage medium, LTFS rewinds the tape and writes to a special partition on the tape. These rewinds result in an increase of wear and tear on the tape. By default, StorNext rewinds and dismounts a tape after five minutes (you can modify this setting during the drive configuration). If you want to avoid rewinding and dismounting the tape often, Quantum recommends that you set the Delay Dismount time for the drive to a minimum value of 20 minutes.