Prevent File System Fragmentation

This section provides technical information with respect to defragmentation fear, uncertainty, and doubt (FUD).

Overview — History and Baseline

File systems are subject to fragmentation due to unsophisticated disk space allocators, poor application I/O behavior, and "aging" (repeated file creation and deletion).

In addition, defragmentation tools used for making individual files contiguous often lead to free space fragmentation. File systems such as XFS, CXFS, GPFS, ETX3 have all suffered in this respect; it is very common in the industry.

Over time, companies have created their own so-called defrag tools to re-order files in directories to improve I/O performance.

The term defrag has also become overloaded in the industry, namely:

• defrag may refer to making the disk space for an individual file contiguous on disk.
• defrag may refer to making the frames for an image sequence contiguous on disk.
• defrag may refer to making the file system free space contiguous.

How does StorNext respond to fragmentation prevention and defragmentation for repairing?

StorNext responds to file system fragmentation in four ways:

1. StorNext has a file-system configuration option (allocation session reservation) you can use with performance sensitive workflows (prevention).

   Allocation Session Reservations (ASR) is patented technique you can use to automatically allocate contiguous space to files created within the same directory. This goes a long way to prevent fragmentation in the first place. StorNext file systems have ASR enabled by default and do not require configuration or modification to applications in order to benefit from its use, provided that frame files are simply written in the sequence they are expected to be played back. See Allocation Session Reservation (ASR) and How ASR Works for additional details.

2. The StorNext file system file defrag (snfsdefrag) utility allows you to rearrange the extents of existing files.

   The utility allows you to make disk space allocation for a file contiguous on disk. For example, this is useful when an application has written out a single large video file in a non-linear fashion resulting in suboptimal allocation which might lead to "dropped frames" on playback. You can also use the snfsdefrag utility to generate a detailed report of a file’s current disk allocations.

   See snfsdefrag in the StorNext 6 Man Pages Reference Guide for additional details.

3. The Video frame allocation inspector and resequencer (vidiomap) utility allows you to rearrange the on-disk space associated with DPX frames (files) within a directory.

   The utility allows you to make the allocations across files within a directory contiguous on disk. For example, if a directory contains DPX frame files, you can reorder the allocations so that play-out is smooth without any drops. If ASR is enabled and frames are written in order, vidiomap might not be necessary. Even so, there can still be use cases. For example, most "copy" programs read and write files in directory hash order (effectively random) instead of processing files in alphanumeric order. After a sequence of frames has been copied into a file system using such tools, it might be necessary to run vidiomap once before play-out to ensure disk space allocation is contiguous.

   See vidiomap in the StorNext 6 Man Pages Reference Guide for additional details.

4. The CLI command, sgdefrag, allows you to defragment file system free space.

   As mentioned previously, the free space of a file system can become fragmented for multiple reasons. This is unavoidable after a file system has "aged". This makes even the best allocators and tools, such as snfsdefrag and vidiomap ineffective because there simply is no remaining contiguous free space to draw from. To correct this, you can run the sgdefrag command to reallocate and provide free space that is more contiguous.

   See sgdefrag in the StorNext 6 Man Pages Reference Guide for additional details.