Overview

To store data on tape, the objects need to be stored in the S3 Glacier or the S3 Glacier Deep Archive Storage Classes via the S3 API. This can be accomplished in two ways:

• Store objects directly in the S3 Glacier Storage Class when writing the object to the system via an S3 PUT request.

• Transition objects to the S3 Glacier Storage Class via an S3 Lifecycle Policy.

Customers can not directly read objects which are stored on tape. Instead, they must launch S3 Restore requests which will restore a copy of the chosen objects on the HDD tier for a user-specified number of days.

This approach to handle cold data via the S3 API is exactly the same as the cold data handling in AWS S3 Storage, making ActiveScale fully compatible with AWS S3 Storage with respect to the handling of cold data. A more detailed explanation of the S3 API for Cold Storage can be found in the S3 API Section.

An ActiveScale Cold Storage system always consists of the following components:

• A regular ActiveScale HDD-based system - This is used to store metadata of all objects and data of the objects stored in the HDD tier. It is also the main entry point into the system for S3 operations and system management.

• 1 or more tape libraries - These host the tape cartridges where cold data will be stored and the tape drives that will be used to read and write data to the tape cartridges.

• 1 or more Tape Connectors - These are physical servers used to facilitate the communication between the ActiveScale system and the tape drives.

The physical layout of an ActiveScale Cold Storage system typically looks like this:

Objects which are stored on the Tape tier have metadata associated with them, just like objects stored on the HDD tier. The metadata of all objects across all storage tiers is stored in ActiveScale’s metadata tier, which is hosted on SSDs for performance reasons. This allows efficient metadata operations like S3 HEAD and S3 LIST for all objects stored within an ActiveScale system, independent of their storage tier.

Moving data which is stored on the HDD tier to the Tape tier is a process that consists of several steps:

1. A large number of objects that need to be stored on Tape media are identified and grouped together. The 2D Erasure Coding is applied to this group of objects and the resulting data fragments are temporarily stored on the HDD tier as well.

2. The original data of the objects is deleted, since all this data is also stored within the 2D Erasure Coded fragments.

3. The 2D Erasure Coded fragments are written to tape, taking into account the data spreading rules. This is done in such a way that we achieve uninterrupted high performance when writing to each tape drive, in order to avoid frequent starting and stopping of the tape media movement (referred to as 'back-hitching' or 'shoe-shining'). End-to-end data integrity checksums are used to ensure the integrity of the data which is written to tape.

4. When the 2D Erasure Coded fragments are safely stored on tape media, they are removed from the HDD tier.

The ActiveScale Software will only start writing data to tape when enough data is ready to be stored on tape. This usually corresponds to a minimum of several TBs of data. Nothing will be written to tape when not enough data is ready, but data will still be retrievable via restore requests like any data written to the S3 Glacier Storage Classes. This allows the Software to only do large streaming writes to tape, which is optimal for performance and also for data durability and integrity.

If the system needs to restore a copy of some objects to the HDD tier, it will try to do that in an optimal way as well.

When S3 restore requests are launched by the S3 client, the system will wait a certain period of time before actually starting to restore data from tape. This will allow the system to potentially batch several restore requests for the same tape media together and complete all of them in the most efficient way possible. The actual time that the system will wait depends on the Retrieval Tier of the restore requests: Expedited, Standard or Bulk. The wait time will be the shortest for Expedited retrievals and the longest for Bulk retrievals. When only a few objects need to be restored, an Expedited retrieval will usually result in the best restore performance. On the other hand, when a large number of objects needs to be restored, a Bulk retrieval will usually result in the best restore performance.

The system will also optimize the actual reading of data from tape. When a tape cartridge is mounted in a tape drive to read data from, all read requests for this tape cartridge will be executed in one go, and they will be executed in an order that optimizes the read performance as best as possible.

The total restore performance that is experienced by an S3 client will thus depend on a number of variables: the number of restore requests, the number of tape drives, the fill rate of the tape libraries, the 2D Erasure Coding policy, the number of tapes from which data needs to be read, and the Retrieval Tier.

The tables below summarize ActiveScale X100 durability policies.

Table 2: Storage policies for a single site ActiveScale X100

Policy Name	Description	Details
Performance optimized, single site	This policy provides the highest number of transactions per second. It provides a usable capacity of 72% of the raw disk capacity. The system will self-heal up to five simultaneously failed drives.	Large files: 18/5 Small files: 7/5 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average object size: 825-1154 KB depending on disk capacity. Overhead: Large files: 1.38x, Small files: 4x Durability: Up to 19 nines Availability: Data is available even if one Storage Node is unavailable and one additional drive fails.
Balanced, single site	This policy provides balance between usable capacity, durability, and throughput. It provides a usable capacity of 75% of the raw disk capacity. The system will self-heal up to five simultaneously failed drives.	Large files: 20/5 Small files: 7/5 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average object size: 856-1199 KB depending on disk capacity. Overhead: Large files: 1.33x, Small files: 4x. Durability: Up to 19 nines Availability: Data is available even if one Storage Node is unavailable and one additional drive fails.
Capacity optimized, single site	This policy provides the highest usable capacity. It provides a usable capacity of 80% of the raw disk capacity. The system will self-heal up to four simultaneously failed drives.	Large files: 20/4 Small files: 6/4 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average object size: 913-1279KB depending on disk capacity. Overhead: Large files: 1.25x, Small files: 3.5x. Durability: Up to 16 nines Availability: Data is available even if one Storage Node is unavailable.
Default, single site		Performance optimized with a small file threshold of 524,288 bytes.

Table 3: Storage policies for a multi-site ActiveScale X100

Policy Name	Description	Details
Availability optimized, multi-site	This policy has the highest availability. When a single site is offline and up to two drives fail in the remaining sites, all data is still available. It provides a usable capacity of 56% of the raw disk capacity. The system will self-heal up to eight simultaneously failed drives.	Large files: 18/8 Small files: 10/8 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average object size: 634-888 KB depending on disk capacity. Overhead: Large files: 1.8x, Small files: 5.5x Durability: Up to 19 nines Availability: Data is available even if one site is unavailable and two additional drives fail.
Balanced, multi-site	This policy provides balance between usable capacity and availability. When a single site is offline and one drive fails in the remaining sites, all data is still available. It provides a usable capacity of 61% of the raw disk capacity. The system will self- heal up to seven simultaneously failed drives.	Large files: 18/7 Small files: 9/7 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average object size: 698-977 KB depending on disk capacity. Overhead: Large files: 1.64x, Small files: 5x Durability: Up to 19 nines Availability: Data is available even if one site is unavailable and one additional drive fails.
Default, multi- site		Availability optimized with a small file threshold of 524,288 bytes.

Note: A multi-site deployment is a single global namespace across three sites. If you have two separate sites in a replication setup, see the storage policies for single site deployments.

The tables below summarize ActiveScale X200 durability policies.

Table 4: Storage policies for a single site ActiveScale X200

Policy Name	Description	Details
Balanced, 2 4U90 Server chassis	This policy is optimized for an X200 system consisting of 2 4U90 Server chassis, providing balance between usable capacity, durability, and throughput. Usable capacity will be 61% of raw.	Large objects: 19/7 Small objects: 9/7 Average client large object size to avoid stranded storage: 0.13-0.48 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.64x, Small objects: 5.2x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Storage Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Storage Node is unavailable and 13 additional drives fail.
Balanced, 3 4U90 Server chassis	This policy is optimized for an X200 system consisting of 3 4U90 Server chassis, providing balance between usable capacity, durability, and throughput. Usable capacity will be 72% of raw.	Large objects: 16/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.12-0.43 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.38x, Small objects: 3.6x. Durability: Up to 14 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 15 additional drives fail.
Balanced, 4 or more 4U90 Server chassis	This policy is optimized for an X200 system consisting of 4 or more 4U90 Server chassis, providing balance between usable capacity, durability, and throughput. Usable capacity will be 77% of raw.	Large objects: 20/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.17-0.61 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.29x, Small objects: 3.6x. Durability: Up to 13 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 15 additional drives fail.
Capacity optimized, 2 4U90 Server chassis	This policy provides the highest usable capacity for an X200 system consisting of 2 4U90 Server chassis. Usable capacity will be 64% of raw.	Large objects: 18/6 Small objects: 8/6 Average client large object size to avoid stranded capacity: 0.14-0.51 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.55x, Small objects: 4.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 11 additional drives fail.
Capacity optimized, 3 or more 4U90 Server chassis	This policy provides the highest usable capacity for an X200 system consisting of 3 or more 4U90 Server chassis. Usable capacity will be 77% of raw.	Large objects: 20/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.13-0.46 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.29x, Small objects: 3.6x. Durability: Up to 12 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 13 additional drives fail.

Table 5: Storage policies for a multi-site ActiveScale X200

Policy Name	Description	Details
Balanced	This policy provides balance between usable capacity and availability. Usable capacity will be 59% of raw.	Large objects: 18/7 Small objects: 9/7 Average client large object size to avoid stranded capacity: 0.10-0.35 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.69x, Small objects: 5.2x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one site is unavailable and one additional drive fails. Write availability: Objects can be written even if one site is unavailable and 13 additional drives fail.
Availability optimized	This policy has the highest availability. Usable capacity will be 54% of raw.	Large objects: 18/8 Small objects: 10/8 Average client large object size to avoid stranded capacity: 0.09-0.32 MB depending on disk capacity and number of disks. Storage overhead: Large objects: 1.86x, Small objects: 5.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one site is unavailable and two additional drives fail. Write availability: Objects can be written even if one site is unavailable and 13 additional drives fail.

The tables below summarize ActiveScale P100 durability policies.

Table 6: Storage policies for a single site ActiveScale P100

Policy Name	Description	Details
Balanced, single site	This policy provides balance between usable capacity, durability, and throughput. Usable capacity will be 69% of raw.	Large files: 13/4 Small files: 6/4 with a threshold 0 to 8,388,608 bytes. Default: 524,288 bytes. Average client large file size to avoid stranded storage: 804-1127 KB depending on disk capacity. Storage overhead: Large files: 1.44x, Small files: 3.5x. Durability: Up to 16 nines Read availability: Objects are retrievable even if one Storage Node is unavailable and one additional drive fails.
Capacity optimized, single site	This policy provides the highest usable capacity. Usable capacity will be 73% raw.	Large files: 15/4 Small files: 6/4 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average client large file size to avoid stranded storage: 804-1194 KB depending on disk capacity. Storage overhead: Large files: 1.36x, Small files: 3.5x. Durability: Up to 14 nines Read availability: Objects are retrievable even if one Storage Node is unavailable and one additional drive fails.
Default, single site		Balanced with a small file threshold of 524,288 bytes.
2-site replication optimized	This policy provides even higher usable capacity than the capacity optimized policy, but trades further durability in favor of capacity. This policy is only recommended in combination with 2-site replication. Usable capacity will be 83% raw.	Large files: 12/2 Small files: 4/2 with a threshold 0 to 8,388,608 bytes. Default: 524,288 bytes. Storage overhead: Large files: 1.2x, Small files: 2.5x. Durability: Up to 8 nines. Read availability: Data is retrievable even if one Storage Node is unavailable.

Table 7: Storage policies for a multi-site ActiveScale P100

Policy Name	Description	Details
Availability optimized, multi-site	This policy has the highest availability. Usable capacity will be 56% of raw.	Large files: 18/8. Small files: 10/8 with a threshold 0 to 8,388,608 bytes. Default: 524,288 bytes. Average client large file size to avoid stranded storage: 645-904 KB depending on disk capacity. Storage overhead: Large files: 1.8x, Small files: 5.5x. Durability: Up to 19 nines. Read availability: Objects are retrievable even if one site is unavailable and two additional drives fail.
Balanced, multi-site	This policy provides balance between usable capacity and availability. Usable capacity will be 61% of raw.	Large files: 18/7 Small files: 9/7 with a threshold of 0 to 8,388,608 bytes. Default: 524,288 bytes. Average client large file size to avoid stranded storage: 710-995 KB depending on disk capacity. Storage overhead: Large files: 1.64x, Small files: 5x. Durability: Up to 19 nines. Read availability: Objects are retrievable even if one site is unavailable and one additional drive fails.
Default, multi-site		Availability optimized with a small file threshold of 524,288 bytes.

Note: A multi-site deployment is a single global namespace across three sites. If you have two separate sites in a replication setup, see the storage policies for single site deployments.

The tables below summarize ActiveScale P200 durability policies.

Table 8: Storage policies for a single site ActiveScale P200

Policy Name	Description	Details
Balanced, 3 Nodes	This policy is optimized for a 3-node P200 system, providing balance between usable capacity, durability, and throughput. Usable capacity will be 45% of raw.	Large objects: 15/8 Small objects: 10/8 Average client large object size to avoid stranded capacity: 0.23-0.29 MB depending on disk capacity. Storage overhead: Large objects: 2.22x, Small objects: 5.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 7 additional drives fail.
Balanced, 4 Nodes	This policy is optimized for a 4-node P200 system, providing balance between usable capacity, durability, and throughput. Usable capacity will be 61% of raw.	Large objects: 19/7 Small objects: 9/7 Average client large object size to avoid stranded capacity: 0.42-0.52 MB depending on disk capacity. Storage overhead: Large objects: 1.64x, Small objects: 5.2x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 10 additional drives fail.
Balanced, 5 Nodes	This policy is optimized for a 5-node P200 system, providing balance between usable capacity, durability, and throughput. Usable capacity will be 68% of raw.	Large objects: 17/5 Small objects: 7/5 Average client large object size to avoid stranded capacity: 0.59-0.72 MB depending on disk capacity. Storage overhead: Large objects: 1.47x, Small objects: 4.0x. Durability: Up to 16 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 10 additional drives fail.
Balanced, 6 or 7 Nodes	This policy is optimized for a P200 system consisting of 6 or 7 Nodes, providing balance between usable capacity, durability, and throughput. Usable capacity will be 72% of raw.	Large objects: 16/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.38-0.54 MB depending on disk capacity. Storage overhead: Large objects: 1.38x, Small objects: 3.6x. Durability: Up to 13 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 12 additional drives fail.
Balanced, 8 or more Nodes	This policy is optimized for a P200 system consisting of 8 or more Nodes, providing balance between usable capacity, durability, and throughput. Usable capacity will be 77% of raw.	Large objects: 20/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.40-0.65 MB depending on disk capacity. Storage overhead: Large objects: 1.29x, Small objects: 3.6x. Durability: Up to 12 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 12 additional drives fail.
Capacity optimized, 3 Nodes	This policy provides the highest usable capacity for a 3-node P200 system. Usable capacity will be 48% of raw.	Large objects: 10/5 Small objects: 7/5 Average client large object size to avoid stranded capacity: 0.25-0.31 MB depending on disk capacity. Storage overhead: Large objects: 2.07x, Small objects: 4.0x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 9 additional drives fail.
Capacity optimized, 4 Nodes	This policy provides the highest usable capacity for a 4-node P200 system. Usable capacity will be 64% of raw.	Large objects: 18/6 Small objects: 8/6 Average client large object size to avoid stranded capacity: 0.45-0.54 MB depending on disk capacity. Storage overhead: Large objects: 1.55x, Small objects: 4.7x. Durability: Up to 18 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 8 additional drives fail.
Capacity optimized, 5 Nodes	This policy provides the highest usable capacity for a 5-node P200 system. Usable capacity will be 72% of raw.	Large objects: 16/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.63-0.76 MB depending on disk capacity. Storage overhead: Large objects: 1.38x, Small objects: 3.6x. Durability: Up to 13 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 10 additional drives fail.
Capacity optimized, 6 or more Nodes	This policy provides the highest usable capacity for a P200 system consisting of 6 or more Nodes. Usable capacity will be 77% of raw.	Large objects: 20/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: 0.40-0.65 MB depending on disk capacity. Storage overhead: Large objects: 1.29x, Small objects: 3.6x. Durability: Up to 12 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 10 additional drives fail.

Table 9: Storage policies for a multi-site ActiveScale P200

Policy Name	Description	Details
Balanced	This policy provides balance between usable capacity and availability. Usable capacity will be 59% of raw.	Large objects: 18/7 Small objects: 9/7 Average client large object size to avoid stranded capacity: 0.31-0.37 MB depending on disk capacity. Storage overhead: Large objects: 1.69x, Small objects: 5.2x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one site is unavailable and one additional drive fails. Write availability: Objects can be written even if one site is unavailable and 10 additional drives fail.
Availability optimized	This policy has the highest availability. Usable capacity will be 54% of raw.	Large objects: 18/8 Small objects: 10/8 Average client large object size to avoid stranded capacity: 0.28-0.34 MB depending on disk capacity. Storage overhead: Large objects: 1.86x, Small objects: 5.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one site is unavailable and two additional drives fail. Write availability: Objects can be written even if one site is unavailable and 10 additional drives fail.

The tables below summarize ActiveScale Z200 durability policies.

Table 10: Storage policies for a single site ActiveScale Z200

Policy Name	Description	Details
Balanced, 3 Nodes	This policy is optimized for a 3-node Z200 system, providing balance between usable capacity, durability, and throughput. Usable capacity will be 45% of raw.	Large objects: 15/8 Small objects: 10/8 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 2.22x, Small objects: 5.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 5 additional drives fail.
Balanced, 4 Nodes	This policy is optimized for a 4-node Z200 system, providing balance between usable capacity, durability, and throughput. Usable capacity will be 61% of raw.	Large objects: 19/7 Small objects: 9/7 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.64x, Small objects: 5.2x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 8 additional drives fail.
Balanced, 5 Nodes	This policy is optimized for a 5-node Z200 system, providing balance between usable capacity, durability, and throughput. Usable capacity will be 68% of raw.	Large objects: 17/5 Small objects: 7/5 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.47x, Small objects: 4.0x. Durability: Up to 18 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 8 additional drives fail.
Balanced, 6 or 7 Nodes	This policy is optimized for a Z200 system consisting of 6 or 7 Nodes, providing balance between usable capacity, durability, and throughput. Usable capacity will be 72% of raw.	Large objects: 16/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.38x, Small objects: 3.6x. Durability: Up to 15 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 10 additional drives fail.
Balanced, 8 or more Nodes	This policy is optimized for a Z200 system consisting of 8 or more Nodes, providing balance between usable capacity, durability, and throughput. Usable capacity will be 77% of raw.	Large objects: 20/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.29x, Small objects: 3.6x. Durability: Up to 14 Nines Read availability: Objects are retrievable even if one Node is unavailable and one additional drive fails. Write availability: Objects can be written even if one Node is unavailable and 10 additional drives fail.
Capacity optimized, 3 Nodes	This policy provides the highest usable capacity for a 3-node Z200 system. Usable capacity will be 48% of raw.	Large objects: 10/5 Small objects: 7/5 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 2.07x, Small objects: 4.0x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 7 additional drives fail.
Capacity optimized, 4 Nodes	This policy provides the highest usable capacity for a 4-node Z200 system. Usable capacity will be 64% of raw.	Large objects: 18/6 Small objects: 8/6 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.55x, Small objects: 4.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 6 additional drives fail.
Capacity optimized, 5 Nodes	This policy provides the highest usable capacity for a 5-node Z200 system. Usable capacity will be 72% of raw.	Large objects: 16/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.38x, Small objects: 3.6x. Durability: Up to 15 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 8 additional drives fail.
Capacity optimized, 6 or more Nodes	This policy provides the highest usable capacity for a Z200 system consisting of 6 or more Nodes. Usable capacity will be 77% of raw.	Large objects: 20/4 Small objects: 6/4 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.29x, Small objects: 3.6x. Durability: Up to 14 Nines Read availability: Objects are retrievable even if one Node is unavailable. Write availability: Objects can be written even if one Node is unavailable and 8 additional drives fail.

Table 11: Storage policies for a multi-site ActiveScale Z200

Policy Name	Description	Details
Balanced	This policy provides balance between usable capacity and availability. Usable capacity will be 59% of raw.	Large objects: 18/7 Small objects: 9/7 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.69x, Small objects: 5.2x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one site is unavailable and one additional drive fails. Write availability: Objects can be written even if one site is unavailable and 8 additional drives fail.
Availability optimized	This policy has the highest availability. Usable capacity will be 54% of raw.	Large objects: 18/8 Small objects: 10/8 Average client large object size to avoid stranded capacity: N/A (NVME capacity will be dynamically used for data or metadata). Storage overhead: Large objects: 1.86x, Small objects: 5.7x. Durability: Up to 19 Nines Read availability: Objects are retrievable even if one site is unavailable and two additional drives fail. Write availability: Objects can be written even if one site is unavailable and 8 additional drives fail.

Encryption in transit is always enabled over HTTPS.

ActiveScale OS supports the following RSA ciphers for encryption in transit using HTTPS:

• TLS_DHE_RSA_WITH_AES_256_GCM_SHA384

• TLS_DHE_RSA_WITH_AES_256_CBC_SHA

• TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

• TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

ActiveScale OS supports system-wide and object-level encryption. This feature may be disallowed in certain countries.

When encryption is enabled, objects are encrypted before they are erasure encoded and stored on disk.

Once encryption is enabled, you cannot disable it.

ActiveScale OS uses the symmetric cipher AES-256-CTR (256-bit Advanced Encryption Standard in Counter mode) to encrypt objects in a bucket. The input consists of plain text data, a 256-bit encryption key, and a 128-bit initialization vector. The output consists of cipher text data with the same size as the plain text data. Compared to AWS, the ActiveScale OS supports only Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3); alternative methods of server-side encryption or client-side encryption are not supported.

ActiveScale OS encrypts both data and metadata, including custom metadata. For the encryption of data, ActiveScale OS uses a 256-bit encryption key and one or more unique initialization vectors. It generates one encryption key per object. It calculates initialization vectors based on the globally unique object identifier. This identifier is a deterministic hash of the object and bucket names. Data is encrypted first, and then erasure encoded.

ActiveScale OS protects data transferred within the system and/or the ActiveScale SM with the following cipher suites:

SSL_RSA_WITH_RC4_128_SHA

TLS_RSA_WITH_RC4_128_SHA

In contrast, if an object is unencrypted, this means that:

During reading and writing of the object, the cleartext data is readable on the network connection between the System Node and the column (this is in the local data center, but might be a connection between different racks).

During reading and writing of the object, the cleartext data is readable on the network connection between columns (this might be a WAN multi-site connection).

SSDs contain unencrypted metadata.

HDDs contain unencrypted data.

For each account, an account-specific encryption phrase is generated by the system at creation time (or when the unit is upgraded to ActiveScale OS version 7.0.2 or later). It’s not accessible by the owner of the account, but the encoding or decoding process can retrieve it from the system’s internal database and keep it in memory for optimization of the encoding/decoding flow.

When encryption is enabled, ActiveScale OSencrypts every object with its own encryption key. To do this, it generates an encryption key for the bucket (when the first object is added to it), and an encryption key for the object.

Both the bucket and the object encryption keys are based on the account encryption key of the account that owns the bucket and object. The system encrypts the generated keys themselves using the account encryption key and stores the encrypted encryption keys in the metadata of the bucket and the object.

Enabling encryption only applies to new incoming data, or updated data. Data that was ingested prior to enabling encryption remains unchanged. If you need to encrypt existing data, you must rewrite it (through the S3 API).

ActiveScale OS writes a daily disaster recovery report into the system bucket. If a per-bucket encryption key gets lost due to a database corruption, Quantum Support can help you recover from this situation.

To recover from the loss of an entire data center in a multi-site system, contact Quantum Support to discuss the process, logistics, and timelines associated with recovery.

 

Can an S3 client specify their own encryption key?

No; an S3 client cannot specify a custom encryption key. There is no support for an external key manager.

 

Can an S3 client enable encryption on the call to create a bucket?

Encryption is managed by the system administrator. There is no S3 API to allow S3 clients to manage per-bucket encryption.

 

Can encryption be enabled by default for all objects?

Yes.

 

When I toggle encryption modes, is existing data decrypted and resaved?

When toggling from System Wide Forced to Object Level, ActiveScale OS no longer encrypts new data unless it is sent with the S3 request header x-amz-server-side-encryption:AES256, but existing data remains encrypted. If you need to unencrypt existing data, you must rewrite it (through the S3 API).

 

When I enable encryption, is existing data encrypted?

When enabling encryption, new incoming data, or updated data, is encrypted. Data that was in any bucket prior to enabling encryption is unchanged. If you need to encrypt existing data, you must rewrite it (through the S3 API).

 

How often do I need to backup the disaster recovery report?

It is advised to backup the disaster recovery report daily to a different system than the ActiveScale system. Note that old versions of the disaster recovery report may be deleted or overwritten by a new version of the disaster recovery report.

 

In a multi-site environment, is an up-to-date copy of the encryption key database maintained at all three sites?

The encryption key database is kept in the distributed key-value store environment metadata, which has a copy at each site. The distributed key-value store cluster needs at least one other member in order to accept an operation. In a multi-site environment, if one site is offline, it will catch up with the other cluster members when it comes back online, without any intervention from the system administrator.

 

In a multi-site environment, is the data encrypted before it is erasure coded and distributed to the other sites, after erasure coding but before distribution, or after distribution at the destination site?

Dynamic Data Placement encrypts the superblock before erasure coding and before distributing to the destination site: the client sends the object to Dynamic Data Placement, which encrypts the superblock, encodes it, and then distributes erasure coded chunks to the storage backend. In a multi-site environment, the erasure coded chunks are sent to storage daemons over the WAN in an encrypted state, and the storage daemons write them to disk.

The reverse process happens for a GET. The storage daemon retrieves the erasure coded chunks from the disk and sends them to Dynamic Data Placement, which decodes the chunks and unencrypts the data prior to sending it to the client.

ActiveScale OS allows the following incoming connections:

• S3 HTTPS - Destination TCP port 443

• S3 HTTP - Destination TCP port 80

• SSH - Destination TCP port 22

• HTTPS - Destination TCP port 10443

• SNMP - Destination UDP port 161

• rpcbind (NFS) - Destination TCP port 111

• NFS - Destination TCP port 2049

• mountd (NFS) - Destination TCP port 16010

• nlockmgr (NFS) - Destination TCP port 16015

• rquotad (NFS) - Destination TCP port 16025

• ping - ICMP type echo-request and echo-reply to allow ping from both the outside and inside

• Neighbour Solicitation and Neighbour Advertisement - IPv6 only. ICMPv6 type neighbour-solicitation and neighbour-advertisement to allow neighbor discovery.

Outgoing connections are allowed by default. However to make this function correctly, the return traffic of the outgoing connections is allowed :

• NTP

• DNS

• SMTP

• HTTP

• HTTPS

• SNMP

Limitations:

• As soon as a System Interconnect goes down, the management and data path clients experience erratic behavior and random failures for a few minutes. However, there is no data loss.

• While a System Interconnect is down:

  • Performance of the system can be lower, especially when reading or writing large objects.

  • Repair speed of the system can be lower, especially for large objects.

  • Certain background jobs do not function; for example, the jobs which generate capacity reports, encryption key backups, and garbage collection.

  • Certain manageability no longer functions; for example, IPMI management of nodes, System Interconnects and PDUs might be unavailable, since IPMI connectivity is not highly available.

  • When the System Interconnect is brought back up, all these limitations automatically disappear within 5 minutes.

Public Network 1 and Public Network 2 connect each System Node to your public network, which you deploy on your own switches. Since each System Node is connected to both Public Network 1 and Public Network 2, or to a single Public Network if you enabled NIC bonding, your system has a total of 6 public IP addresses, or 3 if NIC bonding is enabled.

Public interfaces can be configured to also have an IPv6 address and therefore run in a dual stack configuration.

Private Network 1 and Private Network 2 carry all internal ActiveScale traffic.

Each node is connected to both Private Network 1 and Private Network 2. At scaled-out sites, this network spans a singe system and lives in one site. The nodes use this network to communicate with:

• Local columns (LAN only, never WAN; high bandwidth - datapump)

• Each other (LAN + WAN; low bandwidth - distributed key-value store + the management framework)

• The IPMI ports (LAN + WAN; low bandwidth, Private Network 1 only)

System Network 1 and System Network 2 carry traffic between System Nodes and Storage Nodes.

Each System Node and Storage Node is connected to both System Network 1 and System Network 2. At scaled-out sites, this network can span multiple systems, but lives in one site. System Nodes use this network to communicate with:

• Local columns (LAN only, never WAN; high bandwidth - datapump)

• Each other (LAN + WAN; low bandwidth - distributed key-value store + the management framework)

• The IPMI ports of the System Node and Storage Nodes (LAN + WAN; low bandwidth)

Storage Network 1 and Storage Network 2 are used solely within a column for back-end traffic.

Every Storage Node is connected to both Storage Network 1 and Storage Network 2.

There is no system administrator access to the intelligent platform management interface (IPMI); remote access through IPMI is only available to Quantum Support and is strictly controlled through your quantum.com/support passcode. On the ActiveScale X100/X200/Z200, IPMI access is through the 1Gb Management Interconnect, which uplinks to System Interconnect 1 and connects to all IPMI ports in the system. This means that all IPMI IP addresses are assigned from System/Private Network 1. On the ActiveScale P100/P200, IPMI access is through System/Private Network 1.

ActiveScale OS supports the use of Virtual IP addresses on the Public Interfaces. Virtual IP addresses are only supported for IPv4.

Each Public interface can be assigned an additional VIP address in its Public Network. When a network failure occurs that impacts one or more public interfaces (node down, NIC failure, network cable failure), the remaining members in the failover group will take over the Virtual IP addresses from the affected interfaces.

This configuration can be combined with a simple client-side DNS round-robin load-balancing configuration targeting the ActiveScale VIP addresses. Such a setup ensures High Availability and continuous concurrent access to all nodes without the need for a dedicated (software) load balancer or any additional network reconfiguration. The NFS or S3 clients to simply retry failed/timed-out operations to automatically switch over to another ActiveScale node. Your switch must allow multicast traffic on the ports connected to the Public Network(s) on the site for which the failover group is created.

Note: A Maximum of 30 seconds is expected for Virtual IP failover/failback. NFS/S3 client timeout and retries should be configured accordingly to get uninterrupted service.

Configuring VIP Addresses and Failover Groups for NFS/S3 Traffic

1. Create one failover group per Public Network.

2. Add a virtual IP for each public NIC of each system node for each defined failover group.

3. Apply changes.

Note: The system automatically updates the failover group at each site with any System Expansion Nodes or new System Nodes (scale out) that are added, Virtual IP's must be added to each new System Expansion Node or System Node.

• Site Index_PublicNetwork1

• SiteIndex_PublicNetwork2

• SiteIndex_SystemNetwork1

• SiteIndex_SystemNetwork2

• SiteIndex_ColumnColNumber_StorageNetwork1

• SiteIndex_ColumnColNumber_StorageNetwork2

Where ColNumber is 1-3 for system 01, 4-6 for system 02, and so on.

Examples

• Site1_Column1_StorageNetwork1 is Storage Network 1 at site 01, system 01.

• Site3_Column4_StorageNetwork2 is Storage Network 2 at site 03, system 2.

Figure 2: Network Connectivity for a Single Site, Single Scale-up ActiveScale P100

Figure 3: Network Connectivity for a Single Site ActiveScale X100

Figure 4: Routing on a Multi-Site ActiveScale X100

Figure 5: Default (mode 1) Uplinks for a Multi-Site ActiveScale X100

When ActiveScale is deployed as a single namespace that spans three geographically dispersed sites:

Multi-site networks are always routed.

There are two System Networks per site; in other words, six in total. Each System Network is local to one site. The IP addresses for all nodes on that network are local to that site.

• The three System Network 1's (SiteIndexSystemNetwork1) form a logical group. Each System Network 1 is on a different subnet, but is routed to the System Network 1's at the other two sites. You provide the connectivity using an uplink from System Interconnect 1 to your router.

• The three System Network 2s (SiteIndex_SystemNetwork2) form a logical group. Each System Network 2 is on a different subnet, but is routed to the System Network 2s at the other two sites. You provide the connectivity using an uplink from System Interconnect 2 to your router.

There are two Storage Networks per column per site for ActiveScale™ P100 (ActiveScale™ X100 has 2 storage networks total, regardless of number of columns); in other words, six in total. Each Storage Network is local to one site. The IP addresses for all nodes on that network are local to that site.

• The three Storage Network 1's (SiteIndexColumnColNumber_StorageNetwork1) form a logical group. Each Storage Network 1 is on a different subnet, but is routed to the other Storage Network 1's at the other two sites. You provide the connectivity using an uplink from System Interconnect 1 to your router.

• The three Storage Network 2s (SiteIndex_ColumnColNumber_StorageNetwork2) form a logical group. Each Storage Network 2 is on a different subnet, but is routed to the other Storage Network 2s at the other two sites. You provide the connectivity using an uplink from System Interconnect 2 to your router.

A multi-site column lives in three racks (one per site). For this group of three racks:

• Connectivity is provided by connecting the System Network of System Interconnect 1 to a 10Gb uplink over your network. Traffic over that uplink is untagged. You provide connectivity over the WAN.

1. Configure virtual IP addresses and failover groups for NFS traffic. Refer to Virtual IP Addresses and Failover Groups.

   Note: Quantum strongly recommends configuring virtual IP addresses to ensure high availability for NFS clients.

   Note: NFS is operational without VIP failover but VIP failover is required to achieve high availability (HA).

   Note: When Kerberos is used for authentication for file interface, the Client might see the "Permission Denied" error during VIP failover. After successful failover, NFS operations will return to the operational state automatically.

2. Create a volume using ActiveScale View (see Manage Volumes) or ascli with the following command.

   ascli$ uda volume create --name myvolume --email user@domain.com

3. Create an export within that volume using ActiveScale View (see Manage Volumes) or ascli with the following command.

   ascli$ uda nfs-export create --volume myvolume --path /myexport --create-path --uid 1000 --gid 1000 --perm 755 --squash root_squash --tag mytag

   Note: Best practice is to mount NFS exports using their tag name rather than their export paths. For example, if you want to create multiple exports for the same directory, for example, with different permissions for different clients, use the commands in the CLI Reference Guide to specify a unique tag for each export of the same directory. Mounting the export can then be done by using SystemNodeIP:mytag (without "/" between the IP and the tag).

4. On the NFS Client, create a local directory to mount (for example, /mnt/exportMount)

5. Mount the NFS export either by using the defined tag for the export or using the export path:

   1. Using the defined tag (for example, mytag):

      When connecting over IPv4

      sudo mount -t nfs -o

      nfsvers=3,proto=tcp,async,lookupcache=none,nolock,timeo=600

      SystemNodeIP:mytag /mnt/exportMount

      When connecting over IPv6

      sudo mount -t nfs -o

      nfsvers=3,proto=tcp6,async,lookupcache=none,nolock,timeo=600

      SystemNodeIP:mytag /mnt/exportMount

   2. Alternatively, on the NFS Client:

      List the exported paths using the following command:

      showmount -e SystemNodeIP

      Sample output:

      Export list for SystemNodeIP:

      /legacyexport (everyone)

      /resources (everyone)

      /resources (everyone)

      /resources/photos (everyone)

      /backups (everyone)

      Mount the NFS export:

      When connecting over IPv4

      sudo mount -t nfs -o

      nfsvers=3,proto=tcp,async,lookupcache=none,nolock,timeo=600

      SystemNodeIP:mytag /mnt/exportMount

      When connecting over IPv6

      sudo mount -t nfs -o

      nfsvers=3,proto=tcp6,async,lookupcache=none,nolock,timeo=600

      SystemNodeIP:mytag /mnt/exportMount

6. On your client machines, mount the NFS export(s). Use these options for the Linux mount command:

   Options	Description
   nfsvers=3	NFS version 3 only
   rsize=1048576	Use the default setting of 1MiB for rsize; smaller sizes are unsupported
   wsize=1048576	Use the default setting of 1MiB for wsize; smaller sizes are unsupported
   proto=tcp	Use when mounting over IPv4. Use the default transport protocol, TCP. UDP is unsupported
   proto=tcp6	Use when mounting over IPv6. Use the default transport protocol, TCP. UDP is unsupported
   async	Use the default option, async; sync is unsupported
   lookupcache=none	This sets the kernel's cache management scheme to none, which is not the Linux default. Note: The lookupcache value may be tuned to your use case but consider that this value trades consistency for performance. When enabling lookupcache, it may take some time before a client notices a directory has changed. Refer to NFS client documentation for more information.
   nolock	Use option nolock, because server side file lock is unsupported. Set option nolock for Mac OS X users to avoid unresponsive behavior.
   timeo=600	Use the default timeout value before an NFS client retries a request to 600 deciseconds. (60 seconds).

   Note: The mounted file system behaves as if noatime is specified (in other words, access time is not updated for read operations).

Exports created in ActiveScale 5.4.1 and 5.5 are backward compatible. However, Quantum recommends to remove legacy exports in volume0, recreate those exports and remount them to avoid confusion in the difference between legacy and new-style export paths:

Legacy exports (of volume0, created in ActiveScale 5.4.1 and 5.5) continue to use export path

IP:/legacyexport

New exports created on ActiveScale 5.6 and above use the export path

IP:/volumename/newexport

New exports and recreated exports in volume0 also uses the new style export path

IP:/volume0/newexportinvolume0

Legacy and new exports can be mounted by using tags; in that case the address provided in the mount command is IP:mytag (without "/" between the IP and the tag).

NFS is only supported on Linux-based systems and not on Windows clients.

You can create and export up to 65,535 NFS directories.

You can create a maximum of 4096 NFS client mounts per System Node on a P100 system or 8196 NFS client mounts per System Node on an X100 system.

The maximum file size you can store/retrieve from an NFS export is 16TiB.

ActiveScale OS does not support concurrent multi-protocol access to NFS exports. In other words, clients must read/write to the NFS export using the NFS protocol only, not a combination of NFS and S3.

You cannot enable asynchronous replication for a volume.

NFS can be used on multi-site deployments but requires a maximum latency of 20ms between sites.

NFS supports the fsstat NFS operation but updates the fsstat metrics once a day.

NFS does not support hard links. Symbolic links are supported.

NFS requires all filenames to be in valid UTF-8 format with no generic byte sequences.

Local NFS mount points are not supported. NFS mount points should be set up on client machines.

The fsck operation can be launched by an administrator or by support. It is exposed through an internal debugging interface. It is not exposed over NFS, so it cannot be used by a customer.

System Administrator	The system administrator manages the system and accounts/users through ActiveScale SM, ActiveScale CLI or the management API. The system administrator does not have access to objects stored within buckets, but can manage an account to a certain extent on behalf of an account owner.
Account	An account owns all buckets created by itself or any of its users. An account owns all objects in its buckets, regardless of the identity which performed the put operation. An account can be billed for the used capacity of its buckets. An account also implicitly has a special IAM (Identity and Access Management) user, called the root user.
IAM (Identity and Access Management) User	An IAM user belongs to an account. An IAM user can execute S3 and IAM operations for which it has been granted permissions (the root user always has all permissions for its own resources). When performing S3 or IAM operations, an IAM user identifies itself using an access key. If the root user does not have an access key, the system administrator must provide them with a new access key. IAM users can manage access keys when they have been granted the correct permissions.
Authenticated Users	This group includes identities (accounts/users) who have active S3 API keys.
All Users	This group includes all users, with or without S3 API keys. Members of this group can do anonymous bucket and object operations.

Table 13: A comparison of ActiveScale OS and AWS identities

	ActiveScale OS	AWS
Identity model tiers:	System administrator Account IAM User (Identity and Access Management)	Account IAM user (Identity and Access Management)
The system administrator can:	Manage accounts and users Manage API keys	There is no corresponding identity in AWS.
An account can:	Own buckets Automatically own all objects in the buckets it creates Be billed for used bucket capacity	Own buckets Only own objects it uploads into buckets it creates Be billed for used bucket capacity
An IAM user:	Cannot own objects	Cannot own objects

ActiveScale OS has a simpler permissions model than AWS. ActiveScale OS implements a subset of the Identity and Access Management (IAM) permissions model of AWS. For difference between ActiveScale OS and AWS, refer to ActiveScale IAM API

The S3 API additionally has the concept of granting ACL permissions. These are equivalent to allowing the corresponding IAM permissions for the bucket and its contents. Table 8 describes which IAM permissions correspond to the ACL permissions.

Note: ActiveScale OS does not support object ACLs - when applying an ACL to a bucket, the same ACL is implicitly applied to all objects in the bucket as well.

Note: S3 bucket ACLs are an alternative to S3 bucket policies, but they are less capable/flexible, and their use is discouraged. It is recommended to use S3 bucket policies instead.

Table 14: A Comparison of ActiveScale OS and AWS Permissions

ACL Permission	ActiveScale OS	AWS
READ	s3:ListBucket s3:ListBucketVersions s3:ListBucketMultipartUploads s3:GetObject s3:GetObjectVersion s3:GetObjectVersionForReplication	When granted on buckets: s3:ListBucket s3:ListBucketVersions s3:ListBucketMultipartUploads When granted on objects: s3:GetObject s3:GetObjectVersion
WRITE	s3:DeleteObject s3:PutObject s3:ReplicateDelete s3:ReplicateObject	When granted on buckets: s3:PutObject s3:DeleteObjectVersion (if the grantee is the bucket owner)
READ_ACP	s3:GetBucketAcl s3:GetObjectAcl s3:GetObjectVersionAcl	When granted on buckets: s3:GetBucketAcl When granted on objects: s3:GetObjectAcl s3:GetObjectVersionAcl
WRITE_ACP	s3:PutBucketAcl s3:PutObjectAcl s3:PutObjectVersionAcl	When granted on buckets: s3:PutBucketAcl When granted on objects: s3:PutObjectAcl s3:PutObjectVersionAcl
FULL_CONTROL	Equivalent to granting READ, WRITE, READ_ACP, and WRITE_ACP ACL permissions. Accordingly, this ACL permission maps to a combination of corresponding access policy permissions.	Equivalent to granting READ, WRITE, READ_ACP, and WRITE_ACP ACL permissions. Accordingly, this ACL permission maps to a combination of corresponding access policy permissions.

The ActiveScale S3 Object Lock provides Compliance Mode which prevents an object from being overwritten or deleted. When an object version is locked in Compliance Mode, the retention period cannot be changed, and its retention period cannot be shortened.

Updating an object version's metadata, which occurs when setting Object Lock or altering an Object Lock, does not overwrite the object version or reset its Last-Modified timestamp

When a retention period on an object version is set, S3 Object Lock stores a timestamp in the object version's metadata to indicate when the retention period expires. After the retention period expires, the object version can be deleted.

A retention period can be placed on an object version either explicitly in the S3 API or through a bucket default retention period setting. When applying a retention period to an object version explicitly, a Retain Until Date for the object version is specified. S3 Object Lock stores the Retain Until Date setting in the object version's metadata and protects the object version until the retention period expires.

When using bucket default retention period settings, a duration is specified in either days or years. When an object is placed in the bucket, S3 Object Lock calculates a Retain Until Date for the object version by adding the specified duration to the object version's creation timestamp.

Note: Retention periods apply to individual object versions. Different versions of a single object can have different retention periods.

A retention period can be extended after a retention setting to an object version has been applied. This can be done by submitting a new lock request for the object version with a Retain Until Date that is later than the one currently configured for the object version. S3 Object lock replaces the existing retention period with the new, longer period. Any user with write permissions on the bucket can extend the retention period for a locked object version.

To use Object Lock, it must be enabled for a bucket. A default retention mode and period can be configured and applies to new objects that are placed in the bucket

Before any objects can be locked, the bucket has to be configured to use S3 Object Lock. This is done by enabling Object Lock in the View Create Bucket dialog or by creating the bucket through the S3 API with the x-amz-bucket-object-lock-enabled header. Once a bucket has been configured for Object Lock, objects in that bucket can be locked using retention periods.

Note: Object Lock can only be enabled for new buckets.

When a bucket is created with the S3 Object Lock capability enabled, versioning will automatically be enabled for the bucket.

After a bucket has been created with S3 Lock enabled, it cannot be disabled anymore and versioning for that bucket cannot be suspended.

When S3 Object Lock is enabled for a bucket, the bucket can store protected objects. However, the setting doesn't automatically protect objects that are put into the bucket. To automatically protect object versions that are placed in the bucket, a default retention period must be created. Default retention settings apply to all new objects that are placed in the bucket, unless a different Retain Until Date is specified at the time of creation of an object.

A default retention period is described not as a timestamp, but as a period either in days or in years. When an object version is placed in a bucket with a default retention period, S3 Object Lock calculates the Retain Until Date. It does this by adding the default retention period to the creation timestamp for the object version. The resulting timestamp will be stored in the metadata of the object version as the Retain Until Date.

The default retention period is used for all new object versions put into the bucket. This default can be overridden on a per-object-version basis.

Note: Retain Until Dates are applied to individual object versions. Either the Retain Until Date is specified when placing an object into a bucket or, if this was not specified, the Retain Until Date is derived from the bucket's default retention period. After the object is created, the retention period is independent from the default retention period on the bucket. Changing the default retention period on the bucket does not affect existing object versions.

Note: If a default retention period is configured on a bucket, all requests to upload objects in such a bucket must include the Content-MD5 header.

HTTP 503 Service Unavailable errors will be returned to the client to indicate that the system is operating at maximum capacity. These can be sent on 2 occasions:

• When requests are queued for longer than 20 seconds in the system node, without any data being transferred, ActiveScale will send a HTTP 503 Service Unavailable error to the clients.

• When more connections are being created to a system node than what is configured as the system limit.

On anActiveScale system, there are 3 types of S3 traffic: external client S3 requests, outgoing replication requests and incoming replication requests. A built-in CLI call s3 activity-distribution update defines the distribution among these 3 types. A budget of 100% can be distributed between these 3 types. E.g. 60% for S3 client requests, 20% for incoming replication and 20% for outgoing replication. This makes it possible to, for example, throttle down the budget used for outgoing replication requests, so a larger share of the in progress requests becomes available for S3 client traffic and incoming replication.

The following illustration provides an example of the UI graph that can be used to monitor the total number of active S3 requests on anActiveScale system as a whole and on a per node basis. The total number of active S3 requests only includes S3 requests made by external clients in this screen shot, this system with 3 system nodes is handling 788 concurrent S3 requests from external clients.

Figure 18: S3 Connections Aggregation

Scaling up on the ActiveScale P100 adds additional column(s). The tables below list scale-up/out options for ActiveScale P100. Note that Raw capacity is dependent on drive capacity, whereas Maximum object count is independent of drive capacity.

Scaling out on the ActiveScale P100 adds additional systems to a site. For more information, see the ActiveScale P100 Support Guide.

Table 15: Scalability options for a single site ActiveScale P100

Rack/column count	HDDs per system (10TB, 12TB, or 14TB)	Raw capacity (PB)	Maximum object count (Millions)
1 rack with 1 column	72	0.72 - 1.008	600
1 rack with 2 columns	144	1.44 - 2.016	1,200
1 rack with 3 columns	216	2.16 - 3.024	1,800
2 racks, 4 columns total	288	2.88 - 4.032	2,400
2 racks, 5 columns total	360	3.6 - 5.04	3,000
2 racks, 6 columns total	432	4.32 - 6.048	3,600
3 racks, 7 columns total	504	5.04 - 7.056	4,200
3 racks, 8 columns total	576	5.76 - 8.064	4,800
3 racks, 9 columns total	648	6.48 - 9.072	5,400

Table 16: Scalability options for a multi-site ActiveScale P100

Rack/column count	HDDs per system (10TB, 12TB, or 14 TB)	Raw capacity (PB)	Maximum object count (Millions)
1 rack at each site, with 1 logical column total (6 Storage Nodes at each site for a total of 18 Storage Nodes)	216	2.16 - 3.024	1,800
1 rack at each site, with 2 logical columns total (36 Storage Nodes)	432	4.32 - 6.048	3,600
1 rack at each site, with 3 logical columns total (54 Storage Nodes)	648	6.48 - 9.072	5,400
2 racks, 4 columns at each site, for a total of 12 columns	864	8.64 - 12.096	7,200
2 racks, 5 columns at each site, for a total of 15 columns	1,080	10.8 - 15.12	9,000
2 racks, 6 columns at each site, for a total of 18 columns	1,296	12.96 - 18.144	10,800
3 racks, 7 columns at each site, for a total of 21 columns	1,512	15.12 - 21.168	12,600
3 racks, 8 columns at each site, for a total of 24 columns	1,728	17.28 - 24.192	14,400
3 racks, 9 columns at each site, for a total of 27 columns	1,944	19.44 - 27.216	16,200

Scaling up the ActiveScale X100 is offered in the form of its flexible capacity feature, which allows you to start with as little as one populated sled column (one sled in each Storage Enclosure Basic) and increase the number of populated columns over time. In other words, you can replace all the empty sleds in an entire column with populated sleds. The tables below list scale-up options for the ActiveScale X100. Note that Maximum raw capacity is dependent on drive capacity, whereas Maximum object count is independent of drive capacity.

Scaling out the ActiveScale X100 adds additional systems to a site. For more information, see the ActiveScale X100 Support Guide.

Table 17: Scalability options for a single site ActiveScale X100

Rack/sled count	HDDs per system (10TB, 12TB, or 14TB)	Maximum raw capacity (PB)	Maximum object count (millions)
1 rack with 1 sled column (6 sleds)	84	0.84 - 1.176	714
1 rack with 2 sled columns (12 sleds)	168	1.68 - 2.352	1,429
1 rack with 3 sled columns (18 sleds)	252	2.52 - 3.528	2,143
1 rack with 4 sled columns (24 sleds)	336	3.36 - 4.704	2,857
1 rack with 5 sled columns (30 sleds)	420	4.2 - 5.88	3,571
1 rack with 6 sled columns (36 sleds)	504	5.04 - 7.056	4,286
1 rack with 7 sled columns (42 sleds)	588	5.88 - 8.232	5,000
2 racks with a total of 8 sled columns (48 sleds)	672	6.72 - 9.408	5,714
2 racks with a total of 9 sled columns (54 sleds)	756	7.56 - 10.548	6,429
2 racks with a total of 10 sled columns (60 sleds)	840	8.4 - 11.76	7,143
2 racks with a total of 11 sled columns (66 sleds)	924	9.24 - 12.936	7,857
2 racks with a total of 12 sled columns (72 sleds)	1,008	10.08 - 14.112	8,571
2 racks with a total of 13 sled columns (78 sleds)	1,092	10.92 - 15.288	9,286
2 racks with a total of 14 sled columns (84 sleds)	1,176	11.76 - 16.464	10,000
3 racks with a total of 15 sled columns (90 sleds)	1,260	12.6 - 17.64	10,714
3 racks with a total of 16 sled columns (96 sleds)	1,344	13.44 - 18.816	11,429
3 racks with a total of 17 sled columns (102 sleds)	1,428	14.28 - 19.992	12,143
3 racks with a total of 18 sled columns (108 sleds)	1,512	15.12 - 21.168	12,857
3 racks with a total of 19 sled columns (114 sleds)	1,596	15.95 - 22.344	13,571
3 racks with a total of 20 sled columns (120 sleds)	1,680	16.8 - 23.520	14,286
3 racks with a total of 21 sled columns (126 sleds)	1,764	17.64 - 24.696	15,000

Table 18: Scalability options for a multi-site ActiveScale X100

Rack/sled count	HDDs per system (10TB, 12TB, or 14TB)	Maximum raw capacity (PB)	Maximum object count (millions)
1 rack with 1 sled column at each site (18 sleds)	252	2.52 - 3.528	2,143
1 rack with 2 sled columns at each site (36 sleds)	504	5.04 - 7.056	4,286
1 rack with 3 sled columns at each site (54 sleds)	756	7.56 - 10.584	6,429
1 rack with 4 sled columns at each site (72 sleds)	1,008	10.08 - 14.112	8,571
1 rack with 5 sled columns at each site (90 sleds)	1,260	12.6 - 17.640	10,714
1 rack with 6 sled columns at each site (108 sleds)	1,512	15.12 - 21.168	12,857
1 rack with 7 sled columns at each site (126 sleds)	1,764	17.64 - 24.696	15,000
2 racks at each site, 8 sled columns total (144 sleds)	2,016	20.16 - 28.224	17,143
2 racks at each site, 9 sled columns total (162 sleds)	2,268	22.68 - 31.752	19,286
2 racks at each site, 10 sled columns total (180 sleds)	2,520	25.2 - 35.280	21,429
2 racks at each site, 11 sled columns total (198 sleds)	2,772	27.72 - 38.808	23,571
2 racks at each site, 12 sled columns total (216 sleds)	3,024	30.24 - 42.336	25,714
2 racks at each site, 13 sled columns total (234 sleds)	3,276	32.76 - 45.864	27,857
2 racks at each site, 14 sled columns total (252 sleds)	3,528	35.28 - 49.392	30,000
3 racks at each site, 15 sled columns total (270 sleds)	3,780	37.8 - 52.920	32,143
3 racks at each site, 16 sled columns total (288 sleds)	4,032	40.32 - 56.488	34,286
3 racks at each site, 17 sled columns total (306 sleds)	4,284	42.84 - 59.976	36,429
3 racks at each site, 18 sled columns total (324 sleds)	4,536	45.36 - 63.504	38,571
3 racks at each site, 19 sled columns total (342 sleds)	4,788	47.88 - 67.032	40,714
3 racks at each site, 20 sled columns total (360 sleds)	5,040	50.4 - 70.56	42,857
3 racks at each site, 21 sled columns total (378 sleds)	5,292	52.92 - 74.088	45,000

The UI uses the following terminology to configure VIP:

Table 19: VIP Terminology

Failover Group Name	A name you wish to add, to identify a new Failover group.
Multicast Address	This is the multicast network the Failover Group will use to communicate between nodes in the cluster. This is optional and defaults to 239.0.0.18
Virtual IP	This is the IP address you wish to assign as virtual IP address. It should be within the range of one of your public networks and an available IP address.
System Node	On which system node you wish to apply the VIP.
Failover Group	Select a Failover Group for which you want to add this VIP.

This section provides the procedure for configuring Failover Groups and VIP.

From the Network Configuration screen, scroll down the Name column to find Failover Groups.

Figure 20: Network Configuration Wizard - Failover Groups

1. Click the Add button next to failover groups to start the network configuration wizard.

2. Enter the Failover Group name and multicast address as illustrated in the following figure. The default address is 239.0.0.18.

   Figure 21: Network Configuration Wizard - Add Failover Group

   

3. Click Create to confirm the new failover group and to display the network configuration wizard, confirming the failover group was created.

   Figure 22: Network Configuration Wizard - Failover Group Created

   

4. In the Network Configuration Wizard, click the Add button to add the Virtual IP address, system node of the VIP's home node, and the the Failover Group the VIP belongs to.

   Figure 23: Network Configuration Wizard - Add VIP

   

5. Click the Add button to add the new VIP. Confirm the VIP was successfully created on the Network Configuration screen.

6. Repeat the previous steps to create more Failover Groups and/or add extra Virtual IPs.

7. Add a new Failover Group for each Datacenter or Site and one for each Public network subnet.

   Note: It is advised to add a VIP for each System Node and each of its Public Networks. As the VIP is constrained to the subnet it is part of. If one network should go down (e.g. switch failure), the other VIP on the second network can still be accessed.

8. Once all the desired Failover Group(s) and Virtual IP(s) are created, apply this configuration on the system.

9. Navigate to the Network Configuration wizard, scroll down and you will see the option to apply. Click the Apply button for (one of) the Virtual IP you created.

   Figure 24: Network Configuration Wizard - Apply VIP Configuration

   

10. The confirmation pop-up is displayed. Select Continue to apply the VIP configuration.

    Figure 25: Network Configuration Wizard - Confirm VIP Configuration

    Note: The configuration progress will be displayed while changes are applied.

11. Confirm the configuration in the Network Configuration wizard by refreshing the page and scrolling down to the Virtual IP section. A green check mark will be visible below Latest Applied.

    Figure 26: Network Configuration Wizard - VIP Configuration Confirmed