The space manager’s duties are reduced from the DXi 1.x software design. However, the space manager still plays a pivotal role in keeping the system from running out of disk space. With DXi 2.x software, truncation is no longer required or used. However, space reclamation is still required.

What is space reclamation (aka garbage collection)? 

Space reclamation and garbage collection are the terms used to describe deletion of data on a DXi that is no longer referenced. There are four stages to reclamation:

• Stage 1 - Compaction: Compaction is the term used to describe the stage where data is actually deleted. This stage is in place to delete data that was identified for deletion during a previous reclamation operation. This allows customers to cancel reclamation and recover data without running through the entire operation cycle.

• State 2 – Reconciliation: During this stage two lists are created then compared. A list of tags that are still referenced and a list of tags that are still in the blockpool. The reconciliation thread does the following:
  • Uses the bfst_query api commands to get a list of all tags in the blockpool and dump them to /data/hurricane/bpgc/blockpooltags.
  • Syncs with dedupd to see if there are any files in the middle of a dedup operation.
  • Dumps tags referenced in the file system and replication namespaces to /data/hurricane/bpgc/referencedtags.
  • Compares the two files created previously, blockpooltags and referencedtags. Tags that are in blockpooltags but not in referencedtags are now deletion candidates and are used to create the candidate list in /data/hurricane/bpgc/deleteCandidates.
  • The deleteCandidates file is then broken up into 4 separate files so that bpgc can use 4 deletion threads, one for each file.
     
• Stage 3 – Deletion. There are 4 Deletion Threads (controlled by seer variable MaxStreams). Each worker threads copies its candidate file to a working file. When a worker thread makes a single pass through it’s deletion candidate file, it pauses. The bfst_delete_count API for each tag in the candidate list providing the number of times the tag is referenced at the time the candidate list was generated (ensures the tag is not deleted if it was re-stored)
   
• Stage 4 – Compaction. See Stage 1

SpaceManagement will force space reclamation during system low space conditions. In addition, ingest throttling to allow compaction/namespace reconciliation (space reclamation) is still required. However, the component that implements throttling is BPW.

What's Next?

Troubleshooting Examples >

Additional Resources