Quantum
Blockpool: How to Check Truncation and Dedupe Queues in DXi 1.x Software (DRAFT)

SR Information: (TS: Any specific SRs to list here?)

 

Product / Software Version: DXi systems with 1.x software.

 

Problem Description: Deduplication or truncation is not occurring.

Overview

This topic applies to DXi systems with 1.x software and describes how to use the deduplication/truncation queue to see which files have been deduplicated and/or truncated on the DXi. Use the information in this topic to troubleshoot cases were deduplication and/or truncation is not occurring. You can also use the information if you just want  to learn how to monitor the queue.

 

Procedure

First, get familiar with the deduputil command. Bellow you'll find information about the command's syntax:

 

/opt/DXi/util/deduputil [-h]

/opt/DXi/util/deduputil    <file> [-f] [-s] [-m <mountpoint>] [-r] [-n] [-a]

/opt/DXi/util/deduputil -c <file> [-f] [-s] [-m <mountpoint>] [-r] [-n] [-k]

/opt/DXi/util/deduputil -d <file> [-f] [-s] [-m <mountpoint>] [-q]

/opt/DXi/util/deduputil -t <file> [-f] [-s] [-m <mountpoint>]

 <file>:  The absolute path of the file or directory

 -h:      Print this help

 -r:      If <file> is directory, recurse into its subdirectories

 -f:      Display file system statistics

 -s:      Display summary of all files

 -n:      Only print files that have not been deduped

 -c:      See what's in the candidate list file

 -k:      Used in addition to -c, print dedup statistics for each candidate

 -d:      Store this file to the local blockpool and return the tag

 -q:      Used with -d to set the tag into file system EA

 -t:      Call qbfs_truncate to truncate this file

 -a:      Add tag of stored file to replication candidate list

 -m:      The mount point of QBFS under SNFS, default is /snfs/Q

 

 

Below are some useful examples of how to monitor the deduplication/truncation queue:

 

Note that the output of the command bellow will give you an idea of how many files are waiting in the deduplication/truncation queue. You may want to run the command more than once to check if the queue is moving:

 

[root@vtlpwd8003 ~]# /opt/DXi/util/deduputil -nsr /snfs/Q

Number of directories:           24570

Number of deduped/total files:   1065405/1142519 (93%/100%)

Number of truncated/total files: 862160/1127787 (76%/100%)

Remaining files to be deduped: 77,114

Remaining files to be truncated: 265,627

Number of files not qualified for truncation:

  - VTL meta files: 8927

  - Empty files:    5805

Total file size: 44639056365082 bytes

[root@vtlpwd8003 ~]#

 

 

 

[root@vtlpwd8003 ~]# /opt/DXi/util/deduputil -fsr /snfs/Q

 

F5DD9E505DD2B48EEEB900645D5D6A61 (T)            0 /snfs/Q/shares/ns_12_20_1/d.txt

A5457FF4E3F5AC22F14C01BE72BC3FD4 (T)            0 /snfs/Q/shares/ns_12_20_1/d1/r.txt

D41D8CD98F00B204E9800998ECF8427E (NT)            0 /snfs/Q/shares/ns_12_20_1/ps/r.txt

9C31BB23BF969A427E5D272F42EF07F2 (T)            0 /snfs/Q/shares/ns_12_20_1/fgp.txt

633D4D761DD0B696A262733693BD1357 (T)            0 /snfs/Q/shares/ns_12_20_1/g.txt

7FA9459A31722DDB2D68BB8CEAB9EA5A (NT)          310 /snfs/Q/partitions/VL05CX0816BVA00074/Carts/JPS001_CX0816BVA00074_562975/config.xml

DCC6F527E6FFD513E831B05FA409D5F2 (NT)       512048 /snfs/Q/partitions/VL05CX0816BVA00074/Carts/JPS000_CX0816BVA00074_237498/cmd2

                         <NOTAG> (NT)      8404992 /snfs/Q/partitions/VL05CX0816BVA00074/Carts/JPS000_CX0816BVA00074_237498/ucot

DCC6F527E6FFD513E831B05FA409D5F2 (NT)       512048 /snfs/Q/partitions/VL05CX0816BVA00074/Carts/JPS000_CX0816BVA00074_237498/cmd1

D41D8CD98F00B204E9800998ECF8427E (NT)            0 /snfs/Q/partitions/VL05CX0816BVA00074/Carts/JPS000_CX0816BVA00074_237498/data

360E7F81275B573FBD12827F3BCEAA5F (NT)         5683 /snfs/Q/partitions/VL05CX0816BVA00074/Carts/JPS000_CX0816BVA00074_237498/mam.xml

. . .

D41D8CD98F00B204E9800998ECF8427E (T)            0 /snfs/Q/partitions/VL03CX0816BVA00074/Carts/1289L7_CX0816BVA00074_810711/data

EE16E0C0BC9666C1A1E5717AD955659F (T)            0 /snfs/Q/partitions/VL03CX0816BVA00074/Carts/1289L7_CX0816BVA00074_810711/mam.xml

EC286422B5A690127A348FFBE645CD35 (T)            0 /snfs/Q/partitions/VL03CX0816BVA00074/Carts/1289L7_CX0816BVA00074_810711/config.xml

Number of directories:           1287

Number of deduped/total files:   25534/25548 (99%/100%)

Number of truncated/total files: 24791/24808 (99%/100%)

Remaining files to be deduped: 14

Remaining files to be truncated: 17

Number of files not qualified for truncation:

  - VTL meta files: 399

  - Empty files:    341

Total file size: 946836285 bytes

FileSystem Free/Total: 68206521745408/71224415748096 95.76%

 

 

The output above can be very helpful. You can check which files were truncated/deduplicated. Customers may be able to tell you when those files were sent to the DXi by the backup application and help you to determine how long the file has been in the queue.

 

The second column displays whether or not the file has been deduplicated or truncated:

 

 

 


This page was generated by the BrainKeeper Enterprise Wiki, © 2018