Quantum
Recovering from HoldLinks issues consuming space.

Replication / BPGC

Thanks to Patrick Proske for this information...

 

Where are the holdTag files held ... in 2.x

 

All Files for replication, (continuous, namespace and trigger replication) are kept in /data

 

[root@dxi-ibm tmp]# cd /data
[root@dxi-ibm data]# ls
AttrBallHoldList              == Holds the Namespace Bundles
etc             
HwMonSim     
NS_AUD        
NS_OSTD      
NS_VMM              
TagHoldList                   == List of tags in the hold list (TBD)
AttrBallTagListHoldList       == Map of HoldLink Directories for the Namespace Bundles
HoldTagRefsMap                == Map of Namespace Bundles and associated Tags
HwMonStatus  
NS_BPWAPI     
NS_TRIGGERD  
PrepostTagHoldList            ==  Continuous Files referenced
TagMap                        == Map of Tags (TBD)
conf           
hurricane         
node1-files  
NS_NU_Qnode1  
NS_VCM       
StorNext            
upload

# Looking at the continuous files:

[root@dxi-ibm data]# cat PrepostTagHoldList
/snfs/tmp/trigger/replication/oracle-101373/continuous,/snfs/tmp/HoldLinks/HoldLink7fcd599d2cc7714fa03b164b7496c988,2455916244,18446744073709551615
/snfs/tmp/trigger/replication/oracle-124054/continuous,/snfs/tmp/HoldLinks/HoldLink89c5c62c3f7fce06c785a9b38710ac11,3838413260,18446744073709551615
/snfs/tmp/trigger/replication/tsm-163308/continuous,/snfs/tmp/HoldLinks/HoldLinkeaf7e95cc86ef3ec6fcdb616078876de,5905138861,18446744073709551615
/snfs/tmp/trigger/replication/oracle-165479/continuous,/snfs/tmp/HoldLinks/HoldLinkdbb8d726459cce69e710c9b73d009b3e,6005321161,18446744073709551615
/snfs/tmp/trigger/replication/oracle-169224/continuous,/snfs/tmp/HoldLinks/HoldLinkeed1952bceeff34ce16fc4b1d99a9e48,6185705264,18446744073709551615
/snfs/tmp/trigger/replication/tsm-173198/continuous,/snfs/tmp/HoldLinks/HoldLinkd0ccd93f79f54b227ce6352801cf41e8,6349208230,18446744073709551615
/snfs/tmp/trigger/replication/oracle-204686/continuous,/snfs/tmp/HoldLinks/HoldLinkff723daec26d6e63ce1e2dd1363bf472,1259465705,18446744073709551615
/snfs/tmp/trigger/replication/oracle-240963/continuous,/snfs/tmp/HoldLinks/HoldLink3fb090a578dd3b97e08fbe9f15cb8219,2987154597,18446744073709551615
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/continuous,/snfs/tmp/HoldLinks/HoldLink0ca99401e955cc17081a18de937c7390,3536113166,18446744073709551615

# This continuous file is a left over from a trigger so lets check the age:

 

/snfs/tmp/trigger/replication/oracle-101373/continuous,/snfs/tmp/HoldLinks/HoldLink7fcd599d2cc7714fa03b164b7496c988,2455916244,18446744073709551615

# Ok the Continuous file is old and we do not need it any more

[root@dxi-ibm data]# cd /snfs/tmp/trigger/replication/oracle-101373/
[root@dxi-ibm oracle-101373]# ls -al
total 192
drwxrwxrwx  2 root 4294967295  2048 Nov 10 00:10 .
drwxrwxrwx 10 root 4294967295 42622 Feb  7 10:36 ..
-rw-r--r--  2 root 4294967295   135 Nov 10 00:10 continuous

# in this case we have a bunch of old trigger replication continuous files from older FW releases lets tar them up and remove them

[root@dxi-ibm oracle-101373]# cd ..

[root@dxi-ibm replication]# ls -al
total 640
drwxrwxrwx 10 root 4294967295 42622 Feb  7 10:36 .
drwxrwxrwx  3 root 4294967295  2048 Aug 29 15:42 ..
drwxrwxrwx  2 root 4294967295  2048 Nov 10 00:10 oracle-101373
drwxrwxrwx  2 root 4294967295  2048 Nov 26 00:11 oracle-124054
drwxrwxrwx  2 root 4294967295  2048 Dec 21 02:06 oracle-165479
drwxrwxrwx  2 root 4294967295  2048 Dec 23 04:13 oracle-169224
drwxrwxrwx  2 root 4294967295  2060 Jan 12 02:08 oracle-204686
drwxrwxrwx  2 root 4294967295  2060 Feb  1 02:03 oracle-240963
drwxrwxrwx  2 root 4294967295  2048 Dec 19 22:17 tsm-163308
drwxrwxrwx  2 root 4294967295  2048 Dec 25 01:38 tsm-173198

[root@dxi-ibm replication]# tar -cvf /scratch/TBRtrigger-filesold.tar *
oracle-101373/
oracle-101373/continuous
oracle-124054/
oracle-124054/continuous
oracle-165479/
oracle-165479/continuous
oracle-169224/
oracle-169224/continuous
oracle-204686/
oracle-204686/continuous
oracle-240963/
oracle-240963/continuous
tsm-163308/
tsm-163308/continuous
tsm-173198/
tsm-173198/continuous
[root@dxi-ibm replication]# rm -rf  *

# Now we need to remove them from the reference list ,too

[root@dxi-ibm data]# cat PrepostTagHoldList
/snfs/tmp/trigger/replication/oracle-101373/continuous,/snfs/tmp/HoldLinks/HoldLink7fcd599d2cc7714fa03b164b7496c988,2455916244,18446744073709551615
/snfs/tmp/trigger/replication/oracle-124054/continuous,/snfs/tmp/HoldLinks/HoldLink89c5c62c3f7fce06c785a9b38710ac11,3838413260,18446744073709551615
/snfs/tmp/trigger/replication/tsm-163308/continuous,/snfs/tmp/HoldLinks/HoldLinkeaf7e95cc86ef3ec6fcdb616078876de,5905138861,18446744073709551615
/snfs/tmp/trigger/replication/oracle-165479/continuous,/snfs/tmp/HoldLinks/HoldLinkdbb8d726459cce69e710c9b73d009b3e,6005321161,18446744073709551615
/snfs/tmp/trigger/replication/oracle-169224/continuous,/snfs/tmp/HoldLinks/HoldLinkeed1952bceeff34ce16fc4b1d99a9e48,6185705264,18446744073709551615
/snfs/tmp/trigger/replication/tsm-173198/continuous,/snfs/tmp/HoldLinks/HoldLinkd0ccd93f79f54b227ce6352801cf41e8,6349208230,18446744073709551615
/snfs/tmp/trigger/replication/oracle-204686/continuous,/snfs/tmp/HoldLinks/HoldLinkff723daec26d6e63ce1e2dd1363bf472,1259465705,18446744073709551615
/snfs/tmp/trigger/replication/oracle-240963/continuous,/snfs/tmp/HoldLinks/HoldLink3fb090a578dd3b97e08fbe9f15cb8219,2987154597,18446744073709551615
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/continuous,/snfs/tmp/HoldLinks/HoldLink0ca99401e955cc17081a18de937c7390,3536113166,18446744073709551615

# vi PrepostTagHoldList and remove all references of the continuous files we just removed
# after the edit"

 

[root@dxi-ibm data]# cat PrepostTagHoldList
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/continuous,/snfs/tmp/HoldLinks/HoldLink0ca99401e955cc17081a18de937c7390,3536113166,18446744073709551615

 

# Ok now we can start bpgc and it will start to cleanup the TAGS which where only referenced in those old continuous files and will also remove the files we kept in /snfs/tmp

/snfs/tmp/HoldLinks/HoldLink7fcd599d2cc7714fa03b164b7496c988,2455916244,18446744073709551615
/snfs/tmp/HoldLinks/HoldLink89c5c62c3f7fce06c785a9b38710ac11,3838413260,18446744073709551615
/snfs/tmp/HoldLinks/HoldLinkeaf7e95cc86ef3ec6fcdb616078876de,5905138861,18446744073709551615
/snfs/tmp/HoldLinks/HoldLinkdbb8d726459cce69e710c9b73d009b3e,6005321161,18446744073709551615
/snfs/tmp/HoldLinks/HoldLinkeed1952bceeff34ce16fc4b1d99a9e48,6185705264,18446744073709551615
/snfs/tmp/HoldLinks/HoldLinkd0ccd93f79f54b227ce6352801cf41e8,6349208230,18446744073709551615
/snfs/tmp/HoldLinks/HoldLinkff723daec26d6e63ce1e2dd1363bf472,1259465705,18446744073709551615
/snfs/tmp/HoldLinks/HoldLink3fb090a578dd3b97e08fbe9f15cb8219,2987154597,18446744073709551615

# Now we look at the Namespace bundles these bundles are referenced on this system:

 

[root@dxi-ibm data]# cat AttrBallHoldList
/snfs/tmp/HoldCBC7A32AF54FB4EB617A29347512E81D,/snfs/tmp/HoldCBC7A32AF54FB4EB617A29347512E81D,2851214626,3540337821
/snfs/tmp/Hold5088A15C913F5D81B7B3B5D3D9F038C8,/snfs/tmp/Hold5088A15C913F5D81B7B3B5D3D9F038C8,3125278053,18446744073709551615
/snfs/tmp/HoldE615A6DCEBF1B77F05B11BC8B2B98212,/snfs/tmp/HoldE615A6DCEBF1B77F05B11BC8B2B98212,3129102926,18446744073709551615
/snfs/tmp/Hold1B4D9C729EC3C26587D3C20A2392737B,/snfs/tmp/Hold1B4D9C729EC3C26587D3C20A2392737B,3192575116,18446744073709551615
/snfs/tmp/HoldB3D32D4C03EB90AF4BDD9E2A7C7CF140,/snfs/tmp/HoldB3D32D4C03EB90AF4BDD9E2A7C7CF140,3196117627,18446744073709551615
/snfs/tmp/Hold64E39C05E86E920B909A3A335619FA15,/snfs/tmp/Hold64E39C05E86E920B909A3A335619FA15,3353688633,3540337730
/snfs/tmp/HoldFD78A3601F10F569EEEEF256D30DDDB0,/snfs/tmp/HoldFD78A3601F10F569EEEEF256D30DDDB0,3439089236,18446744073709551615
/snfs/tmp/Hold3E7F3EFA0E4DDF5887C50D08EE9CF53B,/snfs/tmp/Hold3E7F3EFA0E4DDF5887C50D08EE9CF53B,3456031570,18446744073709551615
/snfs/tmp/HoldCFBC5E68CEE843A9C227A31765D42C02,/snfs/tmp/HoldCFBC5E68CEE843A9C227A31765D42C02,3466771078,18446744073709551615
/snfs/tmp/Hold3F105ED41E31EEA72C111A8675FFDDA9,/snfs/tmp/Hold3F105ED41E31EEA72C111A8675FFDDA9,3471818685,18446744073709551615
/snfs/tmp/Hold9525A30C4FDAF33A014DFDCFC50EA39B,/snfs/tmp/Hold9525A30C4FDAF33A014DFDCFC50EA39B,3525581256,18446744073709551615

# In snfs tmp we can find the Hold Directories for each namespace bundle:

rwxr-xr-x  2 root 4294967295   2048 Feb  3 11:07 Hold1B4D9C729EC3C26587D3C20A2392737B
drwxr-xr-x  2 root 4294967295   2048 Feb  6 12:18 Hold3E7F3EFA0E4DDF5887C50D08EE9CF53B
drwxr-xr-x  2 root 4294967295   2048 Feb  6 16:41 Hold3F105ED41E31EEA72C111A8675FFDDA9
drwxr-xr-x  2 root 4294967295   2048 Feb  2 16:25 Hold5088A15C913F5D81B7B3B5D3D9F038C8
drwxr-xr-x  2 root 4294967295   2048 Feb  5 07:52 Hold64E39C05E86E920B909A3A335619FA15
drwxr-xr-x  2 root 4294967295   2052 Feb  7 07:37 Hold9525A30C4FDAF33A014DFDCFC50EA39B
drwxr-xr-x  2 root 4294967295   2052 Feb  3 12:06 HoldB3D32D4C03EB90AF4BDD9E2A7C7CF140
drwxr-xr-x  2 root 4294967295   2048 Jan 30 12:18 HoldCBC7A32AF54FB4EB617A29347512E81D
drwxr-xr-x  2 root 4294967295   2048 Feb  6 15:17 HoldCFBC5E68CEE843A9C227A31765D42C02
drwxr-xr-x  2 root 4294967295   2048 Feb  2 17:29 HoldE615A6DCEBF1B77F05B11BC8B2B98212
drwxr-xr-x  2 root 4294967295   2048 Feb  6 07:36 HoldFD78A3601F10F569EEEEF256D30DDDB0
drwxrwxrwx  2 root root       663168 Feb  7 10:59 HoldLinks
drwxrwxrwx  3 root root         2059 Jan  4 10:14 restore-tsm130
drwxrwxrwx  2 root root         2048 Feb  6 16:43 sync
drwxrwxrwx  3 root 4294967295   2048 Aug 29 15:42 trigger

# Lets link the Hold files to the namespace bundles.  This is the list of namespaces on this unit:

 
[root@dxi-ibm tmp]# find /snfs/replication/ -type f -name tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/169/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/167/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/168/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/14/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/13/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/12/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/13/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/12/tag
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/11/tag

# And here is the list of the TAGs for each of the above namespaces:

[root@dxi-ibm tmp]#
[root@dxi-ibm tmp]# find /snfs/replication/ -type f -name tag -exec cat {} \;
9525A30C4FDAF33A014DFDCFC50EA39B
FD78A3601F10F569EEEEF256D30DDDB0
CFBC5E68CEE843A9C227A31765D42C02
3F105ED41E31EEA72C111A8675FFDDA9
1B4D9C729EC3C26587D3C20A2392737B
E615A6DCEBF1B77F05B11BC8B2B98212
3E7F3EFA0E4DDF5887C50D08EE9CF53B
B3D32D4C03EB90AF4BDD9E2A7C7CF140
5088A15C913F5D81B7B3B5D3D9F038C8

# And the creation date.

[root@dxi-ibm tmp]# find /snfs/replication/ -type f -name tag -exec ls -la {} \;
-rw-r--r-- 1 root 4294967295 33 Feb  7 07:37 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/169/tag
-rw-r--r-- 1 root 4294967295 33 Feb  6 07:35 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/167/tag
-rw-r--r-- 1 root 4294967295 33 Feb  6 15:17 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/168/tag
-rw-r--r-- 1 root 4294967295 33 Feb  6 16:41 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/14/tag
-rw-r--r-- 1 root 4294967295 33 Feb  3 11:07 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/13/tag
-rw-r--r-- 1 root 4294967295 33 Feb  2 17:29 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/12/tag
-rw-r--r-- 1 root 4294967295 33 Feb  6 12:18 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/13/tag
-rw-r--r-- 1 root 4294967295 33 Feb  3 12:06 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/12/tag
-rw-r--r-- 1 root 4294967295 33 Feb  2 16:25 /snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/11/tag



for i in ` find /snfs/replication/ -type f -name tag` ; do echo -en "$i = " ; cat $i ;done

/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/169/tag       = 9525A30C4FDAF33A014DFDCFC50EA39B
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/167/tag       = FD78A3601F10F569EEEEF256D30DDDB0
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/tsm/168/tag       = CFBC5E68CEE843A9C227A31765D42C02
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/14/tag      = 3F105ED41E31EEA72C111A8675FFDDA9
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/13/tag      = 1B4D9C729EC3C26587D3C20A2392737B
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/veeam/12/tag      = E615A6DCEBF1B77F05B11BC8B2B98212
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/13/tag     = 3E7F3EFA0E4DDF5887C50D08EE9CF53B
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/12/tag     = B3D32D4C03EB90AF4BDD9E2A7C7CF140
/snfs/replication/target/dxi-pra.relaish-dns.fr/shares/oracle/11/tag     = 5088A15C913F5D81B7B3B5D3D9F038C8

# Looking at the HASH numbers we can now tell which HOLDLINK directories are referencing the Namespace and can decided which one we need to remove from the list. (more details below)

# The same information is available in HoldTagRefsMap

 

Note:
In order to remove a continuous file we need to remove from the PrepostTagHoldList, the reference to the HoldLink file and the original continous file on the file system
In order to remove a Old Namespace bundle we would need to modify the HoldTagRefsMap, AttrBallHoldList and remove the Namespace bundle from the Filesystem

 

 

 

How to remove the last Namespace bundle from a DXi2.x unit

There are 2 ways of doing this:

 

  1. GUI
  2. CLI

 

The GUI actions

We need to make sure nobody is accessing the SHARE for the time we work on it.
a. Disabel Replication on the Source Unit
b. Go to the Target Unit and remove the Namespace
c. Enable Trigger Base Replication again

Now TBR is working again and with the propergated deletes in 2.1.1 nothing stays behind, a run of bpgc will delete all tags referenced in the deleted namespace and there should be no need for another SYNC run

 

 

CLI actions

Manually Clear a namespace in 2.x
There is one namespace on this unit:
./DXi0-211-src.labs.northampton.uk/shares/ppnfs-so:
total 256
drwxrwxrwx 2 root 4294967295 2065 Apr 24 13:23 6 <[Namespace Bundle ID 6]
-rw-r--r-- 2 root root 0 Apr 27 07:41 continuous
-rw-r--r-- 1 root 4294967295 2 Apr 24 13:23 oneup

./DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/6:
total 384
drwxrwxrwx 2 root 4294967295 2065 Apr 24 13:23 .
drwxrwxrwx 3 root 4294967295 2950 Apr 27 07:41 ..
-rw-rw-rw- 1 root 4294967295 73 Apr 24 13:23 properties
-rw-rw-rw- 1 root 4294967295 52 Apr 24 13:23 status
-rw-r--r-- 1 root 4294967295 33 Apr 24 13:23 tag
-rw-rw-rw- 1 root 4294967295 29 Apr 24 13:23 update

The existing namespaces are referenced here.
/data/
# cat HoldTagRefsMap

 <?xml version="1.0"?>
        <root>
              <tags>
                        <tag>
                                <tag>E9FC96D20B31E17BD5FC39FB1DCC5B24</tag>
                                <id>/snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/6</id>
                       </tag>
            </tags>
      </root>

And here is the link to the Namespace in /snfs/tmp
/data/
# cat AttrBallHoldList
/snfs/tmp/HoldE9FC96D20B31E17BD5FC39FB1DCC5B24,/snfs/tmp/HoldE9FC96D20B31E17BD5FC39FB1DCC5B24,1732818246,18446744073709551615

NOTE: same HASH E9FC96D20B31E17BD5FC39FB1DCC5B24 (will change for each bundle)
looking at snfs tmp

# ls -la /snfs/tmp/
total 33152
drwxrwxrwx 7 root root 2437 Apr 27 09:15 .
drwxrwxrwx 14 root root 2050 Apr 27 09:13 ..
drwxr-xr-x 2 root 4294967295 2052 Apr 24 13:23 HoldE9FC96D20B31E17BD5FC39FB1DCC5B24
drwxrwxrwx 2 root root 23934 Apr 27 08:25 HoldLinks
[root@DXi0-211-tgt data]# ls -la /snfs/tmp/HoldLinks/
total 192
drwxrwxrwx 2 root root 23934 Apr 27 08:25 .
drwxrwxrwx 7 root root 2437 Apr 27 09:15 ..
-rw-rw-r-- 2 root root 33 Apr 4 14:16 HoldLink729a7ff5c86f8a999144f266de8b2280
-rw-r--r-- 2 root root 0 Apr 27 07:41 HoldLink84f6ccc2dc632046484da5b89c5bdfb7

Continuous files are referenced here.

# cat PrepostTagHoldList
/snfs/replication/source/srchost/shares/cifspp-tg/1/tag,/snfs/tmp/HoldLinks/HoldLink729a7ff5c86f8a999144f266de8b2280,8015461,18446744073709551615
/snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/continuous,/snfs/tmp/HoldLinks/HoldLink84f6ccc2dc632046484da5b89c5bdfb7,1971509388,18446744073709551615

.. now doing the old style of a clean up ... we remove the Namespace Dir:

# cd /snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/
# ls -al
drwxrwxrwx 2 root 4294967295 2049 Apr 4 16:33 cifspp-so
drwxrwxrwx 3 root 4294967295 2950 Apr 27 07:41 ppnfs-so
# mv ppnfs-so/ /snfs/namespace-ppnfs-so
# ls -al
drwxrwxrwx 2 root 4294967295 2049 Apr 4 16:33 cifspp-so

Looking at the messages in tsunami. bpgc removes old HoldList files:

INFO - 04/27/12-09:15:02 - bpgc GCAPI.cpp(464) [bpgc] launchManualDeletions() - Attempting to start deletions, signaling bpgc
INFO - 04/27/12-09:15:02 - bpgc SysUtil.cpp(549) [bpgc] signal() - Sending signal:10 to pid:16978
INFO - 04/27/12-09:15:02 - bpgc GCDaemon.cpp(365) [bpgc] interruptWait() - Start Reclamation Request detected, stopping wait
INFO - 04/27/12-09:15:07 - bpgc CleanupStrategyFactory.cpp(122) [bpgc] create() - Selecting the Target Cleanup Strategy
INFO - 04/27/12-09:15:07 - bpgc CompactThread.cpp(318) [bpgc] doProcess() - {1149200704} Starting GC compaction
INFO - 04/27/12-09:15:08 - bpgc CompactThread.cpp(224) [bpgc] updateStatusFile() - {1149200704} 0 bytes of 0 reclaimed (progress: 100.00)
INFO - 04/27/12-09:15:08 - bpgc ReconcileThread.cpp(1121) [bpgc] doProcess() - {1086871872} Reconciling Blockpool Tags
INFO - 04/27/12-09:15:08 - bpgc ReconcileThread.cpp(126) [bpgc] dumpBlockpoolTags() - {1086871872} Dumping the blockpool tags
INFO - 04/27/12-09:15:08 - bpgc ReconcileThread.cpp(257) [bpgc] dumpReferencedTags() - {1086871872} Dumping referenced tags
INFO - 04/27/12-09:15:08 - bpgc GCUtil.cpp(1998) [bpgc] callCleanupHoldList() - Calling cleanupHoldList
INFO - 04/27/12-09:15:08 - bpgc GCUtil.cpp(2002) [bpgc] callCleanupHoldList() - Call to cleanupHoldList completed successfully
INFO - 04/27/12-09:15:08 - BPWAPI.AttrBall AttrBall.cpp(558) [bpgc] getSharitionPolicy() - Path [/snfs/ddup] is not a sharition. Defaulting to RP_NO_REPLICATION.
INFO - 04/27/12-09:15:09 - bpgc ReconcileThread.cpp(836) [bpgc] determineCandidates() - {1086871872} Determining the blockpool deletion candidates
INFO - 04/27/12-09:15:09 - bpgc FileUtil.cpp(352) [bpgc] sortFile() - Sorting file /data/hurricane/bpgc/blockpooltags
INFO - 04/27/12-09:15:09 - bpgc FileUtil.cpp(352) [bpgc] sortFile() - Sorting file /data/hurricane/bpgc/referencedtags
INFO - 04/27/12-09:15:09 - bpgc ReconcileThread.cpp(1180) [bpgc] doProcess() - {1086871872} Cleaning up old replicated namespaces
INFO - 04/27/12-09:15:09 - bpgc ReconcileThread.cpp(1191) [bpgc] doProcess() - {1086871872} Cleaning up old source namespaces
INFO - 04/27/12-09:15:09 - bpgc ReconcileThread.cpp(1216) [bpgc] doProcess() - {1086871872} Reconciling Complete
INFO - 04/27/12-09:15:10 - bpgc CompactThread.cpp(318) [bpgc] doProcess() - {1149200704} Starting GC compaction
INFO - 04/27/12-09:15:10 - bpgc CompactThread.cpp(224) [bpgc] updateStatusFile() - {1149200704} 0 bytes of 0 reclaimed (progress: 100.00)
INFO - 04/27/12-09:15:10 - bpgc GCStatusFile.cpp(680) [bpgc] logState() - Reclamation Results --- StartTime:1335514502; EndTime:1335514510; Running:false; 
Progress:100; TotalProgress:100; Status:Reclamation Completed - No Candidates To Delete; SubStatus:; ActualBytesReclaimed:0; ActualBytesToReclaim:0; UserBytesReclaimed:0; UserBytesToReclaim:0; 
INFO - 04/27/12-09:15:15 - bpgc CleanupStrategyFactory.cpp(127) [bpgc] create() - Selecting the Reconcile-Only Strategy
INFO - 04/27/12-09:15:15 - bpgc CleanupReconcileStrategy.cpp(129) [bpgc] postRun() - Sleeping for 43200 seconds

But by only deleting the Namespace directory we did not remove the information in the hold files and nothing got deleted.
Ok lets do the next step:
Remove the references in the files with a vi:
$ vi /data/HoldTagRefsMap

Before:

          <?xml version="1.0"?>
           <root>
           <tags>
                 <tag>
                        <tag>E9FC96D20B31E17BD5FC39FB1DCC5B24</tag>
                        <id>/snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/6</id>
               </tag>
           </tags>
          </root>

After:

<?xml version="1.0"?>
  <root>
           <tags>
           </tags>
</root>

$ vi /data/AttrBallHoldList

Before:
/snfs/tmp/HoldE9FC96D20B31E17BD5FC39FB1DCC5B24,/snfs/tmp/HoldE9FC96D20B31E17BD5FC39FB1DCC5B24,1732818246,18446744073709551615

After:
#The file should hold the other entrys you might need but this line above is removed

Now run bpgc again:
[root@DXi0-211-tgt data]# /hurricane/bpgc -delete

INFO - 04/27/12-09:22:18 - bpgc GCAPI.cpp(464) [bpgc] launchManualDeletions() - Attempting to start deletions, signaling bpgc
INFO - 04/27/12-09:22:18 - bpgc SysUtil.cpp(549) [bpgc] signal() - Sending signal:10 to pid:16978
INFO - 04/27/12-09:22:18 - bpgc GCDaemon.cpp(365) [bpgc] interruptWait() - Start Reclamation Request detected, stopping wait
INFO - 04/27/12-09:22:23 - bpgc CleanupStrategyFactory.cpp(122) [bpgc] create() - Selecting the Target Cleanup Strategy
INFO - 04/27/12-09:22:23 - bpgc CompactThread.cpp(318) [bpgc] doProcess() - {1149200704} Starting GC compaction
INFO - 04/27/12-09:22:24 - bpgc CompactThread.cpp(224) [bpgc] updateStatusFile() - {1149200704} 0 bytes of 0 reclaimed (progress: 100.00)
INFO - 04/27/12-09:22:24 - bpgc ReconcileThread.cpp(1121) [bpgc] doProcess() - {1086871872} Reconciling Blockpool Tags
INFO - 04/27/12-09:22:24 - bpgc ReconcileThread.cpp(126) [bpgc] dumpBlockpoolTags() - {1086871872} Dumping the blockpool tags
INFO - 04/27/12-09:22:24 - bpgc ReconcileThread.cpp(257) [bpgc] dumpReferencedTags() - {1086871872} Dumping referenced tags
INFO - 04/27/12-09:22:24 - bpgc GCUtil.cpp(1998) [bpgc] callCleanupHoldList() - Calling cleanupHoldList
INFO - 04/27/12-09:22:24 - bpgc GCUtil.cpp(2002) [bpgc] callCleanupHoldList() - Call to cleanupHoldList completed successfully
INFO - 04/27/12-09:22:24 - BPWAPI.AttrBall AttrBall.cpp(558) [bpgc] getSharitionPolicy() - Path [/snfs/ddup] is not a sharition. Defaulting to RP_NO_REPLICATION.
INFO - 04/27/12-09:22:25 - bpgc ReconcileThread.cpp(836) [bpgc] determineCandidates() - {1086871872} Determining the blockpool deletion candidates
INFO - 04/27/12-09:22:25 - bpgc FileUtil.cpp(352) [bpgc] sortFile() - Sorting file /data/hurricane/bpgc/blockpooltags
INFO - 04/27/12-09:22:25 - bpgc FileUtil.cpp(352) [bpgc] sortFile() - Sorting file /data/hurricane/bpgc/referencedtags
INFO - 04/27/12-09:22:25 - bpgc ReconcileThread.cpp(1180) [bpgc] doProcess() - {1086871872} Cleaning up old replicated namespaces
INFO - 04/27/12-09:22:25 - bpgc ReconcileThread.cpp(1191) [bpgc] doProcess() - {1086871872} Cleaning up old source namespaces
INFO - 04/27/12-09:22:25 - bpgc ReconcileThread.cpp(1216) [bpgc] doProcess() - {1086871872} Reconciling Complete
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(526) [bpgc] doProcess() - {1191160128} Starting GC deletions
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(526) [bpgc] doProcess() - {1201649984} Starting GC deletions
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(526) [bpgc] doProcess() - {1212139840} Starting GC deletions
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(526) [bpgc] doProcess() - {1222629696} Starting GC deletions
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(563) [bpgc] doProcess() - {1191160128} Found 1 tags to delete
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(563) [bpgc] doProcess() - {1201649984} Found 0 tags to delete
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(563) [bpgc] doProcess() - {1212139840} Found 0 tags to delete
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(563) [bpgc] doProcess() - {1222629696} Found 0 tags to delete
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1191160128} GC Delete Progress is currently: 1.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1201649984} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(439) [bpgc] deleteCandidates() - {1201649984} Successfully deleted 0 tags
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1212139840} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(439) [bpgc] deleteCandidates() - {1212139840} Successfully deleted 0 tags
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1222629696} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(439) [bpgc] deleteCandidates() - {1222629696} Successfully deleted 0 tags
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1201649984} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1212139840} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(439) [bpgc] deleteCandidates() - {1191160128} Successfully deleted 1 tags
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1222629696} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc DeleteThread.cpp(180) [bpgc] updateStatusFile() - {1191160128} GC Delete Progress is currently: 100.00%
INFO - 04/27/12-09:22:26 - bpgc CompactThread.cpp(318) [bpgc] doProcess() - {1149200704} Starting GC compaction
INFO - 04/27/12-09:22:27 - bpgc CompactThread.cpp(224) [bpgc] updateStatusFile() - {1149200704} 0 bytes of 10240 reclaimed (progress: 1.00)

INFO - 04/27/12-09:23:45 - bpgc GCStatusFile.cpp(680) [bpgc] logState() - Reclamation Results --- StartTime:1335514938; EndTime:1335515025; Running:false; 
Progress:100; TotalProgress:100; Status:Reclamation Completed; SubStatus:; ActualBytesReclaimed:10240; ActualBytesToReclaim:10240; UserBytesReclaimed:10240; UserBytesToReclaim:10240;

OK now we deleted the old namespace data from the blockpool.
Last thing to do is remove the old Namespace from /snfs/tmp:

$ rm HoldE9FC96D20B31E17BD5FC39FB1DCC5B24

Ok other way around:

I replicated another time the same share:
$ ls -la /snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/
total 256
drwxrwxrwx 3 root 4294967295 2066 Apr 27 09:30 .
drwxrwxrwx 4 root 4294967295 2058 Apr 27 09:14 ..
drwxrwxrwx 2 root 4294967295 2065 Apr 27 09:30 1
-rw-r--r-- 1 root 4294967295 2 Apr 27 09:30 oneup

NOTE: The namespace Id 1: (because I removed the whole directory above)
$ ls -la /snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/1/
total 384
drwxrwxrwx 2 root 4294967295 2065 Apr 27 09:30 .
drwxrwxrwx 3 root 4294967295 2066 Apr 27 09:30 ..
-rw-rw-rw- 1 root 4294967295 73 Apr 27 09:30 properties
-rw-rw-rw- 1 root 4294967295 52 Apr 27 09:30 status
-rw-r--r-- 1 root 4294967295 33 Apr 27 09:30 tag
-rw-rw-rw- 1 root 4294967295 53 Apr 27 09:30 update

$ls -la /snfs/tmp
drwxr-xr-x 2 root 4294967295 2052 Apr 27 09:30 HoldDD5DFF6D49395D27518355CC2830FB70

We have the Attribute Ball referenced.

$cat AttrBallHoldList
/snfs/tmp/HoldDD5DFF6D49395D27518355CC2830FB70,/snfs/tmp/HoldDD5DFF6D49395D27518355CC2830FB70,1978031316,18446744073709551615

and the file cat HoldTagRefsMap tells us the Namespace linked to this AttrBall:

<?xml version="1.0"?>
   <root>
             <tags>
                       <tag>
                                <tag>DD5DFF6D49395D27518355CC2830FB70</tag>
                                <id>/snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/1</id>
                      </tag>
           </tags>
 </root>


$ syscli --list repnas

Output data:

List of all replicated NAS share on the target:
Total count = 1
[replicated nas = 1]
ID = 1
NAS Share Name = ppnfs-so
Source Host = DXi0-211-src.labs.northampton.uk
Replication Started = Fri Apr 27 09:30:23 2012
Replication Finished = Fri Apr 27 09:30:27 2012
Replication Status = Success

Command completed successfully.


Now try to delete:

$ syscli --del repnas --repname ppnfs-so --srchost DXi0-211-src.labs.northampton.uk --id 1

This replicated share ppnfs-so cannot be removed until either 
1) The share's replication state is changed to disabled on the source system, or 
2) the source system host entry has been removed from the list of "Allowed Replication Hosts" on this target host.


This would stop the shares being in sync if any modifications are made to the share while we modify the settings in the gui so to avoid this we need to remove the files by running the the cleanup on the cli.


We do this in short:


Remove the Referenced TAG or TAGS from HoldTagRefsMap for the Namespace Bundle we'd like to delete.
$ vi /data/HoldTagRefsMap


Remove the Referenced TAG or TAGS from AttrBallHoldList
$ vi /data/ AttrBallHoldList


Remove the directory HoldBundle /snfs/tmp/Hold{HASH}
Remove the namespace dir in /snfs/replication/target/DXi0-211-src.labs.northampton.uk/shares/ppnfs-so/

Then run GC, /hurricane/bpgc -delete

 


This page was generated by the BrainKeeper Enterprise Wiki, © 2018