Hadoop – HBase Oldwals: What is it, how can I clean it?

hbase(main):005:0 > list_peers
PEER_ID CLUSTER_KEY STATE TABLE_CFS
Indexer_profilesIndexer m1.prod.ps,m2.prod.ps,m3.prod.ps:2181:/ngdata/sep/hbase-slave/Indexer_profilesIndexer DISABLED nil
1 row(s) in 0.0070 seconds

hbase(main):006:0> remove_peer'Indexer_profilesIndexer'
0 row(s) in 0.0050 seconds

 
hbase(main):007:0> list_peers
PEER_ID CLUSTER_KEY STATE TABLE_CFS
0 row(s) in 0.0020 seconds

The space in our small hadoop cluster is insufficient, so I am checking the disk usage on HDFS and I found that most of the space is occupied by the /hbase/oldWALs folder .

I have checked the “HBase authoritative book” and other books, websites, I also searched my question on Google, but I did not find the correct answer…

So I want to know what this folder is, what is its purpose, and how to free up space from this folder without destroying everything…

If it is related to a specific version… we The cluster is under 5.3.00-1.cdh5.3.0.p0.30 of cloudera (hbase 0.98.6).

Thank you for your help!

FYI

I have posted this question in the hbase user list.
The following is Enis Söztutar (an hbase submission者) and how I solved it:

The folder gets cleaned regularly by a chore in master. When a WAL file is
not needed any more for recovery purposes (when HBase can guaratee HBase
has flushed all the data in the WAL file), it is moved to the oldWALs
folder for archival. The log stays there until all other references to the
WAL file are finished. There is currently two services which may keep the
files in the archive dir. First is a TTL process, which ensures that the
WAL files are kept at least for 10 min. This is mainly for debugging . You
can reduce this time by setting hbase.master.logcleaner.ttl configuration
property in master. It is by default 600000. The other one is replication.
If you have replication setup, the replication processes will hang on to
the WAL files until they are replicated. Even if you disabled the
replication, the files are st ill referenced.

You can look at the logs from master from classes (LogCleaner,
TimeToLiveLogCleaner, ReplicationLogCleaner) to see whether the master is
actually running this chore and whether it is getting any exceptions .

Replication cannot be performed on all clusters, but it was enabled in the past because we used hbase-indexer to copy data from HBase to Solr, which is based on replication.

p>

I ran this command on the hbase shell:

hbase(main):005:0> list_peers
PEER_ID CLUSTER_KEY STATE TABLE_CFS
Indexer_profilesIndexer m1.prod.ps,m2.prod.ps,m3.prod.ps:2181:/ngdata/sep/hbase-slave/Indexer_profilesIndexer DISABLED nil
1 row(s) in 0.0070 seconds
 
hbase(main):006:0> remove_peer'Indexer_profilesIndexer'
0 row(s) in 0.0050 seconds


hbase(main):007:0> list_peers
PEER_ID CLUSTER_KEY STATE TABLE_CFS
0 row(s) in 0.0020 seconds

Finally I deleted the oldsWALs folder on hdfs!

The folder no longer grows!