Hello All,   This is an older post which had a few recent followup queries. To close the loop, HBase offers multiple Tools to migrate Data from 1 Cluster to another Cluster like Snapshot, Export-Import, HashTable/SyncTable etc. Most of these Tools relies on MapReduce & uses 1 Mapper per Region of the Source Table. All these Tools works without any concerns. The only part of the ask which can't be answered accurately is the Concurrency/Job Configurations/Mapper Memory etc. These details rely on Customer's Environment Setup & the Bandwidth between the 2 Clusters. As such, Customer can run 1 such HBase MR Job & see the Outcome. Accordingly, Fine-Tune is required.    If any issues are observed while performing the above HBase MR Job, Feel free to post the Q in a Community Post for fellow Community Members to review & share their thoughts.    Regards, Smarak ... View more

  When ranger_audits  collection got created in solr, solr plugin test connection in ranger UI is throwing the error Authentication Required.            ... View more

Since Hadoop 2.8, it is possible to make a directory protected and so all its files cannot be deleted, using : fs.protected.directories property.   From documentation: "A comma-separated list of directories which cannot be deleted even by the superuser unless they are empty. This setting can be used to guard important system directories against accidental deletion due to administrator error."   It does not exactly answer the question but it is a possibility. ... View more

I am using CDH 5.13 and CDK 4.0 (apache 2.1). Getting the same problem. However, consumer is still not getting messages after deleting brokers and topics as suggested by you.  ... View more

@Rakesh Enjala we were getting the similar issue, where all of our blocks under HDFS were coming up as Under Replicated Blockshdfs-under-replicated-blocks.png the default value for ipc.maximum.data.length is 67108864Bytes (64MB) from https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/core-default.xml in our case we were getting this as about 100MB to avoid the issue we have increased the value to 128MB and able to get the cluster back to normal but before this we have done some feasts 🙂 and which caused us some unexpected behaviors in our cluster including the data loss this is happened due to: 1) we were thinking deleting the under replicated blocks using hdfs fsck / -delete will delete only the under replicated blocks which it did but in our case we lost the some of the data as well due to the ipc.maximum.data.length issue NameNode doesn't have the actual metadata because of this we lost the blocks (data) but the files were existing with 0Bytes. 2) One of the design issues we have in our cluster was we only have a single mount point (72TB) for Datanodes which is a big mistake where it have been made at least into 6 each with 12TB. 3) Never run the hdfs fsck / -delete when you see the Requested data length 97568122 is longer than maximum configured RPC length 67108864 from the NameNode logs. Hope this helps someone ... View more