Member since
12-16-2015
235
Posts
6
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1930 | 09-01-2020 04:15 AM |
07-06-2021
10:26 AM
@diplompils It is not necessary that file is lost if you are getting the output as false for recoverLease command. Usually file can't be deleted until it has lease acquired and not explicitly deleted using rm command. You can try below- hdfs debug recoverLease -path <file> -retries 10 Or you may check - https://issues.apache.org/jira/browse/HDFS-8576
... View more
05-12-2021
03:24 AM
1 Kudo
@PrernaU Ambari 2.7.5 is only fully compatible with HDP 3.1.5, while other HDP 3.1.x or 3.0.x versions are partially compatible for upgrade only. Please see.
... View more
05-12-2021
03:22 AM
1 Kudo
@fpaezalban Ambari 2.7.5 is only fully compatible with HDP 3.1.5, while other HDP 3.1.x or 3.0.x versions are partially compatible for upgrade only. Please see.
... View more
05-12-2021
03:20 AM
1 Kudo
@Jans What is the exact HDP version? Ambari 2.7.5 is only fully compatible with HDP 3.1.5, while other HDP 3.1.x version are compatible only for upgrade.
... View more
05-10-2021
03:22 AM
@PrernaU I hope you haven't changed any config in HDFS. Please compare pre & Post upgrade configs using ambari UI. Considering the exception below, I have seen this issue once due to memory issue on a single DataNode. ERROR: Cannot set priority of datanode process 45359 i.e. The available memory was 12GB while the DN heap was set to 16GB & due to this the DN JVM was failing to start. /proc/meminfo:
MemTotal: 131407744 kB
MemFree: 2180792 kB
MemAvailable: 12004080 kB You can try checking which process is using RAM by running below command and try to reduce the RAM utilization and start the DN process #ps aux --sort -rss There could be something else but probably some host level crunch is causing the JVM to not get started properly. Please check DN.out and DN.log file once for more details.
... View more
09-01-2020
08:40 AM
There are several options available to achieve this use case. The easiest and the best approach would be HBase snapshots method to transfer the data.
Note: All actions need to be performed as the HBase user only, to ensure correct permissions.
On source cluster: #hbase shell> snapshot ‘Test’_Table’, ‘Test_Table_SS’
#hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot ‘Test_Table_SS’ -files -stats
#hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot 'Test_Table_SS' -copy-to hdfs://<Destination_NN_hostname>:8020/hbase -mappers 16 -bandwidth 200
On destination cluster: #hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot ‘Test_Table_SS’ -files -stats
#hbase shell> clone_snapshot ‘Test_Table_SS’, ‘Test_Table’
#hbase shell> major_compact 'Test_Table'
Once done, you can choose to delete the snapshots on both ms05 and as05: #hbase shell> delete_snapshot 'Test_Table_SS'
But If you plan to use CopyTable , this will not work without additional configuration.
Communication between an older client and a newer server is not guaranteed, there is currently a workaround to allow this to work by adding the following property to your client configuration:
On the client being used to launch the CopyTable, you can do either:
Command Line:
-Dhbase.meta.replicas.use=true
OR
hbase-site.xml:
hbase.meta.replicas.use
true
... View more
09-01-2020
04:15 AM
1 Kudo
There are several options available for this. If you plan to use CopyTable, this will not work without additional configuration. Communication between an older client and newer server is not guaranteed, there is currently a workaround to allow this to work by adding the following property to your client configuration. On the client being used to launch the CopyTable, you can do either: Command Line: -Dhbase.meta.replicas.use=true OR hbase-site.xml: hbase.meta.replicas.use true Your other option would be to use snapshots to transfer the data. Note- If this helps, please don't forget to click on "Accept as Solution".
... View more
03-27-2020
10:12 AM
@dineshc Please specify where the mentioned workaround property needs to be added. Ams-site.xml or Ams-Hbase-site.xml?
... View more
02-06-2020
09:12 AM
@josh_nicholson NOTE: For Kerberized Cluster use the value of "zookeeper.znode.parent" may be "/ams-hbase-secure" so we can connect to it as following: /usr/hdp/2.5.0.0-1245/phoenix/bin/sqlline.py c6403.ambari.apache.org:61181:/ams-hbase-secure
... View more
01-06-2020
08:57 AM
@aengineer Also noticed from one comment in HDFS-8789 that " balancer doesn't support anything other than the default placement policy (BlockPlacementPolicyDefault). " HDFS-14053 says ability for NN to re-replicate blocks based on policy change is fixed in hadoop 3.3.0 [not sure if it's hadoop version or not, though NN version doesn't make sense], while HDFS-14637 supports above statement until UD get enable.
... View more