About rki_

rki_ · ‎09-13-2023

@newtocm you can't pause the Balancer. You can kill it and start it again and it will try to balance the rest of the DFS data remaining to be balanced.

rki_ · ‎08-25-2023

Generally, the size of data blocks would be 128mb across all the Datanodes. But, if you have small files then you might see smaller blocks on some Datanodes as well. So Datanodes with different disk spaces would have uneven "Number of Blocks" and the Balancing happens based on the difference in the DFS usage and not by the difference in block count.

rki_ · ‎08-24-2023

Did you make any changes at the KDC end prior to seeing this issue? Are there any other services hosted on this node that are working fine?

rki_ · ‎08-24-2023

@Crash you can set up load balance at a disk level for the Datanodes. Refer : https://docs.cloudera.com/documentation/enterprise/5-16-x/topics/admin_dn_storage_balancing.html

rki_ · ‎08-17-2023

Hi @Ben1996, As this is very specific use case, I don't see any Cloudera doc which could cover this. The basic requirements would be to import the hdfs module and provide the HDFS configs. For ex, https://tahiriamine9.medium.com/python-hdfs-cd822199799e#:~:text=Let's%20have%20an%20example%20of,into%20HDFS%20with%20CSV%20format.

rki_ · ‎08-07-2023

I would suggest keeping one jar file at a time and checking. 1) /home/hadoop/hbase/hbase-2.5.5/lib/hbase-shaded-netty-4.1.4.jar!/META-INF/native/liborg_apache_hbase_thirdparty_netty_transport_native_epoll_x86_64.so 2) /home/hadoop/hbase/hbase-2.5.5/lib/phoenix-client-hbase-2.5-5.1.3.jar!/META-INF/native/liborg_apache_hbase_thirdparty_netty_transport_native_epoll_x86_64.so

rki_ · ‎08-04-2023

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/phoenix_installation.html#concept_ysq_t4n_c3b

rki_ · ‎08-04-2023

@Noel_0317 Do you have rack awareness configured for the Datanodes? Also, check for any disk-level issues on the datanode. Try enabling Debug for block placement : log4j.logger.org.apache.hadoop.hdfs.server.blockmanagement. BlockPlacement Policy= DEBUG log4j.logger.org.apache.hadoop.hdfs.protocol. Block StoragePolicy= DEBUG

rki_ · ‎07-30-2023

It requires Hbase cluster as its a SQL layer on top of Hbase

rki_ · ‎07-28-2023

@sra_vis For CDP, In Cloudera Manager, click Home Click to the right of the cluster name and select Add Service . A list of service types display. Select Phoenix from the list and click Continue . Follow the wizard to add the Phoenix service. For HDP : https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.1/bk_command-line-installation/content/ch_install_phoenix_chapter.html

Online	Offline
Last Visited	‎09-26-2023 10:14 AM

Member Since	‎07-30-2020 02:04 AM
Last Visited	‎09-26-2023 10:14 AM
Posts	189
Kudos received	30

Cloudera Community

Cloudera Community

Re: CM-HDFS

Re: How to write to HDFS remotely using pandas

Re: Balancer is not available

Re: HBase querying large table CallTimeoutExceptio...

Re: Small file in hadoop

Re: Can HDFS Rebalancer run without interrupted Pr...

Re: CM-HDFS

Re: Unable to register Impalad: Caused by: KrbExce...

Re: CM-HDFS

Re: How to write to HDFS remotely using pandas

Re: Does Apache Phoenix run on distributed mode or...

Re: Does Apache Phoenix run on distributed mode or...

Re: Datanode low number of blocks

Re: Does Apache Phoenix run on distributed mode or...

Re: Does Apache Phoenix run on distributed mode or...