About madhura2688

madhura2688 · ‎04-24-2019

Hi , We are using 12 Datanodes in our cluster. And we have 3 replication factor. Currently we are doing cost optimization and our total cluster is 35% used. We want to remove one datanode from our cluster, but we want to see first how our cluster performs on 11 datanodes. if it doesn't perform well, we need to restart that datanode again. To experiment this- Do I need to Stop the datanode or Decomission datanode? Please need suggestion.

madhura2688 · ‎03-22-2019

Currently we are seeing our HDFS DFS directory is getting filled up and we have to remove the data at faster rate. We currently have 12 datanodes and 4 masternodes 1 edgenode. Can I delete the files from HDFS from masternodes and edgenodes at once? I have created a script on edgenode which deletes the HDFS files but speed is really slow. How can I delete multiple files at a time ? Can I place that script on multiple server and delete the files?

madhura2688 · ‎02-12-2019

@Geoffrey Shelton Okot Even after doing Rebalance HDFS to 25% threshold value, I still see the disk is 100% , IS hdfs not able to read it from the disk as its full, Also I had to set the DataNode failed disk tolerance to 1 as HDFS service was not coming up on that node. Can we delete the data manually from that particular disk? is there any way?

madhura2688 · ‎02-11-2019

Hi, We currently have 8 Datanodes which has two hdfs disk mounted on each of the datanodes. One of the disk from the datanode is full. On this node HDFS(nodemanager) service was not coming up as it had below error Upon checking into articles found out that we can setup- "DataNode failed disk tolerance" value to 1 and ignore this volume as its 100% full, But I would like to understand how I can cleanup the data from this disk? I tried doing rebalance HDFS but which threshold value I should use? StorageLocation [DISK]file:/hdfs/data1/hadoop/hdfs/data/ org.apache.hadoop.util.DiskChecker$DiskErrorException: Error checking directory /hdfs/data1/hadoop/hdfs/data /dev/xvdl 50G 49G 0 100% /hdfs/data1 /dev/xvdk 50G 27G 21G 56% /hdfs/data0

madhura2688 · ‎10-17-2018

Hi @Aditya Sirna I tried to follow this steps however, Spark2 History server is not available to ADD on another node? I could add Livy for spark2 and Spark2 Thrift Server.. Can you please tell me how I should move the clients? Can you please help ? Thank you

madhura2688 · ‎10-17-2018

No Move button for Spark2 in Ambari . Why? If I have to move spark2. I will need to add service to node and then restart the service right? What happens to the existing spark2-clients in the node. if I try to add another spark2 components?

madhura2688 · ‎10-15-2018

@Jay Kumar SenSharma Can you please help me with this one?

madhura2688 · ‎10-14-2018

Hi All, I have spark components installed on one host. Here are the components- Livy for Spark2 Server Spark2 Thrift Server Spark2 History Server I want to move this components to other host, as I have to stop the current running host which has these components. Can you please let me know how I should perform this moving components and do I need to change anything on application side? Can I move this through Ambari ? Also, any service restart is required? Appreciate your help! Thanks.

madhura2688 · ‎10-08-2018

Hi @Aditya Sirna, I have some followup questions. 1. I have multple hiveserver, so thats why I want to stop one.. but if I stop it I still see the znode. So I am not sure if Zookeeper will select this hiveserver2 randomly. Do I need to delete this hiveserver2 after stopping so it will get deleted from the znode? 2. For hiveserver2 to install is there any steps need to follow? Thanks.

madhura2688 · ‎10-08-2018

I have a node where I have installed hiveserver2. I have to stop the running hiveserver2 on that node, if I just stop the hiveserver2. It still shows up in the zookeeper if I do ls /hiverserver ... So zookeeper will still select that hiveserver even though its stopped? As I know zookeeper will select randomly registered hive server2. Also If I want to install the hiverserver2 again.. I should just install it from Ambari right? is there any other steps need to follow to install hiveserver2?

Online	Offline
Last Visited	‎12-10-2019 10:41 AM

Member Since	‎10-04-2018 04:56 PM
Last Visited	‎12-10-2019 10:41 AM
Posts	39

Cloudera Community

Need suggestion to stop or Decommission Datanode?

Can I delete multiple files from hdfs?

Re: How to cleanup data from single HDFS disk?

How to cleanup data from single HDFS disk?

Re: How to move Spark2 components to another node?

No Move button for Spark2 in Ambari . Why?

Re: How to move Spark2 components to another node?

How to move Spark2 components to another node?

Re: Will Zookeeper select the stopped hiveserver2?

Will Zookeeper select the stopped hiveserver2?