About ripu

ripu · ‎08-01-2016

Thanks a lot Kuldeep i agree and thats why i wanted suggestions from experts like you 🙂

ripu · ‎08-01-2016

Hi Team, I have 3 virtual machines in HDP cluster ,if i have huge capacity in data nodes disk in TBs so can i use the same disk with diff mount points to store Data nodes data, NN namenode data, SN data, JT data (master node data) and /usr and /var . I know then if my disk has some issue then all data will be affected basically i wanted to know if my data node disks have lot of space in TBs, so do you recommend creating diff mounts on same data node disks for diff purposes like /usr,/var and storing NN SN JT data Also each HDP version data is in /usr/hdp

ripu · ‎07-19-2016

We can store application related data and logs on SAN/NAS However SAN/NAS are not at all recommended for I/O sensitive and CPU bound jobs , that is to avoid bottleneck situations while reading data from disk or from network or in processing data So for Logs/application data --> SAN/NAS Data nodes data --> DAS with JBOD configuration NO RAID NN/SN/JT nodes --> should be highly available [ RAID 5/10(depends on usecase) ] Hadoop is a scale out and shared nothing architecture http://www.bluedata.com/blog/2015/12/separating-hadoop-compute-and-storage/ https://community.emc.com/servlet/JiveServlet/previewBody/41473-102-1-132603/Virtualizing%20Hadoop%20in%20Large%20Scale%20Infrastructures.pdf Also I understand sometimes true cost of DAS is also more considering Hadoop replication , but this is how Hadoop is thriving (One of the key tenets of Hadoop is to bring the compute to the storage instead of the storage to the compute.)

ripu · ‎07-19-2016

@Sbandaru: i researched over this more deeply and conclusion is , we don't need edge node We don’t need edge node if Hadoop cluster and application are in same network its only needed when hadoop cluster and application are in diff network , at that time edge node acts as a gateway to hadoop cluster ( like a proxy ) thanks for your inputs

ripu · ‎07-16-2016

Hi Team, We are going to deploy HDP 2.3.4 for Big Env setup Can Some one Please explain me the architecture of Edge node in hadoop . I am able to find only the definition on the internet. I have some queries 1)What is edge node? 2) when and why do we need it ? 3) does every production cluster contain this edge node? 4) Does the edge node a part of the cluster (What advantages do we have if it is inside the cluster . Does it store any blocks of data in hdfs. any performance improvement? 5)Should the edge node be outside the cluster . 6) Please refer any docs where i can know about it. Preferably Hortonworks docs

ripu · ‎07-13-2016

will be helpful

ripu · ‎07-13-2016

thanks nice

ripu · ‎07-13-2016

Thanks a lot

ripu · ‎07-13-2016

answer looks goo d.. Thanks for your answer can you please advise how to decide DiskIO in cluster ? which factors to consider for Disk I/O calculation ?

ripu · ‎07-13-2016

Hi Team, can someone kindly advise How to plan a hortonworks hadoop cluster if my application is not running any map reduce jobs .. and i will be loading 250 GB data in hbase i understand we need to take care of below points How to plan my storage? what to use disks or RAID for NN datanodes? How to plan my CPU? How to plan my memory? How to plan the network bandwidth?

Online	Offline
Last Visited	‎09-27-2016 02:35 AM

Member Since	‎06-17-2015 08:02 AM
Last Visited	‎09-27-2016 02:35 AM
Posts	61
Kudos received	20

Cloudera Community

Re: want to decrease block size in HDP ambari , wh...

Re: can we customize HDP installation in other dir...

Re: issue error :cloudera unsupported major minor ...

Re: HDP 2.3.4 failed, parent directory /usr/hdp/cu...

Re: can i use the same disk with diff mount points...

can i use the same disk with diff mount points to ...

Re: Can I place the OS for the nodes Hadoop Cluste...

Re: Edge node do we really need in Production hort...

Edge node do we really need in Production hortonwo...

Re: DISTCP fails from CHD4.2(Non HA+Non-secure) t...

Re: What causes a datanode to consider a volume as...

Re: How to plan a hortonworks hadoop cluster if my...

Re: How to plan a hortonworks hadoop cluster if my...

How to plan a hortonworks hadoop cluster if my app...