About learninghuman

learninghuman · ‎10-27-2017

I have come across DataPlane service which was announced recently. Few questions in that regard, 1. As far as i understand, it contains Atlas, Ranger and Knox. Is this correct? 2. What is the motivation behind this new product? If i have HDP, i get Ranger, Knox, Atlas anyways. 3. What is the applicability of this product? Can i use this with other Hadoop distributions? 4. Has the sandbox for Dataplane been released? Where can i find the download?

learninghuman · ‎03-14-2017

My question on HDFS using SAN as the backend storage has 3 main parts 1. Is it feasible to use SAN as the back end storage for HDFS? 2. What are the pros and cons of using SAN or NAS for HDFS? 3. Has it been tested for performance and may be other aspects?

learninghuman · ‎01-23-2017

@Devin Pinkston For some reason the link does not work. But essentially, i would to know HWs recommendation of using BlueData for HDP and on that platform i also get the other services running (multi-tenancy).

learninghuman · ‎01-23-2017

I came across this and found the BlueData's BDAAS interesting. Whats Hortownworks recommendation for the same in using HDP on BlueData?

learninghuman · ‎01-06-2017

@Tom McCuch Thanks for the clarification. One another related question is that in general what are the advantages that Mesos would bring over Yarn? Especially given the fact that Hortonworks is making efforts to support HDP on Mesos. I mean why care. If HDP on the cloud, its still YARN thats going to be the cluster manager.

learninghuman · ‎01-05-2017

Is it possible to deploy HDP docker container in Mesos using Marathon? If so, where can i get the docker images from and the Marathon recipes? If its not possible with the combination above, what are the options to deploy HDP on Mesos? How is it going to be better than running on Yarn?

learninghuman · ‎12-30-2016

My understanding along with questions as below, AWS-HDCloud Manual scaling using Ambari or AWS UI possible. Auto Scaling 1. Is it possible to auto-scale in this option (while creating the cluster can i set auto-scaling group)? 1.1. In that case, how is the data re-balanced? i.e. if a new node is added, then compute may not gain data locality. -------------------------------------------------------------------------------------------------------------------------------------------------------------- AWS-HDP on IaaS Manual scaling using Ambari is possible. Auto Scaling-Without CloudBreak 2. Is it possible to auto-scale in this option (while creating the cluster can i set auto-scaling group)? 2.1. In that case, how is the data re-balanced? i.e. if a new node is added, then compute may not gain data locality. Auto Scaling-WithCloudBreak Auto-scaling may be possible, but question 2.1 applies here as well. -------------------------------------------------------------------------------------------------------------------------------------------------------------- Azure-HdInsights Manual scaling using Ambari or Azure UI possible. Auto Scaling 3. Is it possible to auto-scale in this option (while creating the cluster can i set auto-scaling group)? 3.1. In that case, how is the data re-balanced? i.e. if a new node is added, then compute may not gain data locality. -------------------------------------------------------------------------------------------------------------------------------------------------------------- Azure-HDP in MarketPlace Manual scaling using Ambari or Azure UI possible. Auto Scaling 4. Is it possible to auto-scale in this option (while creating the cluster can i set auto-scaling group)? 4.1. In that case, how is the data re-balanced? i.e. if a new node is added, then compute may not gain data locality. -------------------------------------------------------------------------------------------------------------------------------------------------------------- Azure-HDP on IaaS Same questions as AWS-HDP on IaaS

learninghuman · ‎12-30-2016

@Tom McCuch One last question which i got after reading your answer again. WASB in Azure is supported on both HDP on Azure IaaS and HDP in Azure MarketPlace. Does this mean that WASB is natively optimized in Hadoop 2.x? If so, this would also mean that any distribution with Hadoop 2.x deployed on Azure can use WASB for storage?

learninghuman · ‎12-28-2016

@Tom McCuch So to summarize, please correct as appropriate 1. HDI 3.5 - WASB and ADLS 2. Pre HDI 3.5 - Only WASB 3. HDP on Asure IaaS - Only WASB and HDFS on VHD 4. HDP from Azure Marketplace - Only WASB and HDFS on VHD 5. HDCloud 2.5 - S3 Only 6. HDP on AWS IaaS - HDFS on Ephemeral or EBS

learninghuman · ‎12-27-2016

@Tom McCuch Thanks. Can you also please talk a little bit about ADLS? Do you still recommend WASB over ADLS? And i am not clear on the parallelism factor on s3 and WASB. Are you saying that S3 does not offer parallelism and suitable for larger number of smaller files? whats you take on parallelism when it comes to WASB? And can i use WASB, ADLS and S3 when i install HDP on Azure's IaaS (using CloudBreak) as the HDFS layer?

Online	Offline
Last Visited	‎07-05-2016 08:52 AM

Date Registered	‎06-29-2016 09:30 AM
Last Visited	‎07-05-2016 08:52 AM
Total Messages Posted	81
Total Kudos Received	43

Re: Mapreduce and Hcatalog Integration fails to us...

Dataplane service

Feasibility and recommendation for running HDFS on...

Re: Hortonworks recommendation for Blue Data

Hortonworks recommendation for Blue Data

Re: HDP on Mesos using Marathon, Docker

HDP on Mesos using Marathon, Docker

Scaling and Auto-scaling of HDP on AWS and Azure C...

Re: HDP on Cloud (Azure, AWS) - Storage options an...

Re: HDP on Cloud (Azure, AWS) - Storage options an...

Re: HDP on Cloud (Azure, AWS) - Storage options an...