Member since
09-18-2015
191
Posts
81
Kudos Received
40
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1993 | 08-04-2017 08:40 AM | |
5311 | 05-02-2017 01:18 PM | |
1068 | 04-24-2017 08:35 AM | |
1080 | 04-24-2017 08:21 AM | |
1296 | 06-01-2016 08:54 AM |
04-29-2016
12:27 PM
Yes, indeed, you've identified the issues I was thinking about. I'd definately suggest rolling back the snapshots and seeing if you can repeat the situation. I once had a room full of 40 people, with one cluster per 2 people, and a slightly iffy wifi connection, we probably saw this occur on 3 of the 20 clusters, so I know the issue is out there :o/
... View more
04-29-2016
12:19 PM
Hi @ARUNKUMAR RAMASAMY the best resource I've found describing this can be found here: https://0x0fff.com/spark-dataframes-are-faster-arent-they/ I'd be interested to know if this answers your question or not!
... View more
04-29-2016
12:16 PM
Hi @axel couty. I have seen this, and other similar strange behaviour before, usually when there's been some form of network instability between the users browser and the ambari server. It seems to get the Ambari database into a strange condition. If that's the case, you may need to perform some database hacking to fix it, however before you descend into that, have you tried restarting the ambari server first and seeing if once it comes back up the new versions appear now?
... View more
04-29-2016
11:48 AM
2 Kudos
Hi there @Pradeep kumar to remove a service from a host you need to use the Ambari API, this is all detailed in the following community article:
https://community.hortonworks.com/questions/607/how-do-you-remove-a-service-from-hdp-via-ambari.html Hope that helps!
... View more
04-29-2016
10:59 AM
2 Kudos
Hi @Tabrez Basha Syed there are a few points regarding the difference between underlying disk block sizes and HDFS block size. The key here is HDFS block size is a logical abstraction on top of the system filesystem rather than a native filesystem of its own. To understand HDFS block size in more detail, I'd recommend reviewing a few good stackoverflow questions:
http://stackoverflow.com/questions/13012924/large-block-size-in-hdfs-how-is-the-unused-space-accounted-for http://stackoverflow.com/questions/19473772/data-block-size-in-hdfs-why-64mb For your disk/filesystem recommendations take a look here: https://community.hortonworks.com/content/kbentry/14508/best-practices-linux-file-systems-for-hdfs.html Hope that all helps!
... View more
04-29-2016
10:38 AM
Hi @Pedro Rodgers there's quite a lot of good information available at the Hortonworks website that includes project examples and customer success stories. The intro page for a lot of this is: http://hortonworks.com/partner/sas/ A few examples include: Rogers Media - https://youtu.be/wTnkg16jHwg Webinar on Predictive Analytics and Machine Learning using SAS - https://youtu.be/D6YzqFgiRnI
... View more
04-28-2016
07:11 PM
Hi @kavitha velaga Ambar Metrics is just added like any other service in Ambari, the process for adding a new service is documented here: http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Users_Guide/content/_adding_a_service.html For Ambari Metrics tuning, take a look at the guide in the documentation: http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_ambari_reference_guide/content/ch_amb_ref_configuring_ambari_metrics.html All Ambari documentation can be found from here: http://docs.hortonworks.com/HDPDocuments/Ambari/Ambari-2.2.1.1/index.html Hope that helps!
... View more
04-26-2016
06:11 AM
One more thing to add is that HA mode or non HA mode only adds one or two nodes when you're getting started, and isn't dependant on how much data you have. Very large clusters may well have more master nodes to further distribute the services of course.
... View more
04-26-2016
06:09 AM
1 Kudo
Hi @sujitha sanku that's a big question to answer in a short response, but probably the best resource to point you at is the high level sizing calculator which we have on-line here:
http://hortonworks.com/blog/how-to-size-your-hadoop-cluster/ There's also a whitepaper linked from there too. Finally, do check out the cluster planning guide documentation for HDP as well: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_cluster-planning-guide/content/ch_hardware-recommendations_chapter.html Hope that helps!
... View more
04-24-2016
12:00 PM
Also incase anyone else comes across this, the file you need to modify from the root of your metron dir is: ./deployment/amazon-ec2/ansible.cfg
... View more