Member since
09-18-2015
3274
Posts
1159
Kudos Received
426
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2717 | 11-01-2016 05:43 PM | |
| 9323 | 11-01-2016 05:36 PM | |
| 5067 | 07-01-2016 03:20 PM | |
| 8505 | 05-25-2016 11:36 AM | |
| 4654 | 05-24-2016 05:27 PM |
02-09-2016
11:22 PM
@Astronaut Bigdatanova You can backup ambari database as it has details of all the configs. Active active setup will meet your requirements. You can back up server like any other server. We have built in HA for namenode and RM " you can enable it once cluster is installed" Falcon will be backing up and replicating the data. Hdfs replication will provide fault tolerance Again, look into wandisco 🙂
... View more
02-09-2016
10:57 PM
@Astronaut Bigdatanova Using this to complete the answer based on your latest questions: 1)Testing: Link You will see various components listed.Spark is just a component out of 20+ components. You can run basic smoke test for each component.Pick a use case and then pick a tech/component, research on that piece & you will find tons of testing docs. 2)I shared benchmark link as I did not realize that you are asking for app testing 3)Falcon can do HDFS, Hive replication from one location to another.On prem,cloud or wherever your cluster is running.You have to have servers besides storage in DR
... View more
02-09-2016
10:46 PM
@Lester Martin Thanks for testing and confirming. I think you should publish article based on your comments
... View more
02-09-2016
10:37 PM
@Pardeep see this http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_ambari_views_guide/content/section_getting_cluster_config_values_pig_view.html Table 6.2. Pig View Settings for NameNode High Availability
... View more
02-09-2016
10:19 PM
@Astronaut Bigdatanova 1) Testing tools : Really good guide http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/ 2) Yes , we do need separate clusters for dev, QA and Prod ( Same as RDBMS world) for users to test and QA before prod releases. 3) Active-Active (3rd party tools like Wandisco) , Active-Passive , Apache Falcon is best for DR. 4) Get familiar with the Hadoop stack. Get familiar with 100% open source model of Hadoop. There are vendors who are selling Hadoop but not 100% open source. Hortonworks is the only 100% open source enterprise ready hadoop "no vendor lock in". Once you pick the vendor then learn the technology stack Operations, Security, Data operating system ...Please see this blog https://www.linkedin.com/pulse/20141204175510-28584737-rdbms-to-hadoop The above blog covers the sandbox and other details.
... View more
02-09-2016
10:12 PM
@Wes Floyd Great question! @Enis @Josh Elser
... View more
02-09-2016
10:11 PM
H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe" The Ambari view throws the following error: H100 Unable to submit statement show databases like '*': org.apache.thrift.transport.TTransportException:java.net.SocketTimeoutException: Read timed out
And I can no longer view database or run queries in the Ambari view. The problem appears to resolve itself after some time, but then will reappear after running a sequence of queries in the Hive view. 1) This should be fixed in Ambari 2.2 2) Browser Refresh fixed it: When connection was open for too long then there was a broken pipe when you tried to access the hive view. When you did a restart to opened a fresh connection and was able to connect.
... View more
Labels:
02-09-2016
10:01 PM
2 Kudos
@hkropp Details from support: It should be fixed in Ambari 2.2 also We think what probably happened was the connection was open for too long which is why it said there was a broken pipe when you tried to access the hive view. When you did a restart to opened a fresh connection and was able to connect.
Probably just a refresh of the ambari page would have reset the connection as well.
... View more
02-09-2016
08:45 PM
@khushi kalra For lineage and Graph https://issues.apache.org/jira/browse/AMBARI-5837
... View more