Member since
02-23-2017
15
Posts
0
Kudos Received
0
Solutions
01-05-2018
07:08 AM
Hi, Is it technically possible to run Two Cloudera setups in a small POC cluster? This is only to understand the technical feasibility and nothing to do with real-time use cases. For example: We have 5 nodes with decent hardware named node 1, node 2..Node5. If not, may i know why and where it will break? Can we use Node 1, Node 2 as master for first cloudera setup and Node 4 and Node 5 for second clustera setup with completely different paths and port numbers? I know it is a dump qustion but think there is something to learn
... View more
Labels:
- Labels:
-
Manual Installation
10-26-2017
08:42 AM
Useful tips. Thanks Jag for sharing
... View more
02-28-2017
05:36 AM
Thank you Mr. mqureshi for the quick and detailed response. it is interesting to know that HW Ambari or Cloudera Manager is the only way to see the consolidated logs. If so, how HW Ambari and Cloudera Manager collects the logs and show consolidated logs? are they storing the logs consolidately or read the logs from each machine at real time and show? i believe, Ambari or ClouderaM stores consolidated logs somewhere? any links would be great
... View more
02-28-2017
04:41 AM
Hi,
I have below basic questions on log file location
in a ideal HW cluster setup,
1. Where exactly the hadoop system log of hadop master(s) and data nodes are stored?
for example,
a) where can i see consolidated system logs of Name node & individual Data nodes? what is the path? can i see consolidated log on name node or should i refer both name node and data nodes?
a) where can i see consolidated system logs of RM (resource mgr) & NM (Node Mgr)? what is the path? can i see consolidated log on RM & NM or should i refer both RM and NM?
2. Where exactly the hadoop application log of MR, Spark, YARN are stored?
a) where can i see consolidated YARN application logs? assume that there are 100 YARN applications (Hive queries, Pig scripts etc) running in a day. Can i see all the logs on one centralized location?
b) where can i see consolidated SPARK application logs? assume that there are 100 Spark applications running in a day. Can i see all the logs on one centralized location?
I hear, system & application logs can be stored on HDFS? is it a default HW behavior?
Apologies for this dump question. Appreciate your response.
... View more
- Tags:
- Hadoop Core
- logs
02-28-2017
04:26 AM
Thank you Mr. Grahan for your valuable inputs
... View more
02-27-2017
09:31 PM
Hi, I have below basic questions on log file location in a ideal HW cluster setup, 1. Where exactly the hadoop system log of hadop master(s) and data nodes are stored? for example, a) where can i see consolidated system logs of Name node & individual Data nodes? what is the path? can i see consolidated log on name node or should i refer both name node and data nodes? b) where can i see consolidated system logs of RM (resource mgr) & NM (Node Mgr)? what is the path? can i see consolidated log on RM & NM or should i refer both RM and NM? 2. Where exactly the hadoop application log of MR, Spark, YARN are stored? a) where can i see consolidated YARN application logs? assume that there are 100 YARN applications (Hive queries, Pig scripts etc) running in a day. Can i see all the logs on one centralized location? b) where can i see consolidated SPARK application logs? assume that there are 100 Spark applications running in a day. Can i see all the logs on one centralized location? I hear, system & application logs can be stored on HDFS? is it a default HW behavior? Apologies for this dump question. Appreciate your response.
... View more
- Tags:
- logs
Labels:
- Labels:
-
Cloudera Manager
02-27-2017
08:24 PM
Hi, Thank you for your response. We got a query for one of our customer. We can take this question as pure tech question on how to migrate data, code, scripts, properties from Hortonworks to cloudera and vice versa. Thanks
... View more
02-27-2017
08:23 PM
Thank you for your inputs mbigelow 🙂
... View more
02-24-2017
03:51 AM
Can you please so kind to let me know the options for migrating data and scripts from Cloudera to Hortonworks. 1) What are the options/tools available for Data migration from Cloudera to Hortonworks Platform? 2) What are the tools/options available for copying the scripts (Hive, Pig etc) from Cloudera to Hortonworks Platform? 3) What are the tools/options available for copying important properties (Yarn, HDFS, Hive etc) from Cloudera to Hortonworks Platform? 4) What are the tools/options available for copying Rules/Policies (like Ranger) from Cloudera to Hortonworks? Thanks in advance
... View more
Labels:
02-23-2017
11:38 PM
1) What are the options/tools available for Data migration from Hortonworks To Cloudera Platform? 2) What are the tools/options available for copying the scripts (Hive, Pig etc) from Hortonworks To Cloudera Platform? 3) What are the tools/options available for copying important properties (Yarn, HDFS, Hive etc) from Hortonworks To Cloudera Platform? 4) What are the tools/options available for copying Rules/Policies (like Ranger) from Hortonworks To Cloudera? Thanks in advance
... View more
Labels:
- Labels:
-
Cloudera Manager