Member since
02-10-2015
84
Posts
2
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
13444 | 06-04-2015 06:09 PM | |
7349 | 05-22-2015 06:59 AM | |
5996 | 05-13-2015 03:19 PM | |
2434 | 05-11-2015 05:22 AM |
04-02-2019
07:48 AM
Hi Vinod, Can you please start a different thread and share your Master logs with us? JMS
... View more
06-05-2017
05:58 PM
Hi TS, What kind of permissions do you have to set on the /user/history/done folder to resolve the issue? Best, SKA
... View more
02-20-2017
07:09 PM
Hi, When i Run fsck on my cluster i got that several blocks under replicated and the target replication is 3 even i changed the dfs.replication to NN/ DNs and client server to replication factor 2, and mapred.submit.replication changed to 2. tried also: <property> <name>dfs.replication</name> <value>2</value> <final>true</final> </property> I also restarted all service at my cluster including the oozie. Looking at one of the running jobs conf and see the following with replication factor 3: mapreduce.client.submit.file.replication s3.replication kfs.replication dfs.namenode.replication.interval ftp.replication s3native.replication
... View more
02-17-2017
06:45 AM
Thanks for the update MSharma, glad to hear that you were able to resolve the issue!
... View more
11-13-2015
02:41 AM
Hi We have the same problem, we upgraded from Cloudera 5.3.8 to 5.4.8 usign parcels <!-- fs.default.name, fs.defaultFS = 8020 --> <!-- hive.metastore.warehouse.dir = /user/hive/warehouse --> <!-- hdfs://hd01:8020:8020/user/hive/warehouse/ -->
... View more
09-23-2015
08:52 AM
I had tried this but seemed to have some trouble in using things like pyspark, etc. - is there a gist or something somewhere with exact steps for CDH? I will try again and post what I did.
... View more
07-22-2015
10:35 AM
1 Kudo
Hello TS, MySQL is a perfectly great choice to be the metadata store for the entities you mention. CM doesn't have a "default RDBMS" per se, but certain installation methods can pull postgres in for you. It's perfectly fine to elect to use MySQL instead, and I'd encourage it (as well as guiding you toward our documentation which you can cite to your firm showing that it's fine to use [1]). RDBMS choice aside, the most important consideration is making sure that you have planned for and allocated sufficient space (or the ability to easily grow the available space) for the entities that will use the RDBMS. That's the absolute key. Some people love PostgresQL, others are very savvy with MySQL. Yet others may have a mandate to use Oracle 11g in an environment. Great - Cloudera Manager and CDH support any of these options! As for your questions: <Q1> How often CM talks to its backend metadata store? A1 - Cloudera Manager remains in constant contact with its metadata store. <Q2> Is it possible to quantify the amount of data traffic from the data store to the apps? A2 - I've not done this recently, but it would be an interesting exercise. Moreso than just the Cloudera Manager Server though, a few other of the Cloudera Management Services use an RDBMS (which may be considered for placement on the same instance as the one CM uses). <Q3> Should (or must) the database exist into the same server as CM?? (In my view YES, but I am looking for strong justifications!!!) A3 - The database instance is not required to be located on the same node as Cloudera Manager Server, but if that's what makes sense in your deployment then it's fine to do so. Opting for a colocated database can, in some cases, remove network latency from the picture. But, if you have access to a dedicated database admin team that can deploy MySQL and manage it (while also making sure it is backed by reliable and fast storage), then it can also make more sense to use that rather than a non-dedicated disk that's local to the Cloudera Manager Server. Your circumstances will dictate what's best. Refer to the document 'Storage Space Planning for Cloudera Manager', as it will also help you take note of the various services that use an RDBMS and some of the considerations you should take before deployment of same. Regards, -- Mark S. [1] - http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_cm_requirements.html#cmig_topic_4_4_unique_1
... View more
06-04-2015
06:09 PM
Issue has been resolved! I had to create a new sl for /etc/alternatives/hadoop-conf
... View more
05-26-2015
06:11 PM
In CM & CDH 5.4 you should unset it and let it use the one that is there on the nodes. Much faster. Wilfred
... View more
05-25-2015
05:13 PM
1 Kudo
A1: check the Hdfs Design page for details on what is stored were. The edits log and file system image are on the NN. Look for the section on persistence on file system data. For more detail on setting up the cluster follow Cluster Setup. A2: if you have the disk then having a mirrored disk will make it more resilient. Making a backup is still a good idea 😉 Wilfred
... View more