Member since
06-19-2014
5
Posts
0
Kudos Received
0
Solutions
09-11-2014
07:47 AM
Thanks a lot Darren, a very simple solution 🙂
... View more
09-11-2014
07:09 AM
Hello, When setting the parameter "dfs.datanode.max.locked.memory" to a value other than 0, it is supposed to be the user's responsibility to set the MEMLOCK limits accordingly. CM does not leave that for the user and enforces it. (as seen on /proc/<pid>/limits) This parameter is used by HDFS only to limt cache resources, so there is no real reason for CM to enforce this limit on an OS level. By doing so, CM does not allow external libraries (non-cache related) on the datanode to allocate memory past this limit. To make a long story short, I'd like to cancel CM's enforcement of this limit and set the limits myself. Is there any way I could do so? Thanks, Elad
... View more
Labels:
- Labels:
-
Cloudera Manager
-
HDFS
06-19-2014
04:27 AM
Update: I found the parameter: mapreduce.jobhistory.done-dir With it I was able to set the root directory for done history files But that doesn't completely solve my problem Right now the directory structure is /rootdir/year/month/day/000000/jhist-file I'd like to drop the whole year/month/day/000000 part. Is that possible?
... View more
06-19-2014
03:52 AM
Hi, I'm using MR2 with CDH5 and I noticed that the user history location (on HDFS) for completed jobs is /user/history/done I'm trying to automate some jobs and then analyze their job history, so it could be useful for me to set a different location per job. How can I do it? Thanks, Elad I Mellanox Technlogies
... View more
Labels:
- Labels:
-
HDFS