Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Java Profiling of long running MapReduce container

Solved Go to solution
Highlighted

Java Profiling of long running MapReduce container

Rising Star

Hi dear community!

 

i tried to track MR container (mapper) through Java Mission Control (JMC) and through visualvm.

In both cases I got some error:

Untitled.png

 

if i would profile spark - no problem.
Goal of this action to understand which Java classes consume much more CPU resource.
could anybody tell how to fix this profiling or which tool could show which Java Classes use much more resources in MR container.

 

also add example with visualvm with NodeManager (it doesn't works as well as MR container, and Spark worker profile - it's what i want to get)

Untitled.png

 

 

 

 

what I want to get:

Untitled1.png


thanks in advance!

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Java Profiling of long running MapReduce container

Cloudera Employee
What user are the MR and Spark containers run as? The Yarn processes all run as user "yarn", and I wonder if a user-name mismatch is causing issues.

Also, what user are you running the profiling tools?
Karthik Kambatla
Software Engineer, Cloudera Inc.
2 REPLIES 2

Re: Java Profiling of long running MapReduce container

Cloudera Employee
What user are the MR and Spark containers run as? The Yarn processes all run as user "yarn", and I wonder if a user-name mismatch is causing issues.

Also, what user are you running the profiling tools?
Karthik Kambatla
Software Engineer, Cloudera Inc.

Re: Java Profiling of long running MapReduce container

Rising Star
thanks a lot! it works!