Support Questions

Find answers, ask questions, and share your expertise

Java Profiling of long running MapReduce container

avatar
Rising Star

Hi dear community!

 

i tried to track MR container (mapper) through Java Mission Control (JMC) and through visualvm.

In both cases I got some error:

Untitled.png

 

if i would profile spark - no problem.
Goal of this action to understand which Java classes consume much more CPU resource.
could anybody tell how to fix this profiling or which tool could show which Java Classes use much more resources in MR container.

 

also add example with visualvm with NodeManager (it doesn't works as well as MR container, and Spark worker profile - it's what i want to get)

Untitled.png

 

 

 

 

what I want to get:

Untitled1.png


thanks in advance!

 

1 ACCEPTED SOLUTION

avatar
Cloudera Employee
What user are the MR and Spark containers run as? The Yarn processes all run as user "yarn", and I wonder if a user-name mismatch is causing issues.

Also, what user are you running the profiling tools?
Karthik Kambatla
Software Engineer, Cloudera Inc.

View solution in original post

2 REPLIES 2

avatar
Cloudera Employee
What user are the MR and Spark containers run as? The Yarn processes all run as user "yarn", and I wonder if a user-name mismatch is causing issues.

Also, what user are you running the profiling tools?
Karthik Kambatla
Software Engineer, Cloudera Inc.

avatar
Rising Star
thanks a lot! it works!