Reply
Highlighted
Expert Contributor
Posts: 88
Registered: ‎09-17-2014
Accepted Solution

Java Profiling of long running MapReduce container

[ Edited ]

Hi dear community!

 

i tried to track MR container (mapper) through Java Mission Control (JMC) and through visualvm.

In both cases I got some error:

Untitled.png

 

if i would profile spark - no problem.
Goal of this action to understand which Java classes consume much more CPU resource.
could anybody tell how to fix this profiling or which tool could show which Java Classes use much more resources in MR container.

 

also add example with visualvm with NodeManager (it doesn't works as well as MR container, and Spark worker profile - it's what i want to get)

Untitled.png

 

 

 

 

what I want to get:

Untitled1.png


thanks in advance!

 

Cloudera Employee
Posts: 8
Registered: ‎07-08-2013

Re: Java Profiling of long running MapReduce container

What user are the MR and Spark containers run as? The Yarn processes all run as user "yarn", and I wonder if a user-name mismatch is causing issues.

Also, what user are you running the profiling tools?
Karthik Kambatla
Software Engineer, Cloudera Inc.
Expert Contributor
Posts: 88
Registered: ‎09-17-2014

Re: Java Profiling of long running MapReduce container

thanks a lot! it works!
Announcements