Created 11-09-2016 08:48 AM
Hi
I run my java MR job with parameters:
-Dmapreduce.task.profile=true \ -Dmapreduce.task.profile.maps=0 \ -Dmapreduce.task.profile.params="-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,format=b,file=%s" \
Looks good, in webUI i see
Log Type: profile.out Log Upload Time: Wed Nov 09 17:53:39 +0300 2016 Log Length: 2184156 Showing 4096 bytes of 2184156 total. Click here for the full log. ����(��9�����( A/�}� ��'��'��� ��'��'�����'x�'x����4'pw;0����5'pw;0����%p�P1��<$�'' 9�A�C$�'' 9�Q�#��#��� ��" /�-�8��!��!������!��!���{�� �J`S� �� @A@�"�� (�<�2 �#�^ �!�6 `0�!�n `0�!�s `0�!�d `0�!�i `0 �#�� �!ȯ �!ȴ
It looks file created, but i can't download it. I want to open it in visualVM.
When i try
hadoop fs -ls /var/log/hadoop-yarn/apps/a.pilipenko/logs/application_1478695140324_0107/ Found 2 items -rw-r----- 2 a.pilipenko hadoop 2189451 2016-11-09 17:53 /var/log/hadoop-yarn/apps/a.pilipenko/logs/application_1478695140324_0107/srv108_57010 -rw-r----- 2 a.pilipenko hadoop 52479 2016-11-09 17:53 /var/log/hadoop-yarn/apps/a.pilipenko/logs/application_1478695140324_0107/srv37_33143
It's 2 logs - from 1 mapper and from AM, no profile dump, on local server directories there is not such file.
But when i try
yarn logs -applicationId application_1478695140324_0107 -nodeAddress srv108 -containerId container_1478695140324_0107_01_000008
It shows
LogType:profile.out Log Upload Time:Wed Nov 09 17:53:39 +0300 2016 LogLength:2184156 Log Contents: JAVA PROFILE 1.0.1XI?*?,8G??java.lang.ExceptionG???P??G??java.util.WeakHashMap$EntryG???P??G?!?java.util.WeakHashMap$Entry[]G???P??G??java.lang.invoke.MethodTypeG???P??G??java.lang.CloneableG???P??G??java.util.SetG??1P??G??java.lang.System$2G???P??G?java.io.PrintStreamG????G?? java.lang.ThreadLocalG??(P ?? G?"? java.lang.ref.PhantomReferenceG???P ?? G?0? java.util.Collections$SynchronizedCollectionG??]P ?? G?9? java.security.ProtectionDomain$JavaSecurityAccessImplG???P ?? G??java.lang.LinkageErrorG???P??G??sun.misc.CleanerG???P??G?java.security.PrincipalG???P??G??java.security.Principal[]G???P??G?6?sun.reflect.generics.repository.AbstractRepositoryG??DP??G?%?java.lang.SystemClassLoaderActionG???P??G??java.lang.reflect.ModifierG??HP??G??java.util.Hashtable$EntryG????G??java.util.Hashtable$Entry[]G???P??G?'?java.lang.reflect.InvocationHandlerG??sP??G?)?java.lang.ClassLoader$ParallelLoadersG???P??G?#?java.nio.charset.CharsetEncoderG???P??G??java.util.Map$EntryG??P??G??ava.util.Map$Entry[]G???P???java.util.MapG???P??G??java.util.ComparatorG??P??G?"?java.security.cert.CertificateG??
and so on. It looks like my binary hprof file.
But i can't download it separate.
Any ideas?
Created 11-21-2016 11:01 AM
Dear apilipenko,
Here is my custom awk code to do this:
yarn logs -applicationId application_1478695140324_0107 -nodeAddress srv108 -containerId container_1478695140324_0107_01_000008 | awk 'BEGIN {a=0}; /^JAVA PROFILE/{a=1;print $0}; a==1 && !/^LogType/{print $0}; /^LogType/{a=0}' > profile.out
Gabor
Created 11-22-2016 02:37 AM
Thank you!
I've tried script like this:
#!/usr/bin/perl # $p = 0; while (<STDIN>) { if ($_ =~ /^JAVA PROFILE/) { $p = 1; } if ($_ =~ /^LogType\:stderr/) { $p = 0; } print $_ if $p; }
and than
head --bytes -1 profile.hprof.1 > profile.hprof.2
or
head --bytes -2 profile.hprof.1 > profile.hprof.3
if dump can't be open correct because of last bites.
Thank you one more time!