Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Get client's network activity in Hadoop by IP address

Get client's network activity in Hadoop by IP address

New Contributor

Hello. Some client's app connects to the Hadoop cluster. So, I want to get performance metrics from this app activity on Hadoop. Is there any possibility on Cloudera to get some metrics by client's IP Address. Thanks in advance.

1 REPLY 1
Highlighted

Re: Get client's network activity in Hadoop by IP address

Super Mentor

@Iurii 

One of the option will be to parse the HDFS Audit log "/var/log/hadoop/hdfs/hdfs-audit.log" which also contains the ClientIP address information and the kind of operation these clients are performing.  

 

Example:

2019-11-13 09:08:01,505 INFO FSNamesystem.audit: allowed=true	ugi=yarn (auth:SIMPLE)	ip=/172.xxx.67.xx	cmd=listStatus	src=/ats/active	dst=null	perm=null	proto=rpc	callerContext=yarn_ats_server_v1_5
2019-11-13 09:08:01,975 INFO FSNamesystem.audit: allowed=true	ugi=hdfs (auth:SIMPLE)	ip=/172.xxx.67.xx	cmd=getfileinfo	src=/tmp	dst=null	perm=null	proto=rpc	callerContext=CLI
2019-11-13 09:08:02,019 INFO FSNamesystem.audit: allowed=true	ugi=hdfs (auth:SIMPLE)	ip=/172.xxx.67.xx	cmd=listStatus	src=/tmp	dst=null	perm=null	proto=rpc	callerContext=CLI

 

Don't have an account?
Coming from Hortonworks? Activate your account here