How to do remote debugging for Apache Hadoop Client e.g (NamenodeWebHdfsMethods.java) module using Intellij.

[#1]

I am new to Apache Hadoop, so please excuse or apologies me, if my query is silly or this is not the right platform to ask for the same. I know how to do remote debugging using intellij IDE, but not sure how to connect my Hadoop Client (using 2.8.1 binary) by adding JVM parms. My hadoop client is running on some Virtual Machine. I want to do remote debugging so that I can put breakpoints in the code and see the flow of the code. Thanks in advance

[#2]

@Bhushan kumar

In the terminal where you are running the Hadoop Cline you can set the "HADOOP_OPTS" to add the remote debugging option.

Example:

# export HADOOP_OPTS = $HADOOP_OPTS -agentlib:jdwp=transport=dt_socket,address=9999,server=y,suspend=n

# hdfs dfs -ls /

.

[#3]

@Jay SenSharma
Thank you for your response, I tried this but could not connect remotely. The error on IntelliJ - Unable to run debugger.

echo $HADOOP_OPTS on my hadoop client terminal gives me:

-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9999

Am I missing something?

[#4]

I have the same problem (HDP 2.6, vmware).

Reading a lot of related answers I believe that the problem is port forwarding and/or docker configuration.

What has to be done to enable acess to a specific port (9999 in the example above) of the sandbox?

(Probably the same is done for scp - the sandbox is accessed via port 2222 though there is nothing listening at port 2222 in the sandbox.)

Unfortunately the most useful commands for examining this problem are not installed in the docker machine (e.g. ifconfig, netstat, ...)