question Re: [SOLVED]Java Hdfs Program Fails With Connection Refused in Archives of Support Questions (Read Only)

[SOLVED]Java Hdfs Program Fails With Connection Refused

mathfish — Sun, 18 Aug 2019 10:03:28 GMT

Hopefully someone can help me figure out what is going on.

I downloaded Hdp 2.6 for virtualbox and followed the instructions to install and set up. I am running it on Linux Mint 18. I am following along in the 'Hadoop the Definitive Guide' book and I am trying to run one of the examples. Here is the code I wrote from the book:

I set up my /etc/hosts like this, just following the instructions in the 'Learning the Ropes' tutorial.

Following the book I do the following but I am getting a connection refused error. Is something not setup correctly?

Any and all help would be GREATLY appreciated. I was up super late last night trying to figure this out with no luck.

Re: [SOLVED]Java Hdfs Program Fails With Connection Refused

mathfish — Sun, 18 Aug 2019 10:03:09 GMT

So the above works if I do:

hadoop com/thomp/io/FileSystemCat hdfs://sandbox.hortonworks.com:8020/HadoopPrac/Data/sample.txt

or if I do

hadoop com/thomp/io/FileSystemCat hdfs://172.17.0.2:8020/HadoopPrac/Data/sample.txt

Why is that? I set up hdp 2.6 sandbox on my mac and it works when I just do localhost. Why wouldn't it be the same thing since it is the same install?

Here is a screen shot of the /etc/hosts file on the vm:

Re: [SOLVED]Java Hdfs Program Fails With Connection Refused

emattos — Wed, 17 May 2017 10:24:42 GMT

You can execute the following tests bellow, usually the sanbox hdfs is not set to listen to localhost, but to sandbox.hortonworks.com;

hdfs$ hdfs dfs -ls hdfs://sandbox.hortonworks.com/tmp

You can try with localhost and should fail with the same Error:

hdfs$ hdfs dfs -ls hdfs://localhost/tmp

To check, just go to ambari:services:hdfs:configs:Advance and filter for port number 8020;

Re: [SOLVED]Java Hdfs Program Fails With Connection Refused

emattos — Wed, 17 May 2017 10:42:23 GMT

dfs.namenode.datanode.registration.ip-hostname-check

If true (the default), then the namenode requires that a connecting datanode's address must be resolved to a hostname. If necessary, a reverse DNS lookup is performed. All attempts to register a datanode from an unresolvable address are rejected. It is recommended that this setting be left on to prevent accidental registration of datanodes listed by hostname in the excludes file during a DNS outage. Only set this to false in environments where there is no infrastructure to support reverse DNS lookup.

Re: [SOLVED]Java Hdfs Program Fails With Connection Refused

mathfish — Wed, 17 May 2017 23:17:54 GMT

Okay, that makes sense.

And I just checked and on my mac I was actually doing the command:

hadoop com/thomp/io/FileSystemCat /HadoopPrac/Data/sample.txt

which also worked.

Thanks again!