Support Questions
Find answers, ask questions, and share your expertise

Cannot connect to WebHDFS remotely to an AWS EC2 HDP cluster

New Contributor

Hi,

I'm trying to send a file remotely through WebHDFS to AWS EC2 HDP cluster through the public DNS (public_hostname, i.e: ec2-xx-xx-xxx-xx.eu-central-1.compute.amazonaws.com)

Problem is that the "Location" I get sent in the response header points to the internal DNS of the machine (i.e:

ip-xxx-xx-xx-xxx.eu-central-1.compute.internal

which is obviously not reachable from a remote location (that is not inside Amazon VPC).

I changed my hostname to point to the public DNS, but I suspect that somehow WebHDFS is trying to resolve the DNS of the machine from the internal (private IP adress) of the machine, and therefore logically points to the private DNS.

Is there any way to resolve it somehow to PUBLIC_DNS or public_ip through any of the parameters. Thanks a lot for your help.

1 REPLY 1

Super Guru

@Joachim Hagege Sorry i don't follow your question. Is your webhdfs url a public or private url? can it be access via security configurations you have provide. If it points to a internal DNS you will need to use the public DNS or setup another DNS using route53 to point to the webhdfs url. You can always use a ELB infront of it which will give you the public DNS as well. I hope that helps since I was not able to follow your question completly.