Member since
05-19-2018
2
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2266 | 05-20-2018 06:35 AM |
05-20-2018
06:35 AM
@Shu I just figured out it's an issue with dual NIC at AWS instance. I'm trying to connect using public IP, whereas cluster was setup using hostname that resolved to internal IP of AWS instance. So what happens is that when Nifi instance on my Mac tries to write to HDFS using public IP, namenode creates the blocks and sends back internal IP of datanode to write the actual data. The datanode becomes unreachable and write operation fails. I tried running standalone Nifi on same node as AWS HDP instance (though it's not a good practice), and it worked like a charm! In order to make it work outside the cluster, I will probably need to change DNS entries to map the AWS instance hostname to public IP and restart the cluster. I guess this will be a very common issue for users running Nifi outside the cluster and trying to access HDFS on HDP clusters setup on AWS instances with dual NIC. Thank you for your support anyway!
... View more