Created 06-22-2016 08:57 PM
We have HDInsight cluster setup in Azure.
When I do hadoop fs -ls / it shows me
drwxr-xr-x - root supergroup 0 2016-06-17 20:56 /HdiNotebooks drwxr-xr-x - root supergroup 0 2016-06-17 21:00 /HdiSamples drwxr-xr-x - hdfs supergroup 0 2016-06-17 20:48 /ams drwxr-xr-x - hdfs supergroup 0 2016-06-17 20:48 /amshbase drwxrwxrwx - yarn hadoop 0 2016-06-17 20:48 /app-logs drwxr-xr-x - yarn hadoop 0 2016-06-17 20:48 /atshistory drwxr-xr-x - sshuser supergroup 0 2016-06-21 18:38 /data drwxr-xr-x - root supergroup 0 2016-06-17 20:59 /example drwxr-xr-x - hdfs supergroup 0 2016-06-17 20:48 /hdp drwxr-xr-x - hdfs supergroup 0 2016-06-17 20:48 /hive drwxr-xr-x - mapred supergroup 0 2016-06-17 20:48 /mapred drwx------ - sshuser supergroup 0 2016-06-20 14:22 /mapreducestaging drwxrwxrwx - mapred hadoop 0 2016-06-17 20:48 /mr-history drwxr-xr-x - sshuser supergroup 0 2016-06-20 19:20 /sqoop drwxrwxrwx - hdfs supergroup 0 2016-06-17 20:48 /tmp drwxr-xr-x - hdfs supergroup 0 2016-06-17 20:48 /user
But hadoop fs -ls hdfs://mycluster/
shows following result.
root@hn0-haspar:~# hadoop fs -ls hdfs://mycluster/ Found 3 items drwxr-xr-x - root hdfs 0 2016-06-21 18:48 hdfs://mycluster/data drwx-wx-wx - root hdfs 0 2016-06-17 20:57 hdfs://mycluster/tmp drwx------ - root hdfs 0 2016-06-22 17:24 hdfs://mycluster/user
Dont know where this different dir coming.
Cluster has HA configuration.
Created 06-22-2016 09:16 PM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 06-22-2016 09:16 PM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 06-23-2016 12:33 PM
Thanks @Chris Nauroth for explanation. At present We have Namenode HA and we are putting data from Flume into this cluster. We are configuring hdfs://mycluster/flume as destination in Flume sink.
Whats is the correct way to put data into default HDFS storage (WASB) from Flume and make it accessible from hadoop fs -ls / ?
Appreciate help in this.
Created 06-23-2016 04:12 PM
Created 06-23-2016 04:49 PM
Thanks, it worked.