how to check the dfsadmin report in wasb space

[#1]

I've a hortonworks cluster @azure cloud and I need to check the dfsadmin report in blob space.

when I'm trying with hdfs dfsadmin -report this command is giving the message "blob.core.windows.net is not an HDFS file system"

How do i get the dfsadmin report in wasb

[#2]

@Praveen Atmakuri

This is the format of the hdfsadmin command in Azure, where mycluster is your cluster name

hdfs dfsadmin -D "fs.default.name=hdfs://mycluster/" -report

Please revert

[#3]

@Geoffrey Shelton Okot....Getting the same error.

"blob.core.windows.net is not an HDFS file system"

[#4]

@Praveen Atmakuri

Can you give the output of this command

hadoop fs -ls wasb://[email protected]/

The following is a list of configurations that should be modified to configure WASB: I assume you have done the below

fs.defaultFS

wasb://<containername>@<accountname>.blob.core.windows.net 

fs.AbstractFileSystem.wasb.impl

org.apache.hadoop.fs.azure.Wasb 

fs.azure.account.key. blob.core.windows.net

storage_access_key 

Please revert

[#5]

fidn the output of the command: hadoop fs -ls wasb://[email protected]/
18/04/02 08:31:26 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
18/04/02 08:31:26 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
18/04/02 08:31:26 INFO impl.MetricsSystemImpl: azure-file-system metrics system started
Found 3 items
drwxr-xr-x - paplcloudadmin supergroup 0 2017-12-29 09:53 wasb://[email protected]/user/paplcloudadmin/.hiveJars
drwx------ - paplcloudadmin supergroup 0 2018-01-11 13:18 wasb://[email protected]/user/paplcloudadmin/.staging
drwxr-xr-x - paplcloudadmin supergroup 0 2018-01-11 13:28 wasb://[email protected]/user/paplcloudadmin/prod.db
18/04/02 08:31:26 INFO impl.MetricsSystemImpl: Stopping azure-file-system metrics system...
18/04/02 08:31:26 INFO impl.MetricsSystemImpl: azure-file-system metrics system stopped.
18/04/02 08:31:26 INFO impl.MetricsSystemImpl: azure-file-system metrics system shutdown complete.
The following is a list of configurations that should be modified to configure WASB: I assume you have done the below Correct, All configurations has been done properly and verified again...Result all are same.

[#6]

@Praveen Atmakuri

So I can see you have only the user paplcloudamin home directory set

Can you run the following

$ hdfs dfs -D "fs.default.name=hdfs://mycluster/" -mkdir /test

Check if the directory was created, you should be able to see test directory

$ hdfs dfs -D "fs.default.name=hdfs://mycluster/" -ls /

This command should run successfully,can you just copy and paste this command replacing my cluster with your clustername

$ hdfs dfsadmin -D "fs.default.name=hdfs://mycluster/" -report

Please revert

[#7]

$ hdfs dfs -D "fs.default.name=wasb://containername@storage_name.blob.core.windows.net//"-mkdir /test

Done

$ hdfs dfs -D "fs.default.name=wasb://containername@storage_name.blob.core.windows.net/"-ls /

Result:
drwxr-xr-x - paplcloudadmin supergroup 0 2018-04-02 09:33 /test

hdfs dfsadmin -D "fs.default.name=wasb://containername@storage_name.blob.core.windows.net/" -report

18/04/02 09:40:56 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
18/04/02 09:40:56 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
18/04/02 09:40:56 INFO impl.MetricsSystemImpl: azure-file-system metrics system started
report: FileSystem wasb://containername@storage_name.blob.core.windows.net

is not an HDFS file system
Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning]
18/04/02 09:40:56 INFO impl.MetricsSystemImpl: Stopping azure-file-system metrics system...
18/04/02 09:40:56 INFO impl.MetricsSystemImpl: azure-file-system metrics system stopped.
18/04/02 09:40:56 INFO impl.MetricsSystemImpl: azure-file-system metrics system shutdown complete.

[#8 · Accepted solution]

@Praveen Atmakuri

The dfsadmin is run like below

Get a report of status from each name node to show that the name nodes are in Safe Mode or not.

$ hdfs dfsadmin -D 'fs.default.name=hdfs://mycluster/' -safemode get 

A report that shows the details of HDFS state:

$ hdfs dfsadmin -D 'fs.default.name=hdfs://mycluster/' -report 

Get HDFS out of safe mode

$ hdfs dfsadmin -D 'fs.default.name=hdfs://mycluster/' -safemode leave

What's is your HDInsight cluster name

[#9]

I'm getting the same error. This is the response I received form Cloudera support "Only the dfs commands such as ls/put/mv etc works on wasb using the wasb connector. Admin commands such as dfsadmin as well fsck works only with native hadoop/hdfs implementation"