Can we use webhdfs:// as a KNOX scheme instead of http? If yes, how?

[#1]

I am looking specifically, if Knox can support other URI schemes like webhdfs://

If yes then how can I access hdfs using this scheme?

[#2]

@Neha G

Knox works on HTTP (HTTPS) protocol. You can use webhdfs rest calls via knox Via Curl as described in the following link but you can not replace HTTPS:// with webhdfs://

https://cwiki.apache.org/confluence/display/KNOX/Examples+WebHDFS#ExamplesWebHDFS-WebHDFSviacURL

Example:

https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/webhdfs

Knox Gateway is an Application Gateway for interacting with the REST APIs and UIs of Apache Hadoop deployments. The Knox Gateway provides a single access point for all REST and HTTP interactions with Apache Hadoop clusters.

.

[#3]

@Neha G

More detailed explaination and webhdfs access via Knox is explained here: https://knox.apache.org/books/knox-0-7-0/user-guide.html#WebHDFS

Example:

curl -i -k -u guest:guest-password -X PUT 'https://localhost:8443/gateway/sandbox/webhdfs/v1/user/guest/example/README?op=CREATE'

.

[#4]

@Jay SenSharma

Thanks Jay!

But I want to understand a scenario where I have to write a java application to access hdfs (create, put, get..in hdfs) and I have to use filesystem URI to access hdfs as webhdfs://<HOST>:<HTTP_PORT>/<PATH> , then would I be able to access hdfs via Knox gateway with this URI?

[#5]

Hello @Neha G, you cannot access "webhdfs://" protocol using Knox, Knox only supports Http, Https and WS protocols.