Is it possible to configure for hdfs to use proxy server configurations? We have a proxy server and all outgoing traffic needs to go to that proxy server, we want to access azure blob storage with hdfs commands, but it does not seem to work at the moment because hdfs does not use proxy server, but tries to make direct http / https call to the azure blob storage which is not allowed.
Does anybody have idea that which hadoop properties we can configure to make it happen?
Thanks in advance.
Have you seen the HDP Reference Guide at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_HDP_Reference_Guide/bk_HDP_Reference_Guid...? There are instructions for setting up a trusted proxy server. Maybe that will help.
@Beverley Andalora Thanks for sending me this, as I said in my question, we already have a proxy server setup and it works fine locally on linux, so for example we are able to donwload all hortonworks packages. The problem is while using hdfs commands, for example if we have a storage account on azure blob (its a http connection) and we want to access it via hdfs dfs -ls wasb://<storage account>/ , but hdfs tries to go to the URL directly rather than going to the proxy server first and hence no connection. So can we setup something inside hadoop to make it understand to use proxy server. Thanks for your help.