Member since
12-26-2016
15
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2871 | 08-19-2018 03:43 AM |
08-19-2018
03:43 AM
One is large environment with 20+ pb in size and data is completely different from other environment data, and reasons for different lakes are they both fall in different internal departments and data is also different and customers are also different, again depends on the data these cluster(s) servers located in different data centers and one is open for company wide enterprise network and others open for an internal network within enterprise network.
... View more
05-23-2018
10:51 AM
Hi , I am also facing 403 Forbidden issue while reading a file from WEBHDFS (OPEN operation), where i have enabled KNOX and kerberos. But on the sam efile LISTSTATUS opertaion is working fine. Here is the error: https://host:8443/gateway/default/webhdfs/v1/user/admin/hive/querystatus/stdout?op=OPEN <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 403 Forbidden</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /gateway/default/webhdfs/data/v1/webhdfs/v1/user/knox/test/customers.csv. Reason:
<pre> Forbidden</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>
</body>
</html> Please help me with this. Thanks.
... View more
04-25-2017
09:22 PM
Hi @hdpadmin overlandpark Thanks for working with me over the Support portal. I will put the steps here for others reference: The Topology information is CACHED in NN and so, when ever there is change to Topology we need to restart the NN to clear it.
In our case Datanode is originally registered without rack (default-rack) and due to caching, NN will not allow it to join when its online.
So to fix this issue, we have to restart namenode, which will pick up the current updated rack information.
To avoid a NN restart, there is a workaround to add a new DataNode to a specific Rack without the need of restarting the NameNode: a. Add the New Node in Ambari without choosing 'DataNode' component
b. From Ambari Hosts tab, Select the new Node and add 'DataNode' component
c. Click 'Host Actions' -> 'Set Rack' and specify the required Rack Name
d. From the Ambari Hosts tab, go to the NameNode Host and Refresh the client configs by dropping down the Menu next to Clients.
This step will update /etc/hadoop/conf/topology_mappings.data on the NameNode with the new topology information for the DN.
e. Start the DataNode service on the new Node.
f. Once the Datanode is up, confirm its Rack topology information, by running the following command on any node with hdfs client:
# su - hdfs
# hdfs dfsadmin -report
In short, the Topology should be updated before a Datanode is started (at which point it tries to register with the NN)
... View more
03-12-2017
08:54 PM
Hi Jay, Thank you for the response. Actually we got all these things in place, but i realized that our trust store password was incorrect and i was able to fix that issue, later it complained about a self signed cert on the ranger admin server, so I imported hive cert into ranger trust store and did set common name correctly on hive/ranger configuration and finally my issue was resolved. Thanks again for the reply. Subrah.
... View more