Datanode is not staying up on any node of the cluster. I have a seven node cluster with 4 datanodes. What's going on ? Below is what I see when I perform a HDFS check Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/service_check.py", line 146, in <module> HdfsServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/service_check.py", line 67, in service_check action="create_on_execute" File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 402, in action_create_on_execute self.action_delayed("create") File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 399, in action_delayed self.get_hdfs_resource_executor().action_delayed(action_name, self) File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 255, in action_delayed self._create_resource() File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 269, in _create_resource self._create_file(self.main_resource.resource.target, source=self.main_resource.resource.source, mode=self.mode) File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 322, in _create_file self.util.run_command(target, 'CREATE', method='PUT', overwrite=True, assertable_result=False, file_to_put=source, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 210, in run_command raise Fail(err_msg) resource_management.core.exceptions.Fail: Execution of 'curl -sS -L -w '%{http_code}' -X PUT -T /etc/passwd 'http://hdp-m.asotc:50070/webhdfs/v1/tmp/id000a1902_date422016?op=CREATE&user.name=hdfs&overwrite=True'' returned status_code=403. { "RemoteException": { "exception": "IOException", "javaClassName": "java.io.IOException", "message": "Failed to find datanode, suggest to check cluster health." } } ... View more

Trying to explore the best way to manage users in hadoop ecosystem. Basically I am going to provide 3 user interface to user community: a.) EdgeNode - Linux machine. This is where users can use their Linux credentials and use command line to use hadoop clients (spark, sqoop, hdfs etc) b.) Ambari web interface c.) HUE interface d.) Ranger - For Admins to control file and folder permissions Question I have is is it possible to create account in Linux environment and let all other pull it from there and use the same credentials. I read about LDAP but it appears to be difficult and we don't currently have a working LDAP. How can I centrally manage users without using LDAP ? Thanks Prakash ... View more

Successfully installed HUE and modified all the parameter in HUE.INI such as webhdfs_url and set the parameter in HDFS configuration (proxyuser) and enabled webhdfs, But HUE is not able to connect to HDFS file system. It's trying at the URL: http://localhost:50070/webhdfs/v1 BUT correct hostname is listed in hue.ini. it doesnt seem to read the configuration from hue.ini. INI. below is what I am getting from HUE web interface. Potential misconfiguration detected. Fix and restart Hue. hadoop.hdfs_clusters.default.webhdfs_url Current value: http://localhost:50070/webhdfs/v1/ Failed to access filesystem root Resource Manager Failed to contact Resource Manager at http://localhost:8088/ws/v1: HTTPConnectionPool(host='localhost', port=8088): Max retries exceeded with url: /ws/v1/cluster/apps (Caused by : [Errno 111] Connection refused) Beeswax (Hive UI) The application won't work without a running HiveServer2. hcatalog.templeton_url Current value: http://localhost:50111/templeton/v1/ HTTPConnectionPool(host='localhost', port=50111): Max retries exceeded with url: /templeton/v1/status?user.name=hue&doAs=hue (Caused by : [Errno 111] Connection refused) Oozie Editor/Dashboard The app won't work without a running Oozie server ... View more