Member since
02-21-2019
69
Posts
45
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1487 | 06-06-2018 02:51 PM | |
4427 | 10-12-2017 02:48 PM | |
1382 | 08-01-2017 08:58 PM | |
30318 | 06-12-2017 02:36 PM | |
4741 | 02-16-2017 04:58 PM |
06-12-2017
02:36 PM
@Robin Dong In Linux only iptables controls the Kernel based firewall. You might have firewalld in CentOS7 or ufw in Ubuntu but they're just an abstraction layer on top of iptables. So if 'iptables -L' doesn't show anything then it's all good. The Ambari iptables check is rudimentary and it doesn't know if the rules that exist still allow all the traffic. It only checks for 'service iptables status' or 'systemctl status firewalld', which means there are no filter tables. But please be aware of the cloud firewall as well. For example in AWS even instances in the same Security Group are not allowed by default to communicate with each another and this must be enabled explicitly: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-rules-reference.html#sg-rules-other-instances
... View more
06-12-2017
02:04 PM
@Robin Dong Sounds like you're using a systemd based OS so I'm assuming that's CentOS/RHEL 7. In that case the firewall service is being handled by firewalld rather than iptables: systemctl stop firewalld systemctl disable firewalld Depending on the AMI used, this might be already disabled or not installed. And as long as iptables -L doesn't show anything you should also be fine.
... View more
06-12-2017
01:57 PM
@Justin R. This sounds odd. In Linux, non-root users cannot listen on ports bellow 1024, so are you running NiFi as root? That wouldn't be advisable. What is the complete stacktrace of the error? Maybe there is another nifi processor listening?
... View more
02-17-2017
10:28 AM
Could be network although I would have expected a different error. Maybe there are other errors in the logs pointing to a connectivity issue? I still don't see any dispatch in your log, that's the request Knox would make to the NameNode, do you see it in your audit log like in my example? The second check, the curl -i http://our_namenode_host:50070/static/bootstrap-3.0.2/js/bootstrap.min.js, did you do it from the knox host? I've seen weird behaviours in the past when HTTP proxies are configured, if you have one on the knox box, can you make sure the namenode host is on the NO_PROXY / ignore list? With a proxy, your shell session / user / curl might be allowed to correctly make the request but knox would go via the proxy?
... View more
02-16-2017
05:38 PM
It's really strange @Andreas Schild I've replicated your configuration on my cluster and still works fine, IE or Firefox. Let's try the following troubleshooting steps. 1) First make sure the topology file has actually been resolved with the correct values: cat /etc/knox/conf/topologies/default.xml 2) Get from it the URL for HDFSUI, append /static/bootstrap-3.0.2/js/bootstrap.min.js to it and try it out in a curl, should be like this: curl -i http://namenode:50070/static/bootstrap-3.0.2/js/bootstrap.min.js 3) I'd also want to see the full line the gateway-audit.log, mine shows additional information (like the service name) and also the dispatch: 17/02/16 17:15:53 ||c5f07e39-ab12-4a71-9283-f37a35128419|audit|HDFSUI|guest|||dispatch|uri|http://namenode:50070/static/bootstrap-3.0.2/js/bootstrap.min.js?doAs=guest|success|Response status: 200
17/02/16 17:15:53 ||c5f07e39-ab12-4a71-9283-f37a35128419|audit|HDFSUI|guest|||access|uri|/gateway/default/hdfs/static/bootstrap-3.0.2/js/bootstrap.min.js|success|Response status: 200 4) Check under /usr/hdp/current/knox-server/data/services/ if you have all services, I have: ls -l /usr/hdp/current/knox-server/data/services/
total 0
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 ambari
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 ambariui
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 falcon
drwxr-xr-x. 3 knox knox 19 Feb 7 09:41 hbase
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 hbaseui
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 hdfsui
drwxr-xr-x. 3 knox knox 19 Feb 7 09:41 hive
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 jobhistoryui
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 oozie
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 oozieui
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 ranger
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 rangerui
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 sparkhistoryui
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 storm
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 storm-logviewer
drwxr-xr-x. 3 knox knox 19 Feb 7 09:41 webhcat
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 webhdfs
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 yarn-rm
drwxr-xr-x. 3 knox knox 18 Feb 7 09:41 yarnui
5) Stop Knox and delete/move /usr/hdp/2.5.3.0-37/knox/bin/../data/deployments/default*
... View more
02-16-2017
04:58 PM
1 Kudo
@Bhavin Tandel I had a similar issue recently and while the Ambari YARN Queue Manager View doesn't offer a configurable URL, a workaround I found was to add the cluster as a Remote Cluster (using the alias URL when defining the Ambari Cluster URL) and then point the View to the remote cluster. This seemed to work just fine.
... View more
02-16-2017
10:10 AM
What Knox / HDP version do you have? It works fine on my HDP 2.5.3 Knox. Have you also defined WEBHDFS and RESOURCEMANAGER in your topology? Any errors in gateway.log and do you see the same 404s in gateway-audit.log?
... View more
02-16-2017
09:13 AM
You can try and add a / in your browser when you make the request. Otherwise, look at the github code and replace the xmls for the services you have issues with the ones on github, for example replace /usr/hdp/current/knox-server/data/services/hdfsui/2.7.0/ with https://github.com/apache/knox/tree/v0.11.0/gateway-service-definitions/src/main/resources/services/hdfsui/2.7.0
... View more
12-21-2016
09:36 AM
That's great to hear @Narasimma varman ! Can you accept the answer please so we know this issue / thread is closed?
... View more
12-20-2016
09:45 PM
3 Kudos
Hi @Narasimma varman After reading your message again it looks like you're trying to follow https://community.hortonworks.com/articles/7341/nifi-user-authentication-with-ldap.html which at a close look is using the Demo LDAP as part of Knox. The Knox Demo LDAP listens on port 33389 however it's not started automatically when you start Knox. Please make sure you go to Knox in Ambari and select Start Demo LDAP from the Service Actions as per the screenshot from the link above: https://community.hortonworks.com/storage/attachments/956-1.jpg You can verify if the Demo LDAP has started and listening on port 33389 by running: netstat -tnlp|grep 33389 If you see a process listening then you can configure ambari-server setup-ldap with the following options (use admin-password when asked for the Manager password): # ambari-server setup-ldap
Using python /usr/bin/python
Setting up LDAP properties...
Primary URL* {host:port} (localhost:33389): localhost:33389
Secondary URL {host:port} :
Use SSL* [true/false] (false):
User object class* (person): person
User name attribute* (uid): uid
Group object class* (groupofnames): groupofnames
Group name attribute* (cn): cn
Group member attribute* (member): member
Distinguished name attribute* (dn): dn
Base DN* (dc=hadoop,dc=apache,dc=org): dc=hadoop,dc=apache,dc=org
Referral method [follow/ignore] (follow):
Bind anonymously* [true/false] (false): false
Manager DN* (uid=admin,ou=people,dc=hadoop,dc=apache,dc=org): uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
Enter Manager Password* :
Re-enter password:
====================
Review Settings
====================
authentication.ldap.managerDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
authentication.ldap.managerPassword: *****
Save settings [y/n] (y)? y
Saving...done
Ambari Server 'setup-ldap' completed successfully. You might also need to turn off pagination as the Knox LDAP doesn't support it: echo "authentication.ldap.pagination.enabled=false" >> /etc/ambari-server/conf/ambari.properties Now, don't forget to restart Ambari Server and be careful that after running ambari-server sync-ldap --all, the admin user password will change to admin-password Other users can be found by running this: cat /etc/knox/conf/users.ldif|egrep "^uid|^userPassword" And you can add new users by changing Advanced users-ldif under the Knox Config in Ambari. Good luck!
... View more