My default.xml file: root@ambari-mgr0:/home/knox/knox/conf/topologies# cat default.xml <?xml version="1.0" encoding="utf-8"?> <topology> <gateway> <provider> <role>authentication</role> <name>ShiroProvider</name> <enabled>true</enabled> <param> <!-- session timeout in minutes, this is really idle timeout, defaults to 30mins, if the property value is not defined,, current client authentication would expire if client idles contiuosly for more than this value --> <name>sessionTimeout</name> <value>30</value> </param> <param> <name>main.ldapRealm</name> <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value> </param> <param> <name>main.ldapContextFactory</name> <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value> </param> <param> <name>main.ldapRealm.contextFactory</name> <value>$ldapContextFactory</value> </param> <param> <name>main.ldapRealm.userDnTemplate</name> <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value> </param> <param> <name>main.ldapRealm.contextFactory.url</name> <value>ldap://ambari-mgr0:33389</value> </param> <param> <name>main.ldapRealm.contextFactory.authenticationMechanism</name> <value>simple</value> </param> <param> <name>urls./**</name> <value>authcBasic</value> </param> </provider> <provider> <role>identity-assertion</role> <name>Default</name> <enabled>true</enabled> </provider> <provider> <role>hostmap</role> <name>static</name> <enabled>true</enabled> <param> <name>ambari-mgr0</name> <value>sandbox,sandbox.hortonworks.com</value> </param> </provider> </gateway> <service> <role>NAMENODE</role> <url>http://ambari-mgr0:8020</url> </service> <service> <role>JOBTRACKER</role> <url>rpc://ambari-mgr0:8050</url> </service> <service> <role>WEBHDFS</role> <url>http://ambari-mgr0:50070/webhdfs</url> </service> <service> <role>WEBHCAT</role> <url>http://ambari-mgr0:50111/templeton</url> </service> <service> <role>WEBHBASE</role> <url>http://ambari-mgr0:60080</url> </service> <service> <role>HIVE</role> <url>http://ambari-mgr0:10001/cliservice</url> </service> <service> <role>AMBARIUI</role> <url>http://ambari-mgr0:8080</url> </service> <service> <role>YARNUI</role> <url>http://ambari-mgr0:8088</url> </service> <service> <role>HDFSUI</role> <url>http://ambari-mgr0:50070</url> </service> <service> <role>JOBHISTORYUI</role> <url>http://ambari-mgr0:19888</url> </service> <service> <role>HBASEUI</role> <url>http://ambari-mgr0:16010</url> </service> <service> <role>OOZIEUI</role> <url>http://ambari-mgr0:11000/oozie/</url> </service> <service> <role>OOZIE</role> <url>http://ambari-mgr0:11000/oozie</url> </service> <service> <role>RANGER</role> <url>http://ambari-mgr0:6080</url> </service> <service> <role>RANGERUI</role> <url>http://ambari-mgr0:6080</url> </service> <service> <role>SPARKHISTORYUI</role> <url>http://ambari-mgr0:18081</url> </service> <service> <role>AMBARI</role> <url>http://ambari-mgr0:8080</url> </service> <service> <role>YARN</role> <url>http://ambari-mgr0:8088</url> </service> <service> <role>RESOURCEMANAGER</role> <url>http://ambari-mgr0:8088</url> </service> <service> <role>DRUID-COORDINATOR-UI</role> <url>http://ambari-mgr0:8081</url> </service> <service> <role>DRUID-COORDINATOR</role> <url>http://ambari-mgr0:8081</url> </service> <service> <role>DRUID-BROKER</role> <url>http://ambari-mgr0:8082</url> </service> <service> <role>DRUID-ROUTER</role> <url>http://ambari-mgr0:8082</url> </service> <service> <role>DRUID-OVERLORD</role> <url>http://ambari-mgr0:8090</url> </service> <service> <role>DRUID-OVERLORD-UI</role> <url>http://ambari-mgr0:8090</url> </service> </topology> ... View more

@Geoffrey Shelton Okot I had followed the same document and the issue is I am able to access ranger UI with this url "https://<knox_server_ip>:8443/gateway/default/ranger/" which then prompts for username and password but I need this url to be used when i click on quicklinks in ambari UI for ranger. As i went through the files, got to know that quicklinks configuration is not updated in metainfo.xml file. May i know how this xml file is being generated and why it ginores quicklinks configuration for ranger alone. Please find the below files. /var/lib/ambari-server/resources/stacks/HDP/2.3/services/RANGER/quicklinks/quicklinks.json quicklinks.json: { "name": "default", "description": "default quick links configuration", "configuration": { "protocol": { "type":"https", "checks":[ { "property":"ranger.service.https.attrib.ssl.enabled", "desired":"true", "site":"ranger-admin-site" }, { "property":"ranger.service.http.enabled", "desired":"false", "site":"ranger-admin-site" } ] }, "links": [ { "name": "ranger_admin_ui", "label": "Ranger Admin UI", "requires_user_name": "false", "url": "https://{{knox_server_ip}}:8443/gateway/default/ranger", "attributes": ["authenticated", "sso"], "port":{ "http_property": "ranger.service.http.port", "http_default_port": "6080", "https_property": "ranger.service.https.port", "https_default_port": "6182", "regex": "(\\d*)+", "site": "ranger-admin-site" } } ] } } filepath: /var/lib/ambari-server/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml metainfo.xml: <metainfo> <schemaVersion>2.0</schemaVersion> <services> <service> <name>RANGER</name> <displayName>Ranger</displayName> <comment>Comprehensive security for Hadoop</comment> <extends>common-services/RANGER/0.5.0</extends> <version>0.5.0.2.3</version> </service> </services> </metainfo> ... View more

@Geoffrey Shelton Okot I had followed the same document and the issue is I am able to access ranger UI with this url "https://<knox_server_ip>:8443/gateway/default/ranger/" which then prompts for username and password but I need this url to be used when i click on quicklinks in ambari UI for ranger. My quicklinks configuration is not updated in metainfo.xml file. Please find the below files. /var/lib/ambari-server/resources/stacks/HDP/2.3/services/RANGER/quicklinks/quicklinks.json quicklinks.json: { "name": "default", "description": "default quick links configuration", "configuration": { "protocol": { "type":"https", "checks":[ { "property":"ranger.service.https.attrib.ssl.enabled", "desired":"true", "site":"ranger-admin-site" }, { "property":"ranger.service.http.enabled", "desired":"false", "site":"ranger-admin-site" } ] }, "links": [ { "name": "ranger_admin_ui", "label": "Ranger Admin UI", "requires_user_name": "false", "url": "https://{{knox_server_ip}}:8443/gateway/default/ranger", "attributes": ["authenticated", "sso"], "port":{ "http_property": "ranger.service.http.port", "http_default_port": "6080", "https_property": "ranger.service.https.port", "https_default_port": "6182", "regex": "(\\d*)+", "site": "ranger-admin-site" } } ] } } /var/lib/ambari-server/resources/stacks/HDP/2.3/services/RANGER metainfo.xml: <metainfo> <schemaVersion>2.0</schemaVersion> <services> <service> <name>RANGER</name> <displayName>Ranger</displayName> <comment>Comprehensive security for Hadoop</comment> <extends>common-services/RANGER/0.5.0</extends> <version>0.5.0.2.3</version> </service> </services> </metainfo> ... View more

@Geoffrey Shelton Okot I had followed the same document and the issue is I am able to access ranger UI with this url "https://<knox_server_ip>:8443/gateway/default/ranger/" which then prompts for username and password but I need this url to be used when i click on quicklinks in ambari UI for ranger. My quicklinks configuration is not updated in metainfo.xml file. Please find the below files. Could you please tell me how this metainfo.xml file is created and why it ignores quicklinks configuration for ranger alone. /var/lib/ambari-server/resources/stacks/HDP/2.3/services/RANGER/quicklinks/quicklinks.json quicklinks.json: { "name": "default", "description": "default quick links configuration", "configuration": { "protocol": { "type":"https", "checks":[ { "property":"ranger.service.https.attrib.ssl.enabled", "desired":"true", "site":"ranger-admin-site" }, { "property":"ranger.service.http.enabled", "desired":"false", "site":"ranger-admin-site" } ] }, "links": [ { "name": "ranger_admin_ui", "label": "Ranger Admin UI", "requires_user_name": "false", "url": "https://{{knox_server_ip}}:8443/gateway/default/ranger", "attributes": ["authenticated", "sso"], "port":{ "http_property": "ranger.service.http.port", "http_default_port": "6080", "https_property": "ranger.service.https.port", "https_default_port": "6182", "regex": "(\\d*)+", "site": "ranger-admin-site" } } ] } } /var/lib/ambari-server/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml metainfo.xml: <metainfo> <schemaVersion>2.0</schemaVersion> <services> <service> <name>RANGER</name> <displayName>Ranger</displayName> <comment>Comprehensive security for Hadoop</comment> <extends>common-services/RANGER/0.5.0</extends> <version>0.5.0.2.3</version> </service> </services> </metainfo> ... View more

@Geoffrey Shelton Okot Thanks a lot. It worked. When i created topics using kafka-topics.sh, it throwed LEADER_NOT_AVAILABLE on giving producer command. So i tried with auto-created topics and it worked without any error. ... View more

@Geoffrey Shelton Okot Yes i have enabled ranger-kafka plugin. Have tried with this document also (https://community.hortonworks.com/articles/17059/apache-ranger-and-kafka-1.html ). Created 3 policies for broker, producer and consumer as mentioned in that document, but still getting below error message on running the following producer command(./bin/kafka-console-producer.sh --broker-list hadoop-kafka0:6667 --topic test_2). hadoop-kafka0 is the hostname of my broker. Tried by giving ip instead of hostname. That also thrown the same error. "[2019-04-23 10:50:26,907] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 15 : {test_2=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)" ... View more

@Geoffrey Shelton Okot Its not a kerberized cluster. Also could you please tell me if there is any way to enable and use ranger with kafka in a non-kerberized cluster. Screenshots of ranger policy: ... View more