Hi Team, Ranger usersync daemon is not starting in our cluster.We are using HDP-2.5.6. We are getting following error in ranger usersync log file: 19 May 2018 00:01:25 ERROR UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 60000 milliseconds. Error details: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.delXUserGroupInfo(PolicyMgrUserGroupBuilder.java:615) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.delXUserGroupInfo(PolicyMgrUserGroupBuilder.java:600) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addOrUpdateUser(PolicyMgrUserGroupBuilder.java:326) at org.apache.ranger.unixusersync.process.UnixUserGroupBuilder.updateSink(UnixUserGroupBuilder.java:92) at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:58) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) After debugging we found that its an known issue in Ranger: https://issues.apache.org/jira/browse/RANGER-989 As per mentioned in "RANGER-989" we have copied given jar files from ranger-admin lib to usersync lib and restarted Ranger service: cp /usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/lib/commons-httpclient-3.1.jar /usr/hdp/current/ranger-usersync/lib/ cp /usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/lib/commons-codec-1.9.jar /usr/hdp/current/ranger-usersync/lib/ Still we are getting same error. How to resolve it? Please suggest. Thanks, Bhushan ... View more

Hi Team, We have integrated NIFI with Ranger. Also cluster is kerberized. NIFI is SSL enabled but Ranger is not SSL enabled. We are getting following error in nifi-app.log: 2018-06-25 06:44:48,636 INFO [main] o.a.n.r.a.RangerBasePluginWithPolicies Converting Ranger ServicePolicies model into NiFi policy model for viewing purposes in NiFi UI. 2018-06-25 06:44:48,641 WARN [main] o.a.n.r.a.RangerBasePluginWithPolicies Resources [*] include a wildcard value. Skipping policy for viewing purposes. Will still be used for access decisions. 2018-06-25 06:44:48,652 INFO [main] o.a.r.plugin.service.RangerBasePlugin Policies will NOT be reordered based on number of evaluations 2018-06-25 06:44:48,710 ERROR [Thread-15] o.a.ranger.plugin.util.PolicyRefresher PolicyRefresher(serviceName=MMCHDPDEV_nifi): failed to refresh policies. Will continue to use last known version of policies (69) com.sun.jersey.api.client.ClientHandlerException: java.lang.RuntimeException: java.lang.NullPointerException at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) at com.sun.jersey.api.client.Client.handle(Client.java:652) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) at org.apache.ranger.admin.client.RangerAdminRESTClient$3.run(RangerAdminRESTClient.java:122) at org.apache.ranger.admin.client.RangerAdminRESTClient$3.run(RangerAdminRESTClient.java:115) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1678) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:125) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:171) Caused by: java.lang.RuntimeException: java.lang.NullPointerException at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1506) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3036) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:489) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ... 13 common frames omitted Caused by: java.lang.NullPointerException: null at java.util.Base64$Encoder.encode(Base64.java:261) at java.util.Base64$Encoder.encodeToString(Base64.java:315) at sun.net.www.protocol.http.NegotiateAuthentication.setHeaders(NegotiateAuthentication.java:208) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1749) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ... 15 common frames omitted 2018-06-25 06:44:49,821 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded 104 properties from system properties and environment variables 2018-06-25 06:44:49,821 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded a total of 104 properties. Including precedence overrides effective accessible registry key size is 104 Ranger Policies are not getting applied for NIFI. How to resolve it. Please suggest. Thanks, Bhushan ... View more

Hi Team, Our cluster is secured by Kerberos and Ranger. In PublishKafka processor properties we are using security protocol as SASL_PLAINTEXT. We are getting following error in nifi-app.log when we run PublishKafka processor: Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. Make sure -Djava.security.auth.login.config property passed to JVM and the client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)'. Make sure you are using FQDN of the Kafka broker you are trying to connect to. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:74) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:60) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:79) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:271) ... 16 common frames omitted Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. Make sure -Djava.security.auth.login.config property passed to JVM and the client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)'. Make sure you are using FQDN of the Kafka broker you are trying to connect to. not available to garner authentication information from the user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) at sun.reflect.GeneratedMethodAccessor552.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.kafka.common.security.kerberos.Login.login(Login.java:298) at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:104) at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44) at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:55) Given are the content of jass config: Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/nifi.service.keytab" useTicketCache=true principal="nifi/usdf.test.com@CORP.TEST.INT"; }; KafkaClient { com.sun.security.auth.module.Krb5LoginModule required renewTicket=true serviceName="kafka" useKeyTab=true useTicketCache=true keyTab="/etc/security/keytabs/nifi.service.keytab" principal="nifi/usdf.test.com@CORP.TEST.INT"; }; How to resolve it? Please suggest. Thanks in advance. ... View more