Member since
06-27-2020
10
Posts
0
Kudos Received
0
Solutions
07-15-2020
10:24 AM
@Shelton I can see Ambari created Headless Keytab but didn't see how this is being used or configured. Any insight on how Headless Keytab configured? Thanks!
... View more
07-13-2020
02:30 PM
The above solution is working when I do kinit of kafka service keytab. My Kafka jaas file is as below. kafka_jaas.conf KafkaServer { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/home/ec2-user/kafka.service.keytab" principal="kafka/<public DNS>@EXAMPLE.COM"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/home/ec2-user/kafka.service.keytab" principal="kafka/<public DNS>@EXAMPLE.COM"; }; KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/home/ec2-user/kafka.service.keytab" principal="kafka/<public DNS>@EXAMPLE.COM"; }; When I destroyed the ticket, I am getting below error. WARN Error getting policies. secureMode=true, user=root (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=KafkaTest (org.apache.ranger.admin.client.RangerAdminRESTClient) After looking into error, I found the user is coming as root (highlighted above). I don't want to run kinit command explicitly and that's why I provided KafkaClient in my jaas file above. I did export KAFKA_OPTS and pass this jass file. Can you please help me with what I have done wrong here. Thanks for all your help!
... View more
07-13-2020
02:17 PM
I created the wrong principal for Spnego keytab. Principal needs to HTTP/test-dummy-X.openstacklocal@EXAMPLE.COM, not http/test-dummy-X.openstacklocal@EXAMPLE.COM. After making this change.
... View more
07-12-2020
10:14 AM
@Shelton I looked at that blog and then configured Spnego keytab but still, it is not working. Do I need Spnego keytab in the Kafka broker as well? If yes, how to configure (any JAAS conf, any properties file, etc) it? @VR46 - Any help would be appreciated. Thanks!
... View more
07-10-2020
02:22 PM
We are trying to integrate Kafka and Ranger through kerberized env and receiving 401 error while Kafka trying to download policy from Ranger. Error which we received at Kafka is as below. Kafka Logs while starting broker Error getting policies. secureMode=true, user=kafka/<public DNS>@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=KafkaTest (org.apache.ranger.admin.client.RangerAdminRESTClient) Configuration on Kafka server COMPONENT_INSTALL_DIR_NAME=/home/ec2-user/kafka POLICY_MGR_URL=https://<public DNS of Ranger>:6182
REPOSITORY_NAME=KafkaTest Configuration at Ranger side as below: Core-site.xml <configuration> <property> <name>hadoop.security.authentication</name> <value>kerberos</value> <description>Set the authentication for the cluster. Valid values are: simple or kerberos.</description> </property> </configuration> Install.properties spnego_principal=http/<public DNS>@EXAMPLE.COM #spnego_principal=* spnego_keytab=/home/ec2-user/spnego.service.keytab token_valid=30 cookie_domain=<public DNS> cookie_path=/ admin_principal=rangeradmin/<public DNS>@EXAMPLE.COM admin_keytab=/home/ec2-user/rangeradmin.service.keytab lookup_principal=rangerlookup/<public DNS>@EXAMPLE.COM lookup_keytab=/home/ec2-user/rangerlookup.service.keytab hadoop_conf=/etc/hadoop/conf On Ranger-admin UI, we configured below properties policy.download.auth.users=kafka we also tried giving user as below policy.download.auth.users=kafka/<public dns> (basically principal of Kafka broker) Ranger-admin logs "GET /service/plugins/secure/policies/download/KafkaTest?lastKnownVersion=51&lastActivationTime=1594414061001&pluginId=kafka@<internal-IP of Kafka>-KafkaTest&clusterName= HTTP/1.1" 401 - "-" "Java/1.8.0_242" Please let us know what we have done wrong here. Thanks for all your help!
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Ranger
07-01-2020
01:41 PM
@ajinkyapatil We have two observations - First observation Even if we give a file-path which doesn't exist for Keystore & Trustsrore, Kafka Ranger (enable-kafka-plugin.sh)script is not complaining and going ahead with the generation of cred file. But eventually, Kafka is unable to connect to Ranger with the same error defined in the issue.
Second observation We are not sure but even after providing the right path for Keystore & Trustsrore, Kafka-Ranger (enable-kafka-plugin.sh) script seems to not care about the location of Keystore & Truststore and generate a cred file. Attaching few screen prints
... View more
06-29-2020
09:52 AM
@ajinkyapatil Please see the properties which you asked for. I am still getting the same error message. Please let me know if anything is missing here. <property> <name>xasecure.policymgr.clientssl.keystore</name> <value>/etc/hadoop/conf/kafka.admin.keystore.jks</value> <description> Java Keystore files </description> </property> <property> <name>xasecure.policymgr.clientssl.keystore.password</name> <value>password</value> </property> <property> <name>xasecure.policymgr.clientssl.truststore</name> <value>/etc/hadoop/conf/kafka.admin.truststore.jks</value> <description> java truststore file </description> </property> <property> <name>xasecure.policymgr.clientssl.truststore.password</name> <value>password</value> </property> <property> <name>xasecure.policymgr.clientssl.keystore.credential.file</name> <value>jceks://file/etc/ranger/KafkaTest/cred.jceks</value> <description> java keystore credential file </description> </property> <property> <name>xasecure.policymgr.clientssl.truststore.credential.file</name> <value>jceks://file/etc/ranger/KafkaTest/cred.jceks</value> <description> java truststore credential file </description> </property> </configuration>
... View more
06-27-2020
12:25 PM
We are trying to enable SSL connection between Apache Ranger and Kafka cluster. After creating keystore and truststore for both Kafka and Ranger, we are unable to connect Kafka to Ranger and we are getting the following error message: [2020-06-25 20:47:40,013] ERROR Unable to get the Credential Provider from the Configuration (org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider)
java.lang.IllegalArgumentException: The value of property hadoop.security.credential.provider.path must not be null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1134)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1115)
at org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialProviders(RangerCredentialProvider.java:68)
at org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialString(RangerCredentialProvider.java:46)
at org.apache.ranger.plugin.util.RangerRESTClient.getCredential(RangerRESTClient.java:386)
at org.apache.ranger.plugin.util.RangerRESTClient.getKeyManagers(RangerRESTClient.java:272)
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:188)
at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:275)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:171)
[2020-06-25 20:47:40,013] ERROR PolicyRefresher(serviceName=KafkaTest): failed to refresh policies. Will continue to use last known version of policies (51) (org.apache.ranger.plugin.util.PolicyRefresher)
java.lang.IllegalArgumentException: TrustManager is not specified
at org.apache.commons.lang.Validate.notNull(Validate.java:192)
at org.apache.ranger.plugin.util.RangerRESTClient.getSSLContext(RangerRESTClient.java:369)
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:190)
at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:275)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:171)
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Ranger