Support Questions

Find answers, ask questions, and share your expertise

Should Nifi be kerberized in order to access Kerberized Hive?

Contributor

When I am trying to connect to Secure Hive from an Unsecured Nifi, getting the below error -

Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:183)
        at org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:151)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        ... 45 common frames omitted

39548-capture.png

Should Nifi be kerberized in order to access Kerberized Hive? Is any additional required?

12 REPLIES 12

Expert Contributor

Hi @Hemant,

No Nifi doesn't need to be kerberized but you need to install the kerberos client on the os (where nifi is installed) in order to be able to request a ticket.

Michel

Contributor

Hello @msumbul Kerberos client is installed and I am able to access HDFS without any issues but when I am trying to connect to Hive I am facing the error - errorlog.txt

Expert Contributor

Hi @Hemant,

Did you configure the nifi.kerberos.krb5.file in your nifi.properties?

Expert Contributor

For info, I think that once you configure that property, you need to restart nifi

Contributor

Hello @msumbul

nifi.kerberos.krb5.file is configured in the properties file.

,

Expert Contributor

@Hemant

for the user do you have this structure: hive/FQDN@MY_REALM ?

Contributor

@msumbul Yes, The principal is in the standard format

Expert Contributor

@Hemant,

You said that you were able to interact with hdfs from the host that has nifi. How did you get the ticket to interact wit hdfs? Are you able to create a ticket with the user and keytab mentionned in the configuration or the processor? (Just to be sure that the key tab is working well

Expert Contributor

can nifi user access that keytab? try using the keytab with kinit and try to connect with beeline and see if that works. also you can try adding this property to nifi -Dsun.security.krb5.debug=true , that will give you some detailed logs to figure if there is anything wrong with the TGT.

Explorer

I'm having the same problem trying to access Hive from Nifi via Zookeeper. HDFS access from within hive works fine.

I manually installed the clients on my Nifi node (as it's external to my cluster) and copied the core-site and hive-site files over to it. I can connect via beeline (with and without adding 'principal=hive/_HOST@<REALM>' to the connection string. The hive cli throws an error, however. I believe that this error is due to the fact that it's trying to connect remotely to the mysql instance and would need a password (rather than the password less auth on the local hiveserver) which isn't conifgured, so it fails.
Caused by: java.sql.SQLException: Access denied for user 'hive'@'<HOSTNAME>' (using password: YES)

Super Collaborator

@Hemant

How did you solve this? i am having the same issue. can able to upload files to HDFS using kerberos but not able to execute Hive commands.

Regards,

Sai

Explorer

I had to restart my NiFi processes, but that was just a band-aid. As such YMMV. I believe what is happening is that the TGT renewal isn't occurring properly and it causes the whole process to stop.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.