Reply
Contributor
Posts: 52
Registered: ‎10-19-2016

Disable hive cli conflicts spark submit ?

Hi,

 

I can submit spark jobs using user user_1 before disable hive cli.

After I enable sentry service, it recommands disabling hive cli.

However, I can only submit spark jobs using user hive/impala/hue.

Does disable hive cli conflicts with spark submit?

 

Thanks a lot.

Cloudera Employee
Posts: 94
Registered: ‎05-10-2016

Re: Disable hive cli conflicts spark submit ?

No, these are two seperate libraries and disabling hive cli will not affect spark.
Posts: 642
Topics: 3
Kudos: 109
Solutions: 67
Registered: ‎08-16-2016

Re: Disable hive cli conflicts spark submit ?

Yes it will. I am assuming you restricted Hive metastore access to those users per the Sentry doc. This means that normal users can't run the hive CLI since they cant access the metastore. But hue, hs2, and Impala can since you allowed them access. You should do the same for Spark. I haven't tested it so I'm not sure on which user it should be, either spark or yarn (if using spark on yarn). This will let any spark jobs running that use the hivecontext to read metadata from HMS to work.
Announcements