How to disable hive shell for all users (Hive CLI)

[#1]

I have configured ranger authorization for hive and want to force all the users to use beeline and want to block access to hive shell to all the users.

I know one workaround - we can revoke execute access for below file on all hive-clients.

/usr/hdp/current/hive-client/bin/hive 

By doing this it could cause an issue to jobs scheduled via workflow engines like oozie or azkabaan etc.

Is there any other effective way to do this ?

[#2]

I think you have to make sure Storage Based Authorization (SBA) is enabled for the Metastore

• hive.metastore.pre.event.listeners = org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener
• hive.security.metastore.authorization.manager = org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider

And then lock down filesystem permissions for data warehouse dir. I don't think there's a way to actually prevent users from issuing the "hive" command. I may be wrong, though - happy to be corrected.

[#3]

@Ana Gillan Can you provide details on your idea? Do you have any doc or demo?

[#4]

@Neeraj Sabharwal I don't have a doc or demo, but it's simple enough to demonstrate. Enable SBA (should be already by default) and and then give the hive warehouse dir e.g. 750 permissions for hive:hadoop (recursively). Then if people try to access tables through Hive CLI as any other user, they will get a permission denied error. This won't apply to access via HiveServer2 because you'd manage table and DB permissions via Ranger Hive repository or SQL Standard authorisation. @Alex Miller's suggestion goes on top of that as well. To be truly secure, users must come in via Knox.

[#5]

This is correct. You can hide the standard binary from end users, but if they can create and execute arbitrary files on hosts with access to the cluster then there is no workaround that is 100% effective.

You should restrict access to the entire cluster (via a firewall and/or other means), and users who need to access Hive should do so from through HiveServer2 behind a Knox gateway.

[#6]

Thank you @Alex Miller

[#7]

Thank you @Ana Gillan

[#8]

@Kuldeep Kulkarni perhaps remove alias to Hive or create alias for Hive to redirect to beeline¡ Hive shell is in deprecated mode, it will be dropped at some point. Great question.

[#9]

Thanks @Artem Ervits

[#10]

Hi @Kuldeep Kulkarni

Good question because we don't want user to run hive cli as part of the best practice. One of the things that I suggest is not to install hive client in edge node.

I found this thread and I believe this can be done to avoid the hive cli access.

[#11]

Thanks @Neeraj Sabharwal

[#12]

@Alan Gates @gopal

[#13]

@Kuldeep Kulkarni it is simple, we had the same problem and i simply edited hive.distro file

/usr/hdp/2.2.0.0-2041/hive/bin/hive.distro

go to this line and comment it and add below string --- if [ "$SERVICE" = "" ] ; then

if [ "$SERVICE" = "" ] && [ "$USER" = "xxxxxxxx" ] ; then if [ "$SERVICE" = "" ] ; then

xxxxxxx - you can use your shared id /service id

let me how did it go..

[#14]

Thanks @Raja Sekhar Chintalapati

[#15 · Accepted solution]

Suggestion given by @Hajime

We can add below lines in hive-env template via ambari to disable hive-shell

if [ "$SERVICE" = "cli" ]; then
echo "Sorry! I have disabled hive-shell"
exit 1 
fi

After restarting hive services, when you try to run hive shell then you will get below output

[root@sandbox hive]# hive
Sorry! I have disabled hive-shell

[#16]

This might be sufficient to keep honest people honest. For a user that has write access on the filesystem, it's trivial to override hive-env.sh

[#17]

I've seen it recommended to change the first line to:

if [ "$SERVICE" = "cli" ] && [ "$USER" != "ambari-qa" ]; then 

Without this, Ambari won't be able to check Hive metastore state and will throw an alert (at least in HDP 2.4).

[#18]

Hi @kuldeep Kulkarni

This might be silly answer, if i see there is less option as of now to block hive cli wrt specific user, either if there are lot more changes which needs to be made on hadoop configuration side to block hive cli, then i will like to suggest - why not to block hive command from Linux side. For example say sudoers can be one of the way to do this.

[#19]

Not only do we restrict access to hive in our environment but we also make people use a command called 'hql' which is a wrapper around beeline. By default a user (on our kerberised cluster and therefore they have already done kinit) can just type 'hql' and be in the defaults or pass simple options to specify non-defaults

#!/bin/bash
# David M Walker, Data Management & Warehousing & Worldpay
# hql command line for use with a Kerborised cluster

DATABASE="DEFAULT"
QUERY_FILE=""
HOST="localhost"
PORT="10001"
QUEUE="DEFAULT"
REALM="_HOST@REALM"

while getopts :d:h:p:r:q:f: PARAM
do
   case "${PARAM}" in 
      d) DATABASE="${OPTARG}"
         ;; 
      f) QUERY_FILE="${OPTARG}"
         ;;
      h) HOST="${OPTARG}"
         ;;
      p) PORT="${OPTARG}"
         ;;
      q) QUEUE="${OPTARG}"
         ;;
      r) REALM="${OPTARG}"
         ;;
      ?) echo "Usage: hql [-d DATABASE] [-h HOST] [-p PORT] [-q QUEUE] [-r REALM] [-f QUERY_FILE]" 
         exit 1
         ;;
   esac
done
shift $(($OPTIND - 1))

if [ -z "${QUERY_FILE}" ]
then
   beeline -u "jdbc:hive2://${HOST}:${PORT}/${DATABASE};transportMode=http;httpPath=cliservice;principal=hive/${REALM}" --hiveconf tez.queue.name=${QUEUE}
   exit $?
else
   if [ -r "${QUERY_FILE}" ]
   then
      beeline -u "jdbc:hive2://${HOST}:${PORT}/${DATABASE};transportMode=http;httpPath=cliservice;principal=hive/${REALM}" --hiveconf tez.queue.name=${QUEUE} -f ${QUERY_FILE}
      exit $?
   else
      echo "File ${QUERY_FILE} is not readable"
      exit 1
   fi
fi

exit 0

[#20]

@Kuldeep Kulkarni it seems that HIVE-10511 is the long-term plan for this, also see this link.

[#21]

The only way to reliably accomplish this is to prevent users from logging into cluster nodes at all, and force them to use beeline to access HS2 in HTTP mode through Knox.

Every solution recommending changes to hive-env.sh or hive.distro can be overridden by using a modified copy of those files. Those files could even be copied from elsewhere, because this is all open source.