Hive upgrade HDP3 and Beeline

[#1]

I am currently upgrading my cluster from HDP 2.6.5 to HDP 3.0.0 by following this guide. I successfully passed step 5 by adjusting the path but I have no idea what I should do for step 6 and 7.

5. Adjust the classpath in the following example pre-upgrade command to suit your environment, and then execute a dry run of preupgrading by running the command:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /usr/hdp/current/hive-server2-hive2/lib/derby-10.10.2.0.jar:/usr/hdp/current/hive-server2-hive2/lib/*:/usr/hdp/current/hadoop/*:/usr/hdp/current/hadoop/lib/*:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/hadoop-mapreduce-client/lib/*:/usr/hdp/current/hadoop-hdfs/*:/usr/hdp/current/hadoop-hdfs/lib/*:/usr/hdp/current/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/current/hive-client/conf/conf.server:/usr/hdp/current/hive-metastore/lib/hive-metastore.jar:/usr/hdp/current/hive-metastore/lib/libthrift-0.9.3.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hive-client/lib/hive-common.jar:/usr/hdp/current/hive-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute

6. Examine the scripts in the output to understand what running the scripts will do.

7. Login to Beeline as the Hive service user, and run each generated script to prepare the cluster for upgrading.

The Hive service user is usually the hive user. This is hive by default. If you don’t know which user is the Hive service user in your cluster, go to the Ambari Web UI and click Cluster Admin > Service Accounts, and then look for Hive User.

What should I type in beeline to go through with the upgrade?

[#2]

@Jeremy Jean-Jean

It's usually hive! Can you access the old 2.6.5 and invoke the beeline CLI

# su - hive 
$beeline

If you get the prompt then that confirms the user hive !

[#3]

Thank you for your answer.

I just tried and indeed I can invoke beeline from hive user. But what about the scripts that point 7. mentions? Should I write in beeline:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false-cp /usr/hdp/current/hive-server2-hive2/lib/derby-10.10.2.0.jar:/usr/hdp/current/hive-server2-hive2/lib/*:/usr/hdp/current/hadoop/*:/usr/hdp/current/hadoop/lib/*:/usr/hdp/current/hadoop-mapreduce-client/*:/usr/hdp/current/hadoop-mapreduce-client/lib/*:/usr/hdp/current/hadoop-hdfs/*:/usr/hdp/current/hadoop-hdfs/lib/*:/usr/hdp/current/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.0.0.0-1634.jar:/usr/hdp/current/hive-client/conf/conf.server:/usr/hdp/current/hive-metastore/lib/hive-metastore.jar:/usr/hdp/current/hive-metastore/lib/libthrift-0.9.3.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hive-client/lib/hive-common.jar:/usr/hdp/current/hive-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/* org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool -execute

[#4 · Accepted solution]

@Jeremy Jean-Jean

I can see you have the -execute option, this is ONLY used when you want to run the pre-upgrade tool command in Ambari instead of on the Beeline command line which is the recommended method.

The -execute option automatically executes the generated commands interactively, but if you what to run the scripts in beeline you need the command with the -location option where the scripts to be run in beeline will be generated.

So simply said if you don't want to generate and run the scripts in beeline use the -execute option else use -location and re-run in beeline the generated scripts 🙂

HTH

[#5]

This may work

beeline> !run ///C:/ScriptFile

[#6]

Thank you both for your help!