Support Questions
Find answers, ask questions, and share your expertise

How do you pass on Hiveserver2 URL in Sqoop command

Explorer

I've enabled Hive Ssl, Sqoop tries to connect to hive without the ssl information.

How do I specify or set the right jdbc URL for Hiveserver2 in Sqoop.

I tried using --hs2-url but got 'Unrecognized argument'.

5 REPLIES 5

Expert Contributor

Hi,

Have you used the hs2-url something like below?

 

sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table "employees_test" --target-dir "/user/username/importdir/employees_test" --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

 

Can you provide your sqoop command along with its error output?

Explorer

I tried using --hs2-url but got the below error

'ERROR tool.BaseSqoopTool: Error parsing arguments for import:
 ERROR tool.BaseSqoopTool: Unrecognized argument: --hs2-url'

Sqoop command:

sqoop import \
--connect $MYCONN \
--username $MYUSER \
--password $MYPSWD \
--table "employees_test" \
--target-dir "/user/username/importdir/employees_test" \
--delete-target-dir \
--hive-import \
--hs2-url "jdbc:hive2://hs2host1:2181,hs2host2:2181,hs2host3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" \
--hive-database 'test' \
--hive-table 'batch'

Expert Contributor

Hi @pauljoshiva ,

 

Sqoop uses local hive which does automatic connect so you would need to modify the information in beeline-site.xml as referenced in below article:

 

https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-Usingbeeline...

 

Hence please go to /usr/hdp/<VERSION>/hive/conf then open the beeline-site.xml and modify there.

 

Regards,

Chethan YM

Community Manager

@pauljoshiva, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

Cloudera Employee

Hello,

 

Please use the either of the methods and let us know if it helps:-

 

Step 1:- Pass the --hs2-url correctly

 

sqoop import --connect <<Connection string>> --username <<username>> --password <<password>> --table “ta”ble_name --target-dir “<<“directory path>> --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

 

If you are launching the sqoop job from CLI, you don’t need to pass the hs2-user and hs2 key tab. You can remove the parameters and run the sqoop command after doing the kinit from CLI or you can pass in the sqoop command itself like above.

 

Step 2:-  If you don’t want to add the hs2_url

 

Add the following 2 properties in the sqoop configuration and restart sqoop

1. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml :

         Name: sqoop.beeline.env.preserve

         Value: HADOOP_CLIENT_OPTS

2. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh:

        export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal"