Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How do you pass on Hiveserver2 URL in Sqoop command

avatar
Contributor

I've enabled Hive Ssl, Sqoop tries to connect to hive without the ssl information.

How do I specify or set the right jdbc URL for Hiveserver2 in Sqoop.

I tried using --hs2-url but got 'Unrecognized argument'.

5 REPLIES 5

avatar
Master Collaborator

Hi,

Have you used the hs2-url something like below?

 

sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table "employees_test" --target-dir "/user/username/importdir/employees_test" --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

 

Can you provide your sqoop command along with its error output?

avatar
Contributor

I tried using --hs2-url but got the below error

'ERROR tool.BaseSqoopTool: Error parsing arguments for import:
 ERROR tool.BaseSqoopTool: Unrecognized argument: --hs2-url'

Sqoop command:

sqoop import \
--connect $MYCONN \
--username $MYUSER \
--password $MYPSWD \
--table "employees_test" \
--target-dir "/user/username/importdir/employees_test" \
--delete-target-dir \
--hive-import \
--hs2-url "jdbc:hive2://hs2host1:2181,hs2host2:2181,hs2host3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" \
--hive-database 'test' \
--hive-table 'batch'

avatar
Master Collaborator

Hi @pauljoshiva ,

 

Sqoop uses local hive which does automatic connect so you would need to modify the information in beeline-site.xml as referenced in below article:

 

https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-Usingbeeline...

 

Hence please go to /usr/hdp/<VERSION>/hive/conf then open the beeline-site.xml and modify there.

 

Regards,

Chethan YM

avatar
Community Manager

@pauljoshiva, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Expert Contributor

Hello,

 

Please use the either of the methods and let us know if it helps:-

 

Step 1:- Pass the --hs2-url correctly

 

sqoop import --connect <<Connection string>> --username <<username>> --password <<password>> --table “ta”ble_name --target-dir “<<“directory path>> --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

 

If you are launching the sqoop job from CLI, you don’t need to pass the hs2-user and hs2 key tab. You can remove the parameters and run the sqoop command after doing the kinit from CLI or you can pass in the sqoop command itself like above.

 

Step 2:-  If you don’t want to add the hs2_url

 

Add the following 2 properties in the sqoop configuration and restart sqoop

1. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml :

         Name: sqoop.beeline.env.preserve

         Value: HADOOP_CLIENT_OPTS

2. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh:

        export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal"