Skip to main content
pauljoshiva
pauljoshiva
Silver ⭐️⭐️⭐️
October 1, 2021
Question

How do you pass on Hiveserver2 URL in Sqoop command

I've enabled Hive Ssl, Sqoop tries to connect to hive without the ssl information.

How do I specify or set the right jdbc URL for Hiveserver2 in Sqoop.

I tried using --hs2-url but got 'Unrecognized argument'.

5 replies

ChethanYM
Gold ⭐️
ChethanYM
Gold ⭐️
October 4, 2021

Hi,

Have you used the hs2-url something like below?

 

sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table "employees_test" --target-dir "/user/username/importdir/employees_test" --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

 

Can you provide your sqoop command along with its error output?

Translate
pauljoshiva
pauljoshivaAuthor
Silver ⭐️⭐️⭐️
October 4, 2021

I tried using --hs2-url but got the below error

'ERROR tool.BaseSqoopTool: Error parsing arguments for import:
 ERROR tool.BaseSqoopTool: Unrecognized argument: --hs2-url'

Sqoop command:

sqoop import \
--connect $MYCONN \
--username $MYUSER \
--password $MYPSWD \
--table "employees_test" \
--target-dir "/user/username/importdir/employees_test" \
--delete-target-dir \
--hive-import \
--hs2-url "jdbc:hive2://hs2host1:2181,hs2host2:2181,hs2host3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" \
--hive-database 'test' \
--hive-table 'batch'

Translate
ChethanYM
Gold ⭐️
ChethanYM
Gold ⭐️
October 26, 2021

Hi @pauljoshiva ,

 

Sqoop uses local hive which does automatic connect so you would need to modify the information in beeline-site.xml as referenced in below article:

 

https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-Usingbeeline-site.xmltoautomaticallyconnecttoHiveServer2

 

Hence please go to /usr/hdp/<VERSION>/hive/conf then open the beeline-site.xml and modify there.

 

Regards,

Chethan YM

Translate
ShankerSharma
Gold ⭐️
ShankerSharma
Gold ⭐️
October 4, 2021

Hello,

 

Please use the either of the methods and let us know if it helps:-

 

Step 1:- Pass the --hs2-url correctly

 

sqoop import --connect <<Connection string>> --username <<username>> --password <<password>> --table “ta”ble_name --target-dir “<<“directory path>> --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

 

If you are launching the sqoop job from CLI, you don’t need to pass the hs2-user and hs2 key tab. You can remove the parameters and run the sqoop command after doing the kinit from CLI or you can pass in the sqoop command itself like above.

 

Step 2:-  If you don’t want to add the hs2_url

 

Add the following 2 properties in the sqoop configuration and restart sqoop

1. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml :

         Name: sqoop.beeline.env.preserve

         Value: HADOOP_CLIENT_OPTS

2. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh:

        export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal"

Translate
Terms & ConditionsAccessibility statement