question Re: How do you pass on Hiveserver2 URL in Sqoop command in Support Questions

How do you pass on Hiveserver2 URL in Sqoop command

pauljoshiva — Fri, 01 Oct 2021 19:24:34 GMT

I've enabled Hive Ssl, Sqoop tries to connect to hive without the ssl information.

How do I specify or set the right jdbc URL for Hiveserver2 in Sqoop.

I tried using --hs2-url but got 'Unrecognized argument'.

Re: How do you pass on Hiveserver2 URL in Sqoop command

ChethanYM — Mon, 04 Oct 2021 03:50:03 GMT

Hi,

Have you used the hs2-url something like below?

sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table "employees_test" --target-dir "/user/username/importdir/employees_test" --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

Can you provide your sqoop command along with its error output?

Re: How do you pass on Hiveserver2 URL in Sqoop command

ShankerSharma — Mon, 04 Oct 2021 08:49:08 GMT

Hello,

Please use the either of the methods and let us know if it helps:-

Step 1:- Pass the --hs2-url correctly

sqoop import --connect <<Connection string>> --username <<username>> --password <<password>> --table “ta”ble_name --target-dir “<<“directory path>> --delete-target-dir --hive-import --hs2-url "jdbc:hive2://hs2host:10000/default;principal=hive/hs2host@DOMAIN.COM;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" --hs2-user username --hs2-keytab "/path/to/sqooptestkeytab"

If you are launching the sqoop job from CLI, you don’t need to pass the hs2-user and hs2 key tab. You can remove the parameters and run the sqoop command after doing the kinit from CLI or you can pass in the sqoop command itself like above.

Step 2:- If you don’t want to add the hs2_url

Add the following 2 properties in the sqoop configuration and restart sqoop

1. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml :

Name: sqoop.beeline.env.preserve

Value: HADOOP_CLIENT_OPTS

2. Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh:

export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal"

Re: How do you pass on Hiveserver2 URL in Sqoop command

pauljoshiva — Mon, 04 Oct 2021 15:15:10 GMT

I tried using --hs2-url but got the below error

'ERROR tool.BaseSqoopTool: Error parsing arguments for import:
ERROR tool.BaseSqoopTool: Unrecognized argument: --hs2-url'

Sqoop command:

sqoop import \
--connect $MYCONN \
--username $MYUSER \
--password $MYPSWD \
--table "employees_test" \
--target-dir "/user/username/importdir/employees_test" \
--delete-target-dir \
--hive-import \
--hs2-url "jdbc:hive2://hs2host1:2181,hs2host2:2181,hs2host3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;ssl=true;sslTrustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks;trustStorePassword=password" \
--hive-database 'test' \
--hive-table 'batch'

Re: How do you pass on Hiveserver2 URL in Sqoop command

ChethanYM — Tue, 26 Oct 2021 06:11:44 GMT

Hi @pauljoshiva ,

Sqoop uses local hive which does automatic connect so you would need to modify the information in beeline-site.xml as referenced in below article:

https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-Usingbeeline-site.xmltoautomaticallyconnecttoHiveServer2

Hence please go to /usr/hdp/<VERSION>/hive/conf then open the beeline-site.xml and modify there.

Regards,

Chethan YM

Re: How do you pass on Hiveserver2 URL in Sqoop command

VidyaSargur — Thu, 28 Oct 2021 12:23:24 GMT

@pauljoshiva, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.