Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ssl with sqoop 1.4.6

ssl with sqoop 1.4.6

Explorer

Hi Team,

For secure data transfer using sqoop planning to use SSL but sqoop import command has issues.

 

This is my sqoop config file:

import
-Dmapreduce.output.fileoutputformat.compress=false
-Dmapreduce.output.fileoutputformat.compress.type=NONE
-Dhadoop.security.credential.provider.path=jceks://hdfs/user/xxx/key/xx.dbpass.jceks
--connect
jdbc:db2://xxxx:num/db:sslConnection=true;sslTrustStoreLocation=/xxx/truststore.jks;sslTrustStorePassword=xxxx;
--username
xxxx
--password
xxxxx

 

When running my sqoop job i get the following error:

Exception java.io.FileNotFoundException: Error opening socket to server xxxx/xxxx on port xxxx with message: /tmp/truststore.jks (No such file or directory). ERRORCODE=-4499, SQLSTATE=08001
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76

 

Copied the jks file to /tmp ( hdfs ) location but doesnt solve the issue

7 REPLIES 7
Highlighted

Re: ssl with sqoop 1.4.6

Cloudera Employee

Do you have the truststore distributed on all cluster worker nodes? Keep in mind that Sqoop will run as a M/R job.

Highlighted

Re: ssl with sqoop 1.4.6

Explorer

Hi  @mridley Thanks for getting back.

 

Yes i have tried placing the trustore.jks file in hdfs location but still fails with same error.

 

Below the connection string:

jdbc:db2://xxxx:1234/xxxx:sslConnection=true;sslTrustStoreLocation=hdfs://xxx:8020/tmp/truststore.jks;sslTrustStorePassword=xxxx;

Highlighted

Re: ssl with sqoop 1.4.6

Explorer

@here

 

does sqoop 1.4.6 support ssl?  Could you please point me to any cloudera docs.  I have 2.6.4 hdp cluster.

 

Are there any config changes that has to be made in sqoop config file?

 

 

Highlighted

Re: ssl with sqoop 1.4.6

New Contributor

did you find the solution for the above issue, am also facing the similar issue. i need to pass certs as connection string, it is saying file not found exception.

Highlighted

Re: ssl with sqoop 1.4.6

Explorer

Yes the following solution worked...

 

Created a jdbcprops file & it had the following entries

retrieveMessagesFromServerOnGetMessage=true
sslConnection=true
sslTrustStoreLocation=/xxxx/xxxxl/truststore.jks
sslTrustStorePassword=!!!!!!

 

Sqoop string is :

sqoop import -files truststore.jks --connect jdbc:db2://xxxxx:xxxx/xxxx --username abcd --password xxxxx --connection-param-file jdbcprops --query 'select * from table1 where $CONDITIONS fetch first 2 rows only' --target-dir /user/abcd -m 1 –verbose

Highlighted

Re: ssl with sqoop 1.4.6

New Contributor

Thanks we did the same and worked. 

Highlighted

Re: ssl with sqoop 1.4.6

New Contributor

Can you please tell where you placed jdbcprops file? local or hdfs.

same for truststore.jks file.

Don't have an account?
Coming from Hortonworks? Activate your account here