ssl with sqoop 1.4.6

[#1]

Hi Team,

For secure data transfer using sqoop planning to use SSL but sqoop import command has issues.

This is my sqoop config file:

import
-Dmapreduce.output.fileoutputformat.compress=false
-Dmapreduce.output.fileoutputformat.compress.type=NONE
-Dhadoop.security.credential.provider.path=jceks://hdfs/user/xxx/key/xx.dbpass.jceks
--connect
jdbc:db2://xxxx:num/db:sslConnection=true;sslTrustStoreLocation=/xxx/truststore.jks;sslTrustStorePassword=xxxx;
--username
xxxx
--password
xxxxx

When running my sqoop job i get the following error:

Exception java.io.FileNotFoundException: Error opening socket to server xxxx/xxxx on port xxxx with message: /tmp/truststore.jks (No such file or directory). ERRORCODE=-4499, SQLSTATE=08001
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76

Copied the jks file to /tmp ( hdfs ) location but doesnt solve the issue

[#2]

Do you have the truststore distributed on all cluster worker nodes? Keep in mind that Sqoop will run as a M/R job.

[#3]

Hi  @mridley Thanks for getting back.

Yes i have tried placing the trustore.jks file in hdfs location but still fails with same error.

Below the connection string:

jdbc:db2://xxxx:1234/xxxx:sslConnection=true;sslTrustStoreLocation=hdfs://xxx:8020/tmp/truststore.jks;sslTrustStorePassword=xxxx;

[#4]

@here

does sqoop 1.4.6 support ssl?  Could you please point me to any cloudera docs.  I have 2.6.4 hdp cluster.

Are there any config changes that has to be made in sqoop config file?

[#5]

did you find the solution for the above issue, am also facing the similar issue. i need to pass certs as connection string, it is saying file not found exception.

[#6]

Yes the following solution worked...

Created a jdbcprops file & it had the following entries

retrieveMessagesFromServerOnGetMessage=true
sslConnection=true
sslTrustStoreLocation=/xxxx/xxxxl/truststore.jks
sslTrustStorePassword=!!!!!!

Sqoop string is :

sqoop import -files truststore.jks --connect jdbc:db2://xxxxx:xxxx/xxxx --username abcd --password xxxxx --connection-param-file jdbcprops --query 'select * from table1 where $CONDITIONS fetch first 2 rows only' --target-dir /user/abcd -m 1 –verbose

[#7]

Thanks we did the same and worked. 

[#8]

Can you please tell where you placed jdbcprops file? local or hdfs.

same for truststore.jks file.