09-26-2017 03:23 PM
We are facing some problems related to the wallet utilization with Sqoop v1.4.6 in Cloudera CDH 5.8.5. Our scenario:
The wallet is well configured as if I run
I can connect to the db without the password.
The problem is with sqoop. I can connect to the db and list all the tables inside using:
export HADOOP_OPTS="-Doracle.net.tns_admin=/home/myuser/wallet -Doracle.net.wallet_location=/home/myuser/wallet"sqoop list-tables --connect "jdbc:oracle:thin:@MY_TNS_NAME"
If I try to dump a table into HDFS with the following command (as suggested here)
export HADOOP_OPTS="-Doracle.net.tns_admin=/home/myuser/wallet -Doracle.net.wallet_location=/home/myuser/wallet"sqoop import -D mapreduce.map.java.opts='-Doracle.net.tns_admin=. -Doracle.net.wallet_location=.' -files /home/myuser/wallet/cwallet.sso,/home/myuser/wallet/ewallet.p12,/home/myuser/wallet/sqlnet.ora,/home/myuser/wallet/tnsnames.ora -libjars "/var/lib/sqoop/oraclepki.jar,/var/lib/sqoop/osdt_cert.jar,/var/lib/sqoop/osdt_core.jar,/var/lib/sqoop/sqljdbc4.jar" --m 1 --connect "jdbc:oracle:thin:@MY_TNS_NAME" --query "SELECT * FROM MYTABLE WHERE \$CONDITIONS" --target-dir '/tmp/mytabledump'
I get this error:
Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: encountered a problem with the Secret Store. Check the wallet location for the presence of an open wallet (cwallet.sso) and ensure that this wallet contains the correct credentials using the mkstore utility: java.io.IOException: oracle.security.crypto.core.CipherException: Invalid padding string (or incorrect password)...
If I run the same command without using the wallet (providing manually user and password) it works.
It seems there are some problems passing the wallet to the mappers. Any idea how to solve this?
12-01-2017 03:02 AM
We managed to solve the issue.
The problem was related to the wallet.
As far as I know there are 2 types of wallet. The first type can be used only on the machine where it was created. The second type can be used on multiple machines.
Our problem is that our wallet was of the first type.
During map reduce the wallet is distributed among mappers and so with this "non sharable" wallet the mappers (located on different machines) are not able to use it.
We had to create the wallet with the command:
mkstore -wrl "<wallet_folder>" -create
Hope our solution will help someone else in the future.