Sqoop import with wallet

ssandona — Fri, 16 Sep 2022 12:18:09 GMT

Hi all,

We are facing some problems related to the wallet utilization with Sqoop v1.4.6 in Cloudera CDH 5.8.5. Our scenario:

user folder: /home/myuser
wallet folder: /home/myuser/wallet (cwallet.sso, cwallet.sso.lck, ewallet.p12, ewallet.p12.lck, sqlnet.ora, tnsnames.ora)

The wallet is well configured as if I run

sqlplus /@MY_TNS_NAME

I can connect to the db without the password.

The problem is with sqoop. I can connect to the db and list all the tables inside using:

export HADOOP_OPTS="-Doracle.net.tns_admin=/home/myuser/wallet -Doracle.net.wallet_location=/home/myuser/wallet"sqoop list-tables --connect "jdbc:oracle:thin:@MY_TNS_NAME"

If I try to dump a table into HDFS with the following command (as suggested here)

export HADOOP_OPTS="-Doracle.net.tns_admin=/home/myuser/wallet -Doracle.net.wallet_location=/home/myuser/wallet"sqoop import -D mapreduce.map.java.opts='-Doracle.net.tns_admin=. -Doracle.net.wallet_location=.' -files /home/myuser/wallet/cwallet.sso,/home/myuser/wallet/ewallet.p12,/home/myuser/wallet/sqlnet.ora,/home/myuser/wallet/tnsnames.ora  -libjars "/var/lib/sqoop/oraclepki.jar,/var/lib/sqoop/osdt_cert.jar,/var/lib/sqoop/osdt_core.jar,/var/lib/sqoop/sqljdbc4.jar" --m 1 --connect "jdbc:oracle:thin:@MY_TNS_NAME" --query "SELECT * FROM MYTABLE WHERE \$CONDITIONS" --target-dir '/tmp/mytabledump'

I get this error:

Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: encountered a problem with the Secret Store. Check the wallet location for the presence of an open wallet (cwallet.sso) and ensure that this wallet contains the correct credentials using the mkstore utility: java.io.IOException: oracle.security.crypto.core.CipherException: Invalid padding string (or incorrect password)...

If I run the same command without using the wallet (providing manually user and password) it works.

Additional info:

ACLs allows the sqoop user to access /home/myuser
the /home/myuser/wallet folder owner group is the sqoop primary group (rwx permissions).

It seems there are some problems passing the wallet to the mappers. Any idea how to solve this?

Re: Sqoop import with wallet

ssandona — Fri, 01 Dec 2017 11:02:47 GMT

We managed to solve the issue.

The problem was related to the wallet.

As far as I know there are 2 types of wallet. The first type can be used only on the machine where it was created. The second type can be used on multiple machines.

Our problem is that our wallet was of the first type.

During map reduce the wallet is distributed among mappers and so with this "non sharable" wallet the mappers (located on different machines) are not able to use it.

We had to create the wallet with the command:

mkstore -wrl "<wallet_folder>" -create

Hope our solution will help someone else in the future.

Stefano

question Sqoop import with wallet in Archives of Support Questions (Read Only)

Sqoop import with wallet

Re: Sqoop import with wallet