Reply
New Contributor
Posts: 1
Registered: ‎03-31-2016

Oozie workflow with Sqoop action and --password-alias option does not work via Hue

Hi,

      I am having this issue where encrypted password stored inside JKS is not working when invoking sqoop action from oozie workflow via Hue. Please find the exception.

   

    Product Version : Sqoop version: 1.4.6-cdh5.5.2 

 

991 [uber-SubtaskRunner] DEBUG org.apache.sqoop.tool.BaseSqoopTool  - Enabled debug logging.
2992 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Reflecting credential provider classes and methods
2992 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Found org.apache.hadoop.security.alias.CredentialProvider
2993 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Found org.apache.hadoop.security.alias.CredentialProviderFactory
2993 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Found CredentialProvider#createCredentialEntry
2993 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Found CredentialProvider#flush
2994 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Found Configuration#getPassword
2994 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Found CredentialProviderFactory#getProviders
2994 [uber-SubtaskRunner] DEBUG org.apache.sqoop.util.password.CredentialProviderHelper  - Resolving alias with credential provider path set to jceks://hdfs/user/test_user/my.pwd.jceks
3036 [uber-SubtaskRunner] DEBUG org.apache.sqoop.Sqoop  - Error resolving password  from the credential providers 
java.lang.RuntimeException: Error resolving password  from the credential providers 
	at org.apache.sqoop.util.password.CredentialProviderHelper.resolveAlias(CredentialProviderHelper.java:114)
	at org.apache.sqoop.tool.BaseSqoopTool.applyCredentialsOptions(BaseSqoopTool.java:1065)
	at org.apache.sqoop.tool.BaseSqoopTool.applyCommonOptions(BaseSqoopTool.java:997)
	at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:875)
	at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:131)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
	at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
	at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
	at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:378)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:296)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Configuration problem with provider path.
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1970)
	at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:1930)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.sqoop.util.password.CredentialProviderHelper.resolveAlias(CredentialProviderHelper.java:106)
	... 31 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
	at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:867)
	at java.security.KeyStore.load(KeyStore.java:1214)
	at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:110)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:58)
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1950)
	... 37 more

 

 The exact same sqoop command works via command line though.

 

sqoop import -Dhadoop.security.credential.provider.path=jceks://hdfs/user/test_user/my.pwd.jceks \
 --connect jdbc:oracle:thin:@//server1.test.com:1526/DB \
 --username test_user\
 --password-alias my.pwd.alias \
 --table MYDB.CUSTOMER \
 --target-dir /user/projects/customer \
 --as-avrodatafile \
 --compression-codec snappy \
 -m 1

 

   The JKS file is not corrupted if that would have been the real issue the command line wouldn't have worked either.

 

    Appreciate your feedbak or suggestions. 

 

New Contributor
Posts: 5
Registered: ‎10-12-2016

Re: Oozie workflow with Sqoop action and --password-alias option does not work via Hue

Hello,

Add the following configuration property to each Sqoop action that uses keystore and re-run the oozie job.

This would resolve the issue - "Error resolving password  from the credential providers "

 

<property>

<name>oozie.launcher.mapred.map.child.env</name>

<value>HADOOP_CREDSTORE_PASSWORD=none</value>

</property>

 

 

Thanks,

Kumar

New Contributor
Posts: 1
Registered: ‎10-17-2016

Re: Oozie workflow with Sqoop action and --password-alias option does not work via Hue

Hello,
Could you please let me know the reason why do we need to add this property?
Explorer
Posts: 26
Registered: ‎01-19-2017

Re: Oozie workflow with Sqoop action and --password-alias option does not work via Hue

Did you find solution for the problem. I am facing same kind of issue.

Highlighted
New Contributor
Posts: 1
Registered: ‎04-19-2017

Re: Oozie workflow with Sqoop action and --password-alias option does not work via Hue

[ Edited ]

Getting following error message

 

"Error resolving password  from the credential providers"

 

When trying to execute sqoop import job inside oozie workflow via hue. Can anyone please suggest some solution to it.

Announcements
New solutions