Support Questions
Find answers, ask questions, and share your expertise

Issue with Sqoop metastore

Hello All,

I have enabled Sqoop Metastore using MYSQL as the backend database using the instructions in this link. However, my oozie job is running into below error:

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, sqoop-site.xml (Permission denied) sqoop-site.xml (Permission denied) at Method) at<init>( at<init>( at org.apache.oozie.action.hadoop.SqoopMain.setUpSqoopSite( at at at org.apache.oozie.action.hadoop.SqoopMain.main( at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( at sun.reflect.DelegatingMethodAccessorImpl.invoke( at java.lang.reflect.Method.invoke( at at at org.apache.hadoop.mapred.MapTask.runOldMapper( at at org.apache.hadoop.mapred.YarnChild$ at Method) at at at org.apache.hadoop.mapred.YarnChild.main(

I have uploaded sqoop-site.xml in hdfs location - /user/oozie/share/lib/lib_20170119183643/sqoop/sqoop-site.xml and also restarted oozie server.

Find the oozie job log and sqoop-site.xml attached. Please help me to understand what location is it trying to look for sqoop-site.xml.

Thanks in advance.

Shashang Sheth

ooziejoblog.txt sqoop-site.xml


Re: Issue with Sqoop metastore

@Shashang Sheth

The issue might be due to Oozie tries to load the sqoop-site.xml to share/lib directory and fails due to permission for overwriting. Remove sqoop-site.xml from folder /user/oozie/share/lib/lib_20170119183643/sqoop and then try running the Oozie job.

Also, refer similar HCC link.

Re: Issue with Sqoop metastore


After removing sqoop-site.xml, the oozie job is now complaining on the contrary:

2017-02-15 12:51:05,420 WARN ActionStartXCommand:523 - SERVER[] USER[u1453711] GROUP[-] TOKEN[] APP[MDMRDM-RDM-tables] JOB[0000000-170215081728307-oozie-oozi-W] ACTION[0000000-170215081728307-oozie-oozi-W@Sqoop-exec-rdm-jobs] Error starting action [Sqoop-exec-rdm-jobs]. ErrorType [ERROR], ErrorCode [JA008], Message [JA008: File does not exist: hdfs://sandbox-gfat/user/oozie/share/lib/lib_20170119183643/sqoop/sqoop-site.xml#sqoop-site.xml] org.apache.oozie.action.ActionExecutorException: JA008: File does not exist: hdfs://sandbox-gfat/user/oozie/share/lib/lib_20170119183643/sqoop/sqoop-site.xml#sqoop-site.xml at org.apache.oozie.action.ActionExecutor.convertExceptionHelper( at org.apache.oozie.action.ActionExecutor.convertException( at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher( at org.apache.oozie.action.hadoop.JavaActionExecutor.start( at at at at org.apache.oozie.service.CallableQueueService$ at org.apache.oozie.service.CallableQueueService$ at at org.apache.oozie.service.CallableQueueService$ at java.util.concurrent.ThreadPoolExecutor.runWorker( at java.util.concurrent.ThreadPoolExecutor$ at

Re: Issue with Sqoop metastore

@Shashang Sheth

Please add it back and provide permissions for all users to read and execute, let me know.

After adding the sqoop-site.xml, make sure to run oozie admin-sharelibupdate to refresh the sharelib.

Re: Issue with Sqoop metastore

We decided to go with HSsqldb as our vendor wouldn't support sqoop metastore + mysql. Thanks, Sindhu for providing your inputs.