Member since
08-12-2016
1
Post
1
Kudos Received
0
Solutions
08-12-2016
01:45 PM
1 Kudo
I am tried to sqoop the tables from Oracle database to HDFS as ORC File. Using a Java framework application which uses the sqoop1.4.6 jar. Before running hadoop -jar command, I am setting the environment variables as under. The ingestion process runs fine for the first table in the loop (from java code) but fails for the rest of the tables which run under this process with the exception as below. Can anyone please suggest how to fix this issue Env variables: export HCAT_HOME=/usr/hdp/current/hive-webhcat
export HIVE_HOME=/usr/hdp/current/hive-client
export SQOOP_HOME=/usr/hdp/current/sqoop-client
export SQOOP_CONF_DIR=/usr/hdp/current/sqoop-client/conf
export HADOOP_CLASSPATH=$HCAT_HOME/share/hcatalog/hive-hcatalog-core.jar:$SQOOP_HOME/sqoop-1.4.6.2.4.2.0-258.jar:$SQOOP_HOME/lib/*#:$HIVE_HOME/lib/* HADOOP_CLASSPATH=$HADOOP_CLASSPATH:${LIBDIR}/*:/etc/hive/conf:/etc/hbase/conf:/etc/hadoop/conf:/usr/hdp/current/pig-client/*:/usr/hdp/current/pig-client/lib/*:/usr/hdp/current/pig-client/lib/h2/*:/usr/hdp/current/hive-metastore/lib/*:/usr/hdp/current/hive-webhcat/share/hcatalog/*:/usr/hdp/current/hive-client/lib/*
Exception: 2016/08/11 10:45:06.322 [ERROR] main (ImportTool.java) - Encountered
IOException running import job: org.apache.hive.hcatalog.common.HCatException :
2004 : HCatOutputFormat not initialized, setOutput has to be called at
org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:102) at
org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:76) at
org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:65) at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266) at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139) at
org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) at
org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at
org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) at
org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196) at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169) at
org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266) at
org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673) at
org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:445) at
org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497) at
org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) at org.ford.mssap.rdbms.datastore.business.SqoopRdbmsService.executeSqoopJob(SqoopRdbmsService.java:279) at
org.ford.mssap.rdbms.datastore.business.SqoopRdbmsService.ingest(SqoopRdbmsService.java:52) at
org.ford.mssap.ingest.database.DatabaseSourceIngest.execute(DatabaseSourceIngest.java:735) at
org.ford.mssap.ingest.database.DatabaseSourceIngest.ingest(DatabaseSourceIngest.java:550) at
org.ford.mssap.ingest.database.DatabaseSourceIngest.dataIngest(DatabaseSourceIngest.java:133) at com.crr.ingestion.inbound.IngestAction.main(IngestAction.java:50) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
java.lang.reflect.Method.invoke(Method.java:497) at
org.apache.hadoop.util.RunJar.run(RunJar.java:221) at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)
... View more
Labels: