Support Questions
Find answers, ask questions, and share your expertise

Error in sqoop export " error: package org.apache.hadoop.io does not exist" at runtime with spring boot web application

New Contributor

Hi, We are using sqoop/hadoop driver in java. At runtime it's unable to find hadoop dependencies while running jar. Works fine in IDE. Looks like some classpath issue at runtime.

tried every solution to build jar with explicit dependencies in gradle but every things is in vain.

 Can someone help me on this please?

 

stack trace:

'''/tmp/sqoop-purnanandkumar/compile/b12b3a69e526c87cb506dc6f6c6d94be/QueryResult.java:7: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.BytesWritable;
^
/tmp/sqoop-purnanandkumar/compile/b12b3a69e526c87cb506dc6f6c6d94be/QueryResult.java:8: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Text;
^
/tmp/sqoop-purnanandkumar/compile/b12b3a69e526c87cb506dc6f6c6d94be/QueryResult.java:9: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Writable;
^
/tmp/sqoop-purnanandkumar/compile/b12b3a69e526c87cb506dc6f6c6d94be/QueryResult.java:10: error: package org.apache.hadoop.mapred.lib.db does not exist
import org.apache.hadoop.mapred.lib.db.DBWritable;
^
/tmp/sqoop-purnanandkumar/compile/b12b3a69e526c87cb506dc6f6c6d94be/QueryResult.java:622: error: cannot find symbol
Text.writeString(__dataOut, tenancy);
^
symbol: variable Text
location: class QueryResult
/tmp/sqoop-purnanandkumar/compile/b12b3a69e526c87cb506dc6f6c6d94be/QueryResult.java:640: error: cannot find symbol
Text.writeString(__dataOut, vpc_id);
^
symbol: variable Text
location: class QueryResult
100 errors
2021-07-19 15:35:17,028 ERROR reqid=AwsInstance 2021/07/19 15:35:15 thread=org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1 customer=1 o.a.s.t.ImportTool:634 : Import failed: java.io.IOException: Error returned by javac
at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:226)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at com.cloudhealth.asset.cache.exporter.domain.ExtractDataFromAssetDB.executeSqoop(ExtractDataFromAssetDB.java:131)
at com.cloudhealth.asset.cache.exporter.domain.ExtractDataFromAssetDB.extractData(ExtractDataFromAssetDB.java:110)
at com.cloudhealth.asset.cache.exporter.domain.ExtractDataFromAssetDB.runExtractor(ExtractDataFromAssetDB.java:69)
at com.cloudhealth.asset.cache.exporter.service.ExportDataService.commenceExport(ExportDataService.java:41)
at com.cloudhealth.asset.cache.exporter.listener.Listener.AssetDataExportListner(Listener.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.kafka.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:48)
at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:326)
at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:86)
at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:51)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:1696)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:1679)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:1634)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:1577)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:1485)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1235)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:985)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:905)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)

2021-07-19 15:35:17,069 ERROR reqid=AwsInstance 2021/07/19 15:35:15 thread=org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1 customer=1 c.c.a.c.e.d.ExtractDataFromAssetDB:115 : Sqoop failed to run successfully'''

0 REPLIES 0