Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Does sqoop import command write a .jar file?

avatar
New Contributor

I'm new in the area, so sorry if this is a stupid question, but i have a sqoop command in one of my projects that i suspect is writing a .jar file inside /tmp. It should not write into /tmp.

 

this is the command:

 

time sqoop import \

-D mapreduce.job.classloader="${<env_var>}" \

-D mapreduce.job.queuename="${<env_var>}" \

-D mapreduce.job.name="${<env_var>}" \

-D tez.queue.name="${<env_var>}" \

-D org.apache.sqoop.splitter.allow_text_splitter=true \

--driver ${<env_var>} \

--connect ${<env_var>} \

--username ${<env_var>} \

--password-file ${<env_var>} \

--query ${<env_var>}

--split-by ${<env_var>} \

--num-mappers ${<env_var>} \

--target-dir ${<env_var>} \

${DELETE_TARGET_DIR} \

--fields-terminated-by ${<env_var>}

 

I also found in the log the following message: orm.CompilationManager: Wiriting jar file: <file_path>

 

What is this orm.CompilationManager?

 

2 REPLIES 2

avatar
Community Manager

@arnxld Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Sqoop experts @JoseManuel and @jphelps  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Expert Contributor

Hello!

 

SQOOP takes advantage of YARN to create a distributed IMPORT/EXPORT job

To do so, it needs to compile your query for the mappers to actually execute the job

 

Check out the following links for further information:

 

https://sqoop.apache.org/docs/1.4.7/SqoopUserGuide.html - Section 8.2 Table 18

https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/orm/CompilationManager.java#L50

 

Hope this helps,

 

Sincerely,

José M Martínez Poblete

Cloudera Support Hive SME US/East/Shift5

my.cloudera.com/faq/support.html

USA: 1-866-809-4659

INTL: 1-650-425-9421