Support Questions

Find answers, ask questions, and share your expertise

Sqoop import failing with "exception: Java Heap space"

avatar
Rising Star

Sqoop import failing with "exception: Java Heap space" when there are no records in Oracle source table. I used a fetchsize of 75000 in sqoop import.

sqoop import successfully ran when I removed fetch-size 75000 though no records in source table.

I am standardizing the Sqoop import job which can be used for many other tables. And suppose in production, if there are no records, job should not fail. how to avoid this situation. And why it is failing when using bigger fetch-size.

Thanks

2 REPLIES 2

avatar

@Gayathri Reddy G

What are the values that you are setting for the following properties?

Example:

sqoop import -Dmapreduce.map.memory.mb=8192 -Dmapreduce.map.java.opts=-Xmx7200m -Dmapreduce.task.io.sort.mb=2400 

- Also what is the Xmx set for the "HADOOP_CLIENT_OPTS"

Example:

export HADOOP_CLIENT_OPTS=" $HADOOP_CLIENT_OPTS -Xmx2g"

.

avatar

@Gayathri Reddy G

The issue could be related to the mapreduce running outofmemory due to higher fetch size. Use the sqoop command with mapreduce parameters as:

sqoop -import -Dmapreduce.map.memory.mb=8192 -Dmapreduce.map.java.opts=-Xmx7200m ..