Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search

Who agreed with this topic

Sqoop GC overhead limit exceeded after CDH5.2 update

Labels:
avatar
author-rank juergen_thomann
Explorer

Created on ‎10-21-2014 05:45 AM - edited ‎09-16-2022 02:10 AM

Hi,

 

we updated sqoop from cdh5.0.1 to cdh5.2 and now it fails everytime with a GC overhead limit exceeded error.

The old version was able to import over 14GB of data over one mapper and the import fails now when a mapper gets too many rows. I checked a heap dump and the memory was completely used by over 3.5 million rows of data (-Xmx 1700M).

The connector is mysql-jdbc version 5.1.33 and the job imports the data as text file in a have table.

 

Can I avoid this with a setting or is this a bug that should go to jira?

 

Thank you,

Jürgen

10,602
0 Kudos
1
Who agreed with this topic