11-05-2014 12:30 PM
i have simple import sqoop-1 job from MySQL to HDFS. I've migrated it to CDH 5.2 and got problems.
Sqoop-1 runs out of memory during import - it cnsumes more than max allowed memory and is kill by nodeManager. Sqoop ignores
mapreduce.map.memory.mb=2048 setting and consumes more. Then it's killed. There is only 4M rows. I don't understasnd why this job is so memory intensive. It should just get batch of records from MySQL and flush it to mapper output.