Support Questions
Find answers, ask questions, and share your expertise

Importing table with Sqoop works as command but hangs as job

New Contributor

Hi guys,

I'm trying to import a table with +/- 7 million rows from a Oracle database into HBase with Sqoop. When I run it like this it works fine:

sqoop import -Dhadoop.security.credential.provider.path=jceks://mypath/mypassword.jceks --connect jdbc:oracle:thin:@1.1.1.1:1111:XXXX  --username username --password-alias mypassword.password.alias --table mytable --hbase-table mynamespace:mytable --hbase-create-table --hbase-bulkload --column-family d --hbase-row-key myrowkey --check-column lastmodified --incremental append --outdir /myoutputdir -m 1

But when I convert this to a job:

sqoop job -Dhadoop.security.credential.provider.path=jceks://mypath/mypassword.jceks --create myjob -- import --connect jdbc:oracle:thin:@1.1.1.1:1111:XXXX  --username username --password-alias mypassword.password.alias --table mytable --hbase-table mynamespace:mytable --hbase-create-table --hbase-bulkload --column-family d --hbase-row-key myrowkey --check-column lastmodified --incremental append --outdir /myoutputdir -m 1

And run it with "sqoop job --exec myjob" the job keeps hanging. I don't get any error messages it just hangs at "INFO mapreduce.Job: map 0% reduce 0%"

The weird thing is that if I execute the job with a recent timestamp like this:

sqoop job --exec myjob -- --last-value 1533722314

Then it works fine and finishes really quickly. But when i do a full load with no last-value or a timestamp like 1000000000 it hangs.

I have no idea what's causing this. Any help would be much appreciated.

We want to do incremental updates but also sometimes a full load. For the incremental load it just grabs the --last-value from the metastore and if we want to do a full load we would like to execute the job like above.

1 REPLY 1

It would be due to amount of data being processed and with mapper=1, it would performing low. Try increasing the mappers like "-m 20".

; ;