Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Importing table with Sqoop works as command but hangs as job

Importing table with Sqoop works as command but hangs as job

New Contributor

Hi guys,

I'm trying to import a table with +/- 7 million rows from a Oracle database into HBase with Sqoop. When I run it like this it works fine:

sqoop import -Dhadoop.security.credential.provider.path=jceks://mypath/mypassword.jceks --connect jdbc:oracle:thin:@1.1.1.1:1111:XXXX  --username username --password-alias mypassword.password.alias --table mytable --hbase-table mynamespace:mytable --hbase-create-table --hbase-bulkload --column-family d --hbase-row-key myrowkey --check-column lastmodified --incremental append --outdir /myoutputdir -m 1

But when I convert this to a job:

sqoop job -Dhadoop.security.credential.provider.path=jceks://mypath/mypassword.jceks --create myjob -- import --connect jdbc:oracle:thin:@1.1.1.1:1111:XXXX  --username username --password-alias mypassword.password.alias --table mytable --hbase-table mynamespace:mytable --hbase-create-table --hbase-bulkload --column-family d --hbase-row-key myrowkey --check-column lastmodified --incremental append --outdir /myoutputdir -m 1

And run it with "sqoop job --exec myjob" the job keeps hanging. I don't get any error messages it just hangs at "INFO mapreduce.Job: map 0% reduce 0%"

The weird thing is that if I execute the job with a recent timestamp like this:

sqoop job --exec myjob -- --last-value 1533722314

Then it works fine and finishes really quickly. But when i do a full load with no last-value or a timestamp like 1000000000 it hangs.

I have no idea what's causing this. Any help would be much appreciated.

We want to do incremental updates but also sometimes a full load. For the incremental load it just grabs the --last-value from the metastore and if we want to do a full load we would like to execute the job like above.

1 REPLY 1
Highlighted

Re: Importing table with Sqoop works as command but hangs as job

It would be due to amount of data being processed and with mapper=1, it would performing low. Try increasing the mappers like "-m 20".