Hi guys,
I'm trying to import a table with +/- 7 million rows from a Oracle database into HBase with Sqoop. When I run it like this it works fine:
sqoop import -Dhadoop.security.credential.provider.path=jceks://mypath/mypassword.jceks --connect jdbc:oracle:thin:@1.1.1.1:1111:XXXX --username username --password-alias mypassword.password.alias --table mytable --hbase-table mynamespace:mytable --hbase-create-table --hbase-bulkload --column-family d --hbase-row-key myrowkey --check-column lastmodified --incremental append --outdir /myoutputdir -m 1
But when I convert this to a job:
sqoop job -Dhadoop.security.credential.provider.path=jceks://mypath/mypassword.jceks --create myjob -- import --connect jdbc:oracle:thin:@1.1.1.1:1111:XXXX --username username --password-alias mypassword.password.alias --table mytable --hbase-table mynamespace:mytable --hbase-create-table --hbase-bulkload --column-family d --hbase-row-key myrowkey --check-column lastmodified --incremental append --outdir /myoutputdir -m 1
And run it with "sqoop job --exec myjob" the job keeps hanging. I don't get any error messages it just hangs at "INFO mapreduce.Job: map 0% reduce 0%"
The weird thing is that if I execute the job with a recent timestamp like this:
sqoop job --exec myjob -- --last-value 1533722314
Then it works fine and finishes really quickly. But when i do a full load with no last-value or a timestamp like 1000000000 it hangs.
I have no idea what's causing this. Any help would be much appreciated.
We want to do incremental updates but also sometimes a full load. For the incremental load it just grabs the --last-value from the metastore and if we want to do a full load we would like to execute the job like above.
