how to do incremental load using sqoop shell script? By passing parameters?

[#1]

+#!/bin/bash

echo " sqoop import --connect $connectors//$orginServer/$orginDatabase --table $origTable --username $username --password $password -m 1 --hive-import --hive-database $hiveDatabase --hive-table $myTable --target-dir /tmp/sqoop_1 --drivers $drivers"

My sample script is above how to do modification in above script for sqoop incremental using column time stamp and merge id

[#2 · Accepted solution]

To do incremental imports, you need to add a few more arguments to your command. Take a look at the following description from the sqoop docs:

https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_incremental_imports

Inside your shell script, I’m not sure how you’re setting the values of your named parameters, unless you have more code that is mapping the positional parameters to local variables.

When using the lastmodified mode, it requires that a field in your table to be a timestamp, which will be updated with the current timestamp during update. Additionally, the update requires a check column, which can be of type timestamp also.

[#3]

It really depends on what your relational database is. Some relational database does not support incremental imports. Otherwise, the link above provided by bhagan should sufficient.