https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-deal-with-duplicate-values-while-importing-data-from/m-p/195326#M62211 <P>I am using ambari.</P><P>sqoop import --connect "jdbc:sqlserver://localhost:1433;database=db_name;username=user_name;pasword= password" --table table_name --target-dir /Sqoop/output --append --incremental append --check-column ID --last-value 100</P><P>while executing this command, it loading newly added rows in the existing directory but it shows duplicated records also.</P><P>how to avoid these duplicated records using sqoop?</P> Fri, 02 Jun 2017 13:38:03 GMT priyal 2017-06-02T13:38:03Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-deal-with-duplicate-values-while-importing-data-from/m-p/195326#M62211 <P>I am using ambari.</P><P>sqoop import --connect "jdbc:sqlserver://localhost:1433;database=db_name;username=user_name;pasword= password" --table table_name --target-dir /Sqoop/output --append --incremental append --check-column ID --last-value 100</P><P>while executing this command, it loading newly added rows in the existing directory but it shows duplicated records also.</P><P>how to avoid these duplicated records using sqoop?</P> Fri, 02 Jun 2017 13:38:03 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-deal-with-duplicate-values-while-importing-data-from/m-p/195326#M62211 priyal 2017-06-02T13:38:03Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-deal-with-duplicate-values-while-importing-data-from/m-p/195327#M62212 <P>Below posts might help:</P><P><A href="https://community.hortonworks.com/questions/63761/sqoop-import-to-hive-again-stroing-repeted-recorde.html" target="_blank">https://community.hortonworks.com/questions/63761/sqoop-import-to-hive-again-stroing-repeted-recorde.html</A></P><P><A href="https://community.hortonworks.com/questions/51508/sqoop-imported-more-records-than-source.html" target="_blank">https://community.hortonworks.com/questions/51508/sqoop-imported-more-records-than-source.html</A></P> Sat, 03 Jun 2017 03:46:25 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-deal-with-duplicate-values-while-importing-data-from/m-p/195327#M62212 namaheshwari 2017-06-03T03:46:25Z