Support Questions
Find answers, ask questions, and share your expertise

sqoop incremental import to hive changes date automatically in the column

Expert Contributor

It has happened once before and now again did. Earlier it happened when I had restarted one of my servers and now when I replace a worker node with a new one.

Our database has dates stored in UTC. So, my sqoop job looks like this:

sqoop job -Dmapred.reduce.tasks=3 --meta-connect jdbc:hsqldb:hsql://fqdn:16000/sqoop --create sales_flat_order_live_incremental -- import --driver com.mysql.jdbc.Driver --connect 'jdbc:mysql://ip/db?zeroDateTimeBehavior=convertToNull&serverTimezone=IST' --username root --password 'password' --query "SELECT * FROM sales_flat_order WHERE ((customer_email NOT LIKE '' AND customer_email NOT LIKE '')|| customer_email IS NULL) AND \$CONDITIONS" --target-dir /user/hue/location --split-by entity_id --incremental lastmodified --last-value 0 --check-column updated_at --hive-drop-import-delims --null-string '\\N' --null-non-string '\\N' --fields-terminated-by '\001' --input-null-string '\\N' --input-null-non-string '\\N' --input-null-non-string '\\N' --input-fields-terminated-by '\001' --m 10

Now, it has automatically started showing created_at and updated_at column of the table with IST dates. HOw does that happen ? I am completely clueless..