Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Convert sqoop job upper bound and lower bound values to timestamp

Solved Go to solution

Convert sqoop job upper bound and lower bound values to timestamp

Contributor

I'm having an issue I was hoping someone could help me with. I'll try and explain as best as I can.

I'm currently trying to run an incremental sqoop job which pulls new data down based on when the job was last run. To do this I have used the incremental append option. This sqoop job also contains a free form query which is used to clean up columns containing additional whitespaces etc. The problem I am having is that I am doing the incremental import based on a timestamp column and the lower bound and upper bound values being generated by the sqoop job are not of a timestamp value so I am recieving type errors whenever I try and run the job. My question is, is there a way to insert a to_timestamp function around the lower bound and upper bound values so that the values are in the correct format when they are comparred to the timestamp column?

Some example code below:

Current output

WHERE ts > '2014-08-10 10:09:36.094' AND ts <= '2016-03-05 10:09:36.094'

Desired output

WHERE ( ts > TO_TIMESTAMP('2014-08-10 10:09:36.094', 'YYYY-MM-DD HH24:MI:SS.FF') AND ts <= 
TO_TIMESTAMP('2016-03-05 10:09:36.094', 'YYYY-MM-DD HH24:MI:SS.FF') )
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Convert sqoop job upper bound and lower bound values to timestamp

Contributor

I've resolved this issue, turned out the problem came from a driver that was being used

--driver oracle.jdbc.OracleDriver

Removed the driver, problem solved!

2 REPLIES 2

Re: Convert sqoop job upper bound and lower bound values to timestamp

Contributor

I should also note that the desired output appears if I use --table instead of --query, however --query is neccersary in order to tidy up the data.

Highlighted

Re: Convert sqoop job upper bound and lower bound values to timestamp

Contributor

I've resolved this issue, turned out the problem came from a driver that was being used

--driver oracle.jdbc.OracleDriver

Removed the driver, problem solved!