NIFI Puthive3streaming Timestamp Number format exception

[#1]

NIFI Version  : 1.12

Iam able to load the data using PutHive3Streaming processor for tables without timestamp datatype. However if the Hive table contains Timestamp field then the load fails with below error.

WARN [Timer-Driven Process Thread-7] o.a.n.processors.hive.PutHive3Streaming PutHive3Streaming[id=01751042-d64d-1d24-cd20-5b2a9e1ea92b] Error [java.lang.NumberFormatException: For input string: "2018-10-20 21:30:03.493"]

Any help here is highly appreciated.

[#2]

Solution using NiFi 1.11.4

I had this same issue when trying to use an ExecuteSQL processor to get the data and then using a PutHive3Streaming processot to put the data into a table. The way you need to fix this is in the configuration for the ExecuteSQL processor, set the "Use Avro Logical Types" property to true. This has the Avro file store the timestamps as an actual timestamp instead of using a string.

[#3]

It seems the avro schema doesnt match with the table structure. E.g it is string in your flowfile, it is string in the destination table as well? If not, you will get this error. The probable reason is that PutHiveStreaming doesnt implicitly change data types.

If you find the answer helpful please accept this as a solution.

[#4]

@Althotta, Has any of the replies in the post helped you resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.