Support Questions

Find answers, ask questions, and share your expertise
Welcome to the upgraded Community! Read this blog to see What’s New!

Best Practice - JSON to Avro, data type preserving

Super Collaborator

Hi All, What is the best approach to convert a JSON to AVRO preserving the source datatypes. My source JSON has a field with timestamp (value would look like 2017-01-26T00:00:00-05:00) which I need to eventually insert to a hive table with column type timestamp. When I infer the schema, I get String for the timestamp field. Is there some pre-formatting that I can do on the timestamp field so that it gets inferred as timestamp field. Current flow is as below - JSON>>AVRO(infer/manually add schema)>>Streaming Insert to hive



Hi @Arun A K

This is a known issue where the datatypes are not preserved. which talks about Oracle/SQL datatypes not being preserved. You should also check out

As mentioned by @Karthik Narayanan, best approach would be to use Record Oriented processors.