Member since
05-15-2018
7
Posts
1
Kudos Received
0
Solutions
05-18-2018
10:59 AM
I'm using Nifi 1.5. I don't have the create statement for the source (it's an oracle view) but the field in question is a TIMESTAMP(6) in the source DB. The target DB is Impala; the create table for that is create table test
( id STRING, my_value DOUBLE, time_inserted TIMESTAMP, time_updated TIMESTAMP)
STORED AS PARQUET
LOCATION '/blah/test'; As the Avro input fields are in uppercase, the PutDatabaseRecord processor is using Translate Field Names and I've also set it to quote both column and table identifiers. Unmatched fields/columns are set to ignore. I'll have a try later with 1.6 and see if that solves it, or if not will have a go at a 1.7 build. In the meantime I've split it out into two separate QueryDatabaseTable processors, one for null and one for not null (luckily time_updated is the only nullable timestamp!) Thanks, Toby
... View more
05-23-2018
03:49 PM
I left a response on my other answer but will leave it here too in case you hadn't seen it: Looking at the parquet-avro code, I think your suggestion of the workaround to change decimal values to fixed is the right approach (for now). We could update the version of parquet-avro but I didn't see anything in there that would improve your situation, it was Impala that needed to support more incoming types.
... View more