Support Questions

Find answers, ask questions, and share your expertise

How can I fetch parquet file with timestamp column using NiFi FetchParquet processor? (INT96 not yet implemented)

avatar
New Contributor

I'm receiving error "INT96 not yet implemented" while trying to fetch parquet file by NiFi using FetchParquet processor

Parquet file contains 1 column with timestamp. Here is Avro schema:
{ "namespace": "sna_project",
"name": "geo_subscriber",
"type": "record",
"fields": [
{ "name": "ecgi", "type": "string" },
{ "name": "cell_dir", "type": "int" },
{ "name": "service_processed_at", "type": "timestamp" },
{ "name": "year", "type": "int" },
{ "name": "month", "type": "int" },
{ "name": "day", "type": "int" }
]}

I tried to exchange timestamp datatype with other options, like:

- { "name": "service_processed_at", "type": "string" }

- { "name": "service_processed_at", "type": { "type" : "long", "logicalType" : "timestamp-millis"} }

etc.

56522-fetchparquet-dataflow.jpg

56523-fetchparquet-configuration.jpg

56524-fetchparquet-controllerservice.jpg

How can I read such parquet file using Nifi?

1 REPLY 1

avatar
New Contributor

Hi,after these days,I found something;

NIFI use the Apache parquet-avro to parse the parquet file;

Unfortunately,INT96 is not yet implemented in the lastest version of parquet-avro;

99433-qq图片20190127162146.png