Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can I fetch parquet file with timestamp column using NiFi FetchParquet processor? (INT96 not yet implemented)

How can I fetch parquet file with timestamp column using NiFi FetchParquet processor? (INT96 not yet implemented)

New Contributor

I'm receiving error "INT96 not yet implemented" while trying to fetch parquet file by NiFi using FetchParquet processor

Parquet file contains 1 column with timestamp. Here is Avro schema:
{ "namespace": "sna_project",
"name": "geo_subscriber",
"type": "record",
"fields": [
{ "name": "ecgi", "type": "string" },
{ "name": "cell_dir", "type": "int" },
{ "name": "service_processed_at", "type": "timestamp" },
{ "name": "year", "type": "int" },
{ "name": "month", "type": "int" },
{ "name": "day", "type": "int" }
]}

I tried to exchange timestamp datatype with other options, like:

- { "name": "service_processed_at", "type": "string" }

- { "name": "service_processed_at", "type": { "type" : "long", "logicalType" : "timestamp-millis"} }

etc.

56522-fetchparquet-dataflow.jpg

56523-fetchparquet-configuration.jpg

56524-fetchparquet-controllerservice.jpg

How can I read such parquet file using Nifi?

1 REPLY 1
Highlighted

Re: How can I fetch parquet file with timestamp column using NiFi FetchParquet processor? (INT96 not yet implemented)

New Contributor

Hi,after these days,I found something;

NIFI use the Apache parquet-avro to parse the parquet file;

Unfortunately,INT96 is not yet implemented in the lastest version of parquet-avro;

99433-qq图片20190127162146.png