- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How can I fetch parquet file with timestamp column using NiFi FetchParquet processor? (INT96 not yet implemented)
- Labels:
-
Apache NiFi
Created on ‎01-30-2018 03:34 PM - edited ‎08-17-2019 11:10 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm receiving error "INT96 not yet implemented" while trying to fetch parquet file by NiFi using FetchParquet processor
Parquet file contains 1 column with timestamp. Here is Avro schema:
{
"namespace": "sna_project",
"name": "geo_subscriber",
"type":
"record",
"fields":
[
{ "name": "ecgi", "type":
"string" },
{ "name": "cell_dir", "type":
"int" },
{ "name": "service_processed_at",
"type": "timestamp" },
{ "name": "year", "type":
"int" },
{ "name": "month", "type":
"int" },
{ "name": "day", "type":
"int" }
]}
I tried to exchange timestamp datatype with other options, like:
- { "name": "service_processed_at", "type": "string" }
- { "name": "service_processed_at", "type": { "type" : "long", "logicalType" : "timestamp-millis"} }
etc.
How can I read such parquet file using Nifi?
Created on ‎01-27-2019 08:22 AM - edited ‎08-17-2019 11:10 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,after these days,I found something;
NIFI use the Apache parquet-avro to parse the parquet fileï¼›
Unfortunately,INT96 is not yet implemented in the lastest version of parquet-avro;
