- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to export parquet file to csv (without Hive)
- Labels:
-
Apache NiFi
-
Apache Spark
Created ‎07-25-2018 02:38 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I am developping a Nifi WebService to export dataLake content (stored as .parquet) as .csv.
I managed to do it using HiveQL Processor but I want to do it without Hive.
What I imagined was :
- get the .parquet file with WebHDFS (invokeHTTP call from nifi)
- use a nifi processor to convert the .parquet file to .csv
Is there a nifi Processor doing that? The only option I found for now is to use a spark job, which sounds a bit complicated for this purpose.
Thanks.
Created ‎07-25-2018 06:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Currently there is nothing OOTB that will parse Parquet files in NiFi, but I have written NIFI-5455 to cover the addition of a ParquetReader, such that incoming Parquet files may be able to be operated on as other supported formats are. As a workaround, there is a ScriptedReader where you could write your own in Groovy, Javascript, Jython, etc.
Created ‎08-14-2018 03:35 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you very much @Bryan Bende I need to insert each message into sql server. The database table has a clientno and jdonmessage field. I think I has to use Splitrecod to get the clientno from Json and insert the whole json record into the jsonmessage field.

- « Previous
-
- 1
- 2
- Next »