Support Questions

Find answers, ask questions, and share your expertise

NiFi 2.5.0 missing parquet integration

avatar
Contributor

Hi guys,

i've just started to work with parquet files, all is running with database own export logics, but they are not traceable - use NiFi was the Idea. 
Now im just annoyed how bad i am to handle this ... seems no default export available for this, install extensible-bundles ... NoClass here and there etc...

Did anybody of you solved to add Parquet to NiFi 2.5.0?

I've downloaded and provided nifi-parquet-nar-2.5.0.nar, nifi-hadoop-nar-2.5.0.nar and nifi-hadoop-libraries-nar-2.5.0.nar still NoClassDefFoundErrors in this order of log (single named)
- org/apache/nifi/serialization/RecordSetWriterFactory
- org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord
- org/apache/nifi/processors/hadoop/AbstractPutHDFSRecord
- org/apache/nifi/serialization/RecordReaderFactory
- org/apache/nifi/serialization/RecordSetWriterFactory
- org/apache/parquet/io/OutputFile
- org/apache/parquet/io/InputFile

Anybody who can helpt me?


3 REPLIES 3

avatar
Expert Contributor

Hello @mbraunerde

I see you mentioned that you're using NiFi 2.5.0, I think that version is not provided for Cloudera on the CFM right? 
Even the most recent CFM does not have NiFi 2.5.0, the latest is CFM 4.10 with NiFi 2.3.0. 

I ask because the Cloudera provided CFM do have Parquet already included: PutParquet
https://docs.cloudera.com/cfm/4.10.0/release-notes/topics/cfm-supported-processors.html

Now, if you want to add it on a custom NiFi install, you should import those NAR already loaded but also you need nifi-standard-services-api-nar and nifi-record-serialization-services-nar. 
You can take a look here: 
https://mvnrepository.com/artifact/org.apache.nifi/nifi-standard-services-api-nar 
https://mvnrepository.com/artifact/org.apache.nifi/nifi-record-serialization-services-nar 


Regards,
Andrés Fallas
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs-up button.

avatar
Contributor

Hi @vafs ,

>  I think that version is not provided for Cloudera on the CFM right

your right, but this platform was the best one so far for all NiFi questions 🤙

I will update this thread, if i got a solution, either if Cloudera or not

avatar
Expert Contributor

Understood. 

Hopefully the missing NAR's pointed on the previous update help you figure the issue. 


Regards,
Andrés Fallas
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs-up button.