NiFi 2.5.0 missing parquet integration

Question

Hi guys,i've just started to work with parquet files, all is running with database own export logics, but they are not traceable - use NiFi was the Idea. Now im just annoyed how bad i am to handle this ... seems no default export available for this, install extensible-bundles ... NoClass here and there etc...Did anybody of you solved to add Parquet to NiFi 2.5.0?I've downloaded and provided nifi-parquet-nar-2.5.0.nar, nifi-hadoop-nar-2.5.0.nar and nifi-hadoop-libraries-nar-2.5.0.nar still NoClassDefFoundErrors in this order of log (single named)- org/apache/nifi/serialization/RecordSetWriterFactory- org/apache/nifi/processors/hadoop/AbstractFetchHDFSRecord- org/apache/nifi/processors/hadoop/AbstractPutHDFSRecord- org/apache/nifi/serialization/RecordReaderFactory- org/apache/nifi/serialization/RecordSetWriterFactory- org/apache/parquet/io/OutputFile- org/apache/parquet/io/InputFileAnybody who can helpt me?

vafs · Answer

Hello @mbraunerde,

I see you mentioned that you're using NiFi 2.5.0, I think that version is not provided for Cloudera on the CFM right?
Even the most recent CFM does not have NiFi 2.5.0, the latest is CFM 4.10 with NiFi 2.3.0.

I ask because the Cloudera provided CFM do have Parquet already included: PutParquet
https://docs.cloudera.com/cfm/4.10.0/release-notes/topics/cfm-supported-processors.html

Now, if you want to add it on a custom NiFi install, you should import those NAR already loaded but also you need nifi-standard-services-api-nar and nifi-record-serialization-services-nar.
You can take a look here:
https://mvnrepository.com/artifact/org.apache.nifi/nifi-standard-services-api-nar
https://mvnrepository.com/artifact/org.apache.nifi/nifi-record-serialization-services-nar

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.