Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

using flume with nifi

Hello All

I would like to use ExecuteFlumeSource Processor. I have a server that sends files using the Avro client of flume. I want to reads these files into NIFI. Has somebody already try to do this with NIFI 1.3.

I get an error in NIFI:

2017-09-21 15:51:41,862 ERROR [Timer-Driven Process Thread-10] o.a.n.p.flume.ExecuteFlumeSource ExecuteFlumeSource[id=64d53840-bec5-1945-8c16-b4710b5f37a7] ExecuteFlumeSource[id=64d53840-bec5-1945-8c16-b4710b5f37a7] failed to process session due to java.lang.AbstractMethodError: org.apache.avro.specific.SpecificFixed.getSchema()Lorg/apache/avro/Schema;: {}

java.lang.AbstractMethodError: org.apache.avro.specific.SpecificFixed.getSchema()Lorg/apache/avro/Schema;

at org.apache.avro.specific.SpecificFixed.<init>(SpecificFixed.java:36)

at org.apache.avro.ipc.MD5.<init>(MD5.java:16)

at org.apache.avro.ipc.Responder.<init>(Responder.java:73)

at org.apache.avro.ipc.generic.GenericResponder.<init>(GenericResponder.java:45)

at org.apache.avro.ipc.specific.SpecificResponder.<init>(SpecificResponder.java:55)

at org.apache.avro.ipc.specific.SpecificResponder.<init>(SpecificResponder.java:51)

at org.apache.avro.ipc.specific.SpecificResponder.<init>(SpecificResponder.java:43)

at org.apache.flume.source.AvroSource.start(AvroSource.java:230)

Regards,

Chris

2 REPLIES 2

Guru

@chris herssens

I haven't done this myself but it looks like someone has written a related article on HCC:

https://community.hortonworks.com/articles/48271/using-apache-flume-sources-and-sinks-with-apache-n....

That article used NiFi 0.7.0, but the configuration for the Flume processors should be the same in NiFi 1.3.0. Hoping you see something in the configuration screenshot that helps you out.

If I configure the processor for netcat then it works fine. The problem is if I want to use it with flume of type=avro