Skip to main content
shiva239
shiva239
Gold ⭐️
August 13, 2024
Question

Nifi consumer for Rabbitmq streams

Hi,

Currently we are using ConsumeAMQP processor to fetch messages from Rabbitmq queues.

There are some use cases in our system where Rabbitmq streams seem to be a good fit.

I have tested ConsumeAMQP to consume messages from Rabbitmq streams queue, but it doesn't work as expected.   I believe the new functionality related to OFFSET management is not available in ConsumeAMQP processor.

Any insights on this would be helpful.  Am I missing something?  If functionality is not part of current processor and would like it to be enhanced, how should I request?

Thanks in advance.

 

 

2 replies

joseomjr
joseomjr
Gold ⭐️⭐️
November 3, 2024

Kinda neat...didn't know RabbitMQ had introduced "stream" queues...I've always used Kafka for streaming in the past.  However, reading up on it I don't think NiFi has the native Java library/processors to use RabbitMQ streams.  However, the version of the amqp-client that ships with NiFi should be able to support connecting to a streams queue if set up correctly meaning it might be possible to create a custom Groovy processor.

https://www.rabbitmq.com/docs/streams

Translate
    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    November 4, 2024

    @shiva239 

    You can create an apache "NiFi" jira in the Apache community to highlight this new feature and request modification to existing processor to support it.

    https://issues.apache.org/jira/

    The more detail you provide in your jira the better chance someone might take this on in the community.  

    Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

    Thank you,
    Matt

    Translate
      Terms & ConditionsAccessibility statement