Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Expression language for Kafka Broker of PublishKafka_0_10 processor

avatar
Explorer

Hello,

I use a HandleHttpRequest to get a POST request with 2 parameters (kafkaHost and kafkaPort)

I can see my param in the attribute of the data (http.param.kafkaHost and http.param.kafkaPort)

I want to use them in a PublishKafka_0_10 to define kafka Brokers propertie (support expression language)

like this : Kafka Brokers = ${http.param.kafkaHost}:${http.param.kafkaPort}

It's doesn't works, my attributes aren't use (log file : bootstrap.servers = [:])

There is a solution ?

1 ACCEPTED SOLUTION

avatar
Super Mentor
@Sylvain Robert

The PublishKafka processor establishes a connection to the brokers before it even looks at the attributes of a FlowFile. This way it does not need to establish a new connection for every single FlowFile. While the broker property does not support FlowFile attributes, it should support resolving Expression Language statements that pull values from environment variables or the NiFi variable registry (File on disk with key value pairs).

Thank you,

Matt

View solution in original post

5 REPLIES 5

avatar
Super Mentor
@Sylvain Robert

The PublishKafka processor establishes a connection to the brokers before it even looks at the attributes of a FlowFile. This way it does not need to establish a new connection for every single FlowFile. While the broker property does not support FlowFile attributes, it should support resolving Expression Language statements that pull values from environment variables or the NiFi variable registry (File on disk with key value pairs).

Thank you,

Matt

avatar
Explorer

Thanks Matt, i will find another way so.

avatar
Explorer

Hello @sy_robert,

 

were you able to resolve this on how to dynamically assign brokers list from a flowfile attributes?

 

avatar
Community Manager

@sivat as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Explorer

@VidyaSargur thank you for the recommendation.

I already started a new thread https://community.cloudera.com/t5/Support-Questions/trying-to-assign-JMS-Broker-URI-property-value-a... which is almost the same question as this thread.

 

Just wanted to check if @sy_robert found any workaround over this.