Flume configuration -- solrSink MorphlineFile

[#1]

In the cloudera manager/flume configuration, I did the following for the agent

  Specified the Flume-NG solrSink contents -- copy/pasted my morphline configuration into that editor.

  In the flume configuration, specified the sink as solrSink -- for the solrSink, I just specified the morphlineId and removed the morphlineFile, since I had pasted my morphline configuration into the Flume-NF Solr Sink section.

I get missing parameter 'morphlineFile' error.  What value should I have for morphlineFile in the flume configuration for solrSink.

[#2 · Accepted solution]

You can use the $ZK_HOST param *inside of the* Morphlines Config file, not *as the* morphlines config file.

[#3]

Hi Kannan,

Can you please try setting the morphlineFile property in your flume config = $ZK_HOST ?

e.g.

a1.sinks.k1.morphlineFile = $ZK_HOST

-Jeff

[#4]

https://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Manag...

[#5]

The referenced section 'Using Flume with Search' talks about using $ZK_HOST in the Morphlines File property instead of the zookeeper quorum and ClouderaManager will resolve it automatically.

But in the Flume conf, if I specify the morphlineFile for the solrsink  as $ZK_HOST and when I restart flume in CM, it returns an error.

   File Not Found : $ZK_HOST

[#6 · Accepted solution]

You can use the $ZK_HOST param *inside of the* Morphlines Config file, not *as the* morphlines config file.

[#7 · Accepted solution]

Agreed and I understand the use of $ZK_HOST inside Morphlines File.

Earlier what I was missing was what value should I use for the morphlineFile property in flume configuration (becuase I am specifying the morphline configuration in the Morphlines File property under Flume-NG Solr Sink), but re-reading the "Using Flume with Search" pointed me in the right direction -- I had to use morphlines.conf as the value for the morphlineFile property

Thanks for your help