question Re: Flume configuration -- solrSink MorphlineFile in Archives of Support Questions (Read Only)

Flume configuration -- solrSink MorphlineFile

kannan — Fri, 16 Sep 2022 08:52:15 GMT

In the cloudera manager/flume configuration, I did the following for the agent

Specified the Flume-NG solrSink contents -- copy/pasted my morphline configuration into that editor.

In the flume configuration, specified the sink as solrSink -- for the solrSink, I just specified the morphlineId and removed the morphlineFile, since I had pasted my morphline configuration into the Flume-NF Solr Sink section.

I get missing parameter 'morphlineFile' error. What value should I have for morphlineFile in the flume configuration for solrSink.

Re: Flume configuration -- solrSink MorphlineFile

Jeff Lord — Tue, 14 Jan 2014 21:03:48 GMT

Hi Kannan,

Can you please try setting the morphlineFile property in your flume config = $ZK_HOST ?

e.g.

a1.sinks.k1.morphlineFile = $ZK_HOST

-Jeff

Re: Flume configuration -- solrSink MorphlineFile

Jeff Lord — Tue, 14 Jan 2014 21:19:59 GMT

https://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_adding_search_solr.html

Re: Flume configuration -- solrSink MorphlineFile

kannan — Tue, 14 Jan 2014 22:21:47 GMT

The referenced section 'Using Flume with Search' talks about using $ZK_HOST in the Morphlines File property instead of the zookeeper quorum and ClouderaManager will resolve it automatically.

But in the Flume conf, if I specify the morphlineFile for the solrsink as $ZK_HOST and when I restart flume in CM, it returns an error.

File Not Found : $ZK_HOST

Re: Flume configuration -- solrSink MorphlineFile

Jeff Lord — Tue, 14 Jan 2014 23:22:55 GMT

You can use the $ZK_HOST param *inside of the* Morphlines Config file, not *as the* morphlines config file.

Re: Flume configuration -- solrSink MorphlineFile

kannan — Tue, 14 Jan 2014 23:02:41 GMT

Agreed and I understand the use of $ZK_HOST inside Morphlines File.

Earlier what I was missing was what value should I use for the morphlineFile property in flume configuration (becuase I am specifying the morphline configuration in the Morphlines File property under Flume-NG Solr Sink), but re-reading the "Using Flume with Search" pointed me in the right direction -- I had to use morphlines.conf as the value for the morphlineFile property

Thanks for your help