Created on 01-10-2014 10:58 AM - edited 09-16-2022 01:52 AM
In the cloudera manager/flume configuration, I did the following for the agent
Specified the Flume-NG solrSink contents -- copy/pasted my morphline configuration into that editor.
In the flume configuration, specified the sink as solrSink -- for the solrSink, I just specified the morphlineId and removed the morphlineFile, since I had pasted my morphline configuration into the Flume-NF Solr Sink section.
I get missing parameter 'morphlineFile' error. What value should I have for morphlineFile in the flume configuration for solrSink.
You can use the $ZK_HOST param *inside of the* Morphlines Config file, not *as the* morphlines config file.
Created 01-14-2014 03:02 PM
Agreed and I understand the use of $ZK_HOST inside Morphlines File.
Earlier what I was missing was what value should I use for the morphlineFile property in flume configuration (becuase I am specifying the morphline configuration in the Morphlines File property under Flume-NG Solr Sink), but re-reading the "Using Flume with Search" pointed me in the right direction -- I had to use morphlines.conf as the value for the morphlineFile property
Thanks for your help
Created 01-14-2014 01:03 PM
Hi Kannan,
Can you please try setting the morphlineFile property in your flume config = $ZK_HOST ?
e.g.
a1.sinks.k1.morphlineFile = $ZK_HOST
-Jeff
Created 01-14-2014 01:19 PM
Created 01-14-2014 02:21 PM
The referenced section 'Using Flume with Search' talks about using $ZK_HOST in the Morphlines File property instead of the zookeeper quorum and ClouderaManager will resolve it automatically.
But in the Flume conf, if I specify the morphlineFile for the solrsink as $ZK_HOST and when I restart flume in CM, it returns an error.
File Not Found : $ZK_HOST
You can use the $ZK_HOST param *inside of the* Morphlines Config file, not *as the* morphlines config file.
Created 01-14-2014 03:02 PM
Agreed and I understand the use of $ZK_HOST inside Morphlines File.
Earlier what I was missing was what value should I use for the morphlineFile property in flume configuration (becuase I am specifying the morphline configuration in the Morphlines File property under Flume-NG Solr Sink), but re-reading the "Using Flume with Search" pointed me in the right direction -- I had to use morphlines.conf as the value for the morphlineFile property
Thanks for your help