Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Key-Value Store Indexer for NRT with number of morphline configs

Solved Go to solution

Key-Value Store Indexer for NRT with number of morphline configs

Explorer

Dear Team,

 

Presently, I'm using Solr batch indexing on hbase tables to create solr documents. I have to enable NRT feature on 3 hbase tables. How to handle multiple morphlines for "Key-Value Store Indexer" service? I know, Morphlines file holds an array of 'id's, but how do we handle the SOLR_LOCATOR section where we have to specify the collection name? can we give a comma seperated list of collection names?

 

I'm using CDH 5.3.1 and CM 5.3.0.

 

Presently I have 3 morphline files say morphlineX.conf, morphlineY.conf and morphlineZ.conf

 

Thanks,

YBSNR

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Key-Value Store Indexer for NRT with number of morphline configs

Expert Contributor
The SOLR_LOCATOR is a variable that works via simple text substitution (ala unix shell scripts). You can define as many variables as you like within the same morphline config file. For example along these lines:

SOLR_LOCATOR_1 : { collection : collection1, zkHost : ${ZK_HOST} }
SOLR_LOCATOR_2 : { collection : collection2, zkHost : ${ZK_HOST} }

morphlines : [
{
id : morphline1
...
{ loadSolr { solrLocator : ${SOLR_LOCATOR_1} } }
}

{
id : morphline2
...
{ loadSolr { solrLocator : ${SOLR_LOCATOR_2} } }
}
]

Wolfgang

2 REPLIES 2
Highlighted

Re: Key-Value Store Indexer for NRT with number of morphline configs

Expert Contributor
The SOLR_LOCATOR is a variable that works via simple text substitution (ala unix shell scripts). You can define as many variables as you like within the same morphline config file. For example along these lines:

SOLR_LOCATOR_1 : { collection : collection1, zkHost : ${ZK_HOST} }
SOLR_LOCATOR_2 : { collection : collection2, zkHost : ${ZK_HOST} }

morphlines : [
{
id : morphline1
...
{ loadSolr { solrLocator : ${SOLR_LOCATOR_1} } }
}

{
id : morphline2
...
{ loadSolr { solrLocator : ${SOLR_LOCATOR_2} } }
}
]

Wolfgang

Re: Key-Value Store Indexer for NRT with number of morphline configs

Explorer

Thanks for the quick response.

 

After I enable Key-Value Store Indexer service and edit service wide morphline (textbox) config as you suggested above, should I maintain the morphlines.conf file in /etc/hbase-solr/conf directory where I have created for the batch indexing purpose?

 

In other words, if I create morphlinesX.conf, morphlinesY.conf and morphlinesZ.conf, should I update the service wide morphline configuration on KV Store Indexer also?

 

My observation is, when I have those 3 morphlines files in /etc/hbase-solr/conf directory, and enable KV Store indexer service with default configuration, the corresponding 3 collections are active and started generating solr index documents.

 

Further, after updating the KV Store indexer service --> configuration --> servicewide --> morphlines --> morphlines file (textbox), and deploy the client configuration, where it will be updated? Where could I verify the deployed configuraiton from CM admin console?

 

Please clarify.

Don't have an account?
Coming from Hortonworks? Activate your account here