Reply
New Contributor
Posts: 1
Registered: ‎10-31-2016

Solr indexing on hive table

Team,

 

we are planning to index hive tables in cloudera solr to find the relative tables using data search. we don’t find any documents in cloudera site for this setup. we could see some generic document  from below link for how to index hive tables using solr. but the problem is we need to build the JAR with third party tool Gradle and also we are not sure it will support cloudera solr or not.

 

https://github.com/lucidworks/hive-solr

 

Could you please guide me how to index hive tables in cloudera solr. Thanks

Cloudera Employee
Posts: 198
Registered: ‎01-09-2014

Re: Solr indexing on hive table

What format are your hive tables? The MapReduceIndexerTool/morphlines can read the hive table files and import them into solr, depending on the format of those hive tables.

-pd
Explorer
Posts: 15
Registered: ‎09-01-2015

Re: Solr indexing on hive table

I am also having the same scenario, I am not able to find any documents from cloudera regarding this. If you got the solution please share.
Explorer
Posts: 15
Registered: ‎09-01-2015

Re: Solr indexing on hive table

I am trying to create the SOLR indexing on hive views. How can I achieve it? Is there is any document related to it please share. I went through this document (https://www.cloudera.com/documentation/enterprise/5-8-x/topics/search_batch_index_use_mapreduce.html) but it is not helpful in my case.
Expert Contributor
Posts: 94
Registered: ‎02-15-2016

Re: Solr indexing on hive table

did anyone find out a way to index hive table .

Explorer
Posts: 19
Registered: ‎02-15-2016

Re: Solr indexing on hive table

Hi,

 

Has anyone found a working solution so far?

 

Regard,

MG

Highlighted
Posts: 153
Topics: 8
Kudos: 15
Solutions: 16
Registered: ‎07-16-2015

Re: Solr indexing on hive table

[ Edited ]

You have basicaly two options :

- either the file format is simple enough and you can index it directly using the MapReduceIndexerTool as suggested by pdvorak (you access the file directly)

 

- either the file format is too complicated (or dynamic) and then you need to code your own indexer that will run the query on hive, get the result and then push it to solr.

Explorer
Posts: 19
Registered: ‎02-15-2016

Re: Solr indexing on hive table

So, the takeaway is that there isn't an official indexer (just like for mysql) for Hive tables. 

 

Is it possible to see it in the upcoming future? or Does it even make sense?

 

I mean, I see a clear use case behind that. If Solr can index Hive tables, it would become so easy to make your Hadoop data searchable.

 

Regards,

MG

Posts: 153
Topics: 8
Kudos: 15
Solutions: 16
Registered: ‎07-16-2015

Re: Solr indexing on hive table

You should look at this : https://chimpler.wordpress.com/2013/03/20/playing-with-apache-hive-and-solr/

 

The content seems to be what you are looking for.

I have not tested it myself.

 

regards,

Mathieu

Explorer
Posts: 19
Registered: ‎02-15-2016

Re: Solr indexing on hive table

Hi Mathieu,

 

Thanks for sharing that link; I will test with it.

 

However, I am a little skeptical about deploying it in production even if it works.

 

Does Cloudera have any plans to develop and release such a connector/handler/library?

 

As I mentioned previously, this seems to be a valid use case for allowing users to be able to search through Hive tables.

 

Regards,

MG

Announcements