Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

I want to ingest data from flume into hive table and then connect that hive table to elastic search? can anyone expalin me the steps and the format which i can use.

I want to ingest data from flume into hive table and then connect that hive table to elastic search? can anyone expalin me the steps and the format which i can use.

 
1 REPLY 1
Highlighted

Re: I want to ingest data from flume into hive table and then connect that hive table to elastic search? can anyone expalin me the steps and the format which i can use.

Contributor

Hi,

Sure this is a cool one:

First you have to download the JAR FILES required for Elastic

 https://www.elastic.co/downloads/hadoop

next, you can create a Hive table that uses Elastic Index as a storage as follows:

add jar /usr/hdp/2.3.2.0-2950/hive/lib/commons-httpclient-3.0.1.jar;  //try to look for the 
add jar  /usr/hdp/jars/elasticsearch-hadoop-hive-2.2.0-beta1.jar;
drop table traffic_view_elastic;
create external table traffic_view_elastic(

time timestamp,
id	int,
name	string
     )
     STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
     TBLPROPERTIES('es.resource' = 'hive/myUseCase',
     'es.index.auto.create' = 'true',
     'es.nodes' = 'elasticnode-hostname-or-ip-address');

once this is done, you can go to Elastic and check out the newly created index (After the data been insterted into this table).

Hope this works for you, appreciate if you mark this as an accepted answer if it does.

thanks,

Ned

Don't have an account?
Coming from Hortonworks? Activate your account here