I have an set of urls in a file. sample file: localhost:1234/aaaa/uyt localhost:1234/bbbb/grt localhost:1234/cccc/hgd etc.. how can I pass this URL dynamically from that file to invoke HTTP processor. ... View more

I using HDP 2.5 and I add a NIFI service in it, I need to send a data to Elasticsearch 5.X, when i add a processor in NIFI, the putelasticsearch5 processor is missing but it has only putelasticsearch(it supports only elasticsearc 2.X version).How can I add the Putelasticsearch5 processor in nifi. ... View more

i have a data in my HDFS,my date format in that file is Tue Dec 20 10:04:31 +0000 2016,when i create a hive table with field name created time and datatype date it store a null value. how to change that date formate in hive? ... View more

i get the twitter data and put those data into hdfs using nifi,it gives a data,but my date formate is in Tue Dec 20 10:04:31 +0000 2016 and i create a hive table with those twitter data.when i create that table i have created_time field and i need to set datatype as date,it shows an null value then i change the datatype as string it shows an value.then i need to transfer a that table into elasticsearch using es-hadoop but i confused what datatype i am going to use as time field. i tried string to that time filed,it shows a value but when i visualize that field in kibana,in show an error a date type is required, my query is given below: i also tried ->change my created_time field as date but i shows a null value create table if not exists tweets_text_partition( tweet_id bigint, created_unixtime bigint, created_time string, displayname string, msg string, fulltext string ) row format delimited fields terminated by "|" location "/tmp/tweets_staging"; ******************************************************************* curl -XPUT http://sandbox.hortonworks.com:9200/twitter_text1/?pretty -d'{"rels":{"properties":{"tweet_id":{"type":"bigint"},"created_unixtime":{"type":"bigint"},"created_time":{"type":"string"},"displayname":{"type":"string"},"msg":{"type":"string"},"fulltext":{"type":"string"}}}}' *********************************************************************** ADD JAR /root/ elasticsearch-hadoop-5.1.1.jar; ADD JAR /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar; ********************************************************************************** CREATE EXTERNAL TABLE test.twitter_es1 ( tweet_id bigint, created_unixtime bigint, created_time string, displayname string, msg string, fulltext string ) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES ( 'es.resource' = 'twitterdata/data', 'es.nodes'='sandbox.hortonworks.com:9200', 'es.index.auto.create' = 'false', 'es.mapping.id'='tweet_id'); ************************************************************* INSERT OVERWRITE TABLE test.twitter_es1 SELECT tweet_id, created_unixtime, created_time, displayname, msg, fulltext FROM test.tweets_hive1; ... View more

i am new to mongodb.plz share some tutorial links or set-up guide how to load a twitter data into mongodb and then transfer those datas into hive. i use hortonworks 2.4 in vm 5.0.20 ... View more