Can we ingest Rss feeds using apache nifi into hive?

[#1]

Any thoughts on which processors to use in nifi for ingesting rss feeds into hive tables.

http://feeds.bbci.co.uk/news/world/rss.xml

thanks.

[#2 · Accepted solution]

@surender nath reddy kudumula

Use this GetHTTP

http://www.nifi.rocks/getting-started-with-apache-nifi/

The URL we are going to grab a file from is XKCD’s rss feed, rss.xml. The GetHTTP procesor is simple to configure and just needs the URL property to be set to http://xkcd.com/rss.xml. Click on the value across from the URL property and enter http://xkcd.com/rss.xml.

Now drag down another processor, EvaluateXPath. Under the properties for this processor, set the following property-value pairs:

• Destination - flowfile-attribute
• Return Type - auto-detect

[#3]

Thank you @Neeraj Sabharwal. I ll give this a try. Why do we need EvaluateXPath here? Also i beleive at the end use puthdfs to route the data into hdfs. I beleive currently nifi doesnt have a processor to ingest directly into hive table i can see it has putsql. So i beleive best approach is to use puthdfs. Please let me know thanks..

[#4]

@surender nath reddy kudumula I will put data in HDFS and create external Hive table on top of it.

[#5]

thank you @Neeraj Sabharwal

[#6]

@surender nath reddy kudumula As part of the best practice, please accept the best answer to close the thread.

[#7]

@surender nath reddy kudumula - There is a JIRA actively being worked on to add Hive JDBC support to Nifi. https://issues.apache.org/jira/browse/NIFI-981

[#8]

Neeraj, thanks for your advice.