Support Questions
Find answers, ask questions, and share your expertise

Can we ingest Rss feeds using apache nifi into hive?

Contributor

Any thoughts on which processors to use in nifi for ingesting rss feeds into hive tables.

http://feeds.bbci.co.uk/news/world/rss.xml

thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

@surender nath reddy kudumula

Use this GetHTTP

http://www.nifi.rocks/getting-started-with-apache-nifi/

The URL we are going to grab a file from is XKCD’s rss feed, rss.xml. The GetHTTP procesor is simple to configure and just needs the URL property to be set to http://xkcd.com/rss.xml. Click on the value across from the URL property and enter http://xkcd.com/rss.xml.

Now drag down another processor, EvaluateXPath. Under the properties for this processor, set the following property-value pairs:

  • Destination - flowfile-attribute
  • Return Type - auto-detect

View solution in original post

7 REPLIES 7

@surender nath reddy kudumula

Use this GetHTTP

http://www.nifi.rocks/getting-started-with-apache-nifi/

The URL we are going to grab a file from is XKCD’s rss feed, rss.xml. The GetHTTP procesor is simple to configure and just needs the URL property to be set to http://xkcd.com/rss.xml. Click on the value across from the URL property and enter http://xkcd.com/rss.xml.

Now drag down another processor, EvaluateXPath. Under the properties for this processor, set the following property-value pairs:

  • Destination - flowfile-attribute
  • Return Type - auto-detect

View solution in original post

Contributor

Thank you @Neeraj Sabharwal. I ll give this a try. Why do we need EvaluateXPath here? Also i beleive at the end use puthdfs to route the data into hdfs. I beleive currently nifi doesnt have a processor to ingest directly into hive table i can see it has putsql. So i beleive best approach is to use puthdfs. Please let me know thanks..

@surender nath reddy kudumula I will put data in HDFS and create external Hive table on top of it.

Contributor

thank you @Neeraj Sabharwal

@surender nath reddy kudumula As part of the best practice, please accept the best answer to close the thread.

Super Collaborator

@surender nath reddy kudumula - There is a JIRA actively being worked on to add Hive JDBC support to Nifi. https://issues.apache.org/jira/browse/NIFI-981

Neeraj, thanks for your advice.