Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDF - Social Media (Non-Twitter) Examples

HDF - Social Media (Non-Twitter) Examples

Hi,

Does anyone have any examples/templates for using HDF to connect to a social media stream that isn't Twitter (e.g. Facebook, Google+, YouTube, Instagram, etc) ?

Thanks,

Andrew

8 REPLIES 8

Re: HDF - Social Media (Non-Twitter) Examples

Not at the moment, but I am working on a non-twitter example. I guess most of the examples use Twitter because there is already a Twitter Processor available out of the box. Dont think we have processors for Facebook, Google+, etc.

Are you just looking for Social Media examples or are you interested in IoT as well?

Re: HDF - Social Media (Non-Twitter) Examples

Correct, we don't have processors for FB, G+ and the likes. But people would certainly like them and until there are application level processors for those social media sites you will need to use lower level processors and create your own API calls to extract data. I just wasn't sure if anyone had an example of how to do that.

Re: HDF - Social Media (Non-Twitter) Examples

Andrew, each of these services has its own REST APIs. While there is a standard GetTwitter processor, you could just as soon use the GetHTTP processors and point them at the API of your choosing. See the attached screenshot for an example getting foreign exchange quotes from Yahoo's Finance REST API: http://finance.yahoo.com/webservice/v1/symbols/allcurrencies/quote?format=json


screen-shot-2015-11-19-at-60553-pm.png

Re: HDF - Social Media (Non-Twitter) Examples

But the advantage of the Twitter Processor is that its using the hbc-twitter4j library, which offers a lot of features to access and process the data from the Twitter hose, right?

Does the GetHTTP Processor support OAuth?

Re: HDF - Social Media (Non-Twitter) Examples

A typical challenge with all social media APIs are additional requirements around OAuth support, signing requests, etc. Pure REST API can get very tricky (still possible in many cases). In general that should be a starting point and evolve into a native processor using a library if warranted.

Highlighted

Re: HDF - Social Media (Non-Twitter) Examples

New Contributor

@Randy Gelhausen thanks for sharing the yahoo API. I think this one would be good example of a GetHTTP demo, if we can add some additional transformation pieces to it. Do you have any template to share for this?

Re: HDF - Social Media (Non-Twitter) Examples

Yep I am aware that you can use the social media website's REST APIs but I am looking for an example of someone who has actually done it. While Yahoo's Finance is a nice example, it's not a social media website and doesn't deal with any authentication.

Re: HDF - Social Media (Non-Twitter) Examples

Mentor

@Andrew Watson has this been resolved? Can you post your solution or accept best answer?

Don't have an account?
Coming from Hortonworks? Activate your account here