Support Questions
Find answers, ask questions, and share your expertise

Nifi not streaming twitter data

Re: Nifi not streaming twitter data

Explorer

@Wynner Yes, Here is the screenshot of what it shows

39868-screenshot-26.png

Re: Nifi not streaming twitter data

@Elvine Egoko

Are you able to ping this address, stream.twitter.com, from the system running NiFi?

Re: Nifi not streaming twitter data

@Elvine Egoko

That sounds like the issue then. You need to be able to ping that address before you'll be able get twitter data. Check the config of your VPN and/or firewall.

Re: Nifi not streaming twitter data

@Elvine Egoko

Try turning off the windows firewall completely, and then see if the GetTwitter processor can pull data.

Re: Nifi not streaming twitter data

@Elvine Egoko

What does the configuration of the GetTwitter processor look like?

Re: Nifi not streaming twitter data

@Elvine Egoko

You are running NiFi on a Windows 10 system correct?

Also, just install NiFi directly, not via the sandbox.

Re: Nifi not streaming twitter data

Explorer

Hello @Wynner, I did the changes as you said, I discovered that there is this error too


screenshot-16.png

Re: Nifi not streaming twitter data

Explorer

@Wynner, I can't ping it successfully

Re: Nifi not streaming twitter data

Explorer

@Wynner, I first used the wrong url http://stream.twitter.com on command line

39896-screenshot-36.png

. After which I ping using stream.twitter.com

, after I used https://stream.twitter.com on the browser. It required some credentials

, so I entered my twitter credentials. After I entered it, it returned on the browser unknown URL and a link to follow http://dev.twitter.com/pages/streaming_api. But the link does not exist. The screenshots are attached below. Also a scrennshot of vpn and firewall config

39894-screenshot-32.png

39895-screenshot-34.png

39891-screenshot-29.png

39892-screenshot-30.png

Re: Nifi not streaming twitter data

Explorer

@Elvine Egoko can you please ssh into your virtual machine

ssh root@localhost -p 12222
password: hadoop

Then grep the Nifi logs

cat /var/log/nifi/nifi-app.log | grep "ERROR"

See if you can find a more detailed explanation of that error.

Also, I noticed you were using a default template from the Nifi wiki: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates

I tried to replicate the error on my end with the same template and processor to no avail. I am using the latest HDF sandbox with virtual box.

I did have to manually enter my own twitter API keys and access tokens with my application account . You can make one here: https://apps.twitter.com/

Do you have an account? Did you try entering in a new set of keys?

What do your processor properties look like?

39924-screen-shot-2017-10-23-at-23119-pm.png