Nifi not streaming twitter data

[#1]

Hello, I have install HDP2.6 on virtualbox, I configured GetTwitter processor in Nifi and I started the streaming but I got connection error. I went through this link but could not resolve the problem, Here is a screenshot of my error. I am using Hortonwork for my final year project and I am new in the community.

[#2]

@Elvine Egoko

Are you running the processor in a cluster? How many concurrent tasks and what is the run schedule of the processor?

[#3]

I followed this tutorial, I just started with Hortonwork

[#4]

@Elvine Egoko

Change the Run Schedule to a value greater than 0 sec in the GetTwitter processor. I recommend trying 15 secs.

The Scheduling tab is where the property is located

[#5]

@Elvine Egoko

That is a completely different issue. That looks like a port isn't opened. Look on the system where your ambari metrics process is running and make sure that port is open on that system.

[#6]

Concerning the first issues I have change the schedulint time but it still do not stream data. I don't kn ow what the possible solution.

[#7]

@Elvine Egoko

Make sure and set the Run Duration to 0ms also.

[#8]

@Wynner It gives the same error. I wish to precise that, I am using a VPN application in order to access Twitter

[#9]

Please is there something else I could do?

[#10]

@Elvine Egoko

Do you see any bulletins on the GetTwitter processor?

[#11]

@Wynner Yes, Here is the screenshot of what it shows

[#12]

@Elvine Egoko

Are you able to ping this address, stream.twitter.com, from the system running NiFi?

[#13]

@Elvine Egoko

That sounds like the issue then. You need to be able to ping that address before you'll be able get twitter data. Check the config of your VPN and/or firewall.

[#14]

@Elvine Egoko

Try turning off the windows firewall completely, and then see if the GetTwitter processor can pull data.

[#15]

@Elvine Egoko

What does the configuration of the GetTwitter processor look like?

[#16]

@Elvine Egoko

You are running NiFi on a Windows 10 system correct?

Also, just install NiFi directly, not via the sandbox.

[#17]

Hello @Wynner, I did the changes as you said, I discovered that there is this error too

[#18]

@Wynner, I can't ping it successfully

[#19]

@Wynner, I first used the wrong url http://stream.twitter.com on command line

. After which I ping using stream.twitter.com

, after I used https://stream.twitter.com on the browser. It required some credentials

, so I entered my twitter credentials. After I entered it, it returned on the browser unknown URL and a link to follow http://dev.twitter.com/pages/streaming_api. But the link does not exist. The screenshots are attached below. Also a scrennshot of vpn and firewall config

[#20]

@Elvine Egoko can you please ssh into your virtual machine

ssh root@localhost -p 12222
password: hadoop

Then grep the Nifi logs

cat /var/log/nifi/nifi-app.log | grep "ERROR"

See if you can find a more detailed explanation of that error.

Also, I noticed you were using a default template from the Nifi wiki: https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates

I tried to replicate the error on my end with the same template and processor to no avail. I am using the latest HDF sandbox with virtual box.

I did have to manually enter my own twitter API keys and access tokens with my application account . You can make one here: https://apps.twitter.com/

Do you have an account? Did you try entering in a new set of keys?

What do your processor properties look like?

[#21]

@palwell, @Wynner

I am using HDP 2.6

I have reinstall sandbox on centos virtual box I got connection_error again. Here is the screenshot of my configuration