Nifi GetTwitter not working - Authorization Required

[#1]

Hi,

I am using NiFi on a VM on Azure.

I installed HDF and I am trying to use the processor GetTwitter.

I created a Twitter App (which works, I checked), I configured the server with NTP.

I installed SSL for NiFi, Ranger and Kerberos to handle the connection to the UI.

But I have this error : Received Error HTTP_ERROR : HTTP/1.1 401 Authorization Required

Any idea what is wrong?

[#2]

@Raphaël MARY

Which end point are you using for the processor? You should use the sample or filter endpoint. I don't believe you can use the firehouse endpoint unless you pay Twitter for access.

[#3]

@Michael Young

I tried sample and filter endpoint but I am still getting the same error. Do I need to use certificates because I actived ssl?

[#4]

@Raphaël MARY

Yes, more than likely. You can read more about Twitter TLS here: https://dev.twitter.com/overview/api/tls

[#5]

@Michael Young

I added the certificate of twitter (twittercom.crt) in NiFi keystore (with keytool) but get the same error.

Do I have to add it in the keystore or the trustore? I did not find any doc on that.

The error mentions HTTP_ERROR, but if I am using SSL, why is it not HTTPS?

[#6]

@Raphaël MARY

A secured NiFi does not force its processor components to be secured. A secured Nifi can communicate with non secure end-points. Security at the processor component level is handled via configuration on the processor itself and never at the NiFi core level.

That being said, I am really not sure what your twitter connection issue is. If the GetTwitter processor did support TLS, I would expect it to have a processor for pointing at a "SSL Context Service" controller service which it does not. It looks like authorization is purely handled via:

You could try putting the twitter processor in debug and see if it produces any additional output in the nifi-app.log that may lead to a cause.

Thanks,

Matt

[#7]

@Matt Clarke

Even with debug level I get this in nifi-app.log :

2017-04-28 14:05:35,507 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.twitter.GetTwitter GetTwitter[id=60003148-812e-14ce-9c95-0df056d3325d] Received error HTTP_ERROR: HTTP/1.1 401 Authorization Required. Will attempt to reconnect

[#8]

@Raphaël MARY

I use the Twitter processor pretty often (albeit I am not using SSL) and one thing I have often run into that produces the same error is a copy/paste issue with one of the Key's or Secrets.

Can you try copying your key's and secrets to a text editor and then copy/paste them from there into NiFi.

Thanks,

Andrew

[#9]

@apsaltis

I already did and I even tested with another application (in java) and it works.

Do I need to test the connection to Twitter in command line like a ping or something ? I can ping dev.twitter.com

[#10]

@Raphaël MARY

Hmm interesting. It appears that you can reach the host just fine if you are getting back a 401. Have you taken the approach @Matt Clarke mentioned ?