question Re: Nifi GetTwitter not working - Authorization Required in Archives of Support Questions (Read Only)

Nifi GetTwitter not working - Authorization Required

raphamarymtl — Sat, 22 Apr 2017 02:23:55 GMT

Hi,

I am using NiFi on a VM on Azure.

I installed HDF and I am trying to use the processor GetTwitter.

I created a Twitter App (which works, I checked), I configured the server with NTP.

I installed SSL for NiFi, Ranger and Kerberos to handle the connection to the UI.

But I have this error : Received Error HTTP_ERROR : HTTP/1.1 401 Authorization Required

Any idea what is wrong?

Re: Nifi GetTwitter not working - Authorization Required

myoung — Sat, 22 Apr 2017 03:14:16 GMT

@Raphaël MARY

Which end point are you using for the processor? You should use the sample or filter endpoint. I don't believe you can use the firehouse endpoint unless you pay Twitter for access.

Re: Nifi GetTwitter not working - Authorization Required

raphamarymtl — Fri, 28 Apr 2017 21:50:43 GMT

@Michael Young

I tried sample and filter endpoint but I am still getting the same error. Do I need to use certificates because I actived ssl?

Re: Nifi GetTwitter not working - Authorization Required

myoung — Fri, 28 Apr 2017 21:58:53 GMT

@Raphaël MARY

Yes, more than likely. You can read more about Twitter TLS here: https://dev.twitter.com/overview/api/tls

Re: Nifi GetTwitter not working - Authorization Required

raphamarymtl — Sat, 29 Apr 2017 00:33:02 GMT

@Michael Young

I added the certificate of twitter (twittercom.crt) in NiFi keystore (with keytool) but get the same error.

Do I have to add it in the keystore or the trustore? I did not find any doc on that.

The error mentions HTTP_ERROR, but if I am using SSL, why is it not HTTPS?

Re: Nifi GetTwitter not working - Authorization Required

MattWho — Sun, 18 Aug 2019 04:22:12 GMT

@Raphaël MARY

A secured NiFi does not force its processor components to be secured. A secured Nifi can communicate with non secure end-points. Security at the processor component level is handled via configuration on the processor itself and never at the NiFi core level.

That being said, I am really not sure what your twitter connection issue is. If the GetTwitter processor did support TLS, I would expect it to have a processor for pointing at a "SSL Context Service" controller service which it does not. It looks like authorization is purely handled via:

You could try putting the twitter processor in debug and see if it produces any additional output in the nifi-app.log that may lead to a cause.

Thanks,

Matt

Re: Nifi GetTwitter not working - Authorization Required

apsaltis — Sat, 29 Apr 2017 01:02:04 GMT

@Raphaël MARY

I use the Twitter processor pretty often (albeit I am not using SSL) and one thing I have often run into that produces the same error is a copy/paste issue with one of the Key's or Secrets.

Can you try copying your key's and secrets to a text editor and then copy/paste them from there into NiFi.

Thanks,

Andrew

Re: Nifi GetTwitter not working - Authorization Required

raphamarymtl — Sat, 29 Apr 2017 01:06:46 GMT

@Matt Clarke

Even with debug level I get this in nifi-app.log :

2017-04-28 14:05:35,507 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.twitter.GetTwitter GetTwitter[id=60003148-812e-14ce-9c95-0df056d3325d] Received error HTTP_ERROR: HTTP/1.1 401 Authorization Required. Will attempt to reconnect

Re: Nifi GetTwitter not working - Authorization Required

raphamarymtl — Sat, 29 Apr 2017 01:14:38 GMT

@apsaltis

I already did and I even tested with another application (in java) and it works.

Do I need to test the connection to Twitter in command line like a ping or something ? I can ping dev.twitter.com

Re: Nifi GetTwitter not working - Authorization Required

apsaltis — Sat, 29 Apr 2017 01:19:46 GMT

@Raphaël MARY

Hmm interesting. It appears that you can reach the host just fine if you are getting back a 401. Have you taken the approach @Matt Clarke mentioned ?

Re: Nifi GetTwitter not working - Authorization Required

raphamarymtl — Sat, 29 Apr 2017 01:31:36 GMT

I did but I have no more information in debug mode :

Re: Nifi GetTwitter not working - Authorization Required

raphamarymtl — Sat, 29 Apr 2017 02:39:15 GMT

It happened that I imported a template with the processor GetTwitter and the version of the processor seemed to be old.

I removed it and added a new one from the tool bar.

It works now!

Re: Nifi GetTwitter not working - Authorization Required

shivkanaujiya00 — Thu, 12 Apr 2018 04:35:23 GMT

Syncing NTP server is what worked for me(NiFi 1.6) :

on centos 6:

service ntpd stop
ntpd -gq

where -g – requests an update irrespective of the time offset

-q – requests the daemon to quit after updating the date from the NTP server.

service ntpd start

Re: Nifi GetTwitter not working - Authorization Required

GosuNadir — Mon, 02 Jan 2023 19:00:35 GMT

how did you fix the HTTP ERROR??

Re: Nifi GetTwitter not working - Authorization Required

VidyaSargur — Tue, 03 Jan 2023 07:21:29 GMT

@GosuNadir, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.