Skip to main content
MOUROU
MOUROU
Silver ⭐️⭐️
June 15, 2023
Question

Using NIFI OAuth2 access Token provider

Dear all,

 

My NIFI version is 1.16.3. 

I am trying to fetch the status of some processors (NIFI-API) within NIFI it self using InvokeHTTP processor.

My NIFI-API is protected and need authentication to be used. I have a technical user with username and password that could get a token from the token API endpoint:

https://nifi.company.com/nifi-api/access/token.

So I defined a controller service OAuth2AccessTokenProvider to generate access tokens.

MOUROU_0-1686822805218.png

 

From next defined API endpoint, I want to get the proprieties of a NIFI processor by its ID

https://nifi.company.com/nifi-api/processors/ae91429e-es3a-1eq9-fe53-b13680cff1e5

I used for that the InvokeHTTP processor with a GET HTTP method and by linking the OAuth2 Access Token provider field to the previously defined OAuth2AccessTokenProvider controller service.

But I am getting such error

 

InvokeHTTP Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to java.io.UncheckedIOException: OAuth2 access token request failed: java.io.UncheckedIOException: OAuth2 access token request failed
- Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'utzcnd154ffd54gedg6g5fd6gdfg656556gdg5gdgt54894455444454fsdf45fd': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (String)"utzcnd154ffd54gedg6g5fd6gdfg656556gdg5gdgt54894455444454fsdf45fd"[truncated 548 chars]; line: 1, column: 81]

 

I changed the real Token value with a random series of characters.

 

7 replies

VidyaSargur
Community Manager
VidyaSargur
Community Manager
June 15, 2023

@MOUROU, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @steven-matison @MattWho  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
MOUROU
MOUROUAuthor
Silver ⭐️⭐️
June 15, 2023

@VidyaSargur  thank you very much for your warm welcome 😀

Translate
    steven-matison
    Platinum ⭐️⭐️
    steven-matison
    Platinum ⭐️⭐️
    June 15, 2023

    @MOUROU I recently built a nifi flow in version 1.21 that uses the NiFI API from within nifi, and it is NOT necessary to get access token.  From within nifi i am able to just start using the api calls I needed.     It would be worth it to see if 1.16 behaves the same way.

     

     

    That flow is here:

     

    https://github.com/cldr-steven-matison/NiFi-Templates/blob/main/NiFi_Template_XML_to_Flow_Definition_JSON.json

     

     

    Translate
    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    June 15, 2023

    @steven-matison 

    Looking at your flow-definition, I see that your invokeHTTP processor are configured to use an SSLContextService.  I am assuming that SSLContextService is configured with the HTTPS enabled NiFi keystore and Truststore.   When you then access the https rest-api endpoint, NiFi in the TLS exchange WANTs a client certificate which would be provided via the SSLContextService.  I am then guessing your NiFi servers have been authorized to access that rest-api endpoint.

    You are correct that you would not need an Access Token since authentication was handled via a mutual TLS exchange with NiFi.   

    Using certificates is actually the recommend method for interacting with the NiFi rest-api for a number of reasons:
    - No need for extra step to get a token.

    - Token is only valid for the specific NiFi node that issued it.

    Thanks,

    Matt

    Translate
      MattWho
      Titanium ⭐️⭐️⭐️
      MattWho
      Titanium ⭐️⭐️⭐️
      June 15, 2023

      @MOUROU 

      I'd recommend using a clientAuth certificate fro interacting with the NiFi rest-api.  Certificate based authentication via a mutualTLS exchange (always enabled in a secure NiFi) is already how NiFi nodes communicate with one another.  Using certificate does not require the extra step if obtaining a token,  Token is only valid for use with the NiFi node that issues it.  Certificated can be created with long expiration times (typically valid for 1 or 2 years by default)

      If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

      Thank you,

      Matt

      Translate
      MOUROU
      MOUROUAuthor
      Silver ⭐️⭐️
      June 16, 2023

      @steven-matison 

      @MattWho 

      Thank you first for your interaction.

      For my case the SSL Context Service is not necessary to request from any unprotected API.
      Our NIFI-API need authentication to be used also within NIFI UI. InvokeHTTP didn't succeed to request anything from the API without authentication.
      But here I will explain you how it could be possible. I created a pipeline composed of theses processors:

      1- GenerateFlowFile: Create FlowFile with username and password of the technical user
      2- InvokeHTTP: Ask for an access token from the token API endpoint
      3- UpdateAttribute: Format the token and save it as ‘Authorization’ attribute
      4- InvokeHTTP: Use this attribute for a HTTP request.

      MOUROU_0-1686912285932.png

      Here is every processor properties:

      1- GenerateFlowFile: Technical user declaration

      MOUROU_1-1686912344218.png

      2- InvokeHTTP: Ask for access token

      MOUROU_2-1686912398687.png

      MOUROU_9-1686913269494.png

      3- UpdateAttribute: Authorization attribute

      MOUROU_4-1686912763468.png

      4- InvokeHTTP: Request Processor Status

      MOUROU_6-1686912821160.pngMOUROU_7-1686912857424.png

      MOUROU_8-1686912889161.png

       

      But this is not the proper and clean way to do it for such reasons:
      - I need to provide the password of the technical user with a visible way
      - Each time we consult the API, we will generate a new access token. But a token should be used until its expiration time, and not generating so many tokens.

      That is why I tried to use the StandardOauth2AccessTokenProvider controller service.. Who should be responsible of fetching access token and attach it to the header of the InvokeHTTP processor request. But unfortunately I am getting the previous described error.

      Translate
      MattWho
      Titanium ⭐️⭐️⭐️
      MattWho
      Titanium ⭐️⭐️⭐️
      June 16, 2023

      @MOUROU 
      Is your NiFi configured to support Oauth2 based user authentication?
      It looks more like you are using either kerberos-provider or ldap-provider fro user authentication.

      My suggestion to create a client certificate and use a SSLContext service for client authentication for an automated dataflow like this is because:
      1. No need to obtain an token ever.
      2. Certs can be created with long expiration time.
      3. Tokens are NiFi node specific (same token is not valid for a different NiFi node in a the same NiFi cluster).
      4. Same certs works no matter which node in the cluster the invokeHTTP connects with.

      Matt

      Translate
      Terms & ConditionsAccessibility statement