Support Questions

Find answers, ask questions, and share your expertise

Prometheus can't scrape metrics

avatar
Explorer

Hello!

Following the advice from this post:

 https://community.cloudera.com/t5/Support-Questions/Accessing-NIFI-Metrics-endpoint-for-Prometheus-w...

I'm trying to set up certificate authentication/authorization from prometheus server to nifi, but getting 403 Forbidden. 

In Nifi user log i can see that authentication CN of certificate is successful, further  comes the 403 error:

blackboks_0-1759412399877.png

Prometheus scrape config:
blackboks_1-1759412984854.png

blackboks_2-1759413177931.png

 

Can you please tell what is wrong here?

1 ACCEPTED SOLUTION

avatar
Master Mentor

@blackboks 

All actions performed against  secured NiFi require proper authentication and authorization.   It appears you have successful authentication via. mutualTLS exchange, but you are missing the proper authorization needed for the rest-api endpoint you are trying to access.  

The shared nifi-user.log entry tells you which authorization policy is missing for the user that is needed for the request endpoint /nifi-api/flow/metrics/prometheus:

"view the user interface" which authorizes a user to /flow NiFI resource.

I don't know how your NIFi has been configured to for authorization, but the most common setup uses the managed-authorizer.

From the NiFi UI you can access the global policies from the NIFi global menu in the upper right corner of UI.

MattWho_0-1759415595493.png

"Policies" will open a new UI, where you can select "view the user interface" fomr the drop down selection:

MattWho_1-1759415685567.png

Then you can click on the person icon to the right to authorize additional user identities.

MattWho_2-1759415794142.png

Your list of user will be different.  What is important to note is NiFi user and group identities are case sensitive and must match exactly.  So the exact user identity shown in the nifi-user.log is the one that needs to be added to the "view the user interface" policy.

If this user is not in the list, you must first add that user identity and then you will be able to authorize it.  This can be done if you are using the managed-authorizer with the file-user-group-provider.  If so, you can access "Users" from the NiFi global menu to open the NiFi Users UI:

MattWho_3-1759416087894.png

From there you can use the "+" icon to the right to add a new user identity. Remember that user identity string must match exactly case sensitive with what is shown in nifi-user.log; otherwise, it will be treated asa different user.

 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@blackboks 

All actions performed against  secured NiFi require proper authentication and authorization.   It appears you have successful authentication via. mutualTLS exchange, but you are missing the proper authorization needed for the rest-api endpoint you are trying to access.  

The shared nifi-user.log entry tells you which authorization policy is missing for the user that is needed for the request endpoint /nifi-api/flow/metrics/prometheus:

"view the user interface" which authorizes a user to /flow NiFI resource.

I don't know how your NIFi has been configured to for authorization, but the most common setup uses the managed-authorizer.

From the NiFi UI you can access the global policies from the NIFi global menu in the upper right corner of UI.

MattWho_0-1759415595493.png

"Policies" will open a new UI, where you can select "view the user interface" fomr the drop down selection:

MattWho_1-1759415685567.png

Then you can click on the person icon to the right to authorize additional user identities.

MattWho_2-1759415794142.png

Your list of user will be different.  What is important to note is NiFi user and group identities are case sensitive and must match exactly.  So the exact user identity shown in the nifi-user.log is the one that needs to be added to the "view the user interface" policy.

If this user is not in the list, you must first add that user identity and then you will be able to authorize it.  This can be done if you are using the managed-authorizer with the file-user-group-provider.  If so, you can access "Users" from the NiFi global menu to open the NiFi Users UI:

MattWho_3-1759416087894.png

From there you can use the "+" icon to the right to add a new user identity. Remember that user identity string must match exactly case sensitive with what is shown in nifi-user.log; otherwise, it will be treated asa different user.

 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

avatar
Explorer

@MattWho Thanks a lot for your help!

My mistake was that I specified the CN in the username as I did earlier when setting up the nifi-registry user, now I entered it without the CN exactly as it is displayed in the user-log and as you said, and this solved the problem.  Thanks again!

blackboks_0-1759419267567.png