Nifi Dynamic Parameter Context / ListAzureBlobStorage SAS Token

[#1]

Hello, 
I'm quite new to Nifi. I'm using v. 1.24.0

I'm working on a project which requires fetching blobs from Azure Blob Storage and putting them into SFTP.  

To achieve the desired result, there are two major steps : 

1. Get Token Bearer from Authentification Server​
2. Calling a webservice to get SAS key with Token Bearer gotten from previous call.

There are two limitations : 

1. The SAS Key expires every 1440 minutes.
2. The ListAzureBlobStorage processor doesn't accept any incoming flowfiles. 

Therefore, I thought about dynamically updating Parameter Context and using the parameter as property value in the ListAzureBlobStorage. 

Following is the high-level approach of my nifi job : 

1. GenerateFlowFile : scheduled to trigger at every  1430 minutes + contains the body of the InvokeHTTP request
2. InvokeHTTP (Get Bearer Token) : processor to request a bearer token from the authentication server
3. EvaluateJsonPath (Extract Bearer Token): Extract the bearer token from the response
4. InvokeHTTP (Get SAS Token): processor to obtain the SAS token
5. EvaluateJsonPath (Extract SAS Token): Extract the SAS token from the response
6. ReplaceText (Prepare for Parameter Context Update): Prepare a JSON payload with the SAS token and update the flow file content
7. PutDistributedMapCache (Store SAS Token): Store the SAS token in a distributed cache for later retrieval.
8. FetchDistributedMapCache (Retrieve SAS Token): Retrieve the SAS token from the distributed cache.
9. ReplaceText (Update Parameter Context): Prepare a JSON payload to update the parameter context with the new SAS token.
10. InvokeHTTP (Update Parameter Context)
11. EvaluateJsonPath (Check Update Status): Extract the completion status from the response to check if the update is complete.
12. RouteOnAttribute (Check Update Completion): Route the flow based on whether the update is complete or not
13. InvokeHTTP (GET Method to Check Progress): If the update is not complete, send a GET request to check the progress.
14. InvokeHTTP (DELETE Method to Remove Completed Request): If the update is complete, send a DELETE request to remove the completed request from memory.

I took inspirations from the following post : https://community.cloudera.com/t5/Support-Questions/Update-parameter-contexts-dynamically-using-REST...

The SAS Token is a sensitive field in ListAzureBlobStorage and FetchStorageBlobStorage. I'm getting the following error : "SAS Token is invalid because the property SAS Token cannot reference parameter 'sasToken' because the Sensitivity of the parameter does not match the Sensitivity of the property".

I tried the following : 

1. Attempting to update the sensitivity of the parameter 'sasToken' from not sensitive to sensitive by adding a field sensitivity = True in the ReplaceText processor in step 9. 

{
"revision": {
"clientId": "bea2399d-2e37-1fff-1f82-17e8cfbc9514",
"version": 3
},
"component": {
"id": "bea239b6-2e37-1fff-7fe8-a9a01a8d79e7",
"parameters": [
{
"parameter": {
"name": "sasToken",
"sensitive": true,
"value": "${sasToken}"
}
}
]
}
}

2. Pre-set Parameter Sensitivity i.e. ensuring that the 'sasToken' parameter is initially created with the correct sensitivity (i.e., sensitive) in my NiFi parameter context

In both the cases, I get the following error : 

invokehttp.request.url
http://localhost:8080/nifi-api/parameter-contexts/bea239b6-2e37-1fff-7fe8-a9a01a8d79e7/update-requests

invokehttp.response.body
Could not update inherited Parameter Contexts for Parameter Context [ABCD-abcd] because: Cannot update Parameters because doing so would change Parameter 'sasToken' from sensitive to not sensitive

Please let me know what do to ? 

Best regards, 

Deejay

PS : I'm happy to provide more details if needed. 

[#2]

@Deejay Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi expert @steven-matison  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.