My use-case is:   1) Have API credentials 2) Use UpdateAttribute to update (1) schema, (2) s3 bucket/location (my list of reports) 3) Query API endpoint for report 4) API endpoint paginates and gets more records 5) Call MergeRecord 6) Save to s3   Since 3, 4, 5, 6 are all the same, I'm re-using the processors like below (screenshot).  My problem is (5) MergeRecord will try to merge different schemas together, which is obviously a problem.   How can I restructure this?  I'd like to re-use processors as much as possible, but still be able to add more schemas as my needs evolve.   ... View more

I have an avro schema that I can get from a registry:     { "name": "bar_tbl", "fields": [ { "name": "foo1", "type": "string" }, { "name": "foo2", "type": "string" } ] }     I know how to use `EvaluateJsonPath` and get `name` and `fields`. I'd like to set an attribute with `UpdateAttribute` to equal:     select foo1, foo2 from bar_tbl     to use it later downstream in another processor. How can I achieve this?   Or, more directly, I have a REST api that takes an url-encoded sql statement.  How can I feed the sql statement to `InvokeHttp`? ... View more

I have a apache/nifi:latest instance spun inside an Amazon Linux 2 EC2. For reference, see this guide: here I have a QuerySalesforceObject ver. 1.18.0 that makes use of StandardOauth2AccessTokenProvider.  The oauth2 provider url is configured at https://test.salesforce.com/services/oauth2/token I can curl this url from the box and from inside the docker container just fine (I don’t get a timeout). [root@ip-10-229-18-107 \~\]# docker exec -it nifi_container_persistent /bin/sh printenv | grep -i proxy HTTPS_PROXY=http://proxy.MY_DOMAIN.com:3128 no_proxy=localhost,127.0.0.1,MY_DOMAIN.com,.amazonaws.com NO_PROXY=localhost,127.0.0.1, MY_DOMAIN.com,.amazonaws.com https_proxy=http://proxy.MY_DOMAIN.com:3128 http_proxy=http://proxy.MY_DOMAIN.com:3128 HTTP_PROXY=http://proxy.MY_DOMAIN.com:3128 curl https://test.salesforce.com/services/oauth2/token {"error":"unsupported_grant_type","error_description":"grant type not supported"}# But when I run the task, oauth2 fails with an error java.io.UncheckedIOException: OAuth2 access token request failed Caused by: java.net.SocketTimeoutException: connect timed out This leads me to believe the proxy settings are not being honored by the class. How can I fix this? ... View more