Member since
06-16-2020
51
Posts
14
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
486 | 10-23-2024 11:21 AM | |
485 | 10-22-2024 07:59 AM | |
432 | 10-22-2024 07:37 AM | |
247 | 10-21-2024 09:25 AM | |
1974 | 06-16-2023 07:23 AM |
06-01-2023
11:16 AM
Is there a way to convert use expression language to convert the current time to a specific timezone. Here is what I have currently but that's GMT not GMT+10:00. ${now():format("yyyy-MM-dd'T'HH:mm:ss.SSSXXX", "GMT")}
... View more
Labels:
- Labels:
-
Apache NiFi
05-31-2023
03:09 PM
I have a multinode NiFi Cluster running in Docker Containers and I configured the DistributeMapCacheClient and DistributedMapCacheServer to point to a particular container's hostname and set the persistence directory property such that each node in the cluster uses the same Map Cache. I want to make that the MapCache persists even after I restart the container or create a new container. How would I configure that to happen?
... View more
Labels:
- Labels:
-
Apache NiFi
05-02-2023
08:36 AM
I am running trying to run GetSolr Query in NiFi for all the ranger audits being produced. There's a ton of data in the Solr collection. I can't configure any higher amounts of memory footprint due to the VM Solr is running on but I wanted to know if I could configure NiFi more efficiently. Below is a screenshot of what I have.
... View more
Labels:
- Labels:
-
Apache NiFi
04-20-2023
11:24 AM
1 Kudo
I have a NiFi flow that is exporting Ranger Policies from one Cloudera Cluster and another NiFi flow that is importing them into another NiFi cluster. The API call to export them is ${ranger.local.baseurl}/service/plugins/policies/exportJson? and the API call to import them is ${ranger.local.baseurl}/service/plugins/policies/importPoliciesFromFile? Everything works great but once in a blue moon when trying to import the Policies I get - {"statusCode":1,"msgDesc":"XXPolicy not found","messageList":[{"name":"DATA_NOT_FOUND","rbKey":"xa.error.data_not_found","message":"Data not found","objectId":295}]} The best solution I have right now is to restart Ranger Admin and after it works. Does anyone know why this happens or how it gets into this state?
... View more
Labels:
- Labels:
-
Apache Ranger
04-11-2023
06:05 AM
@Kefkoe I really wanted to try and keep it inline and directly in NiFi itself. That way there is no external dependencies to put the script on each node within our NiFi cluster. Good idea as a last resort.
... View more
04-10-2023
01:47 PM
Sorry typo on my part. Even with the */api/v1/admin/topics it works. I cloned the nifi github repo and added some more log statements to the executeStreamProcessor changed the edited the logback.xml to enable DEBUG for that processor. It looks like it stripping the quotes in payload. Here is an example request: curl -X POST -H 'Content-Type:application/json' -d '{"name": "Apple AirPods", "data": {"color": "white", "generation": "3rd", "price": 135}}' https://api.restful-api.dev/objects This works in the command line. If you configure the executeStreamCommand like this: Command Arguments Strategy: Command Arguments Property Command Arguments: -X POST -H 'Content-Type:application/json' -d '{\"name\":\"Apple AirPods\",\"data\":{\"color\":\"white\",\"generation\":\"3rd\",\"price\":135}}' https://api.restful-api.dev/objects Command Path: curl Argument Delimiter: " " (Space) Here's what I see in the logs - Executing and waiting for command: curl, with the arguments: [-X, POST, -H, 'Content-Type:application/json', -d, '{\name\:\Apple AirPods\,\data\:{\color\:\white\,\generation\:\3rd\,\price\:135}}', https://api.restful-api.dev/objects] @cotopaul , @ckumar , @MattWho , @steven-matison - Is there a way to get around this?
... View more
04-10-2023
12:09 PM
@cotopaul The reason why I say that is because if I change the command arguments to this - -X GET -H referer:${Referer} -H 'Content-Type: application/json' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/brokers It returns successfully. The only that changed is the HTTP Request Method and removing the Payload.
... View more
04-10-2023
10:55 AM
@cotopaul I have verified that the same command works from the command line. I think the executeStreamCommand must be doing some sort of formatting prior to sending the request to the Streams Message Manager Server. A GET request works using the ExecuteStreamCommand because there is no payload. I can't use the invokeHTTP because it currently doesn't have configuration to authenticate using kerberos which is why I fell back on using ExecuteStreamCommand.
... View more
04-10-2023
06:35 AM
I am trying to use NiFi to make a CURL call to SMM. Because SMM is using kerberos authentication I want to make a POST request to the SMM API using Kerberos. I am now at the last step where I need to sent a POST request to create a Kafka topic. I am having a problem where when I send the request using the ExecuteStreamCommand I think some formatting happens to the payload before the call gets sent to the SMM API. I tried escaping the quotes in the payload but still no luck. Here are my configurations for the ExecuteStreamCommand: Command Arguments Strategy: Command Arguments Property Command Arguments: -X POST -H referer:${Referer} -H 'Content-Type: application/json' -d '{\"newTopics\": [{\"name\":\"testing123\",\"numPartitions\":3,\"replicationFactor\":3}], \"allTopicNames\":[\"testing123\"]}' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/topics Command Path: curl Ignore STDIN: false Argument Delimiter: " " (Space) Right now it's giving me a 415 - Unsupported Media Type Error. Can someone please help.
... View more
Labels: