Member since
06-16-2020
55
Posts
14
Kudos Received
5
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2000 | 10-23-2024 11:21 AM | |
| 1817 | 10-22-2024 07:59 AM | |
| 1788 | 10-22-2024 07:37 AM | |
| 1153 | 10-21-2024 09:25 AM | |
| 3173 | 06-16-2023 07:23 AM |
06-16-2023
07:05 AM
@Ray82 - I tested out @SAMSAL solution quickly and it worked for me. Make sure in QueryRecord you are referencing the right attribute names.
... View more
06-14-2023
06:57 AM
1 Kudo
@Dracile - Instead I recommend using ForkRecord Processor. Like @steven-matison mentioned, create a Record Reader and Record Writer and add another property with the record path /VisitList, Mode - Split, Include Parent Fields to True. This will result in the next flowfile looking like this - [ {
"employer" : "98765",
"loc_id" : "312",
"topId" : "Management",
"VisitList" : [ {
"S1" : "HR",
"S2" : "Accountant"
} ]
}, {
"employer" : "98765",
"loc_id" : "312",
"topId" : "Management",
"VisitList" : [ {
"S1" : "Manager",
"S2" : "Sr. Manager"
} ]
} ] Then you could split on $ using a SplitJson processor or even better continue using more record-oriented processors for better performance 🙂
... View more
06-02-2023
11:51 AM
Let's say I have a JSON that looks similiar to this [{ "time": "", "value": 92.72, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }, { "time": "", "value": 92.7, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }, { "time": "", "value": { "nil": null }, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": { "TVPMeasurementMetadata": { "qualifier": { "title": null, "href": null } } } }] How would I use a record oriented processor to filter out values that have a the key value equal to this - {"value": { "nil": null } In the example above it the last record in the JSON array would be filtered out so the expected output would look something similar to this - [{ "time": "", "value": 92.72, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }, { "time": "", "value": 92.7, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }] I don't understand how to do filters. I read about it in the RecordPath Guide but would love to see an example.
... View more
Labels:
- Labels:
-
Apache NiFi
06-01-2023
11:16 AM
Is there a way to convert use expression language to convert the current time to a specific timezone. Here is what I have currently but that's GMT not GMT+10:00. ${now():format("yyyy-MM-dd'T'HH:mm:ss.SSSXXX", "GMT")}
... View more
Labels:
- Labels:
-
Apache NiFi
05-31-2023
03:09 PM
I have a multinode NiFi Cluster running in Docker Containers and I configured the DistributeMapCacheClient and DistributedMapCacheServer to point to a particular container's hostname and set the persistence directory property such that each node in the cluster uses the same Map Cache. I want to make that the MapCache persists even after I restart the container or create a new container. How would I configure that to happen?
... View more
Labels:
- Labels:
-
Apache NiFi
04-20-2023
11:24 AM
1 Kudo
I have a NiFi flow that is exporting Ranger Policies from one Cloudera Cluster and another NiFi flow that is importing them into another NiFi cluster. The API call to export them is ${ranger.local.baseurl}/service/plugins/policies/exportJson? and the API call to import them is ${ranger.local.baseurl}/service/plugins/policies/importPoliciesFromFile? Everything works great but once in a blue moon when trying to import the Policies I get - {"statusCode":1,"msgDesc":"XXPolicy not found","messageList":[{"name":"DATA_NOT_FOUND","rbKey":"xa.error.data_not_found","message":"Data not found","objectId":295}]} The best solution I have right now is to restart Ranger Admin and after it works. Does anyone know why this happens or how it gets into this state?
... View more
Labels:
- Labels:
-
Apache Ranger
04-11-2023
06:05 AM
@Kefkoe I really wanted to try and keep it inline and directly in NiFi itself. That way there is no external dependencies to put the script on each node within our NiFi cluster. Good idea as a last resort.
... View more
04-10-2023
01:47 PM
Sorry typo on my part. Even with the */api/v1/admin/topics it works. I cloned the nifi github repo and added some more log statements to the executeStreamProcessor changed the edited the logback.xml to enable DEBUG for that processor. It looks like it stripping the quotes in payload. Here is an example request: curl -X POST -H 'Content-Type:application/json' -d '{"name": "Apple AirPods", "data": {"color": "white", "generation": "3rd", "price": 135}}' https://api.restful-api.dev/objects This works in the command line. If you configure the executeStreamCommand like this: Command Arguments Strategy: Command Arguments Property Command Arguments: -X POST -H 'Content-Type:application/json' -d '{\"name\":\"Apple AirPods\",\"data\":{\"color\":\"white\",\"generation\":\"3rd\",\"price\":135}}' https://api.restful-api.dev/objects Command Path: curl Argument Delimiter: " " (Space) Here's what I see in the logs - Executing and waiting for command: curl, with the arguments: [-X, POST, -H, 'Content-Type:application/json', -d, '{\name\:\Apple AirPods\,\data\:{\color\:\white\,\generation\:\3rd\,\price\:135}}', https://api.restful-api.dev/objects] @cotopaul , @ckumar , @MattWho , @steven-matison - Is there a way to get around this?
... View more
04-10-2023
12:09 PM
@cotopaul The reason why I say that is because if I change the command arguments to this - -X GET -H referer:${Referer} -H 'Content-Type: application/json' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/brokers It returns successfully. The only that changed is the HTTP Request Method and removing the Payload.
... View more