Member since
06-16-2020
55
Posts
14
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1076 | 10-23-2024 11:21 AM | |
1033 | 10-22-2024 07:59 AM | |
1020 | 10-22-2024 07:37 AM | |
608 | 10-21-2024 09:25 AM | |
2446 | 06-16-2023 07:23 AM |
06-16-2023
07:23 AM
@bhadraka What version of NiFi are you using? In NiFi 1.20.0, you can use ReplaceText Processor after reading in the file. Using the line-by-line evaluation mode, there is a drop down "Except-Last-Line". You could then configure it to just replace all previous lines with empty strings. Here's a screenshot of my ReplaceText processor properties.
... View more
06-16-2023
07:05 AM
@Ray82 - I tested out @SAMSAL solution quickly and it worked for me. Make sure in QueryRecord you are referencing the right attribute names.
... View more
06-14-2023
06:57 AM
1 Kudo
@Dracile - Instead I recommend using ForkRecord Processor. Like @steven-matison mentioned, create a Record Reader and Record Writer and add another property with the record path /VisitList, Mode - Split, Include Parent Fields to True. This will result in the next flowfile looking like this - [ {
"employer" : "98765",
"loc_id" : "312",
"topId" : "Management",
"VisitList" : [ {
"S1" : "HR",
"S2" : "Accountant"
} ]
}, {
"employer" : "98765",
"loc_id" : "312",
"topId" : "Management",
"VisitList" : [ {
"S1" : "Manager",
"S2" : "Sr. Manager"
} ]
} ] Then you could split on $ using a SplitJson processor or even better continue using more record-oriented processors for better performance 🙂
... View more
06-02-2023
11:51 AM
Let's say I have a JSON that looks similiar to this [{ "time": "", "value": 92.72, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }, { "time": "", "value": 92.7, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }, { "time": "", "value": { "nil": null }, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": { "TVPMeasurementMetadata": { "qualifier": { "title": null, "href": null } } } }] How would I use a record oriented processor to filter out values that have a the key value equal to this - {"value": { "nil": null } In the example above it the last record in the JSON array would be filtered out so the expected output would look something similar to this - [{ "time": "", "value": 92.72, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }, { "time": "", "value": 92.7, "longitude": "128.732855", "latitude": "-16.1135755", "gathered_time": "1685730397038", "metadata": null }] I don't understand how to do filters. I read about it in the RecordPath Guide but would love to see an example.
... View more
Labels:
- Labels:
-
Apache NiFi
06-01-2023
11:16 AM
Is there a way to convert use expression language to convert the current time to a specific timezone. Here is what I have currently but that's GMT not GMT+10:00. ${now():format("yyyy-MM-dd'T'HH:mm:ss.SSSXXX", "GMT")}
... View more
Labels:
- Labels:
-
Apache NiFi
05-31-2023
03:09 PM
I have a multinode NiFi Cluster running in Docker Containers and I configured the DistributeMapCacheClient and DistributedMapCacheServer to point to a particular container's hostname and set the persistence directory property such that each node in the cluster uses the same Map Cache. I want to make that the MapCache persists even after I restart the container or create a new container. How would I configure that to happen?
... View more
Labels:
- Labels:
-
Apache NiFi
05-02-2023
08:36 AM
I am running trying to run GetSolr Query in NiFi for all the ranger audits being produced. There's a ton of data in the Solr collection. I can't configure any higher amounts of memory footprint due to the VM Solr is running on but I wanted to know if I could configure NiFi more efficiently. Below is a screenshot of what I have.
... View more
Labels:
- Labels:
-
Apache NiFi
04-20-2023
11:24 AM
1 Kudo
I have a NiFi flow that is exporting Ranger Policies from one Cloudera Cluster and another NiFi flow that is importing them into another NiFi cluster. The API call to export them is ${ranger.local.baseurl}/service/plugins/policies/exportJson? and the API call to import them is ${ranger.local.baseurl}/service/plugins/policies/importPoliciesFromFile? Everything works great but once in a blue moon when trying to import the Policies I get - {"statusCode":1,"msgDesc":"XXPolicy not found","messageList":[{"name":"DATA_NOT_FOUND","rbKey":"xa.error.data_not_found","message":"Data not found","objectId":295}]} The best solution I have right now is to restart Ranger Admin and after it works. Does anyone know why this happens or how it gets into this state?
... View more
Labels:
- Labels:
-
Apache Ranger
04-11-2023
06:05 AM
@Kefkoe I really wanted to try and keep it inline and directly in NiFi itself. That way there is no external dependencies to put the script on each node within our NiFi cluster. Good idea as a last resort.
... View more