Member since
04-29-2021
9
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
647 | 06-20-2023 09:03 AM |
06-22-2023
07:14 AM
Hello, I am using the PutElasticsearchHttpRecord processor using grok to push data to Elastic, but this processor is too slow to keep up - has anyone had any joys with this processor or is it a bit rubbish? The version is 1.12.1
... View more
Labels:
06-20-2023
09:03 AM
2 Kudos
No worries, I've found a fix. Setting the jsonwriter to line by line instead of array. 🙂
... View more
06-20-2023
07:40 AM
I have a problem when sending JSON data to Elastic from Nifi - Nifi puts [ square brackets ] around a JSON array, and so Elastic doesn't recognise the data. The current fix is to use a SplitJSON processor, which removes the square brackets, but this will become unworkable when the number of flow files gets a bit big in number. Is there a way of creating an array of JSON that is recognised by Elastic that does not contain square brackets? I am also using an older version of Nifi - Ver 1.12.1
... View more
05-13-2022
03:54 AM
I have a deleteHDFS after a getHDFS processor which removes files older than a certain time period. The getHDFS removes the file no problem, but what I have after is folders within folders that are empty, so I am using a deleteHDFS just after that, that uses the $path attribute - but I then need to specify the folder that contains that $path as I already told the getHDFS processor where to look to expire files - so I use a $custom_path attribute for where to look and add the $path variable to the deleteHDFS processor, but it only deletes the last folder. Is there a way to tell the deleteHDFS processor to delete the $path and keep the $custom_path? Currently in the deleteHDFS processor I have the Path set to: ${custom_path}/${path} and Recursive set to True.
... View more
Labels:
- Labels:
-
Apache NiFi
-
HDFS
06-30-2021
01:36 AM
Every now and then, one of my processors fills the bulletin board with connection error messages from a FetchSentinelData 1.5 processor. Is there a way to suppress only those messges or route them to an output file to prevent them from filling the bulletin board each time?
... View more
Labels:
- Labels:
-
Apache NiFi
05-07-2021
06:28 AM
Thanks, I tried something just like this yesterday, and it would work, but these logs don't have a file extension like what would be needed, (.gz) but I think I have an issue somewhere else, so will carry on trying to fix the overall issue.
... View more
05-06-2021
04:20 AM
I have identified that some of my Ranger logs are coming in in gz format (but without the extension). Is there a way of extracting the log data within the Nifi flow?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Ranger