Member since
05-07-2020
32
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4407 | 03-15-2022 08:41 AM |
03-08-2022
03:10 AM
1 Kudo
Hi, I also have the same problem, I am able to get the 'body' into valid json but want the key value pairs (separated by '=') into json, having no luck?
... View more
01-10-2022
06:21 AM
Thanks Matt!! I hadn't used the advanced tab on the processor until now, worked a treat for my use case. Apologies for the slightly confusing question. I am now able to pick out key words from the hostname attribute to direct them to the correct database. Thanks for the quick response!
... View more
01-10-2022
04:15 AM
Hello all Within NiFi, updateAttribute processor I am trying to change an attribute called 'hive_database' based on the value of another attribute called 'hostname'. For instance if: #1: hostname = Mickey Mouse #2: hostname = James Bond I want to use a 'contains' statement (or similar ) to change the output of new attribute 'hive_database' property: hive_database value: ${hostname:contains('Mickey'):<OUTPUT>('cartoon')} <OR> ${hostname:contains('Mickey'):<OUTPUT>('movie')} So that #1 output would be: property= hive_database value= cartoon The parts I'm stuck on is the OUTPUT and OR parts above, do you know if this would be possible using the update attribute processor? I am trying to avoid having to use routeOnAttribute to break out multiple different 'hosname' values to send data to the relevant 'hive_database' Hope this makes sense, unable to share screenshots etc as sensitive vales. Any help greatly appreciated.
... View more
Labels:
11-25-2021
03:18 AM
Thanks for the reply, is there a way to use Jolt to extract to flowfile attribute?
... View more
11-17-2021
07:28 AM
Hi If one flowfile is processed regularly and the other comes in intermittently do they even need to be merged? it sounds like the flow is so quiet that use of merge is not required?
... View more
11-17-2021
07:23 AM
Hello all I have a NiFi data flow that has flowfiles containing multiple json records (with exactly the same received time) and I'm trying to extract the timestamp as a flowfile attribute as follows: All the received times match as I'm using a PartitionRecord processor prior to EvaluateJsonPath, I have confirmed they are identical. I am aware that I can split > extract > merge but I am trying to avoid that due to the high volume of data passing through this flow, is there any way to just extract the first timestamp for all the records in a single flowfile using EvaluateJsonPath? or another record processor? I am also trying to avoid having to use ExtractText and regex to pull the timestamp, again due to the volume of data in the flow. There is an existing flow working in our production environment using plain/text and regex etc. I am trying to re-design using record processing to improve efficiency. any assistance welcome. 🙂
... View more
Labels:
- Labels:
-
Apache NiFi
07-05-2021
07:36 AM
Thanks for the reply, I am pulling the .rpm file into NiFi using API. How do i convert it to binary? I have not been able to get it to work using invokeHTTP so a colleague of mine has written a Python script for me to use in a excecuteScript processor. This is working in his test environment using a test .rpm file he pulled into using NiFi using GetFile as apposed to API. However in my environment using the API file the flowfile does not move through the processor? again could be as it's not binary?
... View more
06-07-2021
04:21 AM
Thanks for both replies, I managed to get it working last week the same way as you have shown Matt. Cheers
... View more
06-02-2021
11:27 AM
Is there a processor that can detect hidden characters in NiFi? In our test environment I am using an API invoke to deliver a JSON payload, for added security we want to filter out any hidden and potentially malicious text. I have been trying to use RouteText, RouteOnContent and ExtractText processor to only allow alphanumeric and punctuation characters through but can't seem to get the regex to work when looking for 'uncommon' text or characters? any help would be appreciated. Andy
... View more
Labels:
- Labels:
-
Apache NiFi
05-17-2021
06:16 AM
Hello, I have been tasked with POSTing a .rpm file (RedHat package manager) to a server location, this is to automate a manual process that we perform on a weekly basis.
I am able to download the file into NiFi using InvokeHTTP GET. At this stage the 'Content-Type' attribute shows as 'audio/x-pn-realaudio-plugin' , I have read online that this is a common mistake when dealing with .rpm files. I have then used a IdentifyMimeType processor which updates mime type to: application/x-rpm.
In the API documentation is says I require POST /rest/v1/updates as location 'file' as a parameter, see below:
I have been given the following python script that achieves what I am trying to do in Python, so far I have not been able to replicate in Nifi.
I have tried numerous different configs of the processor, manually changed attributes: file, filename, mime.extension, mime.type, Content-Type, Accept-Encoding. I have tried manually adding attributes from the Python script and referencing from the invoke. I have tried to compress to gzip as I was informed it should be in compressed format so I though NiFi had maybe changed it.
So far I have never had anything other than 'no retry' responses.
I currently have set up my invokeHTTP as below:
When I have 'Send Message Body' true - I get the following response:
When I set 'Send Message Body' false - I get:
Any assistance with this at all would be v.much appreciated. I have been trying for a number of days with every different option/config I can think of but cannot get it to POST.
In the script if mentions getting a 'csrftoken' response, am I able to do this using another invokeHTTP?
Thanks in advance
Griggsy
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
- « Previous
- Next »