Member since
12-14-2016
35
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4715 | 12-23-2016 05:41 AM |
01-10-2017
12:59 PM
I am currently using SplitJSON to split incoming JSON into multiple records based on JSON path $.*. Further, I was able to aggregate all the splits into 1 single file with MergeContent. However, now when I am trying to aggregate split files into multiple chunks of 1 MB each, its not working as expected. I have followed some of the other artiles on the same use case, but most of them seems to use 'Minimum Number of Entries' for merging based on records than file size. Below is the MergeContent processor and respective properties. Any quick help or insight is highly appreciated. Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
01-09-2017
08:57 AM
Please advice if the Event driven will be now available/implemented for NiFi 1.x and later versions?
... View more
01-03-2017
01:21 PM
Thank you Matt, I was finally able to generate the JSON as well as the dynamically injected 'ingestionDate'. However the output JSON is all escaped with \" for the strings keys/values, is there a way to strip of those or not have it in the first place? Below is the snippet of JSON and screenshots of the modified processors. Output JSON -
{"items":"[{\"date\":\"2016-11-01T00:00:00\",\"pointsMax\":100,\"respPeriod\":\"2016-11\"},\"details\":[{\"detail\":[{\"meta\":{\"answers\":[{\"text\":\"11/1/2016 \"}]},\"id\":56184,...
...
}}]}]","ingestionDate":"Tue Jan 03 18:17:15 IST 2017"}
EvaluateJSONPath
UpdateAttribute AttributesToJSON
... View more
01-03-2017
12:58 PM
Thank you Don for the simplified explaination, I figured looking at the source code for these eventually.
... View more
01-01-2017
12:07 PM
Thanks Matt for the quick response. I am already using ReplaceText with now{}, however I tried AttributesToJSON but somehow the attribute is not getting updated in the output JSON. Below is what I am trying along with the NiFi screenshot, kindly advice what I might be missing here - Input JSON -
{"eventDetail":{...},"sourceId":14627000,"eventDate":"2016-11-01T00:00:00","ingestionDate":""} Output JSON -
{"eventDetail":{...},"sourceId":14627000,"eventDate":"2016-11-01T00:00:00","ingestionDate":""} PFA screenshots.
... View more
12-31-2016
06:12 AM
1 Kudo
Hello there, Is there a simpler and dynamic way to replace JSON values dynamically than ReplaceText. Currently, below is my JSON and respective ReplaceText properties (PFA). { "orgId": ""
"sourceId": "123"
"eventDate", "31-12-2016",
"eventDetail": "Transaction complete."
"ingestionDate": "",
} I have to add today's date dynamically to ingestionDate below, which I am somehow being able to achieve with ReplaceText, however I believe its at best fragile since it appends it at the end of the JSON. Search Value - (?s:(^.*)}$)
Replacement Value - $1,"ingestionDate":"${ingestionDate}"} However, what if I needed to dynamically assign more values for various other attributes for a complex JSON (I do have another use case for that), and so I need a way to be able to refer to the incoming flow-file JSON attribute via regex (like ingestionDate) and update its value to current date more generically rather than manipulating it as a text. Is there a better way out? Please advice. Thank you and Happy New Year to all! replacetext.pngreplacetext.png
... View more
Labels:
- Labels:
-
Apache NiFi
12-23-2016
05:41 AM
Thank you @Bryan Bende I managed my use case with Jolt Processor, since AVRO would only transform to the schema of incoming file format, and will adjust only in terms of adding or removing some attributes. However, I was looking for a totally different outbound schema than whats incoming.
... View more
12-23-2016
05:38 AM
@Andrew Grande & @Bryan Bende, you guys ROCK!!! It was actually a permissions issue which I fixed. Thank you, appreciate it.
... View more
12-21-2016
06:38 PM
Thanks @Andrew Grande for the quick response. The problem however is with ListFile above, as you can see I have specified the path /home/temp to the 'Remote Path', should that not first enlist all the files in that directory, which then I believe will be sent over to FetchFile as flowfile, right?
... View more
12-21-2016
06:16 PM
I have been struggling with ListSFTP and FetchSFTP (PFA) as none of them seem to be working and ending up with error below - java.io.FileNotFoundException: Could not perform listing on /home/temp/ because could not find the file on the remote server ListSFTP
FetchSFTP
... View more
Labels:
- Labels:
-
Apache NiFi
- « Previous
-
- 1
- 2
- Next »