Member since
06-08-2017
1049
Posts
518
Kudos Received
312
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 11196 | 04-15-2020 05:01 PM | |
| 7096 | 10-15-2019 08:12 PM | |
| 3088 | 10-12-2019 08:29 PM | |
| 11417 | 09-21-2019 10:04 AM | |
| 4312 | 09-19-2019 07:11 AM |
02-06-2018
01:29 PM
2 Kudos
@Varun R
After split text processor use extract text processor and add new property with matching regex, then the extracted attribute will be added as the flowfile attribute. Example:- After split text you are having each address in a flowfile as like this http://aaa.com/q="bigdata"≈i_key="", now you want to know which query param value(ex:bigdata) has been used. Then use extract text add new property query_values
q="(.*?)" output flowfile:- Once the flowfile processed by extract text processor it matches the regex and adds the attribute query_values to the flowfile. By this way you are going to know which query param values are used to get response. . If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.
... View more
02-13-2019
11:31 PM
Thanks for your answer @Shu ! It is very helpful. Is there a way to do wildcard search using this?
... View more
02-05-2018
02:03 PM
@Shu, thank you very much. It worked perfectly!
... View more
02-02-2018
10:45 PM
@Biswajit Chakraborty
If you are using GetFTP processor then after pulling files then processor going to add getftp.remote.source attribute to the flowfile, then you can use this flowfile attribute then prepare filename in update attribute processor Add new property in update attribute filename ${filename}_${getftp.remote.source} //add remote source name to the filename as you can change the way of using expression language to change filename as following ${filename:append(${getftp.remote.source})} //result 711866091328995HDF04-1
(or)
${filename}${getftp.remote.source} //result 711866091328995HDF04-1 Example:- if you are having filename value as 711866091328995 and getftp.remote.source value as HDF04-1 then output flowfile from update attribute will have filename as 711866091328995_HDF04-1 //because we are adding remote source value to filename with underscore (or) if you are having issue with the same filenames and they are getting overwritten, The FlowFile will also have an attribute named uuid, By using UUID(which is a unique identifier for this FlowFile) as filename, will keep every filename as unique so that we are not going to have any overwriting issues. Configs:- filename ${uuid}
... View more
01-30-2018
01:53 PM
1 Kudo
@Anil Reddy In addition to Matt's answer if you want to know all supported Functions in Expression Language for your version of NiFi then click on Right Up Corner GlobalMenu Button And click on Help 3.Then Click on Expression Language Guide on Right Side will shows up all the functions that are supported in your version of NiFi. If you want to implement ${closed_epoch:format("yyyy", "GMT")} this expression even though it is not supported in your version of NiFi then as a work around you can use plus,minus functions. Assuming your closed_epoch attribute value is 1453843201123 ${closed_epoch:minus(86400000)} //this expression will minus 86,400,000 Milliseconds(i.e 24 hrs) from closed_epoch value. New value for closed_epoch will be 1453756801123. *if you are having DayLightSaving time then you need to change the milliseconds value in minus(or) plus function.
... View more
07-17-2018
01:42 PM
@SaiKumar Akula The same flow above will still work for you except swap out the GenerateFlowFile processor for a GetFile processor. Simply set a "run schedule" for how often you want the GetFile to read the source file and change "Keep Source File" property to true. Every time the processor executes it will read the source text file and it can be passed to the splitText processor in above flow and so on... As long as you don't change the source file's filename, you can edit whenever you want to add new lines or remove URL lines. - - Thanks, Matt
... View more
01-27-2018
08:28 PM
@Shu Thanks. This worked. Do you know if i can automate this flow using this external output file in avro format to load into a hive table?
... View more
01-25-2018
04:07 PM
1 Kudo
Thats works very well, thank you once again.
... View more
01-25-2018
09:10 AM
1 Kudo
It works, Thanks alot @Shu
... View more
01-19-2018
02:22 AM
3 Kudos
@Martin Mucha By using RouteOnContent processor we can check the contents of flowfile like is the flowfile message having array in it (or) not, if it is having array then you can route the flowfile to some processors, if not having array route to some other processors. Example:- Let's consider i'm having array of json messages and i'm splitting them individual messages Input:- [{"Id": "1","name":"HDP"},
{"Id": "2","name":"HDF"}] i'm having 2 json messages in an array and i need to add id,name values as attributes for this case i need to split the json array into individual flowfiles. SplitJSON:- JsonPath Expression
$.* this property will splits json array into individual flowfiles. For out input the output would be 2 flowfiles then i will use Evaljson path to extract attributes and add them to the flowfile. But i'm not sure everytime i will get json array (or) not, some times i get only single json message like {"Id": "1","name":"HDP"} for this input my splitjson processor won't give expected results. So for this case i'm going to use RouteOnContent processor:- to check the contents of flowfile and route them accordingly Case1:- if i get json array message then i need to route the flowfile to SplitJson processor. Configs:- Add new property as Array
^\[\{ //we are checking is the content starting with [ and followed by { that means it's an array of json messages in this case i'm going to transfer Array property relation to SplitJson processor. Case2:- if no data is in the json message then i'm checking with below property. NoData
^\{\} //checking the content having empty message in it i'm auto terminating this relation as we are not caring if there is no data. So now the unmatched relation will have single json messages i.e {"Id": "1","name":"HDP"} I'm routing this single json message to other processors(like EvalJsonPath..) to extract attribute values. RouteOnContent Configs:- Sample Flow:- Like this way you can check the contents of flowfile is content having arrays or not. for testing in the above example i'm just checking the starting characters of content, but you can check the whole contents of flowfiles by just adding new property with matching regex and change buffer size if needed. . If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.
... View more