Member since
08-01-2021
58
Posts
14
Kudos Received
7
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3452 | 11-18-2022 09:06 AM | |
| 5164 | 11-15-2022 05:46 PM | |
| 3443 | 10-12-2022 03:18 AM | |
| 2624 | 10-11-2022 08:52 AM | |
| 6479 | 10-08-2022 08:23 AM |
11-19-2022
01:35 PM
I am bumping this question in hopes someone might know of a better solution
... View more
11-18-2022
08:25 AM
A couple of reasons: At the base level, operations in the UI all use the REST API - if you use chrome's devtools and go to the networking tab, you can see the exact REST API route used for every click you do on the canvas. If you'd ever want to automate something you know you can manually do in the UI, using this trick you can perfectly replicate it with with RESTful calls. In the same vain, the REST API has far more routes/operations available. The CLI tools seems to have around ~70 operations, whilst the rest api seems to have over 200. The CLI commands are part of the nifi toolkit. Whilst the toolkit does get updated, I believe it does not get the same level of attention as nifi's main codebase and as such it'd be better not to rely on it completely. This is not to say you can't use the CLI tool - rather, just my opinion on the matter and some insight for how my team writes automations on nifi 🙂
... View more
10-14-2022
02:07 AM
thanks ,Green! your idea helped me solve the problem
... View more
10-08-2022
10:15 PM
1 Kudo
The ReplaceText procesor has a prepend/append mode which might be of help
... View more
10-08-2022
08:23 AM
1 Kudo
Hey @Fredi , I believe the answer for your problem is the processor UpdateRecord. Update record allows you to directly manipulate the fields in your file content. You add dynamic properties to the processor where the key of the property is /<field> (so in your case, '/api_value'), and in the value of this dynamic property you can write down some logic to determine what value to insert into api_value. In the processor, there is a field called "Replacement Value Strategy", which defines how the value of the property will be read. If you set this to "Record Path Value", it means you can now give a path to a different field in your file (url_value!) - I can't test this right now because I'm not at my office, but I'm not entirely sure whether you can manipulate the result after giving a record path (to extract the api_value from the evaluated url_value). Regardless, I'm just about 100% sure this can be done with two processors - One EvaluateJsonPath to extract the url_value into an attribute, then UpdateRecord that uses the 'Literal Value' replacement strategy - with this strategy, you can just add a property with key '/api_value' and value '${url_value}' (or whatever attribute name you gave to the extracted url_value) and once you can access url_value with the expression language (via ${url_value}) you can use all the available functions to manipulate expression language variables. Here's an article with a couple of examples on UpdateRecord: https://community.cloudera.com/t5/Community-Articles/Update-the-Contents-of-FlowFile-by-using-UpdateRecord/ta-p/248267 (I noticed in the article they used some recordPath related functions like "replaceRegex", so I believe there might be a way to use these and then limit the entire issue to just one UpdateRecord processor! Sadly I'm not too familiar with these myself and this was the first time I've seen them) And here's the expression language documentation: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html You can see there are lots of useful functions to extract your api_value once you have ${url_value} as an attribute variable, for example "substring", "find"/"replace", "ifElse", etc. all of which you can try and use to ensure only the api_value is left in the end. Hope this helps! I'm sure using ReplaceText and possibly JoltTransform could provide alternate solutions to the issue, however I believe UpdateRecord is the cleanest solution for this and truly makes use of the processor's abilities. If you struggle to use it correctly, you can reply with an example json and expected output and I'll try to write down the flow when I have time.
... View more
10-06-2022
11:42 AM
Brilliant! That does the trick. Thanks!
... View more
07-12-2022
03:08 PM
@Chakkara As far as i remember , distributed cache is not having consistency . You could use Hbase or HDFS for storing the status of success or failure of the processors for downstream application. Once you saved the Success and Failure at Hbase . You can retrieve it from the Hbase processor using the row ID. Build a REST API NiFi flow to pull the status from Hbase for example HandleHTTPRequest --> FetchHbaseRow - HandleHTTPResponse You can call the HTTP API (Request and Response) via shell script/curl and call the script from Control-M.
... View more
08-10-2021
02:11 PM
1 Kudo
@hegdemahendra I have found this article by Pierre V. Where he goes into deeper detail about the logback.xml file. He mentions something that might be relevant to what you're looking for - the following two passages are what caught my eye: ``` "We can also define new appenders in the log configuration file and change it according to our needs. In particular, we could be interested by the SMTP Appender that can send logs via emails based on quite a large set of conditions. Full documentation here." "Obviously you can also configure this configuration file so that NiFi log files integrate with your existing systems. An idea could be to configure a Syslog appender to also redirect the logs to an external system." ``` I myself have never done something like this, but it sounds like a step in the right direction for directly writing logs from nifi to mongo.
... View more
- « Previous
-
- 1
- 2
- Next »