Use Influx or Prometheus as a "storage" layer. Influx has a better NiFi Processor, than the built-in NiFi. You can download the nar from Influx's site. ... View more

@rafy ,   This is tricky and whatever solution we come up with here cannot be guaranteed to always work. It will always depend on the content that comes in the responseMessage field.   For the particular example that you shared above, this works: The settings are:  Search Value: (?s)(.*\"responseMessage\"\s*:\s*\"[^\"]*|\G[^\"]*)\"(?!,) Replacement Value: $1\\" Evaluation Mode: Entire text   You can check it out here: https://regex101.com/r/paPPhK/1   Cheers, André   ... View more

Thank you. ... View more

You need to implement some kind of check to see which db is running.  A small select would do on db1 from nifi - on success set an attribute (pre-added to the file) to db1 on failure update to db2 then the attribute will always have the value of the running db. Both would lead to UpdateAttribute processors then they can lead to the single update using the attribute for the db to connect to. FYI .. can be a good idea to rename the queues from success and failure to show DB1 and DB2 or whatever makes sense to you. Hope this helps 🙂 ... View more

You can try the following Jolt spec in a JoltTransformRecord processor to change the name of one or more columns: Jolt Transformation DSL: Chain Jolt Specification: [ { "operation": "shift", "spec": { "*": "&", "surname": "lastname" } }, { "operation": "remove", "spec": { "surname": "" } } ]   ... View more

You can set the run schedule of the processor to cron driven [1] and give it your cron expression from there. Note that this cron is not the typical OS cron syntax and its based off quartz cron scheduler [2] To build a cron expression this is a good online tool to do so [3]   [1] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#scheduling-strategy [2] http://www.quartz-scheduler.org/ [3] https://www.freeformatter.com/cron-expression-generator-quartz.html ... View more

@rafy, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. ... View more

@rafy  You'll want to read up on the documentation on the Apache MiNiFi page here: https://nifi.apache.org/minifi/index.html Since MiNiFi does not provide a UI from which you can construct a NiFi dataflow, you will need to build the dataflow that you will use on your MiNiFi using a NiFi installation.   The converter toolkit is what you can  then use to change your NiFi dataflow template into the necessary MiNiFi yaml file. You may also find these community posts helpful: https://community.cloudera.com/t5/Support-Questions/How-send-data-from-nifi-to-minifi-same-config/td-p/325183 https://community.cloudera.com/t5/Community-Articles/Ingesting-Log-data-using-MiNiFi-NiFi/ta-p/248154 If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt ... View more

Hello, I'm a learner & i would like to use the method you made mentioned here to collect logs in a remote server & send to Nifi. Please, can you put me through because i have been battling with how to build a msi before the real implementation. Thank you so much. ... View more