@ushasri, If you are reading an Excel File, you can use an ConvertRecord processor, in which you define an Record Reader (CSVReader most likely) where you tell NiFi how to read the value (value separator). Next, with the help of a Record Writer (I assume CSV again or maybe Avro) you define how your data should look like, including the header. For both the Reader and the Writer you will have to define a schema, where you can do the replace of blank space with underline. ... View more

You do not install the Cloudera version on your laptop 🙂 You need the Cloudera DataFlow for Public Cloud (CDF-PC), meaning that we are talking here about a license and some services. As @steven-matison already provided you with the perfect answer for your question, he might also be in the position to further assist you with everything you need to know about the Cloudera Data Flow and their Public Cloud. Unfortunately I am still learning about what Cloudera offers and how, so I am not the best one to answer your question. If you are going to use NiFi for some real data processing, I strongly recommend you to have a look to Cloudera Data Flow, as this will solve many issues and headaches 🙂 ... View more

@ushasri Since you describe going from 1.20.0-RC1 to 1.15.3-RC1 it is safe to assume those features are not available in 1.15.3. ( Right click on canvas --> Enable all control services & disable all control services) are some of the newer UI features. ... View more

The file can be found within your NiFi logs folder, where you have your other logs as well. ... View more

hi @ushasri,   The file you are looking for is called flow.xml.gz and can be found on each NiFi node, within the conf folder.   More information can be found here: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html flow.xml.gz: Everything the DFM puts onto the NiFi User Interface canvas is written, in real time, to one file called the flow.xml.gz. This file is located in the nifi/conf directory by default. Any change made on the canvas is automatically saved to this file, without the user needing to click a "Save" button. In addition, NiFi automatically creates a backup copy of this file in the archive directory when it is updated. You can use these archived files to rollback flow configuration. To do so, stop NiFi, replace flow.xml.gz with a desired backup copy, then restart NiFi. In a clustered environment, stop the entire NiFi cluster, replace the flow.xml.gz of one of nodes, and restart the node. Remove flow.xml.gz from other nodes. Once you confirmed the node starts up as a one-node cluster, start the other nodes. The replaced flow configuration will be synchronized across the cluster. The name and location of flow.xml.gz, and auto archive behavior are configurable. See the System Administrator’s Guide for further details. ... View more

Hi @ushasri, There are plenty of ways to perform such a thing, but here are three ways I have already implemented the alert in my flows so far: You can have an PutEmail at the end of each failure Queue and in that why, when a message gets rejected you will get notified by email. This can be done strictly in NiFi. You can create a monitoring system which will interrogate your Bulletin Board using NiFi's REST API and send you whatever alerts you want. This can be done either directly inside NiFi or with a shell/python script on your server. You tail the nifi-app.logs on each server and extract only the errors you are interested in. This solution can be implemented both in NiFi and on your local server. ... View more

hi @ushasri, What processor are you using to save the data within your DB and how do you build up the query? If you want to add the filename to the record which gets inserted within your table, you should use an UpdateRecord Processor to generate a new column and populate it using the ${filename}. Next, the generated record should go further down your line up until it gets inserted in your database. ... View more