Member since
04-17-2023
17
Posts
0
Kudos Received
0
Solutions
05-12-2023
04:28 AM
@ushasri, If you are reading an Excel File, you can use an ConvertRecord processor, in which you define an Record Reader (CSVReader most likely) where you tell NiFi how to read the value (value separator). Next, with the help of a Record Writer (I assume CSV again or maybe Avro) you define how your data should look like, including the header. For both the Reader and the Writer you will have to define a schema, where you can do the replace of blank space with underline.
... View more
05-04-2023
06:30 AM
You do not install the Cloudera version on your laptop 🙂 You need the Cloudera DataFlow for Public Cloud (CDF-PC), meaning that we are talking here about a license and some services. As @steven-matison already provided you with the perfect answer for your question, he might also be in the position to further assist you with everything you need to know about the Cloudera Data Flow and their Public Cloud. Unfortunately I am still learning about what Cloudera offers and how, so I am not the best one to answer your question. If you are going to use NiFi for some real data processing, I strongly recommend you to have a look to Cloudera Data Flow, as this will solve many issues and headaches 🙂
... View more
04-25-2023
04:15 AM
1 Kudo
The file can be found within your NiFi logs folder, where you have your other logs as well.
... View more
04-24-2023
12:38 AM
hi @ushasri, The file you are looking for is called flow.xml.gz and can be found on each NiFi node, within the conf folder. More information can be found here: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html flow.xml.gz: Everything the DFM puts onto the NiFi User Interface canvas is written, in real time, to one file called the flow.xml.gz. This file is located in the nifi/conf directory by default. Any change made on the canvas is automatically saved to this file, without the user needing to click a "Save" button. In addition, NiFi automatically creates a backup copy of this file in the archive directory when it is updated. You can use these archived files to rollback flow configuration. To do so, stop NiFi, replace flow.xml.gz with a desired backup copy, then restart NiFi. In a clustered environment, stop the entire NiFi cluster, replace the flow.xml.gz of one of nodes, and restart the node. Remove flow.xml.gz from other nodes. Once you confirmed the node starts up as a one-node cluster, start the other nodes. The replaced flow configuration will be synchronized across the cluster. The name and location of flow.xml.gz, and auto archive behavior are configurable. See the System Administrator’s Guide for further details.
... View more
04-18-2023
02:41 AM
hi @ushasri, Can you provide some more details about your flow? Without knowing what you are doing in your flow, I can only tell you that you can use the Expression Language from NiFi and extract the current time send it into your stream. The current time can be called as an example like: ${now():toNumber():format('yyyy-MM-dd')} Next, you can use an UpdateRecord Processor, add the attribute to your newly defined column and send it to further processing. More about NiFi's Expression Language: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
... View more