Member since
07-30-2019
3432
Posts
1632
Kudos Received
1012
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 105 | 01-27-2026 12:46 PM | |
| 511 | 01-13-2026 11:14 AM | |
| 1116 | 01-09-2026 06:58 AM | |
| 951 | 12-17-2025 05:55 AM | |
| 451 | 12-17-2025 05:34 AM |
05-31-2018
05:37 PM
@Jason Sphar The GetTCP processor is designed to send data to a target TCP port. It sounds like you are trying to recieve data via TCP port in to NiFi? - If that is the case, you want to be using a ListenTCP processor instead that will bind to the tcp port you will be sending your data to. - Thanks, Matt
... View more
05-31-2018
05:32 PM
1 Kudo
@Brandon Spratt You are absolutely correct that older versions of NiFi allowed user to configure both the "nifi.web.http.port" and "nifi.web.https.port" properties in NiFi at the same time. NiFi would at that time then allow any user to access the NiFi UI via the unsecured http URL or via the HTTPS secured URL. - The need client auth property never had anything to do with user access to NiFi. That property was specific to node to node communications only. False simply meant that nodes acting as clients talking securely to other nodes (acting as server) did not need to verify trust of the server's presented certificate in the TLS/SSL handshake. - Allowing user to access a secured NiFi via a http configured URL presents a huge security risk, so that capability was removed. Once NiFi is secured by enabling https, even if http is still configured access will be blocked to the http URL(s). - Can you elaborate on the use case for allowing users to anonymously access a secured NiFi? This opens a security hole where any client (user or other system) can get access to the data passing through that NiFi. - As a work-around you could generate a user certificate that gives all access to every NiFi policy and make that certificate freely available to all your users. - Thank you, Matt - If you find an answer the best addresses your initial question, please take a moment to login to the forum and click the "Accept" link below that provided answer.
... View more
05-31-2018
01:03 PM
2 Kudos
@Rahul Kumar - Where are you getting the nar or jar for this custom influxdb processor from? - Once you have this custom processor nar/jar file, you will want to create a custom lib directory to place it in and then configure your nifi to read that additional custom lib folder location. This is done by adding a new key/value pair to the nifi,properties configuration file: nifi.nar.library.directory.custom-lib=/<path to custom lib dir>/custom-lib - A restart of NiFi will be required before NiFi will load any newly added custom libraries. At that point the new processor shoudl be available to add to the canvas just as you add any other processor. - Thank you, Matt
... View more
05-31-2018
12:52 PM
1 Kudo
@Rahul Kumar - The NiFi "run" directory is where NiFi creates the "nifi.pid" and "nifi.status" files during startup. - You should make sure that the user that owns teh NiFi process has the ability to create files inside the /home/rahul/nifi-1.6.0/run directory. - Your NiFi should have failed to start. Take a look in your nifi-bootstrap.log file to see when the last time you see: org.apache.nifi.bootstrap.Command Starting Apache NiFi... Does the timestamp for that line correspond with your latest attempt to start NiFi? My thought is that their was already a NiFi process running and what you see in browser is really not the same process as to what you think you just started. Run the following command to see what nifi is actually running and how long it has been running: ps -ef|grep "TIME\|nifi-1" - Thank you, Matt
... View more
05-31-2018
10:19 AM
@Mike Wong Does the listFile exhibit the same behavior or does it list your file correctly? - The fact that the logs shows it the processor yielding tells me it found no work to do (meaning no files to list). It yields so that it does not consume not stop CPU looking for work that does not exist. - Did you check your properties for leading or trailing whitespace? Did you try removing the "\" from your file filter? - Thanks, Matt
... View more
05-30-2018
01:05 PM
@Mike Wong You may also want to verify your processor configuration for both input directory and file filter to make sure you do not have any leading or trailing spaces. Spaces are treated as valid characters to NiFi which can result in NiFi not finding the file or even the directory.
... View more
05-30-2018
12:59 PM
1 Kudo
@Mike Wong Did you try becoming the nifi user (sudo su - nifi) and try to navigate the the 2008.csv file and view it? - Another option would be to enable debug logging on the getFile processor to get more details on what is going on here. Doing so requires you to add a new line to the NiFi logback.xml file: - <logger name="org.apache.nifi.processors.standard.GetFile" level="DEBUG"/> - No need to restart Nifi when editing the logback.xml file (this is one of the only conf file in NiFi you can edit that will not require a restart). - Wait 30 seconds after adding this line. Then star tailing the nifi app log: # tail -F ../logs/nifi-app.log - Go to you canvas and start or stop then start your GetFile processor. What output do you see in the nifi-app.log? - Thanks, Matt
... View more
05-30-2018
12:47 PM
@Mike Wong *** Important Forum tip: Please try to avoid responding to an existing "Answer" by starting a new "Answer". Instead use the the "Add comment" to respond to an existing "Answer". The forum offers no guaranteed order to answers which can make following a conversation difficult.
... View more
05-30-2018
12:35 PM
2 Kudos
@Rahul Kumar Upgrading NiFi versions is not a very hard process. - 1. Install the new version in a parallel manor. for example: - lets assume 1.4 is installed at /opt/nifi/nifi-1.4.0/ - Install 1.6 in /opt/nifi/nifi1.6.0/ 2. Use the configurations from the various NiFi configuration files found in the nifi conf directory of 1.4 to configure the same named configuration files in the new 1.6 NiFi conf directory. *** It may be possible at times to simply copy the configuration files from one version to another, but be cautious here of new properties that may be added to these configuration files between releases. NiFi 1.6 should be configured to point at the same 4 repositories (database, content, flowfile, and provenance (if best practices were used these are located on separate disks outside of base NiFi install path)) being used by the old NiFi 1.4. If the This is a NiFi cluster, 1.6 should be using the same external zookeeper as the 1.4 used (If embedded ZK was used, recommend in stalling a n external ZK for new NiFi and migrating its content from embedded ZK to external ZK). New NIFi should also point to same local NiFi state directory. 3. Stop your nifi 1.4 instance/cluster. Copy the flow.xml.gz, users.xml (if secured), and authorizations.xml (if secured) from NiFi 1.4 to NiFi 1.6. 4. Start new NiFi 1.6 version. NiFi 1.6 will start, load the copied flow.xml.gz file, read in Flowfiles from the same flowfile and content repositories nifi 1.4 was using before it was shutdown. Your new NIFi will pickup processing right where the previous version left off when it was shutdown. - *** Be mindful if you have any added custom nars/jars you added tp the previous nifi version. Those will need to be moved to new Nifi as well. Recommend placing any custom nars/jars in a separate custom lib directory rather in NiFi's default lib directory. - Thank you, Matt - If you find an answer that addresses your question best, please take a moment to login to the forum and click the "Accept" link below that provided answer.
... View more
05-30-2018
12:19 PM
1 Kudo
@Siddharth Sa From your image it appears that you are auto terminating the failure relationship on your putSQL processor? - Assuming the misconfiguration in your updateAttribute processor resulting in failure of every FlowFile passed to the putSQL, those FlowFiles would have all been routed to the failure relationship of the putSQL processor. It is rare that user would auto terminate a "failure" relationship as it means data is being deleted. A more typical design is to route "failure" relationships to a dumb processor that is not enabled (like an updateAttribute processor or even a funnel). This would have allowed you to redirect the connection containing that failure relationship back to your fixed updateAttribute processor resulting in all the failed data being reprocessed. - NiFi does if enable archive FlowFiles based on configured thresholds. It is possible to perform a provenance search on the FlowFiles with a "DROP" event recorded by the putSQL processor. The drop event would occur for each FlowFile routed to failure and deleted by putSQL. While not elegant, you may be able to select each failed FlowFile one by one, open the lineage, and replay the FlowFile at the "updateAttribute" point in the lineage history. You would control the sequence of processing by the older in which you replay each FlowFile. There is no bulk replay capability. - Thank you, Matt
... View more