hi @wffger2, I just tested on my local machine and if I download the NiFi Flow definition for a Processor Group, I can find each processor ID in that Processor Group. Are you sure you are not looking in a different export from a different environment? If the same exact flow is imported on a different environment (meaning other servers) and you download the Flow definition, you will not have the same IDs. For example, in your Process Group on Dev you have an Processor PutFile which has the id 1d29eb87-d66f-31ba-d653-93b704036191. If you exported everything on the UAT Environment that Processor Group and the Processor will not have the same exact IDs as on DEV because NiFi generates new UUIDs for those elements --> maybe the ID 1d29eb87-d66f-31ba-d653-93b704036191 is already in use on the UAT environment by another processor. I might be wrong in my post and you could wait for another answer, as somebody might know otherwise. ... View more

Assuming that there would be a way to check for flow files which went through failure, how would you extract them? Manual search within the Data Provenance Menu from within the GUI? Or using the REST API? Now, what I do know is that you cannot identify those files that easy, because they do not write any specific lines within the Data Provenance. If you have a processor right after the failure queue (or if you terminate the failure queue in the processing processor) you can query using that Component ID and identify the type = DROP, meaning that those files have been "processed" (Indicates a provenance event for the conclusion of an object's life for some reason other than object expiration). More about the types can be found here: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.1.2/bk_user-guide/content/provenance_events.html ... View more

Well if I understand your flow logic correctly, what you are trying to achieve is not really possible, so please correct me if I am wrong. In your QueryRecord you receive input from the InputPort and from the QueryDatabaseTable Flow as well. Based on the FlowFile coming from that InputPort, you are trying to filter the data, using a SQL which is built on the input coming from the QueryDatabaseTable Flow, right? Basically you want to filter the InputPort Data (FlowFile1) based on the ${value} from QueryDatabaseTable (FlowFile2). Based on my current experience with NiFi, such a thing is not possible because the processors work on a single FlowFile and not with 2+ FlowFiles (unless merged). But maybe somebody with more experience can tell you otherwise. You would need to rethink your flow and build something else --> hence the idea with LookupRecord/LookupAttribute. ... View more

Have you thought on using LookupRecord instead of QueryRecord? In LookupRecord you can define your RecordReader, to be able to read your JSON input data, as well as a RecordWriter, to write your data in whatever format you want. In addition to this, you have the Lookup Service, where you can use the DatabaseRecordLookupService (as Controller Service) and connect to your database and looking up a Key-Value Combination based on a table name. Or you can try with LookupAttribute if you want to add an attribute in your JSON File with the value you want to use to perform the routing of the file. You can use in LookupAttribute the SimpleDatabaseLookupService where you connect to your DB and define the table in which you query the Key-Value combination. As I did not fully understand your question (nor do I see one), the provided answer is a generic one. ... View more

hi @Sivagopal , What NiFi Version are you running and what property did you set in nifi.sensitive.props.algorithm? As far as I know, since version 1.14, the algorithm PBEWITHMD5AND256BITAES-CBC-OPENSSL might have been deprecated and you need to use a newly supported/updated version, which can be found here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#updating-the-sensitive-properties-algorithm --> Property Encryption Algorithms Section. ... View more

Hi @Ray82 , I am shooting here in the blind, but have you tried setting the command Path = /usr/bin only and the Command arguments = rm -f and_everytyhing_else? I am asking because I am using the ExecuteStreamCommand similar as you are but with python and within the command path i set python and the entire thing I have to execute under the command arguments and it works like a charm. It might not be the recommended way of using this processor but at least I got my things running. ... View more

well if you want to modify the port, you have to stop nifi, modify nifi.properties and restart nifi. But without knowing how you configured NiFi ( content of nifi.properties) nobody can tell you how to solve your issue. Maybe you can add the content of nifi.properties here and afterwards we can guide you further. Nevertheless, how Matt stated below, you need to check your application logs and see if NiFi started correctly. If NiFi starts correctly, you will see a log line saying that NiFi UI (User Interface) is available at a specific URL. You need to take that URL and paste it in your browser. If such a line is not available, than you need to check the logs to see what the error is. ... View more

Without knowing what you have configured, I will try and assume some of your configurations: - based on your installation, I assume that you are using HTTP and not HTTPS, right? If I take a look in your screenshot, I can see that you are running on port 8443, which is the default port configured within nifi.web.https.port. I assume that you wrote the host and the port in the nifi.web.http.host and the nifi.web.http.port properties and not the HTTPS properties, right? If not, this is what might be causing your issue. - besides that, have you check the logs for any errors and so on? I am talking here about the bootstrap logs (nifi-bootstrap.log) and the application logs (nifi-app.log). If everything starts normally, within the application logs you should see the link where NiFi is available. Otherwise, there is an error and you have to take a look. PS: it would help if you could attach the nifi.properties here, with all the confidential information written as standard placeholders (like <hostname>, <password> , etc) ... View more