@Kiranq  This error shared: 2024-10-17 08:35:19,764 ERROR [Timer-Driven Process Thread-5] o.a.n.c.s.StandardControllerServiceNode StandardControllerServiceNode[service=CSVRecordLookupService[id=a8b84b00-b0ee-31c8-dbda-7e7e9795ba4b], name=CSVRecordLookupService, active=true] Encountering difficulty enabling. (Validation State is INVALID: ['CSV File' is invalid because CSV File is required, 'Lookup Key Column' is invalid because Lookup Key Column is required]). Will continue trying to enable. Indicates that NiFi is trying to enable a NiFi Controller services loaded from the flow.json.gz during startup, but cannot because it's configuration is invalid.  It is complaining about the configuration of the "CSV File" and "Lookup Key Column" properties. Have you tried starting your NiFi with the following setting in your nifi.properties file set to "false": nifi.flowcontroller.autoResumeState=false This will start NiFi and all components on the canvas will not be started during startup. Also if you NiFi is at the point it is trying to enable components on the canvas, Your NiFi is up and running. As far as the screenshot error, have you verified ownership and permissions on that directory path.  Permissions can be an issue if you started the NiFI service as different users at some point in time resulting in some files created on startup having different ownership.    Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Abibee04  Please start a new Community question with the details of your query. I am not clear what you mean by "recover the registry", so the more detail you can provide the better in that new community question by sharing what step you performed and what you are looking to accomplish. Thank you, Matt ... View more

If you meant Storing Apache NiFi Versioned Flows in a Git Repository hosted on Azure DevOps, then the steps should be similar to the steps you followed to connect to Github. The appropriate flowPersistenceProvider is GitFlowPersistenceProvider. You can follow the steps as mentioned here: Storing Apache NiFi Versioned Flows in a Git Repos... - Cloudera Community - 248713 Or the video: https://youtu.be/kK7eVppg9Aw   Only changes you would have to do is to generate PAT in Azure DevOps with permissions on code read, write, modify. P.S. The solution is only available for Git under Azure DevOps, and not for TFS. ... View more

Hi @Kiranq , This a basic case of data enrichment and there are multiple ways how you can handle this. If for example instead of the function you have a table with a unique id then you can use LookupRecord with DatabaseRecordLookupService or even SimpleDatabaseLookupService . All you have to do here is basically specify the path where you want the new value to be inserted as dynamic property. However since you have a function instead of simple table\view , then the second option is to use ForkEnrich\JoinEnrich processors . The nice thing about those processors is that you have multiple strategies how you can join the data together as you can read here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apache.nifi.processors.standard.JoinEnrichment/additionalDetails.html   Let me know which one works for you better and let me know if you face any issues regarding using either. If you find this is helpful please accept solution Thanks       ... View more

@Kiranq  all components (processors, controller services, reporting tasks, etc) execute as teh user that owns the running NiFi service.  So that NiFi service user would need to be able to execute the local system command.  The user who authenticates themselves to access the NiFi UI is not the user used to run the components. Have you tried as the NiFi service user to execute your python code from command line? If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

I resolved the issue. I had DepartmentId in my avro schema however, I was getting DepartmentID in the input. Schemas are case sensitive. Now, that the string values are appended in the DepartmentID column, I am using RouteOnContent processors and using ${DepartmentID:equals('Undefined')} to match if the DepartmentID' values are of undefined or null type. It is not identifying the "Undefined" values. is the regex statement correct? Content of flow file is: [ { "TenantId" : "FF369226-0EDA-4D0D-B48C-6B27F404ECCB", "DepartmentID" : "Undefined", "Name": "xyz" } ] ... View more

@Kiranq, What I found is that the JoltTransformRecord expects only single record to work with hence the name. I noticed when I try to pass an array I was getting the error "...error transforming the first record", however if I pass just one json record it works. If you have an array of json\csv and you are looking to split and process each record individually then I would suggest that you split the records before the JoltTransformRecord. If you dont want to split the array then I recommend using JoltTranformJson first and then use Convert Record processor to convert to CSV. ... View more

@Kiranq  The NiFi rest-api endpoint for obtaining token is not rest-api/access. It is rest-api/access/token https://<nifi-hostname>:<nifi-port>/nifi-api/access/token  and data passed like: --data-raw 'username=<username>&password=<user password>' Then the token needs to be passed with every subsequent rest-api request. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

As far as I know, NiFi leverage Jython and would currently limit you to using Python 2.7 compatible code and only modules written in pure Python. Home | Jython ... View more

@Kiranq   Why are you using ExecuteScript?  You can setup a DBCP (DataBaseConnectionPool) controller service with your sql connection and driver file. Make sure that jdbc driver is found on all nifi hosts.   Then, you are able to use any processors that reference a DBCP Controller Service.  For example:  ExecuteSql. ... View more