@Ytch  NiFi is designed to move data and the ListS3 processor is designed to list s3 objects/files and not directories.   On the downstream side, NiFi does not have a processor that just create directories.  Many out processors will create a missing directory if content is to be written must be written to a directory path that does not exist on the target.  But without content those processor will not create any directories. I am clear on what yoru use case is here, but based on yoru description it is not something any of the stock processors would do.  You would need to write something custom (either your own processor or some custom script you can execute via aNiFi scripting processor). Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Ytch  All components on the NiFi canvas are executed as the NiFi service user and not as the user currently authenticated into the NiFi service.   So what you should do is from each host in your NiFi cluster (do on every host since any one of the hosts can be elected as the primary node at any given time), open a command prompt window/console window, become the user that owns the NiFi process, and manually ssh/sftp to the target SFTP server.  You will likely be prompted to add the target SFTP server to your known_hosts file for the NiFi service user.  NiFi SFTP processor has no way of doing this interactive step.  After successfully adding the SFTP to the known_hosts file for the NiFi service user, go back and try to start the GetSFTP or ListSFTP processors again to see if your issue is resolved. If not, please share your GetSFTP and ListSFTP processor component configurations. Also check the nifi-app.log for any exceptions or log output related to these processors. If no log output, you could also try enabling debug in the NiFi logback.xml for these processor classes to see what additional log output may be produced that could be useful here. classes for these processors are: org.apache.nifi.processors.standard.GetSFTP org.apache.nifi.processors.standard.ListSFTP new log lines would look like this that you would add to logback.xml: <logger name="org.apache.nifi.processors.standard.GetSFTP" level="DEBUG"/> <logger name="org.apache.nifi.processors.standard.ListSFTP" level="DEBUG"/> Simply add them in logback.xml where you see similar lines already. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Hi @Ytch , The way I was able to get to work is through the following instruction: 1- After you have download the jdbc drive from here: https://learn.microsoft.com/en-us/sql/connect/jdbc/download-microsoft-jdbc-driver-for-sql-server?view=sql-server-ver15 you should be able to find the following dll "mssql-jdbc_auth-8.2.2.x64.dll" under the path: ..\sqljdbc_8.2\enu\auth\x64 2- Copy the dll and place under the JAVA Home path bin folder: .. \Java\jdk-21\bin 3- Adjust the DB Connection URL in the DBCPconnectionpool  controller service by adding and setting "integrated security"  flag to true: jdbc:sqlserver://{SQL SEVER NAME};databaseName={DB NAme};integratedSecurity=true; 4- Restart Nifi If that helps please accept solution. Thanks   ... View more

hi @Ytch , To use the PutDatabaseRecord you need to have the column names defined in whatever input format you have (CSV, Json....etc.) and the column name should match what is in the DB table ( see also the "Translate Field Option" if not exactly the same). If you know the column order in the CSV file you can add a header using the ReplaceText processor as described here: https://community.cloudera.com/t5/Support-Questions/How-to-add-customer-headers-to-csv-file-in-NIFI/m-p/285457   If you find this is helpful please accept solution. Thanks ... View more