Regarding step 2, You have to determine the HDFS directory where NiFi PutParquet will write the files, and who has access to this directory path on HDFS, that user's user principal and associated keytab is required. I assume if HDFS is secured by Kerberos then the users has to obtain the Kerberos ticket by running kinit with user principal and Keytab to access it at the HDFS  side.     About step 3. No need to install Kerberos service, NiFi needs a Kerberos client on NiFi hosts which is by default installed on most Linux OS. client config files located at /etc/krb5.conf , to which Kerberos server NiFi PutParquet should connect in order to obtain kerbeors ticket using configured user pric/keytab details, user has updated Krb5.conf file with Kerberos Server details. I mean KDC realm details.   If you found this additional response assisted with your issue, please take a moment and click on "Accept as Solution" below this post. Thank you     ... View more

Found it , So you are using this  https://github.com/frank-dkvan/nifi-file-identity-provider-bundle which is a custom identity provider, and that is the reason it is not working if you upgrade the cluster. I would advise rework on you requirement related to how you would like to get Authenticated and Authorized in NiFi and I would suggest using the default ones which come from NiFi.  Thank you      ... View more

By default, there is no such provider name "file-identity-provider" with calsscom.batchiq.nifi.authentication.file.FileIdentityProvider.  nifi.security.user.login.identity.provider was always ldap or kerberos but from Apache nifi 1.14 , SingleUserLoginIdentityProvider is added as by default login.identity.provider. Running NiFi with standard HTTP is basically an anonymous remote code execution platform. Single User mode is added f puts a lock on the door, which offers basic protection.  This is applicable to both Apache NiFI and Cloudera CFM NiFi Versions.         ... View more

Is NiFi is secure? The latest error to do with how you want to setup Authentication in NiFi and I guess the current setting is not known to nifi , Please check what is set for property value in nifi properties file  nifi.security.user.login.identity.provider It should be , ldap-provider or kerberos-provider    Pleaser refer https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication   Thank you  ... View more

Thank you for the updated information. So if you are not listing from each and every sub dir under root dir then you can use a regex pattern that just satisfies directory of your interest, Property Direceoty supports expression language.   Thank you  ... View more

It seems the same question discussed at   https://community.cloudera.com/t5/Support-Questions/Passing-list-of-directories-to-ListHdfs-Processor/td-p/364798 To address the limited number of tables is to limit the number of files in HDFS terms, Listing strategy can only be controlled by File Filter and file-filter-mode on what can be listed. The Listting processing has a two-step -->What to list " controlled by the filter " -->From Whereto list  "where in your case looks like sub directory under root has a widespread and number of nested sub dirs are huge where the processor is spending time on recursive searching.    If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan      ... View more

From provided details, It's not clear how the subdirectory structure looks immediately after root directory. The approach could be using multiple ListHDFS configured with an immediate subdirectory of root with recursive subdirectories as true. The flow will look like a Multiple of ListHDFS (Primary )--->All connected to one FetchHDFS (All Nodes ) ith LB at connection.   If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan        ... View more

@jacektrocinski Yes, Sir, you are right,  I put more focus on read attribute section of the document which has none but missed seeing Additional details.  Kudos!!   ... View more

ConnectWebSocket does not read any details from Flow file attribute from an incoming flow file. If you are a Cloudera customer, feel free to file a support ticket with us, we should be able to help you on this with a longer engagement.     Thank you  ... View more