@medloh  The article you are using for reference is old and bit out of date. As part of the  work that went into NIFI-3921 the schema properties within the putParquet processor were removed.  Before these changes were made at the time of that article you referenced, you had to set the schema properties and they had to match the schema properties set in the recordReader.   With the changes the processor simply gets them from the reader so they do not need to be configured a second time in the processor properties. Also at time of that article there was no parquetReader or ParquetRecordSetWriter controller services.  Now that NiFi has a ParquetReader and writer, you can use the ConvertRecord processor to read a source FlowFiles and convert it parquet within your dataflow and then have freedom to use whatever processor you want downstream in your dataflow to write out the parquet content.  You can think of the putParquet as a combination of ParquetRecordSetWriter and putHDFS with a selectable recordReader only. Hope this helps, Matt ... View more

@BhaveshP  I am in complete agreement with @tusharkathpal response. But you should be able to work around this issue through a configuration change in your nifi.properties file. nifi.web.proxy.host=dev.example.com:<port number> Property description: A comma separated list of allowed HTTP Host header values to consider when NiFi is running securely and will be receiving requests to a different host[:port] than it is bound to. For example, when running in a Docker container or behind a proxy (e.g. localhost:18443, proxyhost:443). By default, this value is blank meaning NiFi should only allow requests sent to the host[:port] that NiFi is bound to. Since the hostname your client is using does not match any SAN in the individual nodes certificates, the above property allows NiFi to accept this additional hostname. The other option is to create new certificates for each of your NiFi nodes where "dev.example.com' is added as an additional SAN entry. Hope this helps, Matt   ... View more

@Umakanth    Any chance you are running a NiFi cluster (multiple NiFi nodes) or you have multiple systems all trying to consume the same data from this same SFTP server? It is possible that one host finished reading the file first and removed it before the other hosts could finsih reading the same file. SFTP is not a cluster friendly protocol and if using this processor in a NiFi Cluster, this processor should be configured to execute on "primary node" only.  Otherwise all nodes in your cluster will be fighting to consume the same source files of which you can expect to see exceptions. The GetSFTP processor is also a deprecated processor in favor of the newer listSFTP and FetchSFTP set of processors.  The newer processors allow you listSFTP (primary node only and produces 0 byte FlowFiles) ---> load balanced connection (balances FlowFiles across all nodes in cluster)--> FetchSFTP ( Execute on all nodes. Retrieves specific content per FlowFile). Hope this helps, Matt ... View more

@Arash    In your 4 node NiFi cluster, what value do you have set in the "nifi.remote.input.host" property in the nifi.properties file for each of the 4 nodes?  It should be the FQDN for each node and not be the same value on all 4 nodes. Form the host where MiNiFi is running, can all 4 of those FQDNs be resolved and reachable over the network?  If not, MiNiFI RPG is only going to be able to send successfully to one FQDN it can reach. When the RPG is started it reaches out to the URL configured in the RPG to obtain S2S details from the target host.  That target host collects the host details for all currently connected nodes in the cluster and communicates that back to the client (MiNiFi).   If all 4 nodes report the same configured FQDN in the "nifi.remote.input.host" property, then client only knows of one FQDN to which it can send FlowFiles over Site-To-Site (S2S). To improve redundancy in the RPG, you can provide a comma separated list of URLS in the RPG configuration so if any one node is down, the RPG can try fetch S2S details from the next host in the comma separated list. Hope this helps, Matt ... View more

@JorgeO    Looking at your nifi-user.log output we see the caused by line as: Caused by: org.springframework.dao.IncorrectResultSizeDataAccessException: Incorrect result size: expected 1, actual 2   This means that your LDAP/AD returned two results instead of only one.  NiFi then has not idea which of those returns is the correct one it should be using. So this either an issue within your LDAP/AD or an issue within your current login-identity-providers.xml filters.  But your last shared file looks fine to me. I would suggest using the ldapsearch command to run a ldap query outside of NiFi to see what returns you get for your admin user (cn=<admin user>) Hope this helps, Matt ... View more

@JorgeO    I am not clear on what "nifi screen does not show any change" means. Some things to keep in mind: 1. NiFi must first be secured before NiFi will even support any form of user authentication or authorization. http://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#security_properties 2. Authentication and authorization processes are handled independently. A user must first successfully authenticate and only then will the user identity be passed to the authorization process to verify authorized policies granted for that user.  3. The ldap-provider does not sync users from ldap, it is only used to validate user credentials supplied at a login prompt with ldap to authenticate that user.  Are you seeing a login window with your NiFi?  (will not see this if NiFi is not yet secured). 4. Once your user successfully authenticates then authorization comes in to play.  Looking at your authorizers.xml, one issue stands out to me: - You configured:   <property name="Initial Admin Identity">cn=admin,dc=example,dc=local</property>   This is set in the "file-access-policy-provider".  This provider is used to seed the initial minimum required policy that an admin user would need in order to access the UI, add additional users to the authorizer through UI and associate those additional users to various NiFi authorization policies. - If you are a multi-node NiFi cluster, the node certificate DNs for each of yoru secured nodes also need to be configured in the "file-access-policy-provider":   <property name="Node Identity 1">CN=node1, OU=nifi</property> <property name="Node Identity 2">CN=node2, OU=nifi</property> etc...   - Problem here is that I don't see where you create that initial admin user.  NiFi can not seed policies for a user that does not exist yet as a known identity to NiFi.  Multiple methods are available for adding users and groups to NiFi for policy assignment. Your setup shared is using the "file-user-group-provider" which uses locally defined user/client identities.  However, you did not add your admin user DN to the file-user-group-provider so that NiFi adds it to the users.xml.  It needs to exist before the "file-access-policy-provider" can associate policies to that user identity.   <property name="Initial User Identity 1">cn=admin,dc=example,dc=local</property>   - Additionally, if you have setup a NiFi multi-node cluster, the DNs from the certificate used to secure each of the nodes must also be added as "Initial User Identity 2,3,4..." properties in the file-user-group-provider also. All authentication and authorization actions would be logged in the nifi-user.log.  So check that to see what is happening if you are having access or authorization exceptions.  If this log does not exist or is empty, odds are that your NiFi was not secured.   In addition to the security properties I provided the link to above, you also need to make sure you set the following additional properties in the nifi.properties: nifi.web.https.host The HTTPS host. It is blank by default. nifi.web.https.port The HTTPS port. It is blank by default. When configuring NiFi to run securely, this port should be configured. The "nifi.web.https.port" is essentially the switch that enables secured NiFi when set. The default is "nifi.web.http.port" which is a non-secured port. Hope this helps, Matt ... View more

@adhishankarit  The issues is being caused by the line returns used in the middle of the NiFi NiFiExpression Language (EL) ifElse() function you are using. The text box where you enter your NiFi EL uses a NiFi editor that highlights to show proper EL format.  You'll notice your EL stops highlighting once you reach first line return. So you'll notice character 32 is the first single quote character.  Since EL breaks at this point it fails to find the matching second expected single quote. This leaves you with two options: 1. Create flat json without the line returns. 2. Looking at result you are trying to achieve, design your NiFi EL differently: Note proper NiFi EL highlighting above. Hope this helps, Matt ... View more