Re: Two NiFi Instances/Clusters Communication Time...

alvinuw · ‎01-18-2018

Hi @Andy LoPresto, Just want to follow up on ticket https://issues.apache.org/jira/browse/NIFI-3834 Is it prioritized for this year? Thanks.

alvinuw · ‎01-18-2018

Hi @Matt Clarke, Kubernetes master ip is different with nifi-0.svc, which is an external pod dns. So nifi-0.svc can't been access outside of Kubernetes. I feel that nifi.web.proxy.host may work for this case. Is there a way to disable the Request Header Check? Thanks.

alvinuw · ‎01-18-2018

Hi @Matt Clarke, Thank you for your clarification. Yes, I use the secure mode. In NiFi v1.4, I setup the two properties as below. nifi.web.https.host= nifi-0.svc nifi.web.https.port= 9443 Please note nifi-0.svc is the pod name of the first nifi node. Then, we have nifi-1.svc, nifi-2.svc in the setting of the other nifi nodes. Normally, if we want to access NiFi cluster outside of Kubernetes, we have to use the master node ip of Kubernetes. e.g. https://master-ip:9443/nifi/ Above worked perfectly before. Now my question is in NiFi v1.5. What should I set for nifi.web.https.host? 1. If I don't do any changes, it gives me "Invalid Host Header" error. 2. If I set it as nifi.web.https.host = master-ip in all three nifi nodes in the cluster. The error in log is: 2018-01-18 17:10:28,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Cannot assign requested address One thing confuses me is: I have three nifi nodes in a cluster. Should I set "nifi.web.https.host" in three nifi.property file to the same ip value? BTW, are we able to disable this request header check? Thanks.

alvinuw · ‎01-18-2018

Hi @Matt Clarke, Thank you for your response. Is it nifi.web.http(s).host or nifi.web.proxy.host? I have already put nifi.web.https.host = the-nifi-node-ip We deploy nifi cluster in Kubernete, What should I change it to? Thanks.

alvinuw · ‎01-18-2018

Hi Guys, I just upgraded NiFi from v1.4 to v1.5. The cluster is set in secure mode and running in Kubernetes. But I found below error in Web UI. System Error The request contained an invalid host header [server:port] in the request [/nifi/]. Check for request manipulation or third-party intercept. Is it due to nifi.web.proxy.host property? How do I setup it? the IP of the nifi node? Thanks.

alvinuw · ‎01-10-2018

Hi @Benoit Rousseau, Thank you for your response. For the jmx_exporter option, my main trouble is where can I find or create the "nifi.yml". I found a couple of template for Kafka and Cassandra. But How do I know the existing jmx metrics in NiFi. I didn't find any documents on it. Would you please share a template? Thanks.

alvinuw · ‎01-08-2018

Hi Guys, I know NiFi provides ReportTask component to export metrics to external monitoring systems. I was wondering whether we can use a generic approach to collect NiFi metrics via JMX. For example, there is a JMX exporter to Prometheus by collecting metrics from java application. May I use it to collect NiFi metrics to Prometheus? Thanks.

alvinuw · ‎10-20-2017

Hi guys, I setup Volatile content repo for my NiFi cluster, the other repos are all file-based. However, It shows me "Content Repository out of space " exception, when I process only 20M dataset. It seems below settings don't work. nifi.content.repository.implementation=org.apache.nifi.controller.repository.VolatileContentRepository nifi.volatile.content.repository.max.size=1 GB nifi.volatile.content.repository.block.size=32 KB From below code comment, I found " If no Backup Repository has been specified, when the threshold is exceeded, an IOException will be thrown" https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/VolatileContentRepository.java My questions are: 1. How to setup the Backup Repository for volatile content repository? 2. Is my above setting for "nifi.volatile.content.repository.max.size" correct? It seems still using the default 100MB. Thanks.

alvinuw · ‎10-19-2017

@Abdelkrim Hadjidj Perfect! Much clearer now. Thanks.

alvinuw · ‎10-18-2017

Hi @Abdelkrim Hadjidj Thank you for your response. Very helpful. I do have retrieve site-to-site details policy set for all nodes in the cluster. And the RPG can connect and refresh. However, the "Retrieve Data via site-to-site" item is grey and disabled for the Input Port. Then, I found below document. https://community.hortonworks.com/questions/67255/how-can-i-set-an-inputoutput-ports-for-a-remote-pr.html It says Input Port must be in the root level to enable S2S. I was wondering is it a limitation? How about I want to put my entire flow in a processor group? BTW, after S2S is enabled. I still find the file distribution is not even. One node processes more than the others. Is there a way to control the distribution? Thanks.

alvinuw · ‎10-17-2017

Hi @Matt Burgess Thank you for your response. I setup RPG and an Input Port. It seems RPG can connect the remote nifi instance. However, when I click Manage Remote Ports, it can't display the input port I added. BTW, I use a secured 3 node cluster with S2S enabled. Please see the attached screenshot. Did I miss anything? Thanks.

alvinuw · ‎10-17-2017

Hi Guys, I have a flow to ingest files from sftp server, by ListSFTP(primary node) -> FetchSFTP(all nodes). I uploaded 20 files(5MB each) to sftp server. I found all files are fetched and processed by the primary node. I was wondering how to make the tasks evenly distributed among all nodes, rather than only primary node. Is it because 20 files is small number or each file is too small? Thanks.

alvinuw · ‎10-17-2017

Hi @Wynner, When I set Volatile content repository properties: nifi.volatile.content.repository.max.size=1 GB nifi.volatile.content.repository.block.size=32 KB It shows me "Content Repository out of space " exception, when I process a 20M dataset. From the code comment, I found " If no Backup Repository has been specified, when the threshold is exceeded, an IOException will be thrown" https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/VolatileContentRepository.java 1. How to setup the Backup Repository for volatile content repository? 2. Is my above setting for "nifi.volatile.content.repository.max.size" correct? It seems still using the default 100MB. Thanks.

alvinuw · ‎09-28-2017

Hi @Bryan Bende I have a new issue when I tried to login Initial Admin Identity by using username and password. In previous test, I successfully made the client cert login work. I have an principle "admin@NIFI.COM" in kerberos, and "admin@NIFI.COM" as Initial Admin in authorizers.xml When I login as "admin", the user log shows: 2017-09-28 14:27:06,112 INFO [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for admin@NIFI.COM 2017-09-28 14:32:13,193 ERROR [NiFi Web Server-19] o.a.nifi.web.security.jwt.JwtService There was an error validating the JWT io.jsonwebtoken.JwtException: Unable to validate the access token. Caused by: io.jsonwebtoken.SignatureException: JWT signature does not match locally computed signature. JWT validity cannot be asserted and should not be trusted. When the nifi cluster is up, I can login in as "admin". After logout, and login again, NiFi gives me above error. Is it caused by the browser cache the token? Thanks.

alvinuw · ‎09-25-2017

Hi @bkosaraju, Below command works to generate a token. $ curl -k 'https://<nifi-server>:9091/nifi-api/access/token'-H 'Accept-Encoding: gzip, deflate, br'-H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8'-H 'Accept: */*'--data 'username=<username>&password=<password>'--compressed However, $ curl -k -X GET 'https://<nifi-server>:9091/nifi-api/cluster/summary' -H 'Authorization: Bearer $token' --compressed returns: Unable to validate the access token. Any idea about it? Do I need to run above commands in one nifi node, or any machine? Thanks.

alvinuw · ‎09-21-2017

Hi @Egor Pushkin, Since we build a secured NiFi cluster with Kerberos for authentication in production. Does this tool support that? I didn't find it from your document. https://github.com/Glympse/terraform-provider-nifi/blob/develop/docs/syntax.md Thanks.

alvinuw · ‎09-20-2017

hi @bkosaraju, Thank you for your response. I did enable the kerberos-provider section. I guess I need to install krb5-user on the machine I will run the above commends, right? I was just wondering whether I need to use keytool to import nifi-cert.pem in my machine? What's the usage of nifi-cert.pem generated by the tls-tool? Thanks.

alvinuw · ‎09-19-2017

Hi guys, I build a secured NiFi cluster authenticated by Kerberos and use TLS tool standalone mode to generate client certificates. I am able to use NiFi Rest Api to access the unsecured cluster. When I want to use username/password of principle created in Kerberos to build a REST call for a secured cluster in curl, I got below error. Unknown user with identity 'anonymous'. Contact the system administrator. I was wondering whether I should install some certificate in my local machine to use curl. Thanks.

alvinuw · ‎09-19-2017

Hi @Arti Wadhwani, Could we use username/password of the principle created in Kerberos to login? Since I don't have kinit in nifi node. Thanks.

alvinuw · ‎09-15-2017

Hi @Bryan Bende, I deleted my previous replies since it's misleading. Let me restate the current issue clearly. The reason I saw below error "Unknown user with identity 'alvin@NIFI.COM'. Contact the system administrator." is due to I login Initial Admin account to add user "CN=alvin, OU=NIFI.COM" in NiFi web. The correct way is to add 'alvin@NIFI.COM' and set its policy. After that my issue is gone. Thanks.

alvinuw · ‎09-08-2017

Hi @Bryan Bende, It seems the authentication works now. But I have issue with authorization. I used the file-based authorization approach. After I type username and password, it says "Unknown user with identity 'alvin@NIFI.COM'. Contact the system administrator." instead of prompting out "Justification" page. Any idea? Thanks. 2017-09-08 19:48:33,294 INFO [NiFi Web Server-29] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for alvin@NIFI.COM 2017-09-08 19:48:33,295 INFO [NiFi Web Server-29] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[alvin@NIFI.COM], groups[none] does not have permission to access the requested resource. Unknown user with identity 'alvin@NIFI.COM'. Returning Forbidden response. 2017-09-08 19:48:33,295 DEBUG [NiFi Web Server-29] o.a.n.w.a.c.AccessDeniedExceptionMapper org.apache.nifi.authorization.AccessDeniedException: Unknown user with identity 'alvin@NIFI.COM'. at org.apache.nifi.web.api.FlowResource.authorizeFlow(FlowResource.java:230) at org.apache.nifi.web.api.FlowResource.getCurrentUser(FlowResource.java:316)

alvinuw · ‎09-08-2017

Hi @Bryan Bende, Thank you for this tutorial. Very helpful. May I ask do we need to also set up below properties in a cluster mode? nifi.kerberos.service.principal nifi.kerberos.service.keytab.location Is kerberos.service and kerberos.spnego one or another? Or both must be set? Since I saw some examples set up kerberos.service rather than kerberos.spnego, e.g. https://community.hortonworks.com/articles/34147/nifi-security-user-authentication-with-kerberos.html Thanks.

alvinuw · ‎09-01-2017

Hi @Wynner, Thanks for your comments. You are right, it's the authorizers.xml format issue. Actually, I am using nifi-1.4-snapshot. That's why my authorizers.xml is different from the 1.3 one. When I switched to the 1.3 version authorizers.xml with nifi-1.4-snapshot, the above issue was gone.This confuses me. Then, when I have below in authorizers.xml file <propertyname="Node Identity 1">CN=nifi-0, OU= NIFI.COM</property> I saw errors from nifi-app.log, it seems the default OU of node identity is "NIFI". 2017-09-01 14:09:08,854 DEBUG [NiFi Web Server-19] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi. at org.apache.nifi.web.api.AccessResource.createAccessTokenFromTicket(AccessResource.java:349) 2017-09-01 14:09:09,045 INFO [NiFi Web Server-127] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<CN=nifiadmin, OU=NIFI.COM><CN=nifi-0, OU=NIFI>) GET https://nifi-0:9443/nifi-api/flow/current-user (source ip: 10.244.1.95) 2017-09-01 14:09:09,048 WARN [NiFi Web Server-127] o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Untrusted proxy CN=nifi-0, OU=NIFI 2017-09-01 14:09:09,048 WARN [NiFi Web Server-127] o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Untrusted proxy CN=nifi-0, OU=NIFI But whenIf I set it as "CN=nifi-0, OU= NIFI" in above file, it works.

alvinuw · ‎08-31-2017

Hi guys, When I secure a three nodes nifi cluster, I got below errors during start up. I use TLS tool to create certificates, and use client-certificate for authentication. The setting in my authorizers.xml is below: <accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class> <property name="User Group Provider">file-user-group-provider</property> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Initial Admin Identity">CN=nifiadmin, OU=NIFI.COM</property> <property name="Legacy Authorized Users File"></property> <property name="Node Identity 1">CN=nifi-0, OU=NIFI.COM</property> <property name="Node Identity 2">CN=nifi-1, OU=NIFI.COM</property> <property name="Node Identity 3">CN=nifi-2, OU=NIFI.COM</property> </accessPolicyProvider> From nifi-user.log ouput, it seems NiFi can locate the DN set in authorizers.xml. However, I don't understand why nifi-bootstrap.log says NiFi can't initial admin. Are users.xml and authorizations.xml auto-created and populated by NiFi? After checking, it seems "Initial Admin Identity" user and administrative policies are NOT added to the users.xml and authorizations.xml files. $ cat nifi-user.log 2017-08-31 18:54:28,424 INFO [main] o.a.n.a.FileUserGroupProvider Creating new users file at /opt/nifi/nifi-1.4.0-SNAPSHOT/./conf/users.xml 2017-08-31 18:54:28,453 INFO [main] o.a.n.a.FileUserGroupProvider Users/Groups file loaded at Thu Aug 31 18:54:28 UTC 2017 2017-08-31 18:54:28,458 INFO [main] o.a.n.a.FileAccessPolicyProvider Creating new authorizations file at /opt/nifi/nifi-1.4.0-SNAPSHOT/./conf/authorizations.xml 2017-08-31 18:54:28,475 WARN [main] org.apache.nifi.authorization.FlowParser Flow Configuration does not exist or was empty 2017-08-31 18:54:28,475 INFO [main] o.a.n.a.FileAccessPolicyProvider Populating authorizations for Initial Admin: CN=nifiadmin, OU=NIFI.COM $ cat nifi-bootstrap.log 2017-08-31 18:54:13,151 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 41600 2017-08-31 18:54:29,443 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Error creating bean with name 'niFiWebApiSecurityConfiguration': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire method: public void org.apache.nifi.web.NiFiWebApiSecurityConfiguration.setJwtAuthenticationProvider(org.apache.nifi.web.security.jwt.JwtAuthenticationProvider); nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtAuthenticationProvider' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'authorizer' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.apache.nifi.authorization.exception.AuthorizerCreationException: org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable to locate initial admin CN=nifiadmin, OU=NIFI.COM to seed policies 2017-08-31 18:54:29,444 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down... 2017-08-31 18:54:30,576 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi

alvinuw · ‎08-22-2017

@Wynner Thank you. I am clear on it now.

alvinuw · ‎08-21-2017

Hi @Wynner, Thank you for your clarification. May I understand the Volatile Content Repository only keeps the content for the flowfiles running in the flow? Once the flowfiles are finished, the contents are wipe out from memory, and can't be found in Repository. In contrast, the File-based Content repository will keep the contents for flowfiles even out of the flow, based on the retention setting. Thanks.

alvinuw · ‎08-18-2017

Hi @Wynner, Thank you for your answers. You said that If the configured memory limit is reached, and more content data can't be added to Volatile Content Repository. But how do we free memory for Volatile Content Repository, if it can't auto clean up? Since this day will come sooner or later. It could be an issue to make the Volatile Content Repository useless. Thanks.

alvinuw · ‎08-18-2017

Hi @Egor Pushkin, This is really fantastic! May I ask what are the main advantages of this Terraform approach, comparing with building a tool by NiFi REST api? Thanks.

alvinuw · ‎08-18-2017

I am curious to know how much risks to use the volatile content repository? My understanding is: If there is a node failure/restart, For data has already been processed/persisted through the flow, no impact on our business or downstreams. But users cannot view and/or replay content via the provenance UI, since the content are gone due to restart. For the content of flowfiles are still in the middle of flow during node failure/restart, we can't replay them from where it fails, when the node is back to normal. Instead, we have to fetch the same files from source again, and reprocess them end to end through the flow. If above is correct, I would say as long as we have source data permanently persisted in somewhere out of NiFi, we can always reprocess it when data in volatile content repository is lost. The only loss is the ability to view/replay them via Provenance UI. BTW, what happens when content exceeds the maximum size of repository? Out of memory exception? Auto purged from memory? auto archived in disk? If I set nifi.content.repository.implementation=org.apache.nifi.controller.repository.VolatileContentRepository Does that mean below properties are auto-disabled? nifi.content.claim nifi.content.repository.archive nifi.content.viewer.url Any comments are appropriated. Thanks.

alvinuw · ‎08-17-2017

Hi @Andy LoPresto, I am curious to know how much risks to use the volatile content repository? My understanding is: If there is a node failure/restart, For data has already been processed/persisted through the flow, no impact on our business or downstreams. But users cannot view and/or replay content via the provenance UI, since the content are gone due to restart. For the content of flowfiles are still in the middle of flow during node failure/restart, we can't replay them from where it fails, when the node is back to normal. Instead, we have to fetch the same files from source again, and reprocess them end to end through the flow. If above is correct, I would say as long as we have source data permanently persisted in somewhere out of NiFi, we can always reprocess it when data in volatile content repository is lost. The only loss is the ability to view/replay them via Provenance UI. BTW, what happens when content exceeds the maximum size of repository? Out of memory exception? Auto purged from memory? auto archived in disk? If I set nifi.content.repository.implementation=org.apache.nifi.controller.repository.VolatileContentRepository Does that mean below properties are auto-disabled? nifi.content.claim nifi.content.repository.archive nifi.content.viewer.url Any comments are appropriated. Thanks.

Online	Offline
Last Visited	‎01-18-2018 07:36 PM

Date Registered	‎07-19-2016 02:09 AM
Last Visited	‎01-18-2018 07:36 PM
Total Messages Posted	91
Total Kudos Received	8

Re: Two NiFi Instances/Clusters Communication Time...

Re: Encrypted NiFi Content Repository

Re: NiFi 1.5 System Error: Invalid host header

Re: NiFi 1.5 System Error: Invalid host header

Re: NiFi 1.5 System Error: Invalid host header

NiFi 1.5 System Error: Invalid host header

Re: How to enable Apache NiFi metrics for JMX

How to enable Apache NiFi metrics for JMX

NiFi Volatile Content Repository out of space

Re: NiFi: All files are processed by SFTP processo...

Re: NiFi: All files are processed by SFTP processo...

Re: NiFi: All files are processed by SFTP processo...

NiFi: All files are processed by SFTP processor on...

Re: NiFi Volatile Content Repository

Re: Apache NiFi 1.0.0 Kerberos Authentication

Re: Use REST API to access a secured NiFi cluster

Re: Terraform Provider NiFi

Re: Use REST API to access a secured NiFi cluster

Use REST API to access a secured NiFi cluster

Re: Use a Kerberos token for accessing NiFi REST A...

Re: Apache NiFi 1.0.0 Kerberos Authentication

Re: Apache NiFi 1.0.0 Kerberos Authentication

Re: Apache NiFi 1.0.0 Kerberos Authentication

Re: NiFi AuthorizerCreationException: Unable to lo...

NiFi AuthorizerCreationException: Unable to locate...

Re: NiFi Volatile Content Repository

Re: NiFi Volatile Content Repository

Re: NiFi Volatile Content Repository

Re: Terraform Provider NiFi

NiFi Volatile Content Repository

Re: Encrypted NiFi Content Repository