Hello, I am trying to send messages to a kerberos ebabled kafka using a standalone NiFi node version nifi-1.1.0.2.1.2.0-10. I followed the instructions listed in the following link: How to configure HDF1.2 to send to and get data from kerberized kafka I inputted the keytab and principal of the kerberized kafka and added the path with the appropriate added line in the bootstrap.conf file of my NiFi instance. However whenever I try sending a message with the PutKafka processor it gives an error saying: "Failed while waiting for acks from kafka". I verified the logs in the NiFi instance and this is what I see: 2017-06-30 15:41:07,166 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress> java.io.EOFException: null at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) at org.apache.kafka.common.network.Selector.poll(Selector.java:248) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) at java.lang.Thread.run(Thread.java:745) 2017-06-30 15:41:07,298 WARN [kafka-producer-network-thread] org.apache.kafka.common.network.Selector Error in I/O with <serveraddress> java.io.EOFException: null Does anyone have any ideas of what could be going on? Any insights in this would be really helpful! ... View more

Hello, I have a 3 node cluster all using the NiFi version nifi-1.2.0.3.0.0.0-453. The cluster has been working fine for the last couple of weeks, however today all of a sudden one of the nodes disconnected from the cluster and won't join the cluster back. I checked the logs and the error I see is the following: ERROR [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl Background retry gave up org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:838) at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2017-06-27 17:54:40,179 ERROR [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl Background operation retry gave up org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:728) at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:857) at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:267) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) The node itself appears to still be running but is disconnected from the cluster. I tried restarting it but the same error keeps appearing over and over again. The other two nodes in the cluster are working fine. Does anyone have any idea of what could be causing this sudden issue? Any insights would be greatly appreciated. ... View more

Hello, I tried upgrading NiFi from version 1.1.0.2.1.4.0-5 to NiFi version 1.2.0.3.0.0.0-453 of HDF 3.0 but I am having an issue when the node joins the cluster. I run a cluster with three NiFi nodes, I was trying a rolling upgrade to not stop all the nodes at the same time. I upgraded one of the nodes and tried to make it join the cluster. The node seemed to start up fine with no issues and said that the cluster had received the heartbeat. However, after the node joined the cluster the UI page in all three nodes started showing an error message saying "Unexpected error please check the logs for any errors". I went to the logs and I found the following warnings: 2017-06-14 11:36:49,279 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.s3.FetchS3Object is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,292 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.s3.PutS3Object is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,294 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.s3.DeleteS3Object is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,301 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.s3.ListS3 is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,310 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.sns.PutSNS is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,481 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.sqs.GetSQS is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,482 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.sqs.PutSQS is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,483 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.sqs.DeleteSQS is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,492 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.lambda.PutLambda is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,497 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.kinesis.firehose.PutKinesisFirehose is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,508 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.dynamodb.GetDynamoDB is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,510 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.dynamodb.PutDynamoDB is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,512 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.dynamodb.DeleteDynamoDB is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,517 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.kinesis.stream.PutKinesisStream is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,523 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.aws.cloudwatch.PutCloudWatchMetric is bundled with its referenced Controller Service APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:49,610 WARN [main] org.apache.nifi.nar.ExtensionManager Controller Service org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService is bundled with its supporting APIs org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderService. The service APIs should not be bundled with the implementations. 2017-06-14 11:36:49,714 WARN [main] org.apache.nifi.nar.ExtensionManager Controller Service org.apache.nifi.jms.cf.JMSConnectionFactoryProvider is bundled with its supporting APIs org.apache.nifi.jms.cf.JMSConnectionFactoryProviderDefinition. The service APIs should not be bundled with the implementations. 2017-06-14 11:36:50,530 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.gcp.storage.PutGCSObject is bundled with its referenced Controller Service APIs org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:50,533 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.gcp.storage.FetchGCSObject is bundled with its referenced Controller Service APIs org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:50,534 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.gcp.storage.DeleteGCSObject is bundled with its referenced Controller Service APIs org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:50,535 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.gcp.storage.ListGCSBucket is bundled with its referenced Controller Service APIs org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:50,544 WARN [main] org.apache.nifi.nar.ExtensionManager Controller Service org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService is bundled with its supporting APIs org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsService. The service APIs should not be bundled with the implementations. 2017-06-14 11:36:52,743 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.couchbase.GetCouchbaseKey is bundled with its referenced Controller Service APIs org.apache.nifi.couchbase.CouchbaseClusterControllerService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:52,746 WARN [main] org.apache.nifi.nar.ExtensionManager Component org.apache.nifi.processors.couchbase.PutCouchbaseKey is bundled with its referenced Controller Service APIs org.apache.nifi.couchbase.CouchbaseClusterControllerService. The service APIs should not be bundled with component implementations that reference it. 2017-06-14 11:36:52,748 WARN [main] org.apache.nifi.nar.ExtensionManager Controller Service org.apache.nifi.couchbase.CouchbaseClusterService is bundled with its supporting APIs org.apache.nifi.couchbase.CouchbaseClusterControllerService. The service APIs should not be bundled with the implementations. However in the app-log file after those warnings everything else seemed to start up fine and the node was connected to the cluster, I was just not able to access the UI from any of the nodes. I saw no errors or warnings in the bootstrap or user logs. I wanted to see if there could be something wrong that I am doing to the cluster with the node configuration or if the problem is that I would have to upgrade all the nodes and start them together all in the same NiFi version 1.2.0.3.0.0.0-453 of HDF 3.0? Any insights on this would be incredible helpful. ... View more

Hello, I am trying to build a flow that gets a file from elasticsearch, however since it is my first time using the FetchElasticSearch processor I am having a doubt about how to configure it. I don't understand what the Document Identifier means and what value am I suppose to give in this configuration. If someone could help in this I would greatly appreciate it. ... View more