Hi All, We're working on setting up a NiFi Cluster; additional requirements for us are 1) capture Provenance data and 2) load balance the incoming data that's being pushed to NiFi; so, we're envisioning the below architecture, where a separate NiFi instance will receive the data that's being pushed by the data producer, for load balancing and it will also be used to archive Provenance data (from the Cluster) to HDFS; I would appreciate your feedback on our design, do you see any flaws, would this work, and any suggestions for better ways of achieving this. Thanks. ... View more

Hi All, My NiFi instance on my local machine is not starting and I'm seeing the following error in nifi-app.log. Any ideas on what could be wrong; it was working fine before; I'm not sure if this caused any issue, but I remember copying the Hive nar file, nifi-hive-nar-1.1.0, into the lib folder the last time and it did startup with no issues; now, even if I remove that nar file, it does not start. Any ideas on what's wrong and how to resolve this. 2017-02-27 13:31:53,763 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.controller.status.history.ComponentStatusRepository: Provider org.apache.nifi.controller.status.history.VolatileComponentStatusRepository could not be instantiated java.util.ServiceConfigurationError: org.apache.nifi.controller.status.history.ComponentStatusRepository: Provider org.apache.nifi.controller.status.history.VolatileComponentStatusRepository could not be instantiated at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_112] at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[na:1.8.0_112] at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[na:1.8.0_112] at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_112] at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_112] at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:116) ~[nifi-nar-utils-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:97) ~[nifi-nar-utils-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.NiFi.<init>(NiFi.java:139) ~[nifi-runtime-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] at org.apache.nifi.NiFi.main(NiFi.java:262) ~[nifi-runtime-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2] Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/util/RingBuffer$ForEachEvaluator at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.8.0_112] at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[na:1.8.0_112] at java.lang.Class.getConstructor0(Class.java:3075) ~[na:1.8.0_112] at java.lang.Class.newInstance(Class.java:412) ~[na:1.8.0_112] at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[na:1.8.0_112] ... 6 common frames omitted Caused by: java.lang.ClassNotFoundException: org.apache.nifi.util.RingBuffer$ForEachEvaluator at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_112] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_112] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_112] ... 11 common frames omitted ... View more

Hi All, just wanted to get your input regarding what tools (preferably open source) are available that can tag data in HDFS; from what I understand Atlas is not able to do that. Heard about Waterline, any others that you guys know of. Thanks. ... View more