No problem... as soon as you added that you were running HDF 2.1.2, it helped. ... View more

The NiFi Spark Receiver itself hasn't needed a true update since before 1.x and the only deps it has is site-to-site[1]. While it would be ideal to upgrade, using the 0.x version of the NiFi Receiver doesn't preclude any blocking features (that I can think of). [1] https://github.com/apache/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-external/nifi-spark-receiver/pom.xml ... View more

Also wanted to mention that site-to-site can only happen with ports on the root canvas, which is done intentionally to make it clear that it is a system decision to send or receive data out of the given instance. This means that the user that sets up the port would need write access to the root canvas, which typically in a multi-tenant environment tenants would be given to a sub-process group, but they would still need some other administrator to setup the site-to-site connection for them. ... View more

This seems to be the desired solution... https://bryanbende.com/development/2018/04/09/apache-nifi-secure-keytab-access ... View more

Thanks for writing this up @Matt Clarke - very helpful. Do you have a rule of thumb for a maximum heap size? Is there a limit where garbage collection will surely cause more problems than any gains from further increases in heap? ... View more

Thanks for confirming my suspicions @kkawamura - looks like I'll have to find another way. ... View more

Great thanks for sharing.. ... View more

Thanks, @Oliver Meyn, I hadn't thought about this approach. It might be a last resort for us, too. ... View more

Thanks Andrew - didn't read deep enough into the docs. ... View more