The Background: I have multitenant clusters. My organization has delivered me a combined truststore and keystore jks file. I will have users who need to hit various external-to-nifi services using SSL/TLS. It is tempting to create an sslcontext controller service at nifi root and let all my users use this service when they need SSL. One problem with this approach that I see is that if I let all my users use the hosts' certs, (keystore and truststore) they could just use a GetHTTP processor and talk to the nifi rest api with full priveleges (or at least whatever privs the node has) So to prevent this I figure that I should get my root CA certs into a separate truststore that is not password protected, and only use this. The question: Should I create an sslcontext service at the root flow with only a truststore, and ask my users to all use the same controller service? Or should I just notify my users of the path to the truststore and have them create controllers within their process groups as needed, what are the pros and cons of each approach? ... View more

I just noticed that ListSFTP can use a distributed cache controller. This is confusing to me because I thought we were supposed to only run ListSFTP on the primary node, and rebalance filenames via S2S RPG. In addition to the distributed cache, it also seems to store state. This is confusing to me because if we use a distributed cache controller, why would the ListSFTP need to store state? What is the current best practice for resiliant, parallelized sftp? If I use a distributed cache, does that mean I can just schedule my ListSFTP to run on all nodes? Can someone help me understand what is going on here? Thanks! ... View more

HDF 3.1 includes nifi 1.5, and the release notes mention that now external LDAP groups can be used in nifi security policies in ranger. It seems that we need to use org.apache.nifi.authorization.ManagedRangerAuthorizer in the authorizers xml but I cannot find any documentation on this. Has anyone successfully used LDAP groups for ranger nifi polices? And is there any documentation? Thanks. PS. I see @Yolanda M. Davis in the nifi git history for this feature, perhaps she can help? ... View more

I wish to programmatically query atlas to provide a list of hive tables that are in certain hive databases. I only want to see hive tables that are in databases that contain a certain string in their name. In the hive_table atlas type, the db property is a reference to an entity of type hive_db, so I cannot use a simple where clause. For example pretend I have many hive databases, some end with '_temp' some end with '_final'. Each database may have several tables. I want to generate a list of all hive tables in databases that end with '_final.' I would also like to exclude hive tables that have been deleted. I have been experimenting with using the /api/atlas/discovery/search/dsl rest endpoint, but I have had no success. There is documentation for the dsl at http://atlas.apache.org/Search.html, but this documentation is very esoteric, and I cannot figure out how to use it. Does anyone have examples of returning lists of entities in atlas bases on properties of referred-to entities? Is there a more user-friendly or complete source of documentation for the atlas query dsl? Also note that I do not wish to query the hive metastore directly, I wish to use atlas. Thank you for any help! ... View more