Hi @Andrew Watson As long as it's installed out of the default system paths, and only called by whatever processing scripts or tools you're thinking of, I don't see that being an issue at all. It'll require careful installation and ongoing care though, just from a security and patching perspective. ... View more

Hi @Smart Solutions. Right now... there isn't really a good, quick, clean, easy way to achieve this. You've already identified the thread that I would otherwise point you towards for some ideas. You're just too good! There are two main approaches that I would recommend thinking about, the first is ceasing to make any changes via the web-ui, and only making changes via the API, that way you can just call both your clusters one after the other to make the configuration changes. The second is to use some of the ideas from the thread you linked to, where you would continue maintaining the configs on your "master" cluster, but then extract the configs from your "master" cluster on a regular basis (or on a trigger of config changed?), diff them between the previous "master" cluster config version, and then push the resulting deltas to your "slave" cluster, again via API calls. Either way, there's quite a bit of automation that would be required there. I'd strongly suggest if you want to go down this path, doing your work out in the open, this is something I see come up now and again, so I think you may well get others who would be interested in working with you on this. Longer term, Ambari will no doubt support multi cluster, and this functionality would be a natural extension of that, but progress on those public JIRA's has been slow, with other more important items taking priority. Happy to hear if you have other ideas too, sorry I couldn't be more direct help, but let me know if you plan on cutting some code moving forward, I'm sure it'd be an interesting project. Many thanks. ... View more

Hi @Smart Solutions. It's tricky to give generic best practice recommendations without knowing a lot of detail about what you are doing or have already done. There are a few things I can think of off the top of my head. Ensure you're using HDFS Data Encryption for especially sensitive locations (though you needn't apply it everywhere). Start looking at things like the refresh rate of your Ranger policies, ensure that's in-line with your expectations, setting the refresh time too low could impact the performance of the Ranger admin so bear that in mind. Make sure that you're actually looking at the logging and auditing that Ranger is creating. Start thinking about how Atlas could start to play a part in your security story, with things like the upcoming tag based policy control, Ranger + Atlas will be a very powerful combination. For more info take a look at: http://hortonworks.com/hadoop-tutorial/tag-based-policies-atlas-ranger/ Standard practices apply, ensure that people have the least permissions they need (both in terms of access to data and services) to complete their job, no more, no less. Hope that helps, the fact that you already have Kerberos, Ranger and Knox in place suggests you're already along the right path. Good luck and hope that helps. ... View more

Hi @Kaliyug Antagonist. I would suggest implementing Knox, with a restricted set of users allowing access to only the set of services you want to expose to those users. Both http://hortonworks.com/apache/knox-gateway/ and http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_Security_Guide/content/perimeter_security_with_apache_knox.html ... should get you started. Hope that helps. ... View more