Hortonworks DataFlow 2.0 comes with the ability to configure TLS for Apache NiFi through Apache Ambari, this is implemented using the tls-toolkit in client/server mode. To demonstrate this functionality, lets set up a 3 node NiFi secured cluster through Ambari locally in Docker containers. First, install docker on your machine. Please note that the default docker-machine vm size is too small for this guide. You should have at least 8 gigs of ram, a few cpus, and 100 gigs of hard drive space allocated to the docker-machine you’re using. Build the Ambari stack: mkdir toolkit-demo-3/ cd toolkit-demo-3/ git clone https://github.com/brosander/dev-dockerfiles.git dev-dockerfiles/ambari/server/centos6/buildStack.sh http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo Generate an ssh key for use talking to the gateway and Ambari: mkdir ambari-ssh-keys ssh-keygen -t rsa -b 4096 -f ambari-ssh-keys/id_rsa Run the Ambari stack and install HDF mpack: wget -P mpack http://public-repo-1.hortonworks.com/HDF/centos6/2.x/updates/2.0.0.0/tars/hdf_ambari_mp/hdf-ambari-mpack-2.0.0.0-579.tar.gz dev-dockerfiles/ambari/server/centos6/runStack.sh -m "`pwd`/mpack/" -p "`pwd`/ambari-ssh-keys/id_rsa.pub" -n 3 -a -g SSH into the gateway at port 2001 and forward local port 1025 for use as a SOCKS proxy (this ip will be where docker exposes the port which may vary based on environment): ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ambari-ssh-keys/id_rsa -p 2001 -D 1025 root@192.168.99.100 Configure your browser to use the ssh connection as a SOCKS proxy (for Firefox this is Settings -> Advanced -> Network -> Connection Settings). I prefer to use Chrome as my main browser so I use Firefox as my “Docker Browser”. Visit http://ambari:8080 in the browser using the SOCKS proxy Login to Ambari (default admin/admin) Launch Installation Wizard Name your cluster -> Next Default versions should be fine -> Next For target hosts enter: centos6[1-3].ambari Select manual registration radio -> Next Verify hosts are green -> Next Deselect Storm and Kafka for the purposes of this tutorial -> Next Click the + by NiFi twice to put it on all 3 hosts -> Next Put NiFi Certificate Authority on centos61.ambari (uncheck wherever it currently is) -> Next On Ambari Metrics tab, enter a Grafana Password, switch to NiFi tab Expand “Advanced nifi-ambari-config”, enter “Sensitive property values encryption password” Expand Advanced nifi-ambari-ssl-config Enter “CN=admin, OU=NIFI” (without quotes) into “Initial Admin Identity” Select “Enable SSL?”, “Clients need to authenticate?” checkboxes Enter NiFi CA Token value Enter the below xml into “Node Identities” Next Deploy Node Identities xml: <property name="Node Identity 1">CN=centos61.ambari, OU=NIFI</property> <property name="Node Identity 2">CN=centos62.ambari, OU=NIFI</property> <property name="Node Identity 3">CN=centos63.ambari, OU=NIFI</property> Wait for the install to finish, at this point you should have a running cluster. Generate your admin client certificate, subsituting the NiFi CA Token you entered on step 16 for YOUR_CA_TOKEN (if you get permissions errors during the docker-run, consider running build.sh passing your uid and gid in): dev-dockerfiles/nifi-toolkit/ubuntu/build.sh docker run -ti --net ambari -v "`pwd`:/opt/toolkit-output" --rm nifi-toolkit tls-toolkit.sh client -c centos61.ambari -D 'CN=admin, OU=NIFI' -p 10443 -T pkcs12 -t YOUR_CA_TOKEN Import the nifi-cert.pem into your browser as a trusted CA. Import The keystore.pkcs12 client certificate file into your browser as a client cert using the keyStorePassword in the generated config.json file Now you should be able to use the NiFi web ui links in Ambari to access your NiFi instances. Congratulations, you’ve used NiFi CA to secure a 3 node HDF cluster in Docker using Ambari! ... View more