Created on 09-22-2016 01:48 PM - edited 08-17-2019 09:52 AM
Hortonworks DataFlow 2.0 comes with the ability to configure TLS for Apache NiFi through Apache Ambari, this is implemented using the tls-toolkit in client/server mode.
To demonstrate this functionality, lets set up a 3 node NiFi secured cluster through Ambari locally in Docker containers.
First, install docker on your machine. Please note that the default docker-machine vm size is too small for this guide. You should have at least 8 gigs of ram, a few cpus, and 100 gigs of hard drive space allocated to the docker-machine you’re using.
Build the Ambari stack:
mkdir toolkit-demo-3/ cd toolkit-demo-3/ git clone https://github.com/brosander/dev-dockerfiles.git dev-dockerfiles/ambari/server/centos6/buildStack.sh http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/126.96.36.199/ambari.repo
Generate an ssh key for use talking to the gateway and Ambari:
mkdir ambari-ssh-keys ssh-keygen -t rsa -b 4096 -f ambari-ssh-keys/id_rsa
Run the Ambari stack and install HDF mpack:
wget -P mpack http://public-repo-1.hortonworks.com/HDF/centos6/2.x/updates/188.8.131.52/tars/hdf_ambari_mp/hdf-ambari-m... dev-dockerfiles/ambari/server/centos6/runStack.sh -m "`pwd`/mpack/" -p "`pwd`/ambari-ssh-keys/id_rsa.pub" -n 3 -a -g
SSH into the gateway at port 2001 and forward local port 1025 for use as a SOCKS proxy (this ip will be where docker exposes the port which may vary based on environment):
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ambari-ssh-keys/id_rsa -p 2001 -D 1025 email@example.com
Configure your browser to use the ssh connection as a SOCKS proxy (for Firefox this is Settings -> Advanced -> Network -> Connection Settings). I prefer to use Chrome as my main browser so I use Firefox as my “Docker Browser”.
Visit http://ambari:8080 in the browser using the SOCKS proxy
Node Identities xml:
<property name="Node Identity 1">CN=centos61.ambari, OU=NIFI</property> <property name="Node Identity 2">CN=centos62.ambari, OU=NIFI</property> <property name="Node Identity 3">CN=centos63.ambari, OU=NIFI</property>
Wait for the install to finish, at this point you should have a running cluster.
Generate your admin client certificate, subsituting the NiFi CA Token you entered on step 16 for YOUR_CA_TOKEN (if you get permissions errors during the docker-run, consider running build.sh passing your uid and gid in):
dev-dockerfiles/nifi-toolkit/ubuntu/build.sh docker run -ti --net ambari -v "`pwd`:/opt/toolkit-output" --rm nifi-toolkit tls-toolkit.sh client -c centos61.ambari -D 'CN=admin, OU=NIFI' -p 10443 -T pkcs12 -t YOUR_CA_TOKEN
Import the nifi-cert.pem into your browser as a trusted CA.
Import The keystore.pkcs12 client certificate file into your browser as a client cert using the keyStorePassword in the generated config.json file
Now you should be able to use the NiFi web ui links in Ambari to access your NiFi instances.
Congratulations, you’ve used NiFi CA to secure a 3 node HDF cluster in Docker using Ambari!