Hortonworks DataFlow 2.0 comes with the ability to configure TLS for Apache NiFi through Apache Ambari, this is implemented using the tls-toolkit in client/server mode.
To demonstrate this functionality, lets set up a 3 node NiFi secured cluster through Ambari locally in Docker containers.
First, install docker on your machine. Please note that the default docker-machine vm size is too small for this guide. You should have at least 8 gigs of ram, a few cpus, and 100 gigs of hard drive space allocated to the docker-machine you’re using.
Build the Ambari stack:
mkdir toolkit-demo-3/ cd toolkit-demo-3/ git clone https://github.com/brosander/dev-dockerfiles.git dev-dockerfiles/ambari/server/centos6/buildStack.sh http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.4.0.1/ambari.repo
Generate an ssh key for use talking to the gateway and Ambari:
mkdir ambari-ssh-keys ssh-keygen -t rsa -b 4096 -f ambari-ssh-keys/id_rsa
Run the Ambari stack and install HDF mpack:
wget -P mpack http://public-repo-1.hortonworks.com/HDF/centos6/2.x/updates/2.0.0.0/tars/hdf_ambari_mp/hdf-ambari-m... dev-dockerfiles/ambari/server/centos6/runStack.sh -m "`pwd`/mpack/" -p "`pwd`/ambari-ssh-keys/id_rsa.pub" -n 3 -a -g
SSH into the gateway at port 2001 and forward local port 1025 for use as a SOCKS proxy (this ip will be where docker exposes the port which may vary based on environment):
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ambari-ssh-keys/id_rsa -p 2001 -D 1025 root@192.168.99.100
Configure your browser to use the ssh connection as a SOCKS proxy (for Firefox this is Settings -> Advanced -> Network -> Connection Settings). I prefer to use Chrome as my main browser so I use Firefox as my “Docker Browser”.
Visit http://ambari:8080 in the browser using the SOCKS proxy
- Login to Ambari (default admin/admin)
- Launch Installation Wizard
- Name your cluster -> Next
- Default versions should be fine -> Next
- For target hosts enter:
centos6[1-3].ambari - Select manual registration radio -> Next
- Verify hosts are green -> Next
- Deselect Storm and Kafka for the purposes of this tutorial -> Next
- Click the + by NiFi twice to put it on all 3 hosts -> Next
- Put NiFi Certificate Authority on centos61.ambari (uncheck wherever it currently is) -> Next
- On Ambari Metrics tab, enter a Grafana Password, switch to NiFi tab
- Expand “Advanced nifi-ambari-config”, enter “Sensitive property values encryption password”
- Expand Advanced nifi-ambari-ssl-config
- Enter “CN=admin, OU=NIFI” (without quotes) into “Initial Admin Identity”
- Select “Enable SSL?”, “Clients need to authenticate?” checkboxes
- Enter NiFi CA Token value
- Enter the below xml into “Node Identities”
- Next
- Deploy
Node Identities xml:
<property name="Node Identity 1">CN=centos61.ambari, OU=NIFI</property> <property name="Node Identity 2">CN=centos62.ambari, OU=NIFI</property> <property name="Node Identity 3">CN=centos63.ambari, OU=NIFI</property>
Wait for the install to finish, at this point you should have a running cluster.
Generate your admin client certificate, subsituting the NiFi CA Token you entered on step 16 for YOUR_CA_TOKEN (if you get permissions errors during the docker-run, consider running build.sh passing your uid and gid in):
dev-dockerfiles/nifi-toolkit/ubuntu/build.sh docker run -ti --net ambari -v "`pwd`:/opt/toolkit-output" --rm nifi-toolkit tls-toolkit.sh client -c centos61.ambari -D 'CN=admin, OU=NIFI' -p 10443 -T pkcs12 -t YOUR_CA_TOKEN
Import the nifi-cert.pem into your browser as a trusted CA.
Import The keystore.pkcs12 client certificate file into your browser as a client cert using the keyStorePassword in the generated config.json file
Now you should be able to use the NiFi web ui links in Ambari to access your NiFi instances.
Congratulations, you’ve used NiFi CA to secure a 3 node HDF cluster in Docker using Ambari!
