Created on 07-28-201603:41 PM - edited 08-17-201911:09 AM
Hortonworks DataFlow (HDF) powered by Apache NiFi, Kafka and
Storm, collects, curates, analyzes and delivers real-time data from the IoAT to
data stores both on-premises and in the cloud. This is the quick installation
guide to install Apache NiFi on AWS EC2 instance. Please refer this document as
supplement guide to official Hortonworks HDF documentation.
The screenshots in this section detail the setup and
configuration of Apache NiFi on EC2 instance.
Refer the NiFi
Admin Guide for the System
requirements. This document covers installation on a Redhat linux (64 bit) EC2
Login to AWS and launch the EC2 instance and OS of your
choice. (Please make sure the selected OS is supported by NiFi). The current
exercise uses the Red Hat Enterprise Linux 7.2 image (HDF EC2 Instance).
Make sure you keep the security private key safe. Under Network and Security configuration, open
the Http ports (e.g. 8081 & 8082 shown below) to access the NiFi web
interface and for the site to site protocol to exchange data between multiple
Download the HDF from HDF Download Page. Either you can download it directly on your EC2 instance or
you can upload the zip file to the EC2 instance from local using scp.
e.g. scp -i HDF.pem HDF-22.214.171.124-1.zip ec2-user@<public-dns-hostname>:/home/ec2-user
where HDF.pem is private key.
Make sure you install the latest java and unzip
yum install unzip
yum install java
Decompress/Unzip zip into desired installation
Make desired edits in nifi.properties file under
e.g. update the site-to-site properties to
include the following