Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Apache Flink Tutorials, Examples and HDP Install Guides

Solved Go to solution

Apache Flink Tutorials, Examples and HDP Install Guides

Hi All,

Does anyone have any steps for installing Flink on HDP? And any tutorials/examples of using Flink? Maybe with Hive ORC tables?

Thanks,

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Apache Flink Tutorials, Examples and HDP Install Guides

I just tried this recently. Couple of options:

1. Compilation on HDP 2.3:

curl -o /etc/yum.repos.d/epel-apache-maven.repo https://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo
yum -y install apache-maven-3.2*
git clone https://github.com/apache/flink.git
cd flink
mvn clean install -DskipTests -Dhadoop.version=2.7.1.2.3.2.0-2950 -Pvendor-repos

2. Download prebuilt tarball:

wget http://www.gtlib.gatech.edu/pub/apache/flink/flink-0.9.1/flink-0.9.1-bin-hadoop27.tgz
tar xvzf  flink-0.9.1-bin-hadoop27.tgz
cd flink-0.9.1
export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/yarn-session.sh -n 1 -jm 768 -tm 1024

3. Experimental Ambari service:

https://github.com/abajwa-hw/ambari-flink-service

Once its up, run word count example

export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/flink run ./examples/flink-java-examples-0.9.1-WordCount.jar
1 REPLY 1

Re: Apache Flink Tutorials, Examples and HDP Install Guides

I just tried this recently. Couple of options:

1. Compilation on HDP 2.3:

curl -o /etc/yum.repos.d/epel-apache-maven.repo https://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo
yum -y install apache-maven-3.2*
git clone https://github.com/apache/flink.git
cd flink
mvn clean install -DskipTests -Dhadoop.version=2.7.1.2.3.2.0-2950 -Pvendor-repos

2. Download prebuilt tarball:

wget http://www.gtlib.gatech.edu/pub/apache/flink/flink-0.9.1/flink-0.9.1-bin-hadoop27.tgz
tar xvzf  flink-0.9.1-bin-hadoop27.tgz
cd flink-0.9.1
export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/yarn-session.sh -n 1 -jm 768 -tm 1024

3. Experimental Ambari service:

https://github.com/abajwa-hw/ambari-flink-service

Once its up, run word count example

export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/flink run ./examples/flink-java-examples-0.9.1-WordCount.jar