Apache Flink Tutorials, Examples and HDP Install Guides

awatson — Thu, 29 Oct 2015 23:16:26 GMT

Hi All,

Does anyone have any steps for installing Flink on HDP? And any tutorials/examples of using Flink? Maybe with Hive ORC tables?

Thanks,

Re: Apache Flink Tutorials, Examples and HDP Install Guides

abajwa — Thu, 29 Oct 2015 23:26:28 GMT

I just tried this recently. Couple of options:

1. Compilation on HDP 2.3:

curl -o /etc/yum.repos.d/epel-apache-maven.repo https://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo
yum -y install apache-maven-3.2*
git clone https://github.com/apache/flink.git
cd flink
mvn clean install -DskipTests -Dhadoop.version=2.7.1.2.3.2.0-2950 -Pvendor-repos

2. Download prebuilt tarball:

wget http://www.gtlib.gatech.edu/pub/apache/flink/flink-0.9.1/flink-0.9.1-bin-hadoop27.tgz
tar xvzf  flink-0.9.1-bin-hadoop27.tgz
cd flink-0.9.1
export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/yarn-session.sh -n 1 -jm 768 -tm 1024

3. Experimental Ambari service:

https://github.com/abajwa-hw/ambari-flink-service

Once its up, run word count example

export HADOOP_CONF_DIR=/etc/hadoop/conf
./bin/flink run ./examples/flink-java-examples-0.9.1-WordCount.jar

question Re: Apache Flink Tutorials, Examples and HDP Install Guides in Support Questions

Apache Flink Tutorials, Examples and HDP Install Guides

Re: Apache Flink Tutorials, Examples and HDP Install Guides