Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Does Cloudera support Apache Beam ?

Does Cloudera support Apache Beam ?

New Contributor

Does Cloudera support Apache Beam project ? Any parcels for installation on CDH ?

2 REPLIES 2

Re: Does Cloudera support Apache Beam ?

Master Guru
Cloudera supports Apache Spark, upon which an Apache Beam runner exists. I assume this is what you'd meant to ask about?

Apache Beam by itself is not a service that needs installation and management (such as via Cloudera Manager), but is rather a programming model that supports various execution modes (one of which is Apache Spark).

You should be able to follow the tutorial on https://beam.apache.org/get-started/quickstart-java/ and https://beam.apache.org/documentation/runners/spark/ without trouble, just ensure to use the CDH version of Apache Spark when configuring your Java application's pom.xml.

There's no direct support offered for Apache Beam SDKs by Cloudera at present, but I see no reason for it to not work on top of your CDH cluster.

Re: Does Cloudera support Apache Beam ?

New Contributor

Thanks Harsh , I was considering the possibility of other runners apart from spark and see how to go about doing it on CDH. I appreciate for confirming no support is offered for SDK.

 

Thanks