Reply
New Contributor
Posts: 2
Registered: ‎07-26-2017

Does Cloudera support Apache Beam ?

Does Cloudera support Apache Beam project ? Any parcels for installation on CDH ?

Posts: 1,567
Kudos: 289
Solutions: 240
Registered: ‎07-31-2013

Re: Does Cloudera support Apache Beam ?

Cloudera supports Apache Spark, upon which an Apache Beam runner exists. I assume this is what you'd meant to ask about?

Apache Beam by itself is not a service that needs installation and management (such as via Cloudera Manager), but is rather a programming model that supports various execution modes (one of which is Apache Spark).

You should be able to follow the tutorial on https://beam.apache.org/get-started/quickstart-java/ and https://beam.apache.org/documentation/runners/spark/ without trouble, just ensure to use the CDH version of Apache Spark when configuring your Java application's pom.xml.

There's no direct support offered for Apache Beam SDKs by Cloudera at present, but I see no reason for it to not work on top of your CDH cluster.
Backline Customer Operations Engineer
New Contributor
Posts: 2
Registered: ‎07-26-2017

Re: Does Cloudera support Apache Beam ?

Thanks Harsh , I was considering the possibility of other runners apart from spark and see how to go about doing it on CDH. I appreciate for confirming no support is offered for SDK.

 

Thanks

 

                     

Announcements