Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Does Cloudera support Apache Beam ?
Labels:
New Contributor
Created on ‎07-26-2017 09:22 AM - edited ‎09-16-2022 08:45 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Does Cloudera support Apache Beam project ? Any parcels for installation on CDH ?
2 REPLIES 2
Mentor
Created ‎07-26-2017 10:01 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Cloudera supports Apache Spark, upon which an Apache Beam runner exists. I assume this is what you'd meant to ask about?
Apache Beam by itself is not a service that needs installation and management (such as via Cloudera Manager), but is rather a programming model that supports various execution modes (one of which is Apache Spark).
You should be able to follow the tutorial on https://beam.apache.org/get-started/quickstart-java/ and https://beam.apache.org/documentation/runners/spark/ without trouble, just ensure to use the CDH version of Apache Spark when configuring your Java application's pom.xml.
There's no direct support offered for Apache Beam SDKs by Cloudera at present, but I see no reason for it to not work on top of your CDH cluster.
Apache Beam by itself is not a service that needs installation and management (such as via Cloudera Manager), but is rather a programming model that supports various execution modes (one of which is Apache Spark).
You should be able to follow the tutorial on https://beam.apache.org/get-started/quickstart-java/ and https://beam.apache.org/documentation/runners/spark/ without trouble, just ensure to use the CDH version of Apache Spark when configuring your Java application's pom.xml.
There's no direct support offered for Apache Beam SDKs by Cloudera at present, but I see no reason for it to not work on top of your CDH cluster.
New Contributor
Created ‎07-26-2017 12:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Harsh , I was considering the possibility of other runners apart from spark and see how to go about doing it on CDH. I appreciate for confirming no support is offered for SDK.
Thanks
