Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP Spark Connector for Google Cloud

Solved Go to solution

HDP Spark Connector for Google Cloud

New Contributor

We are in process to migrate our HDP clusters to Google Cloud. Spark is heavily used by our applications and data scientists. GCS will be also heavily used to fetch raw data and post processed data. One challenge we have is the lack of support for Google Connector and capabilities around GCS and running Spark jobs efficiently on GCS stored data. This is not part of the HDP bundle and there is no information on whether this will be bundled and supported in the next versions. Does Hortonworks have a plan to include it in the following versions, also to support?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDP Spark Connector for Google Cloud

@Jane Becker

True. The connector is not currently bundled or supported. I installed it manually and my preliminary tests were successful when using it with Spark, but I did not anything complicated or at scale. I checked recently with Engineering and there is a good chance that it will be supported in the second part of 2018. As this connector gets more attention and importance from the users community, its priority will increase and there will be a better chance that it will be supported sooner. As you may know, this connector does not seem supported even by Google.

1 REPLY 1

Re: HDP Spark Connector for Google Cloud

@Jane Becker

True. The connector is not currently bundled or supported. I installed it manually and my preliminary tests were successful when using it with Spark, but I did not anything complicated or at scale. I checked recently with Engineering and there is a good chance that it will be supported in the second part of 2018. As this connector gets more attention and importance from the users community, its priority will increase and there will be a better chance that it will be supported sooner. As you may know, this connector does not seem supported even by Google.

Don't have an account?
Coming from Hortonworks? Activate your account here