Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Rising Star

Question

What are the available cluster types in Data Engineering Cluster?

Answer

Currently, Cloudera provides clusters that can run the following Data Engineering job types:

 

- Hive on MapReduce

- Hive on Spark

- Spark on YARN

- MapReduce2

- PySpark

 

Note:  Job submission for a cluster is restricted to the type of job for that various cluster.   For example, if you wanted to run just a MR2 job on a Spark-on-YARN cluster, you cannot submit this, even though Spark is configured to use YARN.   You would need to standup a separate YARN cluster for it to accept a MR2/YARN job.

689 Views