@PDDF_VIGNESH As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks! ... View more

I recently upgraded CM to 5.10.0. Trying to upgrade CDH from the Manager from 5.8.0 throws up the same issue. I cannot see or select the CDH. Attached is a screenshot. Any help in figuring out next steps to get this running will help.                        ... View more

When you say it is not working, what issue does it exhibit?  For Hive on Spark you only need set the Execution Engine within Hive from MapReduce to Spark.  You do need to consider Spark memory setting for Executors in the Spark Service and these must correlate to the YARN container memory settings.  Generally I set the following YARN container settings:   yarn.nodemanager.resource.memory-mb yarn.scheduler.maximum-allocation-mb   To be the same value but greater than the Spark Executor Memory + Overhead .  Check also for the following similar error in the YARN logs:   15/09/17 11:15:09 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (2211 MB per container) Exception in thread "main" java.lang.IllegalArgumentException: Required executor memory (2048+384 MB) is above the max threshold (2211 MB) of this cluster!   Regards Shailesh       ... View more

Excellent that solved the problem along with some unknown fields in the schema.xml where I have now used sanitizeUnknownSolrFields in the Morphline .  Thanks for your help. ... View more