Hi @BlueBerret as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. ... View more

Hi Yuexin   Did you manage to get any resolution for this? I am able to run spark job as a --proxy-user under yarn cluster mode. However I can successfully run using yarn-client mode. This is when using CDH 6.2.1 version of Spark.   There is no problem when using opensource version of Spark with --proxy-user either on client or cluster mode.   Thanks ... View more

this issue resolved ???i am also acing the same issue  please suggest  ... View more

Thanks for the awesome explanation!! This comment from spark explains on the reason for allowing insecure connection https://issues.apache.org/jira/browse/SPARK-26019?focusedCommentId=16719231&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16719231  ... View more

I actually did figure out how to solve this, at least in our case.  Hopefully it's relevant to yours as well.  See here. ... View more

Still happening on CDH 6.3 as well.  See my new post here. ... View more