I have undone all the TLS enabling and still had the same problem. Eventually it occurred to me that some processes might be stuck. So I physically rebooted all the Hadoop machines and that resolved the problem. After that I was able to reenable all the steps for TLS. However, I still have a problem with pyspark and spark-submit: they run with --master=yarn --deploy-mode=client but fail with --master=yarn --deploy-mode=cluster ============== [ivy2@md01 ~]$ pyspark --master=yarn --deploy-mode=cluster Python 2.7.5 (default, Nov 20 2015, 02:00:19) [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Error: Cluster deploy mode is not applicable to Spark shells. Run with --help for usage help or --verbose for debug output Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/spark/python/pyspark/shell.py", line 43, in <module> sc = SparkContext(pyFiles=add_files) File "/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/spark/python/pyspark/context.py", line 112, in __init__ SparkContext._ensure_initialized(self, gateway=gateway) File "/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/spark/python/pyspark/context.py", line 245, in _ensure_initialized SparkContext._gateway = gateway or launch_gateway() File "/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/spark/python/pyspark/java_gateway.py", line 94, in launch_gateway raise Exception("Java gateway process exited before sending the driver its port number") Exception: Java gateway process exited before sending the driver its port number >>> ========== Could TLS intefere with spark?
... View more
