Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

When Run $./sbin/start-master.sh Obtaining Errors

avatar
Contributor

Hello -

 

I have installed a 4 node Spark-3.3.1 running on YARN.  The nodes are:  node-master, node1, node2, node3 and node4.  I am using JAVA 8.

 

Spark is working.

 

I have configured Spark to use 3rd party PKI certificates and successfully created the keystore and truststore.  I copied the keystore and truststore to the Spark/conf directory.

 

I added the following to the spark-defaults.conf file:

 

spark.ssl.enabled                                           true

spark.ssl.enabledAlgorithms                          TLS_RSA_WITH_AES_256_GCM_SHA384

spark.ssl.protocol                                            TLSv1.2

spark.ssl.keyPassword                                   key_password (i entered the correct keypassword)

spark.ssl.keyStore                                          /path/to/my_key_store ( i entered the correct path to include the file name)
spark.ssl.keyStore

spark.ssl.keyStorePassword

spark.ssl.truststore

spark.ssl.trustStorePassword

 

I then started the spark cluster, 

$./sbin/start-master.sh

 

I looked in the log specified when i ran start-master.sh, and this is what the log contained:

INFO Master:  Started daemon with process name: 6096@IP...

INFO SignalUtils:  Registering signal handler for TERM

INFO SignalUtils: Registering signal handler for HUP

INFO SignalUtils: Registering signal for INT

INFO SecurityManager: Changing view acls to:  hadoop

INFO SecurityManager: Changing modify acls to: hadoop

INFO SecurityManager: Changing view acls groups to:

INFO SecurityManager: Changing modify acls groups to:

INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); groups with view permissions: Set(); users with modify permissions: Set(hadoop); groups with modify permissions: Set()

INFO Utils: Successfully started service 'sparkMaster' on port 7077

INFO Master: Starting Spark master at spark://ip...:7077

INFO Master:  Running Spark version 3.3.1

WARN SslContextFactory:  No supported Cipher Suite from [...it appears to be a complete list of the ciphers supported by Java 8...] e.g., TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA2556...and this repeats w/ 45 cipher entries...

INFO Utils: Successfully started service 'MasterUI (HTTPS)' on port 8480

INFO Utils: Successfully started service 'MasterUI' on port 8080

INFO MasterWebUI:  Boiund MasterWebUI to 0.0.0.0 and started https://node-master:8480

INFO Master:  I have been elected leader!  New state:  ALIVE

 

Can anyone shed light on the meaning of the warning above?  And, why is it spitting out all of the Java 8 ciphers?

 

Thank you for any guidance/recommendations anyone can provide.

2 REPLIES 2

avatar
Contributor

And, just to be complete, i did enter the correct values for the following items in the spark-default.conf file:

spark.ssl.keyStore

spark.ssl.keyStorePassword

spark.ssl.truststore

spark.ssl.trustStorePassword

 

avatar
Contributor

Additionally, i copied the keystore and truststore to the node-master.  I did not yet copy the keystore and truststore to node1, node2, node3 and node4.