When Run $./sbin/start-master.sh Obtaining Errors

davehkd — Tue, 16 May 2023 12:45:27 GMT

Hello -

I have installed a 4 node Spark-3.3.1 running on YARN. The nodes are: node-master, node1, node2, node3 and node4. I am using JAVA 8.

Spark is working.

I have configured Spark to use 3rd party PKI certificates and successfully created the keystore and truststore. I copied the keystore and truststore to the Spark/conf directory.

I added the following to the spark-defaults.conf file:

spark.ssl.enabled true

spark.ssl.enabledAlgorithms TLS_RSA_WITH_AES_256_GCM_SHA384

spark.ssl.protocol TLSv1.2

spark.ssl.keyPassword key_password (i entered the correct keypassword)

spark.ssl.keyStore /path/to/my_key_store ( i entered the correct path to include the file name)
spark.ssl.keyStore

spark.ssl.keyStorePassword

spark.ssl.truststore

spark.ssl.trustStorePassword

I then started the spark cluster,

$./sbin/start-master.sh

I looked in the log specified when i ran start-master.sh, and this is what the log contained:

INFO Master: Started daemon with process name: 6096@IP...

INFO SignalUtils: Registering signal handler for TERM

INFO SignalUtils: Registering signal handler for HUP

INFO SignalUtils: Registering signal for INT

INFO SecurityManager: Changing view acls to: hadoop

INFO SecurityManager: Changing modify acls to: hadoop

INFO SecurityManager: Changing view acls groups to:

INFO SecurityManager: Changing modify acls groups to:

INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); groups with view permissions: Set(); users with modify permissions: Set(hadoop); groups with modify permissions: Set()

INFO Utils: Successfully started service 'sparkMaster' on port 7077

INFO Master: Starting Spark master at spark://ip...:7077

INFO Master: Running Spark version 3.3.1

WARN SslContextFactory: No supported Cipher Suite from [...it appears to be a complete list of the ciphers supported by Java 8...] e.g., TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA2556...and this repeats w/ 45 cipher entries...

INFO Utils: Successfully started service 'MasterUI (HTTPS)' on port 8480

INFO Utils: Successfully started service 'MasterUI' on port 8080

INFO MasterWebUI: Boiund MasterWebUI to 0.0.0.0 and started https://node-master:8480

INFO Master: I have been elected leader! New state: ALIVE

Can anyone shed light on the meaning of the warning above? And, why is it spitting out all of the Java 8 ciphers?

Thank you for any guidance/recommendations anyone can provide.

Re: When Run $./sbin/start-master.sh Obtaining Errors

davehkd — Tue, 16 May 2023 10:56:01 GMT

And, just to be complete, i did enter the correct values for the following items in the spark-default.conf file:

spark.ssl.keyStore

spark.ssl.keyStorePassword

spark.ssl.truststore

spark.ssl.trustStorePassword

Re: When Run $./sbin/start-master.sh Obtaining Errors

davehkd — Tue, 16 May 2023 11:26:08 GMT

Additionally, i copied the keystore and truststore to the node-master. I did not yet copy the keystore and truststore to node1, node2, node3 and node4.

question Re: When Run $./sbin/start-master.sh Obtaining Errors in Support Questions

When Run $./sbin/start-master.sh Obtaining Errors

Re: When Run $./sbin/start-master.sh Obtaining Errors

Re: When Run $./sbin/start-master.sh Obtaining Errors