Hello -   I have installed a 4 node Spark-3.3.1 running on YARN.  The nodes are:  node-master, node1, node2, node3 and node4.  I am using JAVA 8.   Spark is working.   I have configured Spark to use 3rd party PKI certificates and successfully created the keystore and truststore.  I copied the keystore and truststore to the Spark/conf directory.   I added the following to the spark-defaults.conf file:   spark.ssl.enabled                                           true spark.ssl.enabledAlgorithms                          TLS_RSA_WITH_AES_256_GCM_SHA384 spark.ssl.protocol                                            TLSv1.2 spark.ssl.keyPassword                                   key_password (i entered the correct keypassword) spark.ssl.keyStore                                          /path/to/my_key_store ( i entered the correct path to include the file name) spark.ssl.keyStore spark.ssl.keyStorePassword spark.ssl.truststore spark.ssl.trustStorePassword   I then started the spark cluster,  $./sbin/start-master.sh   I looked in the log specified when i ran start-master.sh, and this is what the log contained: INFO Master:  Started daemon with process name: 6096@IP... INFO SignalUtils:  Registering signal handler for TERM INFO SignalUtils: Registering signal handler for HUP INFO SignalUtils: Registering signal for INT INFO SecurityManager: Changing view acls to:  hadoop INFO SecurityManager: Changing modify acls to: hadoop INFO SecurityManager: Changing view acls groups to: INFO SecurityManager: Changing modify acls groups to: INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); groups with view permissions: Set(); users with modify permissions: Set(hadoop); groups with modify permissions: Set() INFO Utils: Successfully started service 'sparkMaster' on port 7077 INFO Master: Starting Spark master at spark://ip...:7077 INFO Master:  Running Spark version 3.3.1 WARN SslContextFactory:  No supported Cipher Suite from [...it appears to be a complete list of the ciphers supported by Java 8...] e.g., TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA2556...and this repeats w/ 45 cipher entries... INFO Utils: Successfully started service 'MasterUI (HTTPS)' on port 8480 INFO Utils: Successfully started service 'MasterUI' on port 8080 INFO MasterWebUI:  Boiund MasterWebUI to 0.0.0.0 and started https://node-master:8480 INFO Master:  I have been elected leader!  New state:  ALIVE   Can anyone shed light on the meaning of the warning above?  And, why is it spitting out all of the Java 8 ciphers?   Thank you for any guidance/recommendations anyone can provide. ... View more

Hello - I am new to Apache Spark.  I have installed a 5 node Apache Spark cluster running on YARN on AWS. I am seeking any "prescriptive" vs descriptive information on how to security harden the cluster. I have reviewed the Apache Spark security website, and scoured the internet for detailed, prescriptive information/approaches for security hardening this type of cluster and have not be been able to find information provided by folks with direct, hands on experience.   I have also searched for online courses that focus on implementing security with the Spark on YARN cluster and came up short.   Does anyone have any suggested references that provide/describe how to's, detailed information on hardening procedures for this type of cluster. Any suggested references would be tremendously appreciated!   VR,   Dave ... View more

Is it possible...does it make sense  to place an AWS application load balancer in front of a NiFi cluster?   ... View more

Hello, I have a 3 node NiFi cluster.  I would like to add new users to the cluster that would be administrators i.e., a new user with all of the capabilities of the initial admin identity user (in my case, the initial admin identify is user nifi).   I am using PKI certs as the means of identifty/authenticating users and this is working well.   So, my question is, can I create a handful of users that are administrators that have a policy set equivalent to the initial admin identity?   Is this possible?     ... View more