By increasing the width of the chrome (browser) window on Zeppelin (I am accessing zeppelin on 127.0.0.1:9995/#/), it switched on the zeppelin clone button bar ... View more

Hi Tim, I launched sandbox HDP 2.5. When I go to 127.0.0.1:9995, Zeppelin is an "empty", "static" page with no top bars and no pre-existing zeppelin notebooks... Was it the case when you first launched Zeppelin on HDP2.5 - How do you set Spark 2.0 screenshot-2016-08-23-213704.jpg ... View more

Thank you zblanco. I was able to log into ssh online through 127.0.0.1:4200. I reset the ambari password as per the tutorial. I can access the ambari console as admin with all the various dashboards and componenets. I can not ssh via terminal as I am getting conflict on RSA key from my other HDP2.4 sandbox. Is it possible to add the HDP2.5 key and be able to use both sandboxes through ssh? $ ssh root@127.0.0.1 -p 2222 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:/kRUlZqqoBGnsyfJkjU2jScS/vP1/VEpk5ejg8bnlRI. Please contact your system administrator. Add correct host key in /Users/xxxxx/.ssh/known_hosts to get rid of this message. Offending RSA key in /Users/xxxxx/.ssh/known_hosts:5 RSA host key for [127.0.0.1]:2222 has changed and you have requested strict checking. Host key verification failed. I am able to call spark on the terminal - spark-shell and pyspark. I am getting spark 1.6.2 - How do I select spark 2.0.0 as my default. [root@sandbox ~]# spark-shell SPARK_MAJOR_VERSION is not set, choosing Spark automatically 16/08/23 21:29:14 INFO SecurityManager: Changing view acls to: root 16/08/23 21:29:14 INFO SecurityManager: Changing modify acls to: root 16/08/23 21:29:14 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with vie w permissions: Set(root); users with modify permissions: Set(root) 16/08/23 21:29:14 INFO HttpServer: Starting HTTP Server 16/08/23 21:29:14 INFO Server: jetty-8.y.z-SNAPSHOT 16/08/23 21:29:14 INFO AbstractConnector: Started SocketConnector@0.0.0.0:35616 16/08/23 21:29:14 INFO Utils: Successfully started service 'HTTP class server' on port 35616. Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 1.6.2 /_/ Using Scala version 2.10.5 (OpenJDK 64-Bit Server VM, Java 1.7.0_101) Type in expressions to have them evaluated. Type :help for more information. 16/08/23 21:29:18 INFO SparkContext: Running Spark version 1.6.2 16/08/23 21:29:18 INFO SecurityManager: Changing view acls to: root 16/08/23 21:29:18 INFO SecurityManager: Changing modify acls to: root I am still unable to get a login UI to Zeppelin. In Ambari, Zeppelin is showing green, up and running... Any idea on how to get Zeppelin to work would be appreciated. Thanks. ... View more

HI Pierre, We would need to look at the code. Can you a do a persist just before stage 63 and before stage 65 check the spark UI storage tab and executor tab for data skew. If there is data skew, you will need to add a salt key to your key. You could also look at creating a dataframe from the RDD rdd.toDF() and apply UDF on it. DF manage memory more efficiently. Best, Amit ... View more

Spark 2.0.0 will be available with HDP-2.5 https://community.hortonworks.com/articles/53029/how-to-install-and-run-spark-20-on-hdp-25-sandbox.html. ... View more

Hi Alex, Can you clarify which version of Spark you are using? ... View more

Hi Pierre, How is Object defined and serialized? If fields of your object refers to the RDD, it copies the full RDD and shuffles it. Would you be able to do a persist/cache before the broadcast join and get the Spark UI DAGS and Storage pages. Cheers. Amit ... View more

Cool stuff ... View more