I could re-create the same issue on hdp 2.5 sandbox. it looks to me a bug. ... View more

Spark 2.1 and Zeppelin 0.7 can be run as standalone as follows: Spark Installation Do the followings: 1.Go to http://spark.apache.org/downloads.html and download the latest file. 2.Unzip the file to the appropriate location. 3.Read https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-tips-and-tricks-running-spark-windows.html and follow the instruction. 4.After the installation, go to Spark's bin directory in the command window and run spark-shell to see scala prompt. You can close the command window. *above step 3 summary: - Download winutils.exe binary from https://github.com/steveloughran/winutils repository. (You should select the version of Hadoop the Spark distribution.) - Save winutils.exe binary to a directory of your choice, e.g. c:\hadoop\bin. - Set HADOOP_HOME to reflect the directory with winutils.exe (without bin). e.g. set HADOOP_HOME=c:\hadoop - Set PATH environment variable to include %HADOOP_HOME%\bin as follows: set PATH=%HADOOP_HOME%\bin;%PATH% - Create c:\tmp\hive directory. - Execute winutils.exe chmod -R 777\tmp\hive command and the check with winutils.exe ls \tmp\hive command. Zeppelin Installation Do the followings: 1.Go to http://zeppelin.apache.org/download.html and download the latest file. 2.Unzip the file to the appropriate location. 3.Go to https://github.com/elodina/zeppelin-notebooks/blob/master/conf/interpreter.json. 4.Copy the content of interpreter.json and save it into conf/interpreter.json file. If you don't find the file in conf directory, create it. 5.Learn how to start and stop Zeppelin in http://zeppelin.apache.org/docs/0.7.1/install/install.html. 6.Go to http://localhost:8080 and click anonymous user at the top/right and click Interpreter. Look for Spark section and click edit button at the right. 7.Update master value to local[*], save, and restart the Spark interpreter. Restart button is next to edit button. 8.Don't use the tutorial. It does not work; instead, use Spark's latest tutorial: http://spark.apache.org/docs/latest/sql-programming-guide.html 9.When you code with scala, you don't need to specify any interpreter such as %xyz, but use %sql when you use Spark SQL. ... View more

Does it mean livy.spark.yarn.queue overrides the setting I found? ... View more

I found a partial answer here: https://community.hortonworks.com/questions/35632/how-to-choose-the-queue-in-which-you-want-to-submi.html ... View more

I added: export ZEPPELIN_MEM="-Xms1024m -Xmx2048m -XX:MaxMetaspaceSize=512m" The out of memory issue was fixed; however, I got another new issue related to yarn configuration. I will ask it with "new question". Thanks! ... View more

Yes, that's right. On Ambari, should we add zeppelin_mem, let's say -XmXXXXm to Custom zeppelin-env? ... View more

log file says: INFO [2017-04-07 11:53:58,682] ({pool-2-thread-4} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1491580438681 started by scheduler org.apache.zeppelin.spark.SparkInterpreter283573820 INFO [2017-04-07 11:54:02,145] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Changing view acls to: zeppelin INFO [2017-04-07 11:54:02,145] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Changing modify acls to: zeppelin INFO [2017-04-07 11:54:02,145] ({pool-2-thread-4} Logging.scala[logInfo]:58) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(zeppelin); users with modify permissions: Set(zeppelin) INFO [2017-04-07 11:54:05,322] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Starting HTTP Server INFO [2017-04-07 11:54:05,945] ({pool-2-thread-4} Server.java[doStart]:272) - jetty-8.y.z-SNAPSHOT INFO [2017-04-07 11:54:08,337] ({pool-2-thread-4} AbstractConnector.java[doStart]:338) - Started SocketConnector@0.0.0.0:22900 INFO [2017-04-07 11:54:08,338] ({pool-2-thread-4} Logging.scala[logInfo]:58) - Successfully started service 'HTTP class server' on port 22900. ERROR [2017-04-07 11:54:10,658] ({pool-2-thread-4} Job.java[run]:189) - Job failed java.lang.OutOfMemoryError: GC overhead limit exceeded INFO [2017-04-07 11:54:10,658] ({pool-2-thread-4} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1491580438681 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter283573820 INFO [2017-04-07 11:54:11,954] ({pool-1-thread-18} Logging.scala[logInfo]:58) - Changing view acls to: zeppelin INFO [2017-04-07 11:54:11,954] ({pool-1-thread-18} Logging.scala[logInfo]:58) - Changing modify acls to: zeppelin INFO [2017-04-07 11:54:11,955] ({pool-1-thread-18} Logging.scala[logInfo]:58) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(zeppelin); users with modify permissions: Set(zeppelin) "xxx.out" file says: Exception in thread "qtp686649452-427" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1855) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2068) at org.spark-project.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342) at org.spark-project.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526) at org.spark-project.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44) at org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) Exception in thread "pool-1-thread-4" Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "pool-1-thread-4" Exception in thread "pool-1-thread-7" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-6" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-9" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1834748429-966" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-8" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-13" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-12" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-11" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-1" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-3" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp184565796-1023" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1834748429-962" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp851486649-779" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1459091505-28" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp1613009598-14753" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "pool-1-thread-18" java.lang.OutOfMemoryError: GC overhead limit exceeded ... View more

Move to hdp 2.5 and use Spark action may be easier. ... View more

Thank you for the answer; however, both are not for levelDB, which is used in node manager. Do you have any idea to initialize levelDB. I try to find it, but i can't find any good article. ... View more

Yes, I use Ambari. I have one more question to clear my head. Assume I have nodes that have at least 20GB memory and 4 cores for each node, and each node has node manager installed. If I add one old computer that has 5GB memory and 2 cores, and if I install node manager on this new computer, Yarn memory setting for one node allocation should be reduced at max 5GB? This would be because Yarn configuration is made against all node managers. I am asking because in my cluster, the Ambari Yarn's memory setting, especially for node, the max memory size is the lowest of all nodes that has node manager installed. In other words, if we want to utilize resources, we should add nodes that have similar memory size and number of cores? Thanks you, ... View more

I understand Yarn and HDFS are separately defined. One thing that confuses me is that when I see Yarn's resource manager UI, it shows max memory size and max vcore, I guess, which is not an actual figure from data node servers but from the Yarn configuration? If the yarn has not actual configuration information, how can we know the accurate information of the current processing? I also wonder how resource manager can allocate resource correctly. (sorry about too many questions...) Thank you, ... View more

When we need to have more resources, which means typically, at the time when we add more data node servers? This makes sense to me. What I'm not so clear is let's say without adding anything with my example 5 servers setup, what does it mean to install node manager to all 5 servers? Even if we install more node managers, it does not mean we increase the resources because the number of data node servers stay 3. On the other hand, if we install node manager to all 5 servers, Yarn configuration may give us wrong resource information? Thank you, ... View more

Thank you for the recommendation, but I would like to solve this issue first. We are using hdp 2.5. Previously, we used hdp 2.3 where I could not run Spark with Phoenix. Can hdp 2.5 allow us to use Phoenix in Spark 1.6.2? ... View more

Where did you copy your jdbc driver for Sqoop action? ... View more

As far as I see sqoop action above, I don't see hive-site.xml file. I guess you added it into lib directory in the deployment directory, which will keep Hive action from running and it gives you error something like hive-site.xml permssion error. You should add the hive-site.xml file in "Files" in the Sqoop action. ... View more

When I added Hive action before the Sqoop action, I got hive-site.xml file permission error. To avoid this problem, delete lib/hive-site.xml file, and add this file to inside of Sqoop action as a file. ... View more

When I tried to run an Oozie workflow that contains Hive and Sqoop actions, I had the same problem. In my case, I had lib directory where hive-site.xml exists for Sqoop action. After I move hive-site.xml to the parent directory of lib directory, which is the HDFS deployment directory and I add the hive-site.xml as a file to Sqoop action, the workflow works. By the way, this is for HDP 2.3.2. ... View more

I have successfully run Hive action from Oozie Workflow. My simple Hive operation does: drop table test1; create table test1 as select * from A_BASE; Followings are the steps: 1: run su - oozie from SSH window. 2: run hdfs dfs -put /usr/hdp/2.3.2.0-2950/atlas/hook/hive/* /user/oozie/share/lib/lib_20151027124452/hive (assume HDP 2.3.2 used) 3: create a workflow that contains a Hive action. 4: add a property oozie.action.sharelib.for.hive = hive,hcatalog,sqoop in Oozie parameters. 5: create a hive script like above and upload it from "Script name" in the Hive action edit page. 6: Save the workflow. 7: Run it. 8: It should run. ... View more

It looks like HDP 2.3.2 already has this patch. ... View more

I also tested HiveContext so that Hive processing works in Spark memory. It works. ... View more

Thank you, Artem. ... View more

Since I could see the detail error message from http://localhost:19888/jobhistory/?user.name=hue, I could identify what's wrong. it was due to a temp dir for A_BASE; after I delete this, it works, but since so many things I touched, I will do from the scratch and I will comment my summary here. Thanks, Artem for this with me. ... View more

Since whenever we submit the workflow, it will create a new directory, it does not work. I did sudo -u yarn with sqoop import. I does not return code 1 any more, which is good, but neither HDFS file nor Hive table is created. Any idea? ... View more