Hello every body,   I have a Hive table and trying to find a solution to add an incremental primary key to it. Here is my solution:    create table new_table as select row_number() over () as ID, * from old_table;   It will create a new table with a new incremental column(ID). Its working well on small tables but when I'm running it on a bigger table(20M records/500 columns), it will fail with this message:   Examining task ID: task_1444013233108_0091_r_000558 (and more) from job job_1444013233108_0091 Examining task ID: task_1444013233108_0091_r_000000 (and more) from job job_1444013233108_0091 Task with the most failures(4): ----- Task ID:   task_1444013233108_0091_r_000000 URL:   http://chd.moneyball.guru:8088/taskdetails.jsp?jobid=job_1444013233108_0091&tipid=task_1444013233108_0091_r_000000 ----- Diagnostic Messages for this Task: Exception from container-launch. Container id: container_1444013233108_0091_01_000715 Exit code: 255 Stack trace: ExitCodeException exitCode=255:     at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)     at org.apache.hadoop.util.Shell.run(Shell.java:455)     at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)     at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)     at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)     at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)     at java.util.concurrent.FutureTask.run(FutureTask.java:262)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)     at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 255 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Stage-Stage-1: Map: 142  Reduce: 568   Cumulative CPU: 9791.07 sec   HDFS Read: 38198769932 HDFS Write: 54432 FAIL Total MapReduce CPU Time Spent: 0 days 2 hours 43 minutes 11 seconds 70 msec   ----------------------------- I also tried to limit the number of records in select command:  create table new_table as select row_number() over () as ID, * from old_table limit 1000;   Do you have any idea about this error?   Thanks ... View more

Hi,   We are trying to run a Spark job on YARN. The problem is that usercache direcory(yarn/nm/usercache/ ) is growing too fast and it will fill all of the disk. The size of Hive table is around 70G and total free disk space around 300G. In usercache direcory there are a lot of big folders like blockmgr-b5b55c6f-ef8a-4359-93e4-9935f2390367.   My squestions are: - Is it normal or something is wrong with my Spark or YARN? - If these files are just cached files is there any way to limit the cache size?   Thanks for your help.     ... View more

Hi,   I'm trying to run a Spark application on YARN in a single node instance with 32G RAM. Its working well for a small dataset. But for a bigger table its failing with this error:   Application application_1442094222971_0008 failed 2 times due to AM Container for appattempt_1442094222971_0008_000002 exited with exitCode: 11 For more detailed output, check application tracking page:http://chd.moneyball.guru:8088/proxy/application_1442094222971_0008/Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1442094222971_0008_02_000001 Exit code: 11 Stack trace: ExitCodeException exitCode=11: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)     Container exited with a non-zero exit code 11 Failing this attempt. Failing the application.   -----------------------------   Here is the stdout of container: [2015-09-12T20:53:28.368-04:00] [DataProcessing] [WARN] [] [org.apache.spark.Logging$class] [tid:Driver Heartbeater] [userID:yarn] Error sending message [message = Heartbeat(2,[Lscala.Tuple2;@2c3b1696,BlockManagerId(2, chd.moneyball.guru, 60663))] in 1 attempts java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]     at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)     at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)     at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)     at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)     at scala.concurrent.Await$.result(package.scala:107)     at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:195)     at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:427) [2015-09-12T20:53:32.314-04:00] [DataProcessing] [WARN] [] [org.apache.spark.Logging$class] [tid:Executor task launch worker-0] [userID:yarn] Error sending message [message = GetLocations(rdd_4_1839)] in 1 attempts java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]     at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)     at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)     at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)     at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)     at scala.concurrent.Await$.result(package.scala:107)     at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:195)     at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:221)     at org.apache.spark.storage.BlockManagerMaster.getLocations(BlockManagerMaster.scala:70)     at org.apache.spark.storage.BlockManager.doGetRemote(BlockManager.scala:591)     at org.apache.spark.storage.BlockManager.getRemote(BlockManager.scala:578)     at org.apache.spark.storage.BlockManager.get(BlockManager.scala:622)     at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:44)     at org.apache.spark.rdd.RDD.iterator(RDD.scala:242)     at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)     at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)     at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)     at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)     at org.apache.spark.scheduler.Task.run(Task.scala:64)     at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)     at java.lang.Thread.run(Thread.java:745) [2015-09-12T20:53:42.123-04:00] [DataProcessing] [ERROR] [] [org.apache.spark.Logging$class] [tid:sparkExecutor-akka.actor.default-dispatcher-3] [userID:yarn] Driver Disassociated [akka.tcp://sparkExecutor@chd.moneyball.guru:38443] -> [akka.tcp://sparkDriver@chd.moneyball.guru:43977] disassociated! Shutting down.   Any help?   Thanks ... View more

I have installed CHD5.4 on Ubuntu 14.04 successdully. All of services are ok an just HDFS shows "Under-Replicated Blocks".   When I want to run one of spark samples it will stuck at state:ACCEPTED and show a message in an infinite loop.   Command:   sudo -uhdfs spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode cluster --master yarn /opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/jars/spark-examples-1.3.0-cdh5.4.5-hadoop2.6.0-cdh5.4.5.jar   Output:       SLF4J: Class path contains multiple SLF4J bindings.     SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]     SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/jars/avro-tools-1.7.6-cdh5.4.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]     SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.     SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]     15/08/29 12:07:56 INFO RMProxy: Connecting to ResourceManager at chd2.moneyball.guru/104.131.78.0:8032     15/08/29 12:07:56 INFO Client: Requesting a new application from cluster with 1 NodeManagers     15/08/29 12:07:56 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (1750 MB per container)     15/08/29 12:07:56 INFO Client: Will allocate AM container, with 896 MB memory including 384 MB overhead     15/08/29 12:07:56 INFO Client: Setting up container launch context for our AM     15/08/29 12:07:56 INFO Client: Preparing resources for our AM container     15/08/29 12:07:57 INFO Client: Uploading resource file:/opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/jars/spark-examples-1.3.0-cdh5.4.5-hadoop2.6.0-cdh5.4.5.jar -> hdfs://chd2.moneyball.guru:8020/user/hdfs/.sparkStaging/application_1440861466017_0007/spark-examples-1.3.0-cdh5.4.5-hadoop2.6.0-cdh5.4.5.jar     15/08/29 12:07:57 INFO Client: Setting up the launch environment for our AM container     15/08/29 12:07:57 INFO SecurityManager: Changing view acls to: hdfs     15/08/29 12:07:57 INFO SecurityManager: Changing modify acls to: hdfs     15/08/29 12:07:57 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hdfs); users with modify permissions: Set(hdfs)     15/08/29 12:07:57 INFO Client: Submitting application 7 to ResourceManager     15/08/29 12:07:57 INFO YarnClientImpl: Submitted application application_1440861466017_0007     15/08/29 12:07:58 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:07:58 INFO Client:          client token: N/A          diagnostics: N/A          ApplicationMaster host: N/A          ApplicationMaster RPC port: -1          queue: root.hdfs          start time: 1440864477580          final status: UNDEFINED          tracking URL: http://chd2.moneyball.guru:8088/proxy/application_1440861466017_0007/          user: hdfs     15/08/29 12:07:59 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:00 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:01 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:02 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:03 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:04 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:05 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)     15/08/29 12:08:06 INFO Client: Application report for application_1440861466017_0007 (state: ACCEPTED)   I tried it in several servers but always same error.  Let me know if you need anything else. ... View more