Support Questions
Find answers, ask questions, and share your expertise

Error initialising spark context

i am using hdp 2.4 . when i run this below command. i also tried other commands which is suggested in community, then also no change.

any help?

[root@sandbox ~]# su - hdfs                                                                                                                                                    
[hdfs@sandbox ~]$ hdfs dfs -mkdir /user/root                                                                                                                                   
mkdir: `/user/root': File exists                                                                                                                                               
[hdfs@sandbox ~]$ hdfs dfs -chown -R root:hdfs /user/root                                                                                                                      
[hdfs@sandbox ~]$ exit                                                                                                                                                         
logout                                                                                                                                                                         
[root@sandbox ~]# id                                                                                                                                                           
uid=0(root) gid=0(root) groups=0(root)                                                                                                                                         
[root@sandbox ~]#                                                                                                                                                              
[root@sandbox ~]# ./bin/spark-shell --master yarn-client --driver-memory 512m --executor-memory 512m                                                                           
-bash: ./bin/spark-shell: No such file or directory                                                                                                                            
[root@sandbox ~]# spark-shell                                                                                                                                                  
[root@sandbox ~]# spark-shell

16/04/09 12:01:21 INFO SparkContext: Running Spark version 1.6.0                                                                                                               
16/04/09 12:01:33 INFO SecurityManager: Changing view acls to: root                                                                                                            
16/04/09 12:01:33 INFO SecurityManager: Changing modify acls to: root                                                                                                          
16/04/09 12:01:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(
root)                                                                                                                                                                          
16/04/09 12:02:36 INFO Utils: Successfully started service 'sparkDriver' on port 45756.                                                                                        
16/04/09 12:03:06 INFO Slf4jLogger: Slf4jLogger started                                                                                                                        
16/04/09 12:03:14 INFO Remoting: Starting remoting                                                                                                                             
16/04/09 12:03:34 ERROR SparkContext: Error initializing SparkContext.                                                                                                         
java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds]                                                                                            
        at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)                                                                                               
        at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)                                                                                              
        at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)                                                                                                   
        at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)                                                                                   
        at scala.concurrent.Await$.result(package.scala:107)                                                                                                                   
        at akka.remote.Remoting.start(Remoting.scala:179)                                                                                                                      
        at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)                                                                                           
        at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:620)                                                                                                     
        at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:617)                                                                                                 
        at akka.actor.ActorSystemImpl._start(ActorSystem.scala:617)                                                                                                            
        at akka.actor.ActorSystemImpl.start(ActorSystem.scala:634)                                                                                                             
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)                                                                                                                
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)                                                                                                                
        at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)                                                          
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)                                                                                                
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)                                                                                                
        at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1964)                                                                           
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)                                                                                                    
        at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1955)                                                                                                   
        at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)                                                                                              
        at org.apache.spark.SparkEnv$.create(SparkEnv.scala:266)                                                                                                               
        at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193)                                                                                                      
        at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288)                                                                                                
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:457)                                                                                                        
        at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1017)                                                                                          
        at $line3.$read$$iwC$$iwC.<init>(<console>:15)                                                                                                                         
        at $line3.$read$$iwC.<init>(<console>:24)                                                                                                                              
        at $line3.$read.<init>(<console>:26)                                                                                                                                   
        at $line3.$read$.<init>(<console>:30)                                                                                                                                  
        at $line3.$read$.<clinit>(<console>)                                                                                                                                   
        at $line3.$eval$.<init>(<console>:7)                                                                                                                                   
        at $line3.$eval$.<clinit>(<console>)                                                                                                                                   
        at $line3.$eval.$print(<console>)                                                                                                                                      
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)                                                                                       
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                               
        at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                    
        at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)                                                                                          
        at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)                                                                                          
        at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)                                                                                              
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)                                                                                                    
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)                                                                                                    
        at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)                                                                                            
        at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)                                                                                        
        at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)                                                                                                      
        at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:125)                                                                     
        at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)                                                                     
        at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)                                                                                                
        at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)                                                                                
        at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)                                                                                               
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)                    
        at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:159)                                                                                      
        at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)                                                                                                     
        at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)                                                                             
        at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)                                                                                            
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)                                            
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)                                                   
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)                                                   
        at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)                                                                              
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)                                                                    
        at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)                                                                                                     
        at org.apache.spark.repl.Main$.main(Main.scala:31)                                                                                                                     
        at org.apache.spark.repl.Main.main(Main.scala)                                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)                                                                                       
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                               
        at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                    
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)                                                            
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)                                                                                             
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)                                                                                                  
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)                                                                                                    
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)                                                                                                         
16/04/09 12:03:35 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.                                                                                 
16/04/09 12:03:38 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.                                         
16/04/09 12:03:44 INFO SparkContext: Successfully stopped SparkContext                                                                                                         
16/04/09 12:03:50 WARN RemoteActorRefProvider$RemotingTerminator: unhandled event Failure(akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://sparkDriverActorSyst
em/system/endpointManager#2102592260]] after [10000 ms]) in state WaitTransportShutdown                                                                                        
16/04/09 12:03:54 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports                                                             
java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds]                                                                                            
        at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)                                                                                               
        at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)                                                                                              
        at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)                                                                                                   
        at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)                                                                                   
        at scala.concurrent.Await$.result(package.scala:107)                                                                                                                   
        at akka.remote.Remoting.start(Remoting.scala:179)                                                                                                                      
        at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)                                                                                           
        at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:620)                                                                                                     
        at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:617)                                                                                                 
        at akka.actor.ActorSystemImpl._start(ActorSystem.scala:617)                                                                                                            
        at akka.actor.ActorSystemImpl.start(ActorSystem.scala:634)                                                                                                             
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)                                                                                                                
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)                                                                                                                
        at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)                                                          
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)                                                                                                
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)                                                                                                
        at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1964)                                                                           
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)                                                                                                    
        at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1955)                                                                                                   
        at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)                                                                                              
        at org.apache.spark.SparkEnv$.create(SparkEnv.scala:266)                                                                                                               
        at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193)                                                                                                      
        at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288)                                                                                                
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:457)                                                                                                        
        at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1017)                                                                                          
        at $iwC$$iwC.<init>(<console>:15)                                                                                                                                      
        at $iwC.<init>(<console>:24)                                                                                                                                           
        at <init>(<console>:26)                                                                                                                                                
        at .<init>(<console>:30)                                                                                                                                               
        at .<clinit>(<console>)                                                                                                                                                
        at .<init>(<console>:7)                                                                                                                                                
        at .<clinit>(<console>)                                                                                                                                                
        at $print(<console>)                                                                                                                                                   
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)                                                                                       
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                               
        at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                    
        at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)                                                                                          
        at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)                                                                                          
        at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)                                                                                              
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)                                                                                                    
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)                                                                                                    
        at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)                                                                                            
        at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)                                                                                        
        at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)                                                                                                      
        at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:125)                                                                     
        at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)                                                                     
        at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)                                                                                                
        at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)                                                                                
        at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)                                                                                               
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)                    
        at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:159)                                                                                      
        at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)                                                                                                     
        at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)                                                                             
        at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)                                                                                            
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)                                            
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)                                                   
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)                                                   
        at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)                                                                              
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)                                                                    
        at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)                                                                                                     
        at org.apache.spark.repl.Main$.main(Main.scala:31)                                                                                                                     
        at org.apache.spark.repl.Main.main(Main.scala)                                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)                                                                                       
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                               
        at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                    
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)                                                            
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)                                                                                             
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)                                                                                                  
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)                                                                                                    
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)                                                                                                         
                                                                                                                                                                               
java.lang.NullPointerException                                                                                                                                                 
        at org.apache.spark.sql.SQLContext$.createListenerAndUI(SQLContext.scala:1367)                                                                                         
        at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:101)                                                                                                 
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)                                                                                               
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)                                                                        
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)                                                                
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)                                                                                                     
        at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1028)                                                                                            
        at $iwC$$iwC.<init>(<console>:15)                                                                                                                                      
        at $iwC.<init>(<console>:24)                                                                                                                                           
        at <init>(<console>:26)                                                                                                                                                
        at .<init>(<console>:30)                                                                                                                                               
        at .<clinit>(<console>)                                                                                                                                                
        at .<init>(<console>:7)                                                                                                                                                
        at .<clinit>(<console>)                                                                                                                                                
        at $print(<console>)                                                                                                                                                   
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)                                                                                       
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                               
        at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                    
        at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)                                                                                          
        at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)                                                                                          
        at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)                                                                                              
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)                                                                                                    
        at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)                                                                                                    
        at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)                                                                                            
        at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)                                                                                        
        at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)                                                                                                      
        at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:132)                                                                     
        at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:124)                                                                     
        at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)                                                                                                
        at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:124)                                                                                
        at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)                                                                                               
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)                    
        at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:159)                                                                                      
        at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)                                                                                                     
        at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:108)                                                                             
        at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)                                                                                            
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)                                            
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)                                                   
        at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)                                                   
        at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)                                                                              
        at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)                                                                    
        at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)                                                                                                     
        at org.apache.spark.repl.Main$.main(Main.scala:31)                                                                                                                     
        at org.apache.spark.repl.Main.main(Main.scala)                                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                         
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)                                                                                       
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                               
        at java.lang.reflect.Method.invoke(Method.java:606)                                                                                                                    
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)                                                            
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)                                                                                             
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)                                                                                                  
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)                                                                                                    
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)                                                                                                         
                                                                                                                                                                               
<console>:16: error: not found: value sqlContext                                                                                                                               
         import sqlContext.implicits._                                                                                                                                         
                ^                                                                                                                                                              
<console>:16: error: not found: value sqlContext                                                                                                                               
         import sqlContext.sql                                                                                                                                                 
                ^                                                                                                                                                              
2 REPLIES 2

Re: Error initialising spark context

New Contributor

same error. meet this during busy hours of my cluster

Re: Error initialising spark context

@Ned Qian

try

 [root@sandbox ~]# cd /usr/hdp/current/spark-client

[root@sandbox spark-client]# spark-shell