- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
sqlContext not started after spark-shell command issued at console of CDH5.5
- Labels:
-
Apache Spark
Created on ‎08-04-2016 01:53 PM - edited ‎09-16-2022 03:32 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi All,
My CDH 5.5 was running all fine but now when i typed spark-shell command ,i see below issue of sqlcontext,can anyone suggest what should i do to remove this issue,spark-shell was running perfectly till this issue.
Please note i restarted Cloudera manager from admin console
16/08/04 13:38:51 ERROR Utils: Uncaught exception in thread main
java.lang.NullPointerException
at org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152)
16/08/04 13:38:51 INFO SparkContext: Successfully stopped SparkContext
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory /user/cloudera/.sparkStaging/application_1470339377450_0002. Name node is in safe mode.
The reported blocks 919 needs additional 2 blocks to reach the threshold 0.9990 of total blocks 921.
The number of live datanodes 1 has reached the minimum number 0. Safe mode will be turned off automatically once the thresholds have been reached.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1416)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNames
<console>:10: error: not found: value sqlContext
import sqlContext.implicits._
^
<console>:10: error: not found: value sqlContext
import sqlContext.sql
Created ‎08-04-2016 07:42 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created ‎08-04-2016 07:42 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
got solved by below
sudo -u hdfs hdfs dfsadmin -safemode leave
Created ‎08-16-2016 07:58 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i am using cloudera vm 5.7 when i type spark-shell i am getting following error.
<console>:16: error: not found: value sqlContext
import sqlContext.implicits._
^
<console>:16: error: not found: value sqlContext
import sqlContext.sql
Created ‎08-16-2016 08:09 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
5.5 or 5.7? the title and text disagree. 5.5 would have Spark 1.4, and I am not sure whether SQLContext was exposed as sqlContext by the shell by default like that. It should be in Spark 1.6 (= CDH 5.6+)
