Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Exception in thread "main" java.lang.RuntimeException: core-site.xml not found

Highlighted

Exception in thread "main" java.lang.RuntimeException: core-site.xml not found

Explorer

Hi,

 

    I'm trying to run example scala project in CDSW. When I try to put data to hdfs I have and exception:

Exception in thread "main" java.lang.RuntimeException: core-site.xml not found

 

    I printed HADOOP_CONF_DIR in my scala code and it looks good:

 

println(sys.env("HADOOP_CONF_DIR"))
/etc/hadoop/conf:/etc/hive/conf

 

    Files exist in docker container.

 

   Spark also see configration files. Lines return proper output in my opinion :

 

import org.apache.hadoop.fs._
import org.apache.hadoop.conf._
import scala.collection.JavaConverters._

val fs = FileSystem.get(sc.hadoopConfiguration)
fs.getConf.iterator().asScala.foreach(println)

 

 

    I'm using CDSW 1.5.0 and CM 6.2.0.

 

    Any advices why spark does not see core-site.xml?

 

Regards,

 

Bart