@Sami Ahmad, in terms of supported configurations, Spark versions 1.6.2 and 2.0 (in technical preview) can be installed together on HDP 2.5.0. See the following for more info: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_spark-component-guide/content/install-spark-over-ambari.html. ... View more

this build.sbt fix the issue and now it compiles the package fine [root@hadoop1 TwitterPopularTags]# more build.sbt name := "TwitterPopularTags" version := "1.0" scalaVersion := "2.11.8" val sparkVersion = "1.6.1" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % sparkVersion, "org.apache.spark" %% "spark-streaming" % sparkVersion, "org.apache.spark" %% "spark-streaming-twitter" % sparkVersion ) resolvers += "Akka Repository" at "http://repo.akka.io/releases/" [root@hadoop1 TwitterPopularTags]# ... View more

@Sami Ahmad I would put your Flume related questions in Data Ingestion and Streaming. ... View more

This will give you commands to control your services: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_Reference_Guide/content/ch_controlling_hdp_svcs_manually.html It should work for whatever recent version of HDP you are running on. ... View more

Gouri you were right it was the privileges issue on Linux /tmp/hive folder , I was changing the permission of the hdfs /tmp/hive folder . I can access beeline now and can connect to the hive store , I have other issues though for which I will open a new post. thanks for your help ... View more