About ambariCloud

ambariCloud · ‎09-14-2016

Tested below in AWS. Looks good. Thank you //read error JSON file from cluster 1 val erorDF = spark.read.json("hdfs://master:8020/user/ubuntu/error.json") erorDF.registerTempTable("erorDFTBL") //read file from cluster 2 val erorDF2 = spark.read.json("hdfs://master2:8020/user/ubuntu/errors") erorDF2.registerTempTable("erorDFTBL2")

ambariCloud · ‎09-14-2016

Thank you Becker. Will there be any setup I need to do in Zeppelin. I am running my Zeppelin in cluster 1.

ambariCloud · ‎09-13-2016

Hi I want to source data from two hadoop clusters and join in Spark. Will it possible as shown below //data from cluster1 val erorDF = spark.read.json("hdfs://master:8020//user/ubuntu/error.json") erorDF.registerTempTable("erorDFTBL") //data from cluster2 val erorDF2 = spark.read.json("hdfs://master2:8020//user/ubuntu/error.json") erorDF2.registerTempTable("erorDFTBL2")

ambariCloud · ‎07-14-2016

Hi Spark 2.0 now had structured streaming. How it is different from NIFI file streaming.

Online	Offline
Last Visited	‎03-01-2017 10:35 AM

Member Since	‎01-19-2017 10:19 AM
Last Visited	‎03-01-2017 10:35 AM
Posts	5
Kudos received	1

Cloudera Community

Re: Spark connecting two hadoop clusters

Re: Spark connecting two hadoop clusters

Spark connecting two hadoop clusters

Spark Structured Streaming vs NIFI