Reply
New Contributor
Posts: 1
Registered: ‎02-03-2019

running spark function from java on cdh 6

hi 

i'm trying to run a simple mllib function (fpgrowth) from java from a remote computer on CDH 6 community version.

as default i tries to connect like this :

SparkConf conf = new SparkConf().setAppName("FPGrowth").setMaster("spark://some ip:7077").set("spark.cores.max", "10");

but the connection fails and i also checked netstat -plnt and there is no program listening on 7077 port.

is there a new way to connect a sparkcontext on cdh 6? i guess it is now integrated on yarn but how am i supposed to connect to t and make a sparkcontext ?

thanks