Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

running spark function from java on cdh 6

running spark function from java on cdh 6

New Contributor


i'm trying to run a simple mllib function (fpgrowth) from java from a remote computer on CDH 6 community version.

as default i tries to connect like this :

SparkConf conf = new SparkConf().setAppName("FPGrowth").setMaster("spark://some ip:7077").set("spark.cores.max", "10");

but the connection fails and i also checked netstat -plnt and there is no program listening on 7077 port.

is there a new way to connect a sparkcontext on cdh 6? i guess it is now integrated on yarn but how am i supposed to connect to t and make a sparkcontext ?