Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Py4JJavaError: An error occurred while calling o767.fit

Highlighted

Py4JJavaError: An error occurred while calling o767.fit

New Contributor
 

I tried to fit a random forest classifier in pyspark but i'm getting this error:

Py4JJavaError: An error occurred while calling o767.fit. : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 30.0 failed 1 times, most recent failure: Lost task 0.0 in stage 30.0 (TID 853, localhost, executor driver): java.lang.OutOfMemoryError: Java heap space

Can anyone help me please

My code :

from pyspark.ml.tuning import ParamGridBuilderrf = RandomForestClassifier(labelCol="label", featuresCol="features")
paramGrid = (ParamGridBuilder() .addGrid(rf.numTrees, [100]) .build())
crossval = CrossValidator(estimator=rf, estimatorParamMaps=paramGrid,
evaluator
=BinaryClassificationEvaluator(),
numFolds
=10)
cvModel = crossval.fit(trainingData)
predictions = crossval.transform(testData)
predictions.printSchema()

 

1 REPLY 1
Highlighted

Re: Py4JJavaError: An error occurred while calling o767.fit

Cloudera Employee

Hi,

 

From the above Error "java.lang.OutOfMemoryError: Java heap space" this seems to be memory issue.Could you please increase the memory settings in your job and rerun the job again?

 

Thanks

AKR

Don't have an account?
Coming from Hortonworks? Activate your account here