Member since
06-27-2019
1
Post
1
Kudos Received
0
Solutions
11-19-2019
05:31 AM
1 Kudo
Hello. It's possible to fix it i think. I did it but need to test it deeply. You need to find your pyspark installation, for your case: /opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/spark/python/pyspark/context.py And comment the following lines (arround lines 115). Depending the version 2.3.3, 2.4.0, 2.4.x they can look a little bit different as they changed the code several times there, but the idea is to comment the raise ValueError part: #if gateway is not None and gateway.gateway_parameters.auth_token is None:
# raise ValueError(
# "You are trying to pass an insecure Py4j gateway to Spark. This"
# " is not allowed as it is a security risk.") In some other versions it may be necessary also to comment something similar in the java_gateway.py file. Regards, Albert
... View more