Created on 05-01-2019 01:33 PM - edited 09-16-2022 07:21 AM
After launching pyspark, I'm occasionally seeing an exception that's being print on my pyspark shell -
File "/usr/lib64/python2.7/SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib64/python2.7/SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__
self.handle()
File "/data0/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/lib/spark/python/pyspark/accumulators.py", line 266, in handle
poll(authenticate_and_accum_updates)
File "/data0/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/lib/spark/python/pyspark/accumulators.py", line 241, in poll
if func():
File "/data0/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/lib/spark/python/pyspark/accumulators.py", line 263, in authenticate_and_accum_updates
"The value of the provided token to the AccumulatorServer is not correct.")
Exception: The value of the provided token to the AccumulatorServer is not correct.
However, when i hit Enter key, it prompts for input and I can execute a command.
Not sure why it's printing the exception though? Any turnarounds for the issue?
Created 05-14-2019 10:21 AM
Hi,
We understand that after you launching pyspark getting this error. We need to look into the error more details to get the error resolved. We need to look the command line you're using to start pyspark, And also enabling Debug mode in the pyspark job will give you more information.
Created 08-24-2019 06:54 AM
Hi,
Please check in Logs whether any java.net.BindException: Address already in use is arrived, If it is seen
It clearly resembles that port is already in use, (i.e) something is trying to connect to the port and is blocking which we are not sure.
Possible causes Maybe
1. Some "port scanner" or something is blocking this port (or) Some other Service is running on the port like a port scanner.
2. The accumulator is still running on those ports from an earlier run
3. Several Services may be running on this port.
Please make sure what services (or) applications is running on this port and try to stop it.