Thanks, @Sandeep Nemuri ... View more

Hi @Michael Mester, Can you please see if this commend helps you. login and accept this answer if it did. 🙂 ... View more

@Muhammad Taimoor Yousaf You'd need to have a valid ticket in in the machine you are launching pyspark shell (which is test-m1). Looking at the exception it is clearly saying 'No valid credentials provided'. Hope this helps. ... View more

@Sandeep Nemuri Thanks ... View more

Thank you. ... View more

@Ronnie 10 , Do consider accepting the answer if it helped you 🙂 ... View more

Thanks for the link! It helped to clear up most of my questions. ... View more

@rinu shrivastav The split size is calculated by the formula:- max(mapred.min.split.size, min(mapred.max.split.size, dfs.block.size)) Say, HDFS block size is 64 MB and min.input.size is set to 128MB, then there will be split size would be 128MB. To read 256MB of data, there will be two mappers. To increase the number of mappers, then you could decrease min.input.size till the HDFS block size. split size=max(128,min(256,64)) ... View more

@saichand akella That should be ok 🙂 you can accept this answer. ... View more