Thanks for your reply. I found in the UI the path to /var/log/spark/spark-worker. That is the spark executor's own log. I am actually looking for the logs that I added to functions that I wrote to map RDDs. Please forgive me if I am using the wrong terminology here. ... View more

This is just Java's locking library, it's not specific to the project. This is a lock that supports many readers at one time, but, only one writer at a time (and no readers while a writer has the write lock). You have to acquire the write lock to mutate the shared state, but also need to acquire the read lock to read it -- but, you won't exclude other readers. ... View more

Have you set it to start a generation based on the amount of input received? that could be triggering the new computation.   That said are you sure it only has part of the input? it's possible the zipped file sizes aren't that comparable.   Yes, you simply don't have enough memory allocated to your JVM. Your system memory doesn't matter if you haven't let the JVM use much of it. This is in local mode right? you need to use -Xmx to give more heap.   Yes it will use different tmp directories for different jobs. That's normal. ... View more

ya. It looks my local issue to locate Guava. Thanks for your reply. ... View more

Thanks sowen for your cooperation ... View more