Member since
04-06-2020
5
Posts
0
Kudos Received
0
Solutions
04-07-2020
10:53 AM
Hi Team, I have upgraded to spark 2.2.1 but spark.sql.codegen.wholeStage=false doesn't give any improvement in performance
... View more
04-06-2020
08:34 PM
Hi All, I have solved the issue after upgrading to hadoop client 2.7.2 and excluded transitive dependancy hadoop client 2.6.5 coming from spark jar spark-core_2.11
... View more
04-06-2020
02:40 AM
I get the same error even after removing guava dependancy used in our project. I'm not sure how guava stopwatch is being started twice.. I still get the same error java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
... View more
04-06-2020
02:14 AM
Hi Team,
I'm upgrading spark 2.1.1 to 2.2.1 and we had hadoop of 2.2.0. After changing spark 2.1.1 , I had to change hadoop version 2.7.2. Then it started throwing the following execption.
java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
In our pom.xml we already have a version of guava.
27.0.1-jre
So I have added guava to exclusion list with no luck. I still get the same issue while running Junit (com.intellij.rt.junit.JUnitStarter.main)
<dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>${hadoop.version}</version> <exclusions> <exclusion> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> </exclusion>
Any help would be very much appreciated
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Spark