Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

Running Spark applications on oozie

avatar
New Contributor

We added an oozie service to our cloudera cluster.

I am running spark applicaitons using oozie and had already several times exceptions that happen randomly that are caused because of noit matched jars. The exception come from the spark code and not my code so I can't change dependency.

 

The most wierd thing is that it doesn't happen all the time but randomly.

Is there some way I can define the rigth dependency ?

 

The examples of the exceptions are:

 

Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.collect.MapMaker.makeComputingMap(Lcom/google/common/base/Function;)Ljava/util/concurrent/ConcurrentMap; from class com.jolbox.bonecp.BoneCPDataSource
	at com.jolbox.bonecp.BoneCPDataSource.<init>(BoneCPDataSource.java:64)
	at org.datanucleus.store.rdbms.datasource.BoneCPDataSourceFactory.makePooledDataSource(BoneCPDataSourceFactory.java:73)
	at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:217)
	at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:110)
	at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)


Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
	at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:445)
	at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:366)
	at org.apache.spark.metrics.sink.MetricsServlet.<init>(MetricsServlet.scala:49)

 

Who agreed with this topic