Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎01-17-2018

Running Spark applications on oozie

We added an oozie service to our cloudera cluster.

I am running spark applicaitons using oozie and had already several times exceptions that happen randomly that are caused because of noit matched jars. The exception come from the spark code and not my code so I can't change dependency.

 

The most wierd thing is that it doesn't happen all the time but randomly.

Is there some way I can define the rigth dependency ?

 

The examples of the exceptions are:

 

Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.collect.MapMaker.makeComputingMap(Lcom/google/common/base/Function;)Ljava/util/concurrent/ConcurrentMap; from class com.jolbox.bonecp.BoneCPDataSource
	at com.jolbox.bonecp.BoneCPDataSource.<init>(BoneCPDataSource.java:64)
	at org.datanucleus.store.rdbms.datasource.BoneCPDataSourceFactory.makePooledDataSource(BoneCPDataSourceFactory.java:73)
	at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:217)
	at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:110)
	at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)


Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
	at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:445)
	at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:366)
	at org.apache.spark.metrics.sink.MetricsServlet.<init>(MetricsServlet.scala:49)

 

Announcements