Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark history server API does not work

Spark history server API does not work

Explorer

In CDH 5.5, spark history server throws an exception when I call GET on http://server.com:18088/api/v1/application_12345/logs:

 

The exception is:

org.spark-project.jetty.servlet.ServletHolder$1: java.lang.reflect.InvocationTargetException
	at org.spark-project.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:496)
	at org.spark-project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:543)
	at org.spark-project.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:415)
	at org.spark-project.jetty.servlet.ServletHolder.handle(ServletHolder.java:657)
	at org.spark-project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
	at org.spark-project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.spark-project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.spark-project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.spark-project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.spark-project.jetty.server.handler.GzipHandler.handle(GzipHandler.java:264)
	at org.spark-project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
	at org.spark-project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.spark-project.jetty.server.Server.handle(Server.java:370)
	at org.spark-project.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.spark-project.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
	at org.spark-project.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
	at org.spark-project.jetty.http.HttpParser.parseNext(HttpParser.java:644)
	at org.spark-project.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
	at org.spark-project.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.spark-project.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
	at org.spark-project.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at org.spark-project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
	at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:728)
	at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:678)
	at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:203)
	at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:373)
	at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:556)
	at javax.servlet.GenericServlet.init(GenericServlet.java:244)
	at org.spark-project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:532)
	... 22 more
Caused by: java.lang.NoSuchMethodError: com.sun.jersey.core.reflection.ReflectionHelper.getOsgiRegistryInstance()Lcom/sun/jersey/core/osgi/OsgiRegistry;
	at com.sun.jersey.spi.scanning.AnnotationScannerListener$AnnotatedClassVisitor.getClassForName(AnnotationScannerListener.java:217)
	at com.sun.jersey.spi.scanning.AnnotationScannerListener$AnnotatedClassVisitor.visitEnd(AnnotationScannerListener.java:186)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at com.sun.jersey.spi.scanning.AnnotationScannerListener.onProcess(AnnotationScannerListener.java:136)
	at com.sun.jersey.core.spi.scanning.JarFileScanner.scan(JarFileScanner.java:97)
	at com.sun.jersey.core.spi.scanning.uri.JarZipSchemeScanner$1.f(JarZipSchemeScanner.java:78)
	at com.sun.jersey.core.util.Closing.f(Closing.java:71)
	at com.sun.jersey.core.spi.scanning.uri.JarZipSchemeScanner.scan(JarZipSchemeScanner.java:75)
	at com.sun.jersey.core.spi.scanning.PackageNamesScanner.scan(PackageNamesScanner.java:223)
	at com.sun.jersey.core.spi.scanning.PackageNamesScanner.scan(PackageNamesScanner.java:139)
	at com.sun.jersey.api.core.ScanningResourceConfig.init(ScanningResourceConfig.java:80)
	at com.sun.jersey.api.core.PackagesResourceConfig.init(PackagesResourceConfig.java:104)
	at com.sun.jersey.api.core.PackagesResourceConfig.<init>(PackagesResourceConfig.java:78)
	at com.sun.jersey.api.core.PackagesResourceConfig.<init>(PackagesResourceConfig.java:89)
	... 33 more

 

1 REPLY 1
Highlighted

Re: Spark history server API does not work

Contributor

Hi Martins,

 

Thanks for bringing this to our attention.  Is the cluster that you're currently running CDH 5.5 also running CM 5.5.0 or CM 5.5.1 with a Parcels-based installation?  If so, there's a CM bug that's present in those releases that prevents the API portion of Spark from properly loading due to classpath assembly issues.  This doesn't affect the operation of Spark other than the access and implementation through the Spark API.  Unfortunately, there's no available workaround at the moment for CM 5.5.0 or CM 5.5.1, although this has been fixed in CM 5.5.2 which should be released soon (it's the next upcoming release).

 

 

This should not occur on a package-based install, however please let us know if that is the case, further investigation may be needed.