Reply
Expert Contributor
Posts: 162
Registered: ‎07-29-2013

CM[4.6.3] -> Activities, can't get list of oozie activities

Hi. I do click on 'activites tab"

The Iselect type='oozie' and click "Search"

Then error appears:

A server error has occurred. Send the following information to Cloudera.

Path: http://localhost:7180/cmf/services/4/monitor/contextTreeTable

Version: Cloudera Standard 4.6.3 (#192 built by jenkins on 20130812-1221 git: fa61cf8559fbefeb5af7f223fd02164d1a0adfdb)

org.apache.avro.AvroRuntimeException:org.apache.avro.AvroRuntimeException: Unknown datum type: java.lang.NullPointerException
at SpecificRequestor.java line 126
in org.apache.avro.ipc.specific.SpecificRequestor readError()
Stack Trace:
  1. SpecificRequestor.java line 126
    in org.apache.avro.ipc.specific.SpecificRequestor readError()
  2. Requestor.java line 554
    in org.apache.avro.ipc.Requestor$Response getResponse()
  3. Requestor.java line 359
    in org.apache.avro.ipc.Requestor$TransceiverCallback handleResult()
  4. Requestor.java line 322
    in org.apache.avro.ipc.Requestor$TransceiverCallback handleResult()
  5. Transceiver.java line 73
    in org.apache.avro.ipc.Transceiver transceive()
  6. Requestor.java line 147
    in org.apache.avro.ipc.Requestor request()
  7. Requestor.java line 101
    in org.apache.avro.ipc.Requestor request()
  8. SpecificRequestor.java line 72
    in org.apache.avro.ipc.specific.SpecificRequestor invoke()
  9. line -1
    in $Proxy57 getActivities()
  10. TimeoutNozzleIPC.java line 248
    in com.cloudera.cmon.TimeoutNozzleIPC getActivities()
  11. NozzleIPCWrapper.java line 384
    in com.cloudera.cmon.NozzleIPCWrapper getActivities()
  12. NozzleIPCWrapper.java line 363
    in com.cloudera.cmon.NozzleIPCWrapper getActivities()
  13. ContextTableModel.java line 372
    in com.cloudera.server.web.cmon.ContextTableModel <init>()
  14. ContextTableController.java line 116
    in com.cloudera.server.web.cmon.ContextTableController contextTable()
  15. <generated> line -1
    in com.cloudera.server.web.cmon.ContextTableController$$FastClassByCGLIB$$492bbd5d invoke()
  16. MethodProxy.java line 191
    in net.sf.cglib.proxy.MethodProxy invoke()
  17. Cglib2AopProxy.java line 617
    in org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor intercept()
  18. <generated> line -1
    in com.cloudera.server.web.cmon.ContextTableController$$EnhancerByCGLIB$$f47fd155 contextTable()
  19. line -1
    in sun.reflect.GeneratedMethodAccessor906 invoke()
  20. DelegatingMethodAccessorImpl.java line 25
    in sun.reflect.DelegatingMethodAccessorImpl invoke()
  21. Method.java line 597
    in java.lang.reflect.Method invoke()
  22. HandlerMethodInvoker.java line 176
    in org.springframework.web.bind.annotation.support.HandlerMethodInvoker invokeHandlerMethod()
  23. AnnotationMethodHandlerAdapter.java line 436
    in org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter invokeHandlerMethod()
  24. AnnotationMethodHandlerAdapter.java line 424
    in org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter handle()
  25. DispatcherServlet.java line 790
    in org.springframework.web.servlet.DispatcherServlet doDispatch()
  26. DispatcherServlet.java line 719
    in org.springframework.web.servlet.DispatcherServlet doService()
  27. FrameworkServlet.java line 669
    in org.springframework.web.servlet.FrameworkServlet processRequest()
  28. FrameworkServlet.java line 585
    in org.springframework.web.servlet.FrameworkServlet doPost()
  29. HttpServlet.java line 727
    in javax.servlet.http.HttpServlet service()
  30. HttpServlet.java line 820
    in javax.servlet.http.HttpServlet service()
  31. ServletHolder.java line 511
    in org.mortbay.jetty.servlet.ServletHolder handle()
  32. ServletHandler.java line 1221
    in org.mortbay.jetty.servlet.ServletHandler$CachedChain doFilter()
  33. UserAgentFilter.java line 78
    in org.mortbay.servlet.UserAgentFilter doFilter()
  34. GzipFilter.java line 131
    in org.mortbay.servlet.GzipFilter doFilter()
  35. ServletHandler.java line 1212
    in org.mortbay.jetty.servlet.ServletHandler$CachedChain doFilter()
  36. JAMonServletFilter.java line 48
    in com.jamonapi.http.JAMonServletFilter doFilter()
  37. ServletHandler.java line 1212
    in org.mortbay.jetty.servlet.ServletHandler$CachedChain doFilter()
  38. JavaMelodyFacade.java line 109
    in com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter doFilter()
  39. ServletHandler.java line 1212
    in org.mortbay.jetty.servlet.ServletHandler$CachedChain doFilter()
  40. FilterChainProxy.java line 311
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  41. FilterSecurityInterceptor.java line 116
    in org.springframework.security.web.access.intercept.FilterSecurityInterceptor invoke()
  42. FilterSecurityInterceptor.java line 83
    in org.springframework.security.web.access.intercept.FilterSecurityInterceptor doFilter()
  43. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  44. ExceptionTranslationFilter.java line 113
    in org.springframework.security.web.access.ExceptionTranslationFilter doFilter()
  45. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  46. SessionManagementFilter.java line 101
    in org.springframework.security.web.session.SessionManagementFilter doFilter()
  47. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  48. AnonymousAuthenticationFilter.java line 113
    in org.springframework.security.web.authentication.AnonymousAuthenticationFilter doFilter()
  49. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  50. RememberMeAuthenticationFilter.java line 146
    in org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter doFilter()
  51. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  52. SecurityContextHolderAwareRequestFilter.java line 54
    in org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter doFilter()
  53. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  54. RequestCacheAwareFilter.java line 45
    in org.springframework.security.web.savedrequest.RequestCacheAwareFilter doFilter()
  55. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  56. AbstractAuthenticationProcessingFilter.java line 182
    in org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter doFilter()
  57. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  58. LogoutFilter.java line 105
    in org.springframework.security.web.authentication.logout.LogoutFilter doFilter()
  59. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  60. SecurityContextPersistenceFilter.java line 87
    in org.springframework.security.web.context.SecurityContextPersistenceFilter doFilter()
  61. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  62. ConcurrentSessionFilter.java line 125
    in org.springframework.security.web.session.ConcurrentSessionFilter doFilter()
  63. FilterChainProxy.java line 323
    in org.springframework.security.web.FilterChainProxy$VirtualFilterChain doFilter()
  64. FilterChainProxy.java line 173
    in org.springframework.security.web.FilterChainProxy doFilter()
  65. DelegatingFilterProxy.java line 237
    in org.springframework.web.filter.DelegatingFilterProxy invokeDelegate()
  66. DelegatingFilterProxy.java line 167
    in org.springframework.web.filter.DelegatingFilterProxy doFilter()
  67. ServletHandler.java line 1212
    in org.mortbay.jetty.servlet.ServletHandler$CachedChain doFilter()
  68. CharacterEncodingFilter.java line 88
    in org.springframework.web.filter.CharacterEncodingFilter doFilterInternal()
  69. OncePerRequestFilter.java line 76
    in org.springframework.web.filter.OncePerRequestFilter doFilter()
  70. ServletHandler.java line 1212
    in org.mortbay.jetty.servlet.ServletHandler$CachedChain doFilter()
  71. ServletHandler.java line 399
    in org.mortbay.jetty.servlet.ServletHandler handle()
  72. SecurityHandler.java line 216
    in org.mortbay.jetty.security.SecurityHandler handle()
  73. SessionHandler.java line 182
    in org.mortbay.jetty.servlet.SessionHandler handle()
  74. ContextHandler.java line 766
    in org.mortbay.jetty.handler.ContextHandler handle()
  75. WebAppContext.java line 450
    in org.mortbay.jetty.webapp.WebAppContext handle()
  76. HandlerWrapper.java line 152
    in org.mortbay.jetty.handler.HandlerWrapper handle()
  77. StatisticsHandler.java line 53
    in org.mortbay.jetty.handler.StatisticsHandler handle()
  78. HandlerWrapper.java line 152
    in org.mortbay.jetty.handler.HandlerWrapper handle()
  79. Server.java line 326
    in org.mortbay.jetty.Server handle()
  80. HttpConnection.java line 542
    in org.mortbay.jetty.HttpConnection handleRequest()
  81. HttpConnection.java line 928
    in org.mortbay.jetty.HttpConnection$RequestHandler headerComplete()
  82. HttpParser.java line 549
    in org.mortbay.jetty.HttpParser parseNext()
  83. HttpParser.java line 212
    in org.mortbay.jetty.HttpParser parseAvailable()
  84. HttpConnection.java line 404
    in org.mortbay.jetty.HttpConnection handle()
  85. SelectChannelEndPoint.java line 410
    in org.mortbay.io.nio.SelectChannelEndPoint run()
  86. QueuedThreadPool.java line 582
    in org.mortbay.thread.QueuedThreadPool$PoolThread run()
Cloudera Employee
Posts: 35
Registered: ‎08-08-2013

Re: CM[4.6.3] -> Activities, can't get list of oozie activities

Can you trigger this again and see if there's any error in the ActivityMonitor log? You can see the log by CM -> Mgmt service -> ActivityMonitor -> Log File (top left summary region).
Cloudera Employee
Posts: 4
Registered: ‎10-31-2013

Re: CM[4.6.3] -> Activities, can't get list of oozie activities

Hi Sergey

 

I believe this is a known issue that was fixed in cm 4.7.3 that is about to release very soon. Can you look in the activity monitor log and check if you see something similar to the following:

 

java.lang.NullPointerException
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
    at com.cloudera.cmon.tree.db.TreeNozzleDao.stringMetricValue(TreeNozzleDao.java:678)
    at com.cloudera.cmon.tree.db.TreeNozzleDao.toAvroMetrics(TreeNozzleDao.java:605)
    at com.cloudera.cmon.tree.db.TreeNozzleDao.toAvro(TreeNozzleDao.java:538)
    at com.cloudera.cmon.tree.db.TreeNozzleDao.queryActivitiesWithPartitions(TreeNozzleDao.java:1219)
    at com.cloudera.cmon.tree.db.TreeNozzleDao.queryActivities(TreeNozzleDao.java:1057)
    at com.cloudera.cmon.tree.TreeNozzleImpl.getActivities(TreeNozzleImpl.java:96)
    at com.cloudera.cmon.tree.TreeNozzleImpl.getActivities(TreeNozzleImpl.java:75)
    at sun.reflect.GeneratedMethodAccessor902.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
    at org.apache.avro.ipc.Responder.respond(Responder.java:149)
    at org.apache.avro.ipc.Responder.respond(Responder.java:99)
    at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)

 

Thanks,

G.

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: CM[4.6.3] -> Activities, can't get list of oozie activities

Sorry, I've missed the reply.

I've checked activity server log I do see this:

11:06:13.580ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing createdTime 'Mon, 18 Nov 2013 01:05:05 GMT' from Oozie
11:06:13.580ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing lastModTime 'Mon, 18 Nov 2013 01:09:59 GMT' from Oozie
11:06:13.580ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing endTime 'Mon, 18 Nov 2013 01:09:59 GMT' from Oozie
11:06:13.580ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing startTime 'Mon, 18 Nov 2013 01:05:05 GMT' from Oozie
11:06:13.580ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing createdTime 'Mon, 18 Nov 2013 01:02:07 GMT' from Oozie
11:06:13.580ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing lastModTime 'Mon, 18 Nov 2013 01:06:45 GMT' from Oozie
11:06:13.581ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing endTime 'Mon, 18 Nov 2013 01:06:45 GMT' from Oozie
11:06:13.581ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing startTime 'Mon, 18 Nov 2013 01:02:07 GMT' from Oozie
11:06:13.581ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing createdTime 'Mon, 18 Nov 2013 01:02:06 GMT' from Oozie
11:06:13.581ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing lastModTime 'Mon, 18 Nov 2013 01:03:19 GMT' from Oozie
11:06:13.581ERRORcom.cloudera.cmon.firehose.OozieWorkflow
Error parsing endTime 'Mon, 18 Nov 2013 01:03:19 GMT' from Oozie

And there are plenty of such ERRORs.

 

I did find exception you've mentioned. please see attached screenshot.

 

Выделение_162.png

Announcements