cstefan
cstefan
Explorer
My Accepted Solutions
Show All

Atlas limited support for kafka/storm

by Explorer cstefan in Support Questions
‎12-14-2018 11:28 AM
‎12-14-2018 11:28 AM
Hello all, I was reading the Hortonworks documentation about data governance and more precisely about atlas. As I saw, Atlas claims limited support for Kafka/storm. Does anyone know, what limited support stands for? I haven 't find anything on the web yet. Thanks in advance https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/atlas-overview/content/apache_atlas_features.html ... View more

Re: superset sql lab preview table returns state f...

by Explorer cstefan in Support Questions
‎11-23-2018 01:22 PM
‎11-23-2018 01:22 PM
As I see, if I try the below query, superset returns the results successfully SELECT `added`, `channel`, `cityname`, `comment`, `countryisocode`, `countryname`, `deleted`, `delta`, `isanonymous`, `isminor`, `isnew`, `isrobot`, `isunpatrolled`, `metrocode`, `namespace`, `page`, `regionisocode`, `regionname`, `user` FROM default.druid_table_cstefan_test LIMIT 100 So I guess that the issue is due to the column __time , this column is auto-created from druid when I create the "druid datasource". __time data_type is "timestamp with local time zone" Is there any workaround? ... View more

superset sql lab preview table returns state fail ...

by Explorer cstefan in Support Questions
‎11-22-2018 11:14 AM
‎11-22-2018 11:14 AM
Hello HC, I have successfully created an external hive table from a druid datasource. CREATE EXTERNAL TABLE druid_table_cstefan_test STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' TBLPROPERTIES ("druid.datasource" = "wikipedia"); I am able to run queries from the hive-interactive shell. However, if I run queries ( or preview table ) from superset SQL lab it returns fail 22 ( see attached photo ). Does anyone know anything about this error? And if yes, how can I fix this? Unfortunately, I can't find something relative in the web. Thanks in advance, every help is appreciated hive-interactive = superset database name (backend: hive) druid_table_cstefan_test = table name ... View more
Labels:

Unable to create hive table from df in Zeppelin

by Explorer cstefan in Support Questions
‎10-31-2018 07:43 AM
‎10-31-2018 07:43 AM
Hello community, I am using hdp3.0.1 and I can not create a hive table from a df in Zeppelin. I have seen the below articles: https://community.hortonworks.com/content/kbentry/223626/integrating-apache-hive-with-apache-spark-hive-war.html https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/integrating-hive/content/hive_hivewarehousesession_api_operations.html I am able to execute the most of HiveWarehouseSession API operations (like show/create/drop table),however, I am unable to create a hive table from an existing df I am getting the following error after running this command: df.write.format(HIVE_WAREHOUSE_CONNECTOR).option("table", "new_hive_table").save() java.util.NoSuchElementException: None.get at scala.None$.get(Option.scala:347) at scala.None$.get(Option.scala:345) at com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector.createWriter(HiveWarehouseConnector.java:51) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:254) ... 61 elided<br> If you require any other information, please let me know. Thanks in advance ... View more

Re: Zeppelin : Not able to connect Hive Databases ...

by Explorer cstefan in Support Questions
‎10-29-2018 05:12 PM
‎10-29-2018 05:12 PM
@Shantanu Sharma I was not aware of that, thank you very much for sharing this ... View more

Re: Zeppelin : Not able to connect Hive Databases ...

by Explorer cstefan in Support Questions
‎10-29-2018 02:45 PM
‎10-29-2018 02:45 PM
Hello community, I am facing the same problem as @Shantanu Sharma, I am able to access Hive database from pyspark and spark-shell, but I am also getting the same error with spark-sql Is there any update on this issue? Thanks in advance ... View more

Re: configure zeppelin so spark interpreter can cr...

by Explorer cstefan in Support Questions
‎10-29-2018 08:25 AM
‎10-29-2018 08:25 AM
After editing JDBC interpreter and adding hive password solved the issue. ... View more

configure zeppelin so spark interpreter can create...

by Explorer cstefan in Support Questions
‎10-26-2018 03:02 PM
‎10-26-2018 03:02 PM
Hello community, I am using HDP 3.0.1 and Zeppelin 0.8.0 I would like to create hive tables, however, I get the following error: org.apache.spark.sql.AnalysisException: Hive support is required to CREATE Hive TABLE (AS SELECT); I found that I have to add spark.sql.catalogImplementation=hive If I execute spark.sessionState.catalog from Zeppelin, I get: org.apache.spark.sql.catalyst.catalog.SessionCatalog = org.apache.spark.sql.catalyst.catalog.SessionCatalog@4c3b5588 However, If I run spark from cli using the following command: spark-shell --master yarn --conf spark.sql.catalogImplementation=hive and then type spark.sessionState.catalog I get the following result org.apache.spark.sql.catalyst.catalog.SessionCatalog = org.apache.spark.sql.hive.HiveSessionCatalog@5a384aef Running the same code, I can create a hive table from cli, but I can't create it from Zeppelin I would like to create hive tables from Zeppelin. What configuration do I miss? Thanks in advance. ... View more

Re: Copy users from Ambari 2.6 to Ambari 2.7

by Explorer cstefan in Support Questions
‎10-18-2018 05:53 PM
‎10-18-2018 05:53 PM
@Aditya Sirna Thank you for your reply. Yes I mean ambari users. Unfortunately this is not working. ... View more

Copy users from Ambari 2.6 to Ambari 2.7

by Explorer cstefan in Support Questions
‎10-18-2018 04:32 PM
‎10-18-2018 04:32 PM
Hello community, I would like to know if it is possible to copy Ambari 2.6 users to Ambari 2.7. If yes, how can I do this? Thanks in advance ... View more

Re: Prevent HDFS from creating data directories

by Explorer cstefan in Support Questions
‎10-18-2018 10:29 AM
‎10-18-2018 10:29 AM
@Aditya Sirna Thank you so much for your help, this solved my problem. ... View more

Prevent HDFS from creating data directories

by Explorer cstefan in Support Questions
‎10-18-2018 09:08 AM
‎10-18-2018 09:08 AM
Hello community, I am using HDP 3.0 and I have configured HDFS DataNode Directories as follow: /grid0/hadoop/hdfs/data,/grid1/hadoop/hdfs/data,/grid/hadoop/hdfs/data/ One of my hosts (lets call him hostA) has only one mount point (/grid) and the other host (hostB) has 2 mount points (/grid0 & /grid1) HDFS automatically creates the directories grid0 and grid1 in "hostA" and grid in "hostB". Is there any way to prevent HDFS from doing that? Thanks in advance ... View more
Labels:

Re: Service 'hdfs' check failed: java.lang.NullPoi...

by Explorer cstefan in Support Questions
‎10-16-2018 08:23 AM
‎10-16-2018 08:23 AM
@Jay Kumar SenSharma Thank you so much for your assistance. I update the cluster_handle and then restart the server. This worked and now the problem is solved. ... View more

Re: Service 'hdfs' check failed: java.lang.NullPoi...

by Explorer cstefan in Support Questions
‎10-16-2018 07:33 AM
‎10-16-2018 07:33 AM
@Jay Kumar SenSharma Thank you for your answer. Creating a new "File View Instance" solved the problem. However, I was wondering, if I can do something to associate "File View" with the correct cluster ID I also tried to remove the following directory on ambari server and then restart ambari # rm -rf /var/lib/ambari-server/resources/views/work/FILES\{1.0.0\}/ # ambari-server restart but it was ineffective. ... View more

Service 'hdfs' check failed: java.lang.NullPointer...

by Explorer cstefan in Support Questions
‎10-15-2018 12:41 PM
‎10-15-2018 12:41 PM
Hello Community, I am using hdp 3.0 / Ambari 2.7. HDFS is up and running and if I run a service check, it completes successfully. However, if I click the "Files view" I can't enter and I get the following error: Service 'hdfs' check failed: java.lang.NullPointerException at org.apache.ambari.server.view.ClusterImpl.getConfigByType(ClusterImpl.java:72) at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.copyPropertiesBySite(ConfigurationBuilder.java:217) at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.buildConfig(ConfigurationBuilder.java:325) at org.apache.ambari.view.utils.hdfs.HdfsApi.<init>(HdfsApi.java:75) at org.apache.ambari.view.utils.hdfs.HdfsUtil.getHdfsApi(HdfsUtil.java:157) at org.apache.ambari.view.utils.hdfs.HdfsUtil.connectToHDFSApi(HdfsUtil.java:145) at org.apache.ambari.view.commons.hdfs.HdfsService.hdfsSmokeTest(HdfsService.java:145) at org.apache.ambari.view.filebrowser.HelpService.hdfsStatus(HelpService.java:95) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:291) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:135) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:94) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:73) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:53) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.ambari.server.view.AmbariViewsMDCLoggingFilter.doFilter(AmbariViewsMDCLoggingFilter.java:56) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.ambari.server.view.ViewThrottleFilter.doFilter(ViewThrottleFilter.java:161) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:130) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:130) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:51) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:724) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:221) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:210) at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:151) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang.Thread.run(Thread.java:745) I am able to enter hdfs from CLI and execute commands, like: su - hdfs hadoop fs -du -h -s / Any idea on how to fix this? Thanks in advance. ... View more
Contact Me
Online
Offline
Last Visited
‎12-21-2018 10:15 AM
Public Statistics
Date Registered ‎10-10-2018 04:16 PM
Last Visited ‎12-21-2018 10:15 AM
Total Messages Posted 15