Member since
10-10-2018
15
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1220 | 10-29-2018 08:25 AM |
11-23-2018
01:22 PM
As I see, if I try the below query, superset returns the results successfully SELECT
`added`,
`channel`,
`cityname`,
`comment`,
`countryisocode`,
`countryname`,
`deleted`,
`delta`,
`isanonymous`,
`isminor`,
`isnew`,
`isrobot`,
`isunpatrolled`,
`metrocode`,
`namespace`,
`page`,
`regionisocode`,
`regionname`,
`user`
FROM default.druid_table_cstefan_test
LIMIT 100 So I guess that the issue is due to the column __time , this column is auto-created from druid when I create the "druid datasource". __time data_type is "timestamp with local time zone" Is there any workaround?
... View more
11-22-2018
11:14 AM
Hello HC, I have successfully created an external hive table from a druid datasource. CREATE EXTERNAL TABLE druid_table_cstefan_test
STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
TBLPROPERTIES ("druid.datasource" = "wikipedia");
I am able to run queries from the hive-interactive shell. However, if I run queries ( or preview table ) from superset SQL lab it returns fail 22 ( see attached photo ). Does anyone know anything about this error? And if yes, how can I fix this? Unfortunately, I can't find something relative in the web. Thanks in advance, every help is appreciated hive-interactive = superset database name (backend: hive) druid_table_cstefan_test = table name
... View more
Labels:
- Labels:
-
Apache Hive
-
Druid
10-31-2018
07:43 AM
Hello community, I am using hdp3.0.1 and I can not create a hive table from a df in Zeppelin. I have seen the below articles: https://community.hortonworks.com/content/kbentry/223626/integrating-apache-hive-with-apache-spark-hive-war.html https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/integrating-hive/content/hive_hivewarehousesession_api_operations.html I am able to execute the most of HiveWarehouseSession API operations (like show/create/drop table),however, I am unable to create a hive table from an existing df I am getting the following error after running this command: df.write.format(HIVE_WAREHOUSE_CONNECTOR).option("table", "new_hive_table").save()
java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:347)
at scala.None$.get(Option.scala:345)
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector.createWriter(HiveWarehouseConnector.java:51)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:254)
... 61 elided<br> If you require any other information, please let me know. Thanks in advance
... View more
Labels:
10-29-2018
05:12 PM
@Shantanu
Sharma
I was not aware of that, thank you very much for sharing this
... View more
10-29-2018
02:45 PM
Hello community, I am facing the same problem as @Shantanu Sharma, I am able to access Hive database from pyspark and spark-shell, but I am also getting the same error with spark-sql Is there any update on this issue? Thanks in advance
... View more
10-29-2018
08:25 AM
After editing JDBC interpreter and adding hive password solved the issue.
... View more
10-26-2018
03:02 PM
Hello community, I am using HDP 3.0.1 and Zeppelin 0.8.0 I would like to create hive tables, however, I get the following error: org.apache.spark.sql.AnalysisException: Hive support is required to CREATE Hive TABLE (AS SELECT); I found that I have to add spark.sql.catalogImplementation=hive If I execute spark.sessionState.catalog from Zeppelin, I get: org.apache.spark.sql.catalyst.catalog.SessionCatalog = org.apache.spark.sql.catalyst.catalog.SessionCatalog@4c3b5588 However, If I run spark from cli using the following command: spark-shell --master yarn --conf spark.sql.catalogImplementation=hive and then type spark.sessionState.catalog I get the following result org.apache.spark.sql.catalyst.catalog.SessionCatalog = org.apache.spark.sql.hive.HiveSessionCatalog@5a384aef Running the same code, I can create a hive table from cli, but I can't create it from Zeppelin I would like to create hive tables from Zeppelin. What configuration do I miss? Thanks in advance.
... View more
10-18-2018
05:53 PM
@Aditya Sirna Thank you for your reply. Yes I mean ambari users. Unfortunately this is not working.
... View more
10-18-2018
04:32 PM
Hello community, I would like to know if it is possible to copy Ambari 2.6 users to Ambari 2.7. If yes, how can I do this? Thanks in advance
... View more
Labels:
10-18-2018
10:29 AM
@Aditya Sirna Thank you so much for your help, this solved my problem.
... View more
10-18-2018
09:08 AM
Hello community, I am using HDP 3.0 and I have configured HDFS DataNode Directories as follow: /grid0/hadoop/hdfs/data,/grid1/hadoop/hdfs/data,/grid/hadoop/hdfs/data/ One of my hosts (lets call him hostA) has only one mount point (/grid) and the other host (hostB) has 2 mount points (/grid0 & /grid1) HDFS automatically creates the directories grid0 and grid1 in "hostA" and grid in "hostB". Is there any way to prevent HDFS from doing that? Thanks in advance
... View more
Labels:
- Labels:
-
Apache Hadoop
10-16-2018
08:23 AM
@Jay Kumar SenSharma Thank you so much for your assistance. I update the cluster_handle and then restart the server. This worked and now the problem is solved.
... View more
10-16-2018
07:33 AM
@Jay Kumar SenSharma Thank you for your answer.
Creating a new "File View Instance" solved the problem. However, I was wondering, if I can do something to associate "File View" with the correct cluster ID I also tried to remove the following directory on ambari server and then restart ambari # rm -rf /var/lib/ambari-server/resources/views/work/FILES\{1.0.0\}/
# ambari-server restart but it was ineffective.
... View more
10-15-2018
12:41 PM
Hello Community, I am using hdp 3.0 / Ambari 2.7. HDFS is up and running and if I run a service check, it completes successfully. However, if I click the "Files view" I can't enter and I get the following error: Service 'hdfs' check failed:
java.lang.NullPointerException
at org.apache.ambari.server.view.ClusterImpl.getConfigByType(ClusterImpl.java:72)
at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.copyPropertiesBySite(ConfigurationBuilder.java:217)
at org.apache.ambari.view.utils.hdfs.ConfigurationBuilder.buildConfig(ConfigurationBuilder.java:325)
at org.apache.ambari.view.utils.hdfs.HdfsApi.<init>(HdfsApi.java:75)
at org.apache.ambari.view.utils.hdfs.HdfsUtil.getHdfsApi(HdfsUtil.java:157)
at org.apache.ambari.view.utils.hdfs.HdfsUtil.connectToHDFSApi(HdfsUtil.java:145)
at org.apache.ambari.view.commons.hdfs.HdfsService.hdfsSmokeTest(HdfsService.java:145)
at org.apache.ambari.view.filebrowser.HelpService.hdfsStatus(HelpService.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:291)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:135)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:94)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:73)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:53)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.ambari.server.view.AmbariViewsMDCLoggingFilter.doFilter(AmbariViewsMDCLoggingFilter.java:56)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.ambari.server.view.ViewThrottleFilter.doFilter(ViewThrottleFilter.java:161)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:51)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:724)
at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:221)
at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:210)
at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:151)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:745) I am able to enter hdfs from CLI and execute commands, like: su - hdfs
hadoop fs -du -h -s / Any idea on how to fix this? Thanks in advance.
... View more
Labels: