Member since
02-16-2017
45
Posts
2
Kudos Received
0
Solutions
10-12-2017
09:10 PM
Insert overwite query on LLAP is not working but it runs through normal Hive. The same query works through hive but fails through LLAP throwing this error Vertex failed, vertexName=Map 1, vertexId=vertex_1507670392589_0073_1_00, diagnostics=» Vertex vertex_1507670392589_0073_1_00» Map 1
killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:70)
at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:151)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:148)
at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:121)
at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4031)
at org.apache.tez.dag.app.dag.impl.VertexImpl.access$3100(VertexImpl.java:204)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:2855)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2802)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2784)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:59)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1925)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:203)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2290)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2276)
at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:115)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
... 25 more
Caused by: java.lang.IllegalArgumentException: No running LLAP daemons! Please check LLAP service status and zookeeper configuration
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
at org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:47)
at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.(HiveSplitGenerator.java:121)
... 30 more
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1507670392589_0073_1_01, diagnostics=» Vertex received Kill in NEW state., Vertex vertex_1507670392589_0073_1_01» Reducer 2
killed/failed due to:OTHER_VERTEX_FAILURE
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
... View more
Labels:
- Labels:
-
Apache Hive
06-09-2017
03:11 PM
Does these configuration mentioned in this page work on TEZ engine .I could see SMB working only on MR
... View more
05-19-2017
05:35 PM
I am trying to run a query in hive which is like shown below select * from (with View as (select * from table) select * from view) t1 I am able to execute the subquery but unable to run the full query .Hive throws an error saying NoViableAltException(26@[146:1: fromSource : Does hive not allow to use select from a view created with 'with clause'
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
04-12-2017
04:32 PM
Is there a link showing how I can upgrade Ambari to 2.5
... View more
04-12-2017
03:06 PM
I am currently using HDP 2.5 on my cluster. HDP 2.6 is recently released but am not sure if I can do an upgrade from HDP 2.5 to 2.6 using Ambari. The most latest version I can find in Ambari is 2.5,Is Ambari yet to provide upgrade for 2.6 ?
... View more
Labels:
03-17-2017
06:48 PM
I do not see it showing as org.apache.slider for me. I have connected to the hive interactive server through beeline to run the queries. It shows the application type as TEZ for me. Do I need to set some parameter to get it running.
... View more
03-17-2017
02:39 PM
I fixed it. I am having 4 nodes and trying to run 4 daemons, But I set the daemon container size to its full memory. This does not give any space for application manager to run on any node. Or If application manager runs on a node, One of the daemon which is supposed to run on that node does not get its full container memory which it was set to.
... View more
03-16-2017
10:25 PM
I have recently upgraded HDP 2.5 and enabled Interactive Query for LLAP. I am trying to start the Hive2 Interactive query on my 4-node cluster and it fails.giving the below error. Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 512, in check_llap_app_status
status = do_retries()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/decorator.py", line 55, in wrapper
return function(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 501, in do_retries
llap_app_info['desiredInstances']))
Fail: App state is RUNNING_PARTIAL. Live Instances : '3', Desired Instance : '4'
2017-03-16 17:00:40,921 - LLAP app 'llap0' deployment unsuccessful.
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 535, in <module>
HiveServerInteractive().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py", line 123, in start
raise Fail("Skipping START of Hive Server Interactive since LLAP app couldn't be STARTED.")
resource_management.core.exceptions.Fail: Skipping START of Hive Server Interactive since LLAP app couldn't be STARTED.
~
... View more
Labels:
- Labels:
-
Apache Hive
02-16-2017
07:45 PM
Can you explain what does set hive.mv.thread=0 does? I assume it removes parallelism option,does that reduce performance of the insert query.
... View more
02-16-2017
07:42 PM
Does setting hive.mv.file.thread=0 reduce the performance of the insert query.Can you explain what does setting this configration has to do with HDP 2.5 upgrade? Is the move task parallelism not present in the previous version
... View more
- « Previous
-
- 1
- 2
- Next »