Member since
12-13-2016
33
Posts
3
Kudos Received
0
Solutions
12-18-2017
12:20 PM
i change that flow.xml.gz from an older version,i also remove that flow.xml.gz.but still my nifi wont start.
... View more
12-18-2017
12:09 PM
I have an set of urls in a file. sample file: localhost:1234/aaaa/uyt localhost:1234/bbbb/grt localhost:1234/cccc/hgd etc.. how can I pass this URL dynamically from that file to invoke HTTP processor.
... View more
Labels:
- Labels:
-
Apache NiFi
12-16-2017
07:00 AM
nifi-bootstrap.log
2017-12-16 06:29:38,004 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,065 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:38,162 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,175 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:38,482 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,500 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:38,779 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,801 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:39,925 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:39,943 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:40,576 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:40,588 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:44,728 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Unable to start Flow Controller.
2017-12-16 06:29:44,729 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2017-12-16 06:29:45,653 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
nifi-app.log
2017-12-16 06:29:44,836 INFO [Thread-1] o.a.n.w.c.ApplicationStartupContextListener Initiating shutdown of flow service...
2017-12-16 06:29:44,837 INFO [Thread-1] o.a.n.w.c.ApplicationStartupContextListener Flow service termination completed.
2017-12-16 06:29:44,837 INFO [Thread-1] /nifi-api Closing Spring root WebApplicationContext
2017-12-16 06:29:45,075 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@3ed2a30c{/nifi-api,null,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.1.1.war}
2017-12-16 06:29:45,230 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@526e8108{/nifi,null,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.1.1.war}
2017-12-16 06:29:45,243 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@413bef78{/nifi-jolt-transform-json-ui-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.1.1.war}
2017-12-16 06:29:45,247 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@4ed15347{/nifi-image-viewer-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-media-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.1.1.war}
2017-12-16 06:29:45,253 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@20ead579{/nifi-update-attribute-ui-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-update-attribute-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.1.1.war}
2017-12-16 06:29:45,257 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@33eb6758{/nifi-standard-content-viewer-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.1.1.war}
2017-12-16 06:29:45,259 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
... View more
Labels:
- Labels:
-
Apache NiFi
12-05-2017
05:47 AM
thanks, it works for me and also install nifi 1.1.0 version, in that I have the PutElasticsearch5 processor.that also works well with transport client.
... View more
12-02-2017
08:59 AM
I using HDP 2.5 and I add a NIFI service in it, I need to send a data to Elasticsearch 5.X, when i add a processor in NIFI, the putelasticsearch5 processor is missing but it has only putelasticsearch(it supports only elasticsearc 2.X version).How can I add the Putelasticsearch5 processor in nifi.
... View more
Labels:
12-01-2017
05:55 AM
whats the difference between, when i entering (elasticsearch,solr ) in Terms to Filter On inGET TWITTER processor and this article for creating a process group
... View more
11-02-2017
05:29 AM
Is it possible to secure Kibana UI using Ranger?
... View more
Labels:
- Labels:
-
Apache Ranger
04-24-2017
11:52 AM
i set the number of reducers to 1.
... View more
04-24-2017
11:29 AM
hi,i create a hive table,when i go to fileview in hdfs (path: apps/hive/warehouse/database.db/tablename/000000_0) but i need to change the filename from 00000_0 to tablename. how can i do it?
... View more
- Tags:
- Data Processing
- Hive
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
- Labels:
-
Apache Hive
12-24-2016
08:15 AM
2 Kudos
i have a data in my HDFS,my date format in that file is Tue Dec 20 10:04:31 +0000 2016,when i create a hive table with field name created time and datatype date it store a null value.
how to change that date formate in hive?
... View more
- Tags:
- Hadoop Core
- HDFS
- Hive
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
12-23-2016
12:12 PM
i get the twitter data and put those data into hdfs using nifi,it gives a data,but my date formate is in Tue Dec 20 10:04:31 +0000 2016 and i create a hive table with those twitter data.when i create that table i have created_time field and i need to set datatype as date,it shows an null value then i change the datatype as string it shows an value.then i need to transfer a that table into elasticsearch using es-hadoop but i confused what datatype i am going to use as time field. i tried string to that time filed,it shows a value but when i visualize that field in kibana,in show an error a date type is required, my query is given below: i also tried ->change my created_time field as date but i shows a null value create table if not exists tweets_text_partition(
tweet_id bigint,
created_unixtime bigint,
created_time string,
displayname string,
msg string,
fulltext string
)
row format delimited fields terminated by "|"
location "/tmp/tweets_staging";
*******************************************************************
curl -XPUT http://sandbox.hortonworks.com:9200/twitter_text1/?pretty -d'{"rels":{"properties":{"tweet_id":{"type":"bigint"},"created_unixtime":{"type":"bigint"},"created_time":{"type":"string"},"displayname":{"type":"string"},"msg":{"type":"string"},"fulltext":{"type":"string"}}}}'
***********************************************************************
ADD JAR /root/ elasticsearch-hadoop-5.1.1.jar;
ADD JAR /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar;
**********************************************************************************
CREATE EXTERNAL TABLE test.twitter_es1
(
tweet_id bigint,
created_unixtime bigint,
created_time string,
displayname string,
msg string,
fulltext string
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES (
'es.resource' = 'twitterdata/data',
'es.nodes'='sandbox.hortonworks.com:9200',
'es.index.auto.create' = 'false',
'es.mapping.id'='tweet_id');
*************************************************************
INSERT OVERWRITE TABLE test.twitter_es1
SELECT
tweet_id,
created_unixtime,
created_time,
displayname,
msg,
fulltext
FROM
test.tweets_hive1;
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
12-21-2016
07:54 AM
it resolve my problem but whenever i use the init 6 the session is
closed and i am unable to enable the sandbox.service without reboot my
vm.
... View more
12-21-2016
07:40 AM
add jar /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar when i add the above jar it works fine but it wont works when i try for new table. afterwards i restart the hive session that jar works fine. how i permanently add that jar in hive
... View more
12-21-2016
07:16 AM
i am new to mongodb.plz share some tutorial links or set-up guide how to load a twitter data into mongodb and then transfer those datas into hive. i use hortonworks 2.4 in vm 5.0.20
... View more
Labels:
- Labels:
-
Apache Hive
12-19-2016
08:30 AM
thanks it works fine.
... View more
12-19-2016
07:35 AM
thanks for your reply but the same error will shows again
... View more
12-19-2016
06:47 AM
i have a vertex failed error while insert overwrite table into elasticsearch using es-haddop jar.i follow the tutorial link below https://hadoopist.wordpress.com/2015/01/03/how-to-ingest-the-data-into-elasticsearch-through-hive/ my hive log is Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1481976368061_0022_1_00, diagnostics=[Task failed, taskId=task_1481976368061_0022_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1481976368061_0022_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1481976368061_0022_1_00, diagnostics=[Task failed, taskId=task_1481976368061_0022_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1481976368061_0022_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
hive>
... View more
Labels:
- Labels:
-
Apache Hive
12-19-2016
05:00 AM
2016-12-19 04:43:41,466 ERROR [main]: exec.Task (TezTask.java:execute(212)) - Failed to execute tez graph.
org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0017 to YARN : Application application_1481976368061_0017 submitted by user root to unknown queue: agent
at org.apache.tez.client.TezClient.start(TezClient.java:413)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:196)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:271)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:151)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1720)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1477)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0017 to YARN : Application application_1481976368061_0017 submitted by user root to unknown queue: agent
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271)
at org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72)
at org.apache.tez.client.TezClient.start(TezClient.java:408)
... 22 more
2016-12-19 04:43:41,466 INFO [main]: hooks.ATSHook (ATSHook.java:<init>(90)) - Created ATS Hook
2016-12-19 04:43:41,466 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,470 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1482122621466 end=1482122621470 duration=4 from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,471 ERROR [main]: ql.Driver (SessionState.java:printError(932)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
2016-12-19 04:43:41,471 INFO [main]: ql.Driver (Driver.java:execute(1621)) - Resetting the caller context to
2016-12-19 04:43:41,471 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=Driver.execute start=1482122620489 end=1482122621471 duration=982 from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,471 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,471 INFO [ATS Logger 0]: hooks.ATSHook (ATSHook.java:createPostHookEvent(193)) - Received post-hook notification for :root_20161219044340_094281c1-c0f2-4cc9-83a5-962b067937f7
2016-12-19 04:43:41,488 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1482122621471 end=1482122621488 duration=17 from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,505 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,505 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1482122621505 end=1482122621505 duration=0 from=org.apache.hadoop.hive.ql.Driver>
... View more
12-19-2016
04:57 AM
ERROR [main]: exec.Task (TezTask.java:execute(212)) - Failed to execute tez graph.org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0016 to YARN : Application application_1481976368061_0016 submitted by user root to unknown queue: agent at org.apache.tez.client.TezClient.start(TezClient.java:413) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:196) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeAndOpen(TezSessionPoolManager.java:311) at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:453) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:180) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1720) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1477) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0016 to YARN : Application application_1481976368061_0016 submitted by user root to unknown queue: agent at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271) at org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72) at org.apache.tez.client.TezClient.start(TezClient.java:408) ... 23 more2016-12-17 14:02:51,678 INFO [main]: hooks.ATSHook (ATSHook.java:<init>(90)) - Created ATS Hook2016-12-17 14:02:51,678 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,679 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1481983371678 end=1481983371679 duration=1 from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,679 ERROR [main]: ql.Driver (SessionState.java:printError(932)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask2016-12-17 14:02:51,679 INFO [main]: ql.Driver (Driver.java:execute(1621)) - Resetting the caller context to 2016-12-17 14:02:51,679 INFO [ATS Logger 0]: hooks.ATSHook (ATSHook.java:createPostHookEvent(193)) - Received post-hook notification for :root_20161217140249_635f254a-e5f1-4a5a-9703-2a2bd9612d252016-12-17 14:02:51,679 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=Driver.execute start=1481983369588 end=1481983371679 duration=2091 from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,679 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,685 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1481983371679 end=1481983371685 duration=6 from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,693 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,693 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1481983371693 end=1481983371693 duration=0 from=org.apache.hadoop.hive.ql.Driver>
... View more
12-17-2016
12:57 PM
this is log file of hive
... View more
12-17-2016
11:56 AM
thanks for your answer. ya i use HDP 2.5 Docker Sandbox.your link will resolve my problem. now its works fine
... View more
12-17-2016
07:53 AM
hive log path is /var/log/hive/hiveserver2.log my error log is in below link http://pastie.org/private/fnmpqvqquf19tjjptmjjg
... View more
12-16-2016
06:30 AM
@Santhosh B Gowda hive> INSERT OVERWRITE TABLE default.DRIVERDATA_ES_JAN11
SELECT
KEY,
CLAIMNUMBER,
POL_NBR,
VEHICLE_NBR,
EXP_NBR,
NAME,
COVERAGE,
ADDRESS
FROM
default.DRIVERDATA_TMP1; Query ID = root_20161216053225_fd9bffd2-f2ed-46ec-aa7f-a4095f4775b4
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
... View more
12-16-2016
06:17 AM
@Santhosh B Gowda
MismatchedTokenException(5!=313)
at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormatFieldIdentifier(HiveParser.java:34846)
at org.apache.hadoop.hive.ql.parse.HiveParser.rowFormatDelimited(HiveParser.java:33756)
at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormat(HiveParser.java:34040)
at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:5361)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2648)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1658)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1117)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:202)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:428)
... View more
12-16-2016
05:08 AM
i am using hive and tez.whenever i am performing insert query its
returns following error:
execution error, return code 1 from
org.apache.hadoop.hive.ql.exec.tez.teztask
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Tez
12-13-2016
10:42 AM
I use hdp-2.5,I follow the tutorial in the link -"https://community.hortonworks.com/articles/56121/how-to-pull-data-from-twitter-and-push-data-to-ela.html" in that i Successfully install and run the elasticsearch,it wont show any error,when i use the curl it will show the output but my browser wont get it, i also add the network port in vm But when i follow the same step in HDP-2.4 the web-browser UI works fine when i add a Network-port in VM ware.How i resolve it?
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
