Member since
12-13-2016
33
Posts
3
Kudos Received
0
Solutions
12-18-2017
12:20 PM
i change that flow.xml.gz from an older version,i also remove that flow.xml.gz.but still my nifi wont start.
... View more
12-18-2017
12:09 PM
I have an set of urls in a file. sample file: localhost:1234/aaaa/uyt localhost:1234/bbbb/grt localhost:1234/cccc/hgd etc.. how can I pass this URL dynamically from that file to invoke HTTP processor.
... View more
Labels:
12-16-2017
07:00 AM
nifi-bootstrap.log
2017-12-16 06:29:38,004 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,065 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:38,162 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,175 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:38,482 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,500 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:38,779 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:38,801 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:39,925 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:39,943 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:40,576 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :2061:17: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{processGroup, remoteProcessGroup, connection, controllerService}' is expected.
2017-12-16 06:29:40,588 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :8129:38: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{connection, controllerService}' is expected.
2017-12-16 06:29:44,728 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Unable to start Flow Controller.
2017-12-16 06:29:44,729 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2017-12-16 06:29:45,653 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
nifi-app.log
2017-12-16 06:29:44,836 INFO [Thread-1] o.a.n.w.c.ApplicationStartupContextListener Initiating shutdown of flow service...
2017-12-16 06:29:44,837 INFO [Thread-1] o.a.n.w.c.ApplicationStartupContextListener Flow service termination completed.
2017-12-16 06:29:44,837 INFO [Thread-1] /nifi-api Closing Spring root WebApplicationContext
2017-12-16 06:29:45,075 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@3ed2a30c{/nifi-api,null,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.1.1.war}
2017-12-16 06:29:45,230 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@526e8108{/nifi,null,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.1.1.war}
2017-12-16 06:29:45,243 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@413bef78{/nifi-jolt-transform-json-ui-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.1.1.war}
2017-12-16 06:29:45,247 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@4ed15347{/nifi-image-viewer-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-media-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.1.1.war}
2017-12-16 06:29:45,253 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@20ead579{/nifi-update-attribute-ui-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-update-attribute-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.1.1.war}
2017-12-16 06:29:45,257 INFO [Thread-1] o.e.jetty.server.handler.ContextHandler Stopped o.e.j.w.WebAppContext@33eb6758{/nifi-standard-content-viewer-1.1.1,null,UNAVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.1.1.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.1.1.war}
2017-12-16 06:29:45,259 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
... View more
Labels:
12-05-2017
05:47 AM
thanks, it works for me and also install nifi 1.1.0 version, in that I have the PutElasticsearch5 processor.that also works well with transport client.
... View more
12-02-2017
08:59 AM
I using HDP 2.5 and I add a NIFI service in it, I need to send a data to Elasticsearch 5.X, when i add a processor in NIFI, the putelasticsearch5 processor is missing but it has only putelasticsearch(it supports only elasticsearc 2.X version).How can I add the Putelasticsearch5 processor in nifi.
... View more
12-01-2017
05:55 AM
whats the difference between, when i entering (elasticsearch,solr ) in Terms to Filter On inGET TWITTER processor and this article for creating a process group
... View more
04-24-2017
11:52 AM
i set the number of reducers to 1.
... View more
04-24-2017
11:29 AM
hi,i create a hive table,when i go to fileview in hdfs (path: apps/hive/warehouse/database.db/tablename/000000_0) but i need to change the filename from 00000_0 to tablename. how can i do it?
... View more
- Tags:
- Data Processing
- Hive
- Upgrade to HDP 2.5.3 : ConcurrentModificationException When Executing Insert Overwrite : Hive
Labels:
01-19-2017
07:29 AM
i create a two tables without header: loc2=loc1.map(lambda (x,y,z): ('key', {'location': str(x), 'city':str(y), 'state' : str(z)})) soc1=soc.map(lambda (a,b,c,d,f,g): ('key', {'id': str(a), 'firstname':str(b),'secondname':str(c),'lastname':str(d), 'city' :str(f), 'state':str(g)})) how can i join two tables soc1 and loc2 with the common city field.
... View more
01-12-2017
04:16 AM
please share some sample document to load a sample csv file to elasticsearch using pyspark
... View more
01-10-2017
10:21 AM
i try to load a csv file into elasticsearch using pyspark.when i execute the spark submit,it show an error i try the sample code below from pyspark import SparkContext, SparkConf
from pyspark.sql import SQLContext
if __name__ == "__main__":
conf = SparkConf().setAppName("WriteToES")
sc = SparkContext(conf=conf)
sqlContext = SQLContext(sc)
es_conf = {"es.nodes" : "sandbox.hortonworks.com","es.port" : "9200","es.nodes.client.only" : "true","es.resource" : "spark/data"}
es_df_p = sc.textFile("/sample/hello.csv").map(lambda line: line.split(","))
es_df_pf= es_df_p.groupBy("element_id").count().map(lambda (a,b): ('id',{'element_id': a,'count': b}))
es_df_pf.saveAsNewAPIHadoopFile(
path='-',
outputFormatClass="org.elasticsearch.hadoop.mr.EsOutputFormat",
keyClass="org.apache.hadoop.io.NullWritable",
valueClass="org.elasticsearch.hadoop.mr.LinkedMapWritable",
conf=es_conf)
spark-submit --master yarn-cluster --jars /root/elasticsearch-spark-20_2.10-5.1.1.jar /root/es_spark_write.py
... View more
12-28-2016
08:10 AM
1 Kudo
i need to customise the tile-map in kibana visualization.i successfully created a tile-map UI in kibana but when I click on it,it will show only the no of counts (i attach the cover shot of the output in kibana),i want what are the data’s present in that particular location. For e.g. if you say LA is 500 and if i select LA, then I should see the 500 data in a grid/ tabular form. If I switch another city NY City then the data should be displayed in that grid/ tabular form.
... View more
12-24-2016
08:15 AM
2 Kudos
i have a data in my HDFS,my date format in that file is Tue Dec 20 10:04:31 +0000 2016,when i create a hive table with field name created time and datatype date it store a null value.
how to change that date formate in hive?
... View more
- Tags:
- Hadoop Core
- HDFS
- Hive
Labels:
12-23-2016
12:12 PM
i get the twitter data and put those data into hdfs using nifi,it gives a data,but my date formate is in Tue Dec 20 10:04:31 +0000 2016 and i create a hive table with those twitter data.when i create that table i have created_time field and i need to set datatype as date,it shows an null value then i change the datatype as string it shows an value.then i need to transfer a that table into elasticsearch using es-hadoop but i confused what datatype i am going to use as time field. i tried string to that time filed,it shows a value but when i visualize that field in kibana,in show an error a date type is required, my query is given below: i also tried ->change my created_time field as date but i shows a null value create table if not exists tweets_text_partition(
tweet_id bigint,
created_unixtime bigint,
created_time string,
displayname string,
msg string,
fulltext string
)
row format delimited fields terminated by "|"
location "/tmp/tweets_staging";
*******************************************************************
curl -XPUT http://sandbox.hortonworks.com:9200/twitter_text1/?pretty -d'{"rels":{"properties":{"tweet_id":{"type":"bigint"},"created_unixtime":{"type":"bigint"},"created_time":{"type":"string"},"displayname":{"type":"string"},"msg":{"type":"string"},"fulltext":{"type":"string"}}}}'
***********************************************************************
ADD JAR /root/ elasticsearch-hadoop-5.1.1.jar;
ADD JAR /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar;
**********************************************************************************
CREATE EXTERNAL TABLE test.twitter_es1
(
tweet_id bigint,
created_unixtime bigint,
created_time string,
displayname string,
msg string,
fulltext string
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES (
'es.resource' = 'twitterdata/data',
'es.nodes'='sandbox.hortonworks.com:9200',
'es.index.auto.create' = 'false',
'es.mapping.id'='tweet_id');
*************************************************************
INSERT OVERWRITE TABLE test.twitter_es1
SELECT
tweet_id,
created_unixtime,
created_time,
displayname,
msg,
fulltext
FROM
test.tweets_hive1;
... View more
Labels:
12-21-2016
07:54 AM
it resolve my problem but whenever i use the init 6 the session is
closed and i am unable to enable the sandbox.service without reboot my
vm.
... View more
12-21-2016
07:40 AM
add jar /usr/hdp/current/hive-client/lib/commons-httpclient-3.0.1.jar when i add the above jar it works fine but it wont works when i try for new table. afterwards i restart the hive session that jar works fine. how i permanently add that jar in hive
... View more
12-21-2016
07:16 AM
i am new to mongodb.plz share some tutorial links or set-up guide how to load a twitter data into mongodb and then transfer those datas into hive. i use hortonworks 2.4 in vm 5.0.20
... View more
Labels:
12-19-2016
08:30 AM
thanks it works fine.
... View more
12-19-2016
07:35 AM
thanks for your reply but the same error will shows again
... View more
12-19-2016
06:47 AM
i have a vertex failed error while insert overwrite table into elasticsearch using es-haddop jar.i follow the tutorial link below https://hadoopist.wordpress.com/2015/01/03/how-to-ingest-the-data-into-elasticsearch-through-hive/ my hive log is Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1481976368061_0022_1_00, diagnostics=[Task failed, taskId=task_1481976368061_0022_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1481976368061_0022_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1481976368061_0022_1_00, diagnostics=[Task failed, taskId=task_1481976368061_0022_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
], TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:261)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException
at org.elasticsearch.hadoop.hive.HiveUtils.structObjectInspector(HiveUtils.java:61)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:81)
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:102)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:356)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:230)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.URIException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 28 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1481976368061_0022_1_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
hive>
... View more
Labels:
12-19-2016
05:00 AM
2016-12-19 04:43:41,466 ERROR [main]: exec.Task (TezTask.java:execute(212)) - Failed to execute tez graph.
org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0017 to YARN : Application application_1481976368061_0017 submitted by user root to unknown queue: agent
at org.apache.tez.client.TezClient.start(TezClient.java:413)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:196)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:271)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:151)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1720)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1477)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0017 to YARN : Application application_1481976368061_0017 submitted by user root to unknown queue: agent
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271)
at org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72)
at org.apache.tez.client.TezClient.start(TezClient.java:408)
... 22 more
2016-12-19 04:43:41,466 INFO [main]: hooks.ATSHook (ATSHook.java:<init>(90)) - Created ATS Hook
2016-12-19 04:43:41,466 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,470 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1482122621466 end=1482122621470 duration=4 from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,471 ERROR [main]: ql.Driver (SessionState.java:printError(932)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
2016-12-19 04:43:41,471 INFO [main]: ql.Driver (Driver.java:execute(1621)) - Resetting the caller context to
2016-12-19 04:43:41,471 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=Driver.execute start=1482122620489 end=1482122621471 duration=982 from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,471 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,471 INFO [ATS Logger 0]: hooks.ATSHook (ATSHook.java:createPostHookEvent(193)) - Received post-hook notification for :root_20161219044340_094281c1-c0f2-4cc9-83a5-962b067937f7
2016-12-19 04:43:41,488 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1482122621471 end=1482122621488 duration=17 from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,505 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2016-12-19 04:43:41,505 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1482122621505 end=1482122621505 duration=0 from=org.apache.hadoop.hive.ql.Driver>
... View more
12-19-2016
04:57 AM
ERROR [main]: exec.Task (TezTask.java:execute(212)) - Failed to execute tez graph.org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0016 to YARN : Application application_1481976368061_0016 submitted by user root to unknown queue: agent at org.apache.tez.client.TezClient.start(TezClient.java:413) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:196) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeAndOpen(TezSessionPoolManager.java:311) at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:453) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:180) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1720) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1477) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1481976368061_0016 to YARN : Application application_1481976368061_0016 submitted by user root to unknown queue: agent at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:271) at org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:72) at org.apache.tez.client.TezClient.start(TezClient.java:408) ... 23 more2016-12-17 14:02:51,678 INFO [main]: hooks.ATSHook (ATSHook.java:<init>(90)) - Created ATS Hook2016-12-17 14:02:51,678 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,679 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1481983371678 end=1481983371679 duration=1 from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,679 ERROR [main]: ql.Driver (SessionState.java:printError(932)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask2016-12-17 14:02:51,679 INFO [main]: ql.Driver (Driver.java:execute(1621)) - Resetting the caller context to 2016-12-17 14:02:51,679 INFO [ATS Logger 0]: hooks.ATSHook (ATSHook.java:createPostHookEvent(193)) - Received post-hook notification for :root_20161217140249_635f254a-e5f1-4a5a-9703-2a2bd9612d252016-12-17 14:02:51,679 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=Driver.execute start=1481983369588 end=1481983371679 duration=2091 from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,679 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,685 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1481983371679 end=1481983371685 duration=6 from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,693 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(135)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>2016-12-17 14:02:51,693 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(162)) - </PERFLOG method=releaseLocks start=1481983371693 end=1481983371693 duration=0 from=org.apache.hadoop.hive.ql.Driver>
... View more
12-17-2016
12:57 PM
this is log file of hive
... View more
12-17-2016
11:56 AM
thanks for your answer. ya i use HDP 2.5 Docker Sandbox.your link will resolve my problem. now its works fine
... View more
12-17-2016
07:53 AM
hive log path is /var/log/hive/hiveserver2.log my error log is in below link http://pastie.org/private/fnmpqvqquf19tjjptmjjg
... View more
12-16-2016
06:30 AM
@Santhosh B Gowda hive> INSERT OVERWRITE TABLE default.DRIVERDATA_ES_JAN11
SELECT
KEY,
CLAIMNUMBER,
POL_NBR,
VEHICLE_NBR,
EXP_NBR,
NAME,
COVERAGE,
ADDRESS
FROM
default.DRIVERDATA_TMP1; Query ID = root_20161216053225_fd9bffd2-f2ed-46ec-aa7f-a4095f4775b4
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
... View more
