07-24-2018 02:57 AM
Can anyone help me?
I upload the CSV files into HDFS and then throw them in the Hive as:
create temporary external table test
row format delimited fields terminated by '|'
stored as textfile
insert into table my_table partition(name, md) select dt, uid, name, md from test;
if the files are small then everything is fine, but if the size is large the process falls on the insert with an error:
Error during job, getting debugging information ...
Examining task ID: task_1522750657070_0091_m_000001 (and more) from job job_1522750657070_0091
Examining task ID: task_1522750657070_0091_r_000001 (and more) from job job_1522750657070_0091
Task with the most failures (4):
Diagnostic Messages for this Task:
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
in the YARN logs I see the following:
WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child: java.net.ConnectException: Call From hadoop-datanode02.prod.analytics.wz-ams.lo.mobbtech.com/127.0.1.1 to hadoop- datanode02.prod.analytics.wz-ams.lo.mobbtech.com:22232 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
all other messages INFO only
is this the cause of the insert error?
what else can I look to solve the problem?
I met the mention that the error code 2 is a memory problem ... Is it so?