I have a job which execute an impala query but failed on
java.net.SocketException: Broken pipe (Write failed)
Log Type: stdout
97945830404_12947_01_000001/*:/etc/hadoop/conf.cloudera.yarn:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-yarn/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-yarn/lib/*::
PRELAUNCH_ERR=/data7/yarn/container-logs/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/prelaunch.err:
HADOOP_TOKEN_FILE_LOCATION=/data13/yarn/nm/usercache/lrtech/appcache/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/container_tokens:
NM_AUX_SERVICE_spark_shuffle=:
LOCAL_USER_DIRS=/data10/yarn/nm/usercache/lrtech/,/data1/yarn/nm/usercache/lrtech/,/data11/yarn/nm/usercache/lrtech/,/data12/yarn/nm/usercache/lrtech/,/data13/yarn/nm/usercache/lrtech/,/data2/yarn/nm/usercache/lrtech/,/data3/yarn/nm/usercache/lrtech/,/data4/yarn/nm/usercache/lrtech/,/data5/yarn/nm/usercache/lrtech/,/data6/yarn/nm/usercache/lrtech/,/data7/yarn/nm/usercache/lrtech/,/data8/yarn/nm/usercache/lrtech/,/data9/yarn/nm/usercache/lrtech/:
HADOOP_HOME=/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop:
OOZIE_ACTION_CONF_XML=/data13/yarn/nm/usercache/lrtech/appcache/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/action.xml:
HOME=/home/:
SHLVL=1:
CONTAINER_ID=container_e182_1697945830404_12947_01_000001:
MALLOC_ARENA_MAX=4:
=================================================================
>>> Invoking Shell command line now >>
Stdoutput +-----------------------+
Stdoutput | summary |
Stdoutput +-----------------------+
Stdoutput | Table does not exist. |
Stdoutput +-----------------------+
Exit code of the Shell command 1
<<< Invocation of Shell command completed <<<
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
at org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
Caused by: org.apache.oozie.action.hadoop.LauncherMainException
at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:77)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:64)
... 16 more
Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://nameservice1/user/lrtech/oozie-oozi/0029675-230929111906311-oozie-oozi-W/impala-1031--shell/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification succeeded
Log Type: prelaunch.out
Log Upload Time: Tue Oct 24 18:01:11 -0700 2023
Log Length: 70
Setting up env variables
Setting up job resources
Launching container
Log Type: syslog
Log Upload Time: Tue Oct 24 18:01:12 -0700 2023
Log Length: 0
Log Type: stderr
Log Upload Time: Tue Oct 24 18:01:17 -0700 2023
Log Length: 5007
Showing 4096 bytes of 5007 total. Click here for the full log.
),interval 15 minutes),'MST')) as MV90DT
,IF(concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))='0000','2400',
concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))) as TE
from mdms.drumeterasset d
,mdms.eisrtu a
,mdms.eismptfolder f
,mdms.eischannel c
,mdms.xchanneldata b
where a.id = d.device
and f.id = a.folderid
and c.rtuid = a.id
and b.chnid = c.id
and d.installed = 1
and TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST'))=to_date(DATE_SUB(current_date(),1))
ERROR: TransactionException: java.net.SocketException: Broken pipe (Write failed)
Could not execute command: create table if not exists dept_load_research.IT_MDMS_STAGE_M01A AS
select distinct
a.name as mtrnumbr
,c.name as chnm
,c.channel
,c.multiplier as chnlmult
,b.utcstamp
,b.value
,b.mod_date
,b.dt
,from_unixtime(CAST(b.utcstamp AS BIGINT)) AS UTC_TMSTMP
,from_utc_timestamp(from_unixtime(CAST(b.utcstamp AS BIGINT)),'MST') AS LCL_TMSTMP
,TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST')) as MV90DT
,IF(concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))='0000','2400',
concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))) as TE
from mdms.drumeterasset d
,mdms.eisrtu a
,mdms.eismptfolder f
,mdms.eischannel c
,mdms.xchanneldata b
where a.id = d.device
and f.id = a.folderid
and c.rtuid = a.id
and b.chnid = c.id
and d.installed = 1
and TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST'))=to_date(DATE_SUB(current_date(),1))
Any guidance or help would be greatly appreciated.
