Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

TransactionException: java.net.SocketException: Broken pipe (Write failed)

avatar
New Contributor

I have a job which execute an impala query but failed on 

java.net.SocketException: Broken pipe (Write failed)
Log Type: stdout
 
97945830404_12947_01_000001/*:/etc/hadoop/conf.cloudera.yarn:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-yarn/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-yarn/lib/*::
PRELAUNCH_ERR=/data7/yarn/container-logs/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/prelaunch.err:
HADOOP_TOKEN_FILE_LOCATION=/data13/yarn/nm/usercache/lrtech/appcache/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/container_tokens:
NM_AUX_SERVICE_spark_shuffle=:
LOCAL_USER_DIRS=/data10/yarn/nm/usercache/lrtech/,/data1/yarn/nm/usercache/lrtech/,/data11/yarn/nm/usercache/lrtech/,/data12/yarn/nm/usercache/lrtech/,/data13/yarn/nm/usercache/lrtech/,/data2/yarn/nm/usercache/lrtech/,/data3/yarn/nm/usercache/lrtech/,/data4/yarn/nm/usercache/lrtech/,/data5/yarn/nm/usercache/lrtech/,/data6/yarn/nm/usercache/lrtech/,/data7/yarn/nm/usercache/lrtech/,/data8/yarn/nm/usercache/lrtech/,/data9/yarn/nm/usercache/lrtech/:
HADOOP_HOME=/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop:
OOZIE_ACTION_CONF_XML=/data13/yarn/nm/usercache/lrtech/appcache/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/action.xml:
HOME=/home/:
SHLVL=1:
CONTAINER_ID=container_e182_1697945830404_12947_01_000001:
MALLOC_ARENA_MAX=4:
=================================================================
 
>>> Invoking Shell command line now >>
 
Stdoutput +-----------------------+
Stdoutput | summary               |
Stdoutput +-----------------------+
Stdoutput | Table does not exist. |
Stdoutput +-----------------------+
Exit code of the Shell command 1
<<< Invocation of Shell command completed <<<
 
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
at org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
Caused by: org.apache.oozie.action.hadoop.LauncherMainException
at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:77)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:64)
... 16 more
Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://nameservice1/user/lrtech/oozie-oozi/0029675-230929111906311-oozie-oozi-W/impala-1031--shell/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification succeeded
 
Log Type: prelaunch.out
 
Log Upload Time: Tue Oct 24 18:01:11 -0700 2023
 
Log Length: 70
 
Setting up env variables
Setting up job resources
Launching container
 
Log Type: syslog
 
Log Upload Time: Tue Oct 24 18:01:12 -0700 2023
 
Log Length: 0
 
 
Log Type: stderr
 
Log Upload Time: Tue Oct 24 18:01:17 -0700 2023
 
Log Length: 5007
 
Showing 4096 bytes of 5007 total. Click here for the full log.
 
),interval 15 minutes),'MST')) as MV90DT
,IF(concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))='0000','2400',
concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))) as TE
from mdms.drumeterasset d
     ,mdms.eisrtu a
     ,mdms.eismptfolder f
     ,mdms.eischannel c
     ,mdms.xchanneldata b
where a.id = d.device
and f.id = a.folderid
and c.rtuid = a.id
and b.chnid = c.id
and d.installed = 1
and TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST'))=to_date(DATE_SUB(current_date(),1))
 
Query submitted at: 2023-10-24 18:01:17 (Coordinator: http://hdw001.psr.com:25000)
ERROR: TransactionException: java.net.SocketException: Broken pipe (Write failed)
 
Could not execute command: create table if not exists dept_load_research.IT_MDMS_STAGE_M01A AS
select distinct 
a.name as mtrnumbr 
,c.name as chnm 
,c.channel 
,c.multiplier as chnlmult 
,b.utcstamp 
,b.value 
,b.mod_date 
,b.dt
,from_unixtime(CAST(b.utcstamp AS BIGINT)) AS UTC_TMSTMP
,from_utc_timestamp(from_unixtime(CAST(b.utcstamp AS BIGINT)),'MST') AS LCL_TMSTMP
,TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST')) as MV90DT
,IF(concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))='0000','2400',
concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)),
reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))) as TE
from mdms.drumeterasset d 
     ,mdms.eisrtu a 
     ,mdms.eismptfolder f 
     ,mdms.eischannel c 
     ,mdms.xchanneldata b 
where a.id = d.device 
and f.id = a.folderid 
and c.rtuid = a.id 
and b.chnid = c.id 
and d.installed = 1
and TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST'))=to_date(DATE_SUB(current_date(),1))
 
Any guidance or help would be greatly appreciated.
2 REPLIES 2

avatar
Community Manager

@cloudnewston, Welcome to our community! To help you get the best possible answer, I have tagged in our @ChethanYM @SVB @willx  experts who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
New Contributor

I solved the issue by restarting the Impala service in Cloudera Mgr. Still not sure the root cause.