Member since
10-24-2023
2
Posts
1
Kudos Received
0
Solutions
10-25-2023
08:34 PM
1 Kudo
I solved the issue by restarting the Impala service in Cloudera Mgr. Still not sure the root cause.
... View more
10-24-2023
08:11 PM
I have a job which execute an impala query but failed on java.net.SocketException: Broken pipe (Write failed) Log Type: stdout 97945830404_12947_01_000001/*:/etc/hadoop/conf.cloudera.yarn:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-yarn/*:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop-yarn/lib/*:: PRELAUNCH_ERR=/data7/yarn/container-logs/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/prelaunch.err: HADOOP_TOKEN_FILE_LOCATION=/data13/yarn/nm/usercache/lrtech/appcache/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/container_tokens: NM_AUX_SERVICE_spark_shuffle=: LOCAL_USER_DIRS=/data10/yarn/nm/usercache/lrtech/,/data1/yarn/nm/usercache/lrtech/,/data11/yarn/nm/usercache/lrtech/,/data12/yarn/nm/usercache/lrtech/,/data13/yarn/nm/usercache/lrtech/,/data2/yarn/nm/usercache/lrtech/,/data3/yarn/nm/usercache/lrtech/,/data4/yarn/nm/usercache/lrtech/,/data5/yarn/nm/usercache/lrtech/,/data6/yarn/nm/usercache/lrtech/,/data7/yarn/nm/usercache/lrtech/,/data8/yarn/nm/usercache/lrtech/,/data9/yarn/nm/usercache/lrtech/: HADOOP_HOME=/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/lib/hadoop: OOZIE_ACTION_CONF_XML=/data13/yarn/nm/usercache/lrtech/appcache/application_1697945830404_12947/container_e182_1697945830404_12947_01_000001/action.xml: HOME=/home/: SHLVL=1: CONTAINER_ID=container_e182_1697945830404_12947_01_000001: MALLOC_ARENA_MAX=4: ================================================================= >>> Invoking Shell command line now >> Stdoutput +-----------------------+ Stdoutput | summary | Stdoutput +-----------------------+ Stdoutput | Table does not exist. | Stdoutput +-----------------------+ Exit code of the Shell command 1 <<< Invocation of Shell command completed <<< java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413) at org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55) at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) Caused by: org.apache.oozie.action.hadoop.LauncherMainException at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:77) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107) at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:64) ... 16 more Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.ShellMain], exit code [1] Oozie Launcher, uploading action data to HDFS sequence file: hdfs://nameservice1/user/lrtech/oozie-oozi/0029675-230929111906311-oozie-oozi-W/impala-1031--shell/action-data.seq Stopping AM Callback notification attempts left 0 Callback notification trying http://hdm002.psr.com:11000/oozie/callback?id=0029675-230929111906311-oozie-oozi-W@impala-1031&status=FAILED Callback notification to http://hdm002.psr.com:11000/oozie/callback?id=0029675-230929111906311-oozie-oozi-W@impala-1031&status=FAILED succeeded Callback notification succeeded Log Type: prelaunch.out Log Upload Time: Tue Oct 24 18:01:11 -0700 2023 Log Length: 70 Setting up env variables Setting up job resources Launching container Log Type: syslog Log Upload Time: Tue Oct 24 18:01:12 -0700 2023 Log Length: 0 Log Type: stderr Log Upload Time: Tue Oct 24 18:01:17 -0700 2023 Log Length: 5007 Showing 4096 bytes of 5007 total. Click here for the full log. ),interval 15 minutes),'MST')) as MV90DT ,IF(concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)), reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))='0000','2400', concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)), reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))) as TE from mdms.drumeterasset d ,mdms.eisrtu a ,mdms.eismptfolder f ,mdms.eischannel c ,mdms.xchanneldata b where a.id = d.device and f.id = a.folderid and c.rtuid = a.id and b.chnid = c.id and d.installed = 1 and TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST'))=to_date(DATE_SUB(current_date(),1)) Query submitted at: 2023-10-24 18:01:17 (Coordinator: http://hdw001.psr.com:25000) ERROR: TransactionException: java.net.SocketException: Broken pipe (Write failed) Could not execute command: create table if not exists dept_load_research.IT_MDMS_STAGE_M01A AS select distinct a.name as mtrnumbr ,c.name as chnm ,c.channel ,c.multiplier as chnlmult ,b.utcstamp ,b.value ,b.mod_date ,b.dt ,from_unixtime(CAST(b.utcstamp AS BIGINT)) AS UTC_TMSTMP ,from_utc_timestamp(from_unixtime(CAST(b.utcstamp AS BIGINT)),'MST') AS LCL_TMSTMP ,TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST')) as MV90DT ,IF(concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)), reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))='0000','2400', concat(reverse(substr(reverse(concat("0",cast(hour(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)), reverse(substr(reverse(concat("0",cast(minute(from_utc_timestamp(cast(b.utcstamp as timestamp),'MST')) as string))),1,2)))) as TE from mdms.drumeterasset d ,mdms.eisrtu a ,mdms.eismptfolder f ,mdms.eischannel c ,mdms.xchanneldata b where a.id = d.device and f.id = a.folderid and c.rtuid = a.id and b.chnid = c.id and d.installed = 1 and TO_DATE(from_utc_timestamp(date_sub(cast(b.utcstamp as timestamp),interval 15 minutes),'MST'))=to_date(DATE_SUB(current_date(),1)) Any guidance or help would be greatly appreciated.
... View more
Labels: