question Re: oozie- attach the job logs to email (email action) in Archives of Support Questions (Read Only)

oozie- attach the job logs to email (email action)

AlinaGHERMAN — Fri, 16 Sep 2022 09:49:42 GMT

Hello,

Is there a way to attach or add in the content of the email (email action in oozie) the job logs? (the logs from all the actions of the job)

I didn't find any wf parameter that could help ..

thank you!

Re: oozie- attach the job logs to email (email action)

naane — Fri, 20 Nov 2015 19:35:06 GMT

Oozie Rest APIs to the resucue (Which Hue too uses)

Else

one can write a shell or .ksh which captures external_Id and its related launcher attemp ids, and get the logs of individual attemp ids.

We did the later one.

Re: oozie- attach the job logs to email (email action)

AlinaGHERMAN — Fri, 27 Nov 2015 10:32:33 GMT

Thank you for your answer. I will try to do a schell script for gathering the logs.

However, is there a way to add the logs in the attachement?

Thank you!

Alina

Re: oozie- attach the job logs to email (email action)

naane — Wed, 09 Dec 2015 20:00:19 GMT

email action has <attachment> element which can be used to attach an hdfs file or a comma sperated hdfs files.

However, make sure the logs are added/put to an hdfs file/files before attaching thereof.

Re: oozie- attach the job logs to email (email action)

AlinaGHERMAN — Fri, 11 Dec 2015 07:41:18 GMT

Hello,

I saw that there is an attachement element, however, I cannot add it in hue...

Thank you!

Re: oozie- attach the job logs to email (email action)

naane — Fri, 11 Dec 2015 16:48:07 GMT

Could be related to CDH version/environment as mentioned here. and its related jira.

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

Best regards

Re: oozie- attach the job logs to email (email action)

AlinaGHERMAN — Mon, 14 Dec 2015 13:28:11 GMT

You were right, this is linked to :

OOZIE-2160 - Oozie email action now supports attachments with an <attachment> element

I am in CDH 5.3

Thank you!

Re: oozie- attach the job logs to email (email action)

AlinaGHERMAN — Tue, 15 Dec 2015 07:48:03 GMT

Hello,

I told myself that if I cannot add an attachement, I'll just add a link to my log files from HDFS.

So I did a :

yarn application -list -appStates FINISHED |grep 'my_workflow_name' |grep -Po 'application_\d+_\d+' | sed 's/.*application://' | tail -n 1

in order to find the application id ($my_application_id) that I needed. Afterwards I wanted to do a :

yarn logs -applicationId $my_application_id

However, this doesn't return any logs if it is not executed with a user that has the rights to read the logs. So I wanted to change it into:

sudo -u hdfs yarn logs -applicationId $application_id

but then I got the ERROR:

 sudo: sorry, you must have a tty to run sudo

Is there a proper way to find out the logs without changing the level of securiyt security?

(http://unix.stackexchange.com/questions/122616/why-do-i-need-a-tty-to-run-sudo-if-i-can-sudo-without-a-password )

Thank you!

Re: oozie- attach the job logs to email (email action)

Harsh J — Tue, 15 Dec 2015 08:45:50 GMT

On non-kerberized (insecure) clusters, you can do the below:

export HADOOP_USER_NAME=username
[command]

For ex:

export HADOOP_USER_NAME=hdfs
yarn logs -applicationId $application_id

Re: oozie- attach the job logs to email (email action)

AlinaGHERMAN — Tue, 15 Dec 2015 15:51:01 GMT

Hello,

I tried to do :

export HADOOP_USER_NAME=my_user
load_events=`yarn logs -applicationId $application_id`

I also tried:

export HADOOP_USER_NAME=hdfs
load_events=`yarn logs -applicationId $application_id`

However I get :

Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.

And this is the message that I get when I do this command with un unauthorised user..

Re: oozie- attach the job logs to email (email action)

naane — Wed, 16 Dec 2015 21:56:04 GMT

can you see the application ID logs in your "http://<<YourHost>>:<<port>>/cluster/apps"??

Re: oozie- attach the job logs to email (email action)

AlinaGHERMAN — Thu, 17 Dec 2015 09:38:06 GMT

Hello,

Yes I can see my application_id logs . However, frstly it shows the message :

Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.

and then it recharge the page and it shows the good logs:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.3.5-1.cdh5.3.5.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/disk2/yarn/nm/filecache/7107/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Dec 15, 2015 10:16:44 AM com.google.inject.servlet.InternalServletModule$BackwardsCompatibleServletContextProvider get
WARNING: You are attempting to use a deprecated API (specifically, attempting to @Inject ServletContext inside an eagerly created singleton. While we allow this for backwards compatibility, be warned that this MAY have unexpected behavior if you have more than one injector (with ServletModule) running in the same JVM. Please consult the Guice documentation at http://code.google.com/p/google-guice/wiki/Servlets for more information.
Dec 15, 2015 10:16:44 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Dec 15, 2015 10:16:44 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Dec 15, 2015 10:16:44 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering 
.......

(What I'm searching for..)

Thank you!

Alina