oozie- attach the job logs to email (email action)

[#1]

Hello,

Is there a way to attach or add in the content of the email (email action in oozie) the job logs? (the logs from all the actions of the job)

I didn't find any wf parameter that could help ..

thank you!

[#2]

Oozie Rest APIs to the resucue (Which Hue too uses)

Else

one can write a shell or .ksh which captures external_Id and its related launcher attemp ids, and get the logs of individual attemp ids.

We did the later one.

[#3]

Thank you for your answer. I will try to do a schell script for gathering the logs. 

However, is there a way to add the logs in the attachement?

Thank you!

Alina

[#4]

email action has <attachment> element which can be used to attach an hdfs file or a comma sperated hdfs files.

However, make sure the logs are added/put to an hdfs file/files before attaching thereof.

[#5]

Hello,

I saw that there is an attachement element, however, I cannot add it in hue... 

Thank you!

[#6 · Accepted solution]

Could be related to CDH version/environment as mentioned here. and its related jira.

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

Best regards

[#7]

You were right, this is linked to :

• OOZIE-2160 - Oozie email action now supports attachments with an <attachment> element

I am in CDH 5.3

Thank you!

[#8]

Hello,

I told myself that if I cannot add an attachement, I'll just add a link to my log files from HDFS.

So I did a :

yarn application -list -appStates FINISHED |grep 'my_workflow_name' |grep -Po 'application_\d+_\d+' | sed 's/.*application://' | tail -n 1

in order to find the application id ($my_application_id) that I needed. Afterwards I wanted to do a :

yarn logs -applicationId $my_application_id

However, this doesn't return any logs if it is not executed with a user that has the rights to read the logs. So I wanted to change it into:

sudo -u hdfs yarn logs -applicationId $application_id

but then I got the ERROR:

 sudo: sorry, you must have a tty to run sudo

Is there a proper way to find out the logs without changing the level of securiyt security?

(http://unix.stackexchange.com/questions/122616/why-do-i-need-a-tty-to-run-sudo-if-i-can-sudo-without... ) 

Thank you!

[#9]

On non-kerberized (insecure) clusters, you can do the below:

export HADOOP_USER_NAME=username
[command]

For ex:

export HADOOP_USER_NAME=hdfs
yarn logs -applicationId $application_id

[#10]

Hello,

I tried to do :

export HADOOP_USER_NAME=my_user
load_events=`yarn logs -applicationId $application_id` 

I also tried:

export HADOOP_USER_NAME=hdfs
load_events=`yarn logs -applicationId $application_id` 

However I get :

Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.

And this is the message that I get when I do this command with un unauthorised user..