Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

oozie- attach the job logs to email (email action)

avatar
Champion Alumni

Hello,

 

Is there a way to attach or add in the content of the email (email action in oozie) the job logs? (the logs from all the actions of the job)

I didn't find any wf parameter that could help ..

 

thank you!

GHERMAN Alina
1 ACCEPTED SOLUTION

avatar
Contributor

Could be related to CDH version/environment as mentioned here. and its related jira.

 

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

 

Best regards

View solution in original post

11 REPLIES 11

avatar
Contributor

Oozie Rest APIs to the resucue (Which Hue too uses)

 

Else

 

one can write a shell or .ksh which captures external_Id and its related launcher attemp ids, and get the logs of individual attemp ids.

 

We did the later one.

avatar
Champion Alumni

Thank you for your answer. I will try to do a schell script for gathering the logs. 

However, is there a way to add the logs in the attachement?

 

 

Thank you!

 

Alina

GHERMAN Alina

avatar
Contributor

email action has <attachment> element which can be used to attach an hdfs file or a comma sperated hdfs files.

 

However, make sure the logs are added/put to an hdfs file/files before attaching thereof.

avatar
Champion Alumni

Hello,

 

I saw that there is an attachement element, however, I cannot add it in hue... 

 

 

Thank you!

 

GHERMAN Alina

avatar
Contributor

Could be related to CDH version/environment as mentioned here. and its related jira.

 

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

 

Best regards

avatar
Champion Alumni

You were right, this is linked to :

  • OOZIE-2160 - Oozie email action now supports attachments with an <attachment> element

 

I am in CDH 5.3

 

 

Thank you!

GHERMAN Alina

avatar
Champion Alumni

Hello,

 

I told myself that if I cannot add an attachement, I'll just add a link to my log files from HDFS.

So I did a :

 

yarn application -list -appStates FINISHED |grep 'my_workflow_name' |grep -Po 'application_\d+_\d+' | sed 's/.*application://' | tail -n 1

in order to find the application id ($my_application_id) that I needed. Afterwards I wanted to do a :

 

 

 

yarn logs -applicationId $my_application_id

However, this doesn't return any logs if it is not executed with a user that has the rights to read the logs. So I wanted to change it into:

sudo -u hdfs yarn logs -applicationId $application_id

but then I got the ERROR:

 

 sudo: sorry, you must have a tty to run sudo

 

 

Is there a proper way to find out the logs without changing the level of securiyt security?

(http://unix.stackexchange.com/questions/122616/why-do-i-need-a-tty-to-run-sudo-if-i-can-sudo-without...

 

 

Thank you!

 

 

GHERMAN Alina

avatar
Mentor
On non-kerberized (insecure) clusters, you can do the below:

export HADOOP_USER_NAME=username
[command]

For ex:

export HADOOP_USER_NAME=hdfs
yarn logs -applicationId $application_id

avatar
Champion Alumni

Hello,

 

I tried to do :

 

export HADOOP_USER_NAME=my_user
load_events=`yarn logs -applicationId $application_id` 

I also tried:

export HADOOP_USER_NAME=hdfs
load_events=`yarn logs -applicationId $application_id` 

 

However I get :

 

 

Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.

And this is the message that I get when I do this command with un unauthorised user..

 

 

 

GHERMAN Alina