Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie- attach the job logs to email (email action)

SOLVED Go to solution
Highlighted

oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

Is there a way to attach or add in the content of the email (email action in oozie) the job logs? (the logs from all the actions of the job)

I didn't find any wf parameter that could help ..

 

thank you!

GHERMAN Alina
1 ACCEPTED SOLUTION

Accepted Solutions

Re: oozie- attach the job logs to email (email action)

Explorer

Could be related to CDH version/environment as mentioned here. and its related jira.

 

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

 

Best regards

11 REPLIES 11

Re: oozie- attach the job logs to email (email action)

Explorer

Oozie Rest APIs to the resucue (Which Hue too uses)

 

Else

 

one can write a shell or .ksh which captures external_Id and its related launcher attemp ids, and get the logs of individual attemp ids.

 

We did the later one.

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Thank you for your answer. I will try to do a schell script for gathering the logs. 

However, is there a way to add the logs in the attachement?

 

 

Thank you!

 

Alina

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Explorer

email action has <attachment> element which can be used to attach an hdfs file or a comma sperated hdfs files.

 

However, make sure the logs are added/put to an hdfs file/files before attaching thereof.

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

I saw that there is an attachement element, however, I cannot add it in hue... 

 

 

Thank you!

 

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Explorer

Could be related to CDH version/environment as mentioned here. and its related jira.

 

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

 

Best regards

Re: oozie- attach the job logs to email (email action)

Champion Alumni

You were right, this is linked to :

  • OOZIE-2160 - Oozie email action now supports attachments with an <attachment> element

 

I am in CDH 5.3

 

 

Thank you!

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

I told myself that if I cannot add an attachement, I'll just add a link to my log files from HDFS.

So I did a :

 

yarn application -list -appStates FINISHED |grep 'my_workflow_name' |grep -Po 'application_\d+_\d+' | sed 's/.*application://' | tail -n 1

in order to find the application id ($my_application_id) that I needed. Afterwards I wanted to do a :

 

 

 

yarn logs -applicationId $my_application_id

However, this doesn't return any logs if it is not executed with a user that has the rights to read the logs. So I wanted to change it into:

sudo -u hdfs yarn logs -applicationId $application_id

but then I got the ERROR:

 

 sudo: sorry, you must have a tty to run sudo

 

 

Is there a proper way to find out the logs without changing the level of securiyt security?

(http://unix.stackexchange.com/questions/122616/why-do-i-need-a-tty-to-run-sudo-if-i-can-sudo-without...

 

 

Thank you!

 

 

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Master Guru
On non-kerberized (insecure) clusters, you can do the below:

export HADOOP_USER_NAME=username
[command]

For ex:

export HADOOP_USER_NAME=hdfs
yarn logs -applicationId $application_id

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

I tried to do :

 

export HADOOP_USER_NAME=my_user
load_events=`yarn logs -applicationId $application_id` 

I also tried:

export HADOOP_USER_NAME=hdfs
load_events=`yarn logs -applicationId $application_id` 

 

However I get :

 

 

Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.

And this is the message that I get when I do this command with un unauthorised user..

 

 

 

GHERMAN Alina