Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

oozie- attach the job logs to email (email action)

SOLVED Go to solution

oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

Is there a way to attach or add in the content of the email (email action in oozie) the job logs? (the logs from all the actions of the job)

I didn't find any wf parameter that could help ..

 

thank you!

GHERMAN Alina
1 ACCEPTED SOLUTION

Accepted Solutions

Re: oozie- attach the job logs to email (email action)

Explorer

Could be related to CDH version/environment as mentioned here. and its related jira.

 

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

 

Best regards

11 REPLIES 11

Re: oozie- attach the job logs to email (email action)

Explorer

Oozie Rest APIs to the resucue (Which Hue too uses)

 

Else

 

one can write a shell or .ksh which captures external_Id and its related launcher attemp ids, and get the logs of individual attemp ids.

 

We did the later one.

Highlighted

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Thank you for your answer. I will try to do a schell script for gathering the logs. 

However, is there a way to add the logs in the attachement?

 

 

Thank you!

 

Alina

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Explorer

email action has <attachment> element which can be used to attach an hdfs file or a comma sperated hdfs files.

 

However, make sure the logs are added/put to an hdfs file/files before attaching thereof.

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

I saw that there is an attachement element, however, I cannot add it in hue... 

 

 

Thank you!

 

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Explorer

Could be related to CDH version/environment as mentioned here. and its related jira.

 

However, if possible take a chance to edit manually your workflow for the email action to include <attachment>.. element and check whether attachment functionality works fine for your CDH environment.

 

Best regards

Re: oozie- attach the job logs to email (email action)

Champion Alumni

You were right, this is linked to :

  • OOZIE-2160 - Oozie email action now supports attachments with an <attachment> element

 

I am in CDH 5.3

 

 

Thank you!

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

I told myself that if I cannot add an attachement, I'll just add a link to my log files from HDFS.

So I did a :

 

yarn application -list -appStates FINISHED |grep 'my_workflow_name' |grep -Po 'application_\d+_\d+' | sed 's/.*application://' | tail -n 1

in order to find the application id ($my_application_id) that I needed. Afterwards I wanted to do a :

 

 

 

yarn logs -applicationId $my_application_id

However, this doesn't return any logs if it is not executed with a user that has the rights to read the logs. So I wanted to change it into:

sudo -u hdfs yarn logs -applicationId $application_id

but then I got the ERROR:

 

 sudo: sorry, you must have a tty to run sudo

 

 

Is there a proper way to find out the logs without changing the level of securiyt security?

(http://unix.stackexchange.com/questions/122616/why-do-i-need-a-tty-to-run-sudo-if-i-can-sudo-without...

 

 

Thank you!

 

 

GHERMAN Alina

Re: oozie- attach the job logs to email (email action)

Master Guru
On non-kerberized (insecure) clusters, you can do the below:

export HADOOP_USER_NAME=username
[command]

For ex:

export HADOOP_USER_NAME=hdfs
yarn logs -applicationId $application_id

Re: oozie- attach the job logs to email (email action)

Champion Alumni

Hello,

 

I tried to do :

 

export HADOOP_USER_NAME=my_user
load_events=`yarn logs -applicationId $application_id` 

I also tried:

export HADOOP_USER_NAME=hdfs
load_events=`yarn logs -applicationId $application_id` 

 

However I get :

 

 

Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.

And this is the message that I get when I do this command with un unauthorised user..

 

 

 

GHERMAN Alina