Is there a way to attach or add in the content of the email (email action in oozie) the job logs? (the logs from all the actions of the job)
I didn't find any wf parameter that could help ..
Oozie Rest APIs to the resucue (Which Hue too uses)
one can write a shell or .ksh which captures external_Id and its related launcher attemp ids, and get the logs of individual attemp ids.
We did the later one.
Thank you for your answer. I will try to do a schell script for gathering the logs.
However, is there a way to add the logs in the attachement?
email action has <attachment> element which can be used to attach an hdfs file or a comma sperated hdfs files.
However, make sure the logs are added/put to an hdfs file/files before attaching thereof.
I saw that there is an attachement element, however, I cannot add it in hue...
I told myself that if I cannot add an attachement, I'll just add a link to my log files from HDFS.
So I did a :
yarn application -list -appStates FINISHED |grep 'my_workflow_name' |grep -Po 'application_\d+_\d+' | sed 's/.*application://' | tail -n 1
in order to find the application id ($my_application_id) that I needed. Afterwards I wanted to do a :
yarn logs -applicationId $my_application_id
However, this doesn't return any logs if it is not executed with a user that has the rights to read the logs. So I wanted to change it into:
sudo -u hdfs yarn logs -applicationId $application_id
but then I got the ERROR:
sudo: sorry, you must have a tty to run sudo
Is there a proper way to find out the logs without changing the level of securiyt security?
I tried to do :
export HADOOP_USER_NAME=my_user load_events=`yarn logs -applicationId $application_id`
I also tried:
export HADOOP_USER_NAME=hdfs load_events=`yarn logs -applicationId $application_id`
However I get :
Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled.
And this is the message that I get when I do this command with un unauthorised user..