How to get logs of shell scripts in oozie

[#1]

I have a shell script in HDFS. I want to schedule this script in oozie.

     #!/bin/bash
     LOG_LOCATION=/home/$USER/logs
     exec 2>&1

     [ $# -ne 1 ] && { echo "Usage : $0 table ";exit 1; }

     table=$1

     TIMESTAMP=`date "+%Y-%m-%d"`
     touch /home/$USER/logs/${TIMESTAMP}.success_log
     touch /home/$USER/logs/${TIMESTAMP}.fail_log
     success_logs=/home/$USER/logs/${TIMESTAMP}.success_log
     failed_logs=/home/$USER/logs/${TIMESTAMP}.fail_log

     #Function to get the status of the job creation
     function log_status
     {
     status=$1
     message=$2
     if [ "$status" -ne 0 ]; then
     echo "`date +\"%Y-%m-%d %H:%M:%S\"` [ERROR] $message [Status] $status : failed" | tee -a      "${failed_logs}"
     #echo "Please find the attached log file for more details"
    exit 1
    else
    echo "`date +\"%Y-%m-%d %H:%M:%S\"` [INFO] $message [Status] $status : success" | tee -a    "${success_logs}"
    fi
    }

    `hive -e "create table testing.${table} as select * from database.${table}"`

    g_STATUS=$?
    log_status $g_STATUS "Hive create ${table}"

    *******************************************************************************************************************************

I have a some questions regarding using oozie to schedule shell scripts.

1) In my script I have failed and success logs which give me the result of the script whether it is successful or failed. Can we have this kind of feature in HDFS also while using oozie?

2) IN the script I am also collecting the stdout logs as you can see in the 2nd and 3rd lines after the shebang in my script. Can this also be achieved in HDFS?

If so how can we achieve these both in `HDFS` while scheduling shell scripts using oozie.

Could anyone explain please

If there are bettere ways to do things in oozie please let me know

[#2]

I mean how can we check the folders on datanodes. I don't know how to do it

[#3]

Well, one way to do it would be to connect yourself in SSH to the datanodes.

For exemple, using putty or winscp.

[#4]

@mathieu.d Here I am creating files in /tmp folder on datanodes right.

Can I do hdfs dfs -put /tmp/$USER/...success.log /user/$USER/logs/...success.log

Will this work ? 

[#5]

@mathieu.d @mbigelow Thank you both of you I was able to achieve the desired result.

First stored the logs in local and then uploaded them to HDFS

[#6]

how did you stored logs in local? can you please provide me the way you did it.