Hello experts; I have this query is getting PST time in Hive but how can I convert this PST time to UTC time? Please help select from_unixtime(unix_timestamp()-1*60*60*4, 'yyyyMMddHH') as pst_time; Thanks in advance JN ... View more

Hello all, I am getting core-site.xml (Permission denied) below when using Oozie-Spark workflow. I have access to this directory /opt/data/yarn/local/usercache/oozie/appcache drwxrwxrwx 2 yarn hadoop 10 Jul 20 11:30 appcache Could anyone please help me why I am getting the Permission denied? I really appreciated. Thanks Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied) java.io.FileNotFoundException: /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied) ... View more

Hello all, I am getting Permission denied error below when launching Oozie-Spark workflow. Also the directory with read, write access /opt/data/yarn/local/usercache/oozie/appcache drwxrwxrwx 2 yarn hadoop 10 Jul 20 11:30 appcache Could anyone please help me with below error? I really appreciated. Thanks Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied) java.io.FileNotFoundException: /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied) ... View more

hi all, I am getting this error in Oozie by using sqoop and Hive import. [main] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: Cannot run program "hive": error=2, No such file or directory Caused by: java.io.IOException: error=2, No such file or directory Below is my Oozie workflow: <arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:@xxx</arg> <arg>--username</arg> <arg>abc</arg> <arg>--password</arg> <arg>abc</arg> <arg>--query</arg> <arg>SELECT STARTTIME FROM TEST WHERE STARTTIME BETWEEN TO_DATE('2017-06-08 14:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-06-08 14:10:59','YYYY-MM-DD HH24:MI:SS') AND $CONDITIONS)</arg> <arg>--target-dir</arg> <arg>/${examplesRoot}/output-data/sqoop/mytest</arg> <arg>--hive-import</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>stest_tmp</arg> <arg>-m</arg> <arg>1</arg> <file>hdfs://localhost/sqoop/hive-site.xml</file> Could you please help me with the error? What files that I am missing? Thanks ... View more

Hello all, I have this below action Oozie workflow but I keep getting these error: ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing arguments for import: ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: --hive-drop-import-delimns ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: --hive-overwrite ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: --hive-table ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: test_tmp ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: --target-dir ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: /apps/hive/warehouse/test ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: -m ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: 1 ----------------------------- Oozie Workflow ---------------------------- <action name="sqoop-node"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="hdfs://localhost/test-oozie/examples/output-data/sqoop"/> <mkdir path="hdfs://localhost/test-oozie/examples/output-data"/> </prepare> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:</arg> <arg>--username</arg> <arg>user</arg> <arg>--password</arg> <arg>pwd</arg> <arg>--query</arg> <arg>SELECT * FROM MYTASK WHERE STARTTIME BETWEEN TO_DATE('2017-06-08 14:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-06-08 14:10:59','YYYY-MM-DD HH24:MI:SS') AND $CONDITIONS)</arg> <arg>--hive-import</arg> <arg>--hive-drop-import-delimns</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>test_tmp</arg> <arg>--target-dir</arg> <arg>/apps/hive/warehouse/test</arg> <arg>-m</arg> <arg>1</arg> Would you please help me why I got the Unrecognized argument ? Thank you for your help. JT ... View more

Hello all, I am new to oozie. I have oozie wf working successful with sqoop command but when I added hive import into oozie workflow I got it failed as "Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]" Can anyone please help me? I really appreciated. Thanks Here is my workflow: /************ <?xml version="1.0" encoding="UTF-8"?> <workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-wf"> <start to="sqoop-node"/> <action name="sqoop-node"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="test-oozie/examples/output-data/sqoop"/> <mkdir path="test-oozie/examples/output-data"/> </prepare> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> <property> <name>oozie.hive.defaults</name> <value>/examples/apps/sqoop/hive-site.xml</value> </property> </configuration> <arg>import</arg> <arg>--connect</arg> <arg>jdbc:oracle:thin:@hostname</arg> <arg>--username</arg> <arg>usename</arg> <arg>--password</arg> <arg>pwd</arg> <arg>--query</arg> <arg><![CDATA[SELECT ID, STARTTIME FROM TEST WHERE STARTTIME BETWEEN TO_DATE('2017-05-30 13:00:00', 'YYYY-MM-DD HH24:MI:SS') AND TO_DATE('2017-05-30 13:15:59','YYYY-MM-DD HH24:MI:SS') AND (1=1 OR $CONDITIONS)]]></arg> <arg>--hive-import</arg> <arg>--hive-drop-import-delims</arg> <arg>--hive-overwrite</arg> <arg>--hive-table</arg> <arg>task_detailstest_tmp</arg> <arg>--target-dir</arg> <arg>/apps/hive/warehouse/task_detailstest</arg> <arg>-m</arg> <arg>1</arg> </sqoop> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app> ... View more

Hello all, I am successfully able to run the sqoop command. But when I added to oozie workflow xml I am not able to do with oozie. I received STATUS as KILLED. The error code is JA018. Here is my workflow: <workflow-app xmlns="uri:oozie:workflow:0.4" name="oozie-wf"> <start to="sqoop-wf"/> <action name="sqoop-wf"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <command>import --connect jdbc:oracle:thin:@dbservername:1521:database --username user --password passwprd --query "SELECT TASKID FROM TASKS WHERE TASKTIMESTAMP BETWEEN TO_DATE('2017-02-13 00:00:00', 'yyyy-mm-dd hh24:mi:ss') AND TO_DATE('2017-02-13 23:59:59', 'yyyy-mm-dd hh24:mi:ss') AND \$CONDITIONS" --fields-terminated-by '|' -m 1 --target-dir /dbdata/mytest</command> </sqoop> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Failed, Error Message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app> Could someone please help me? I am new to oozie and I stuck with this error for awhile. Thanks in advance ... View more