When I try to run a hive action I get error number E0729 Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [12] with nothing more in log file. How fix this? log: 2016-06-30 18:33:37,930 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@:start:] Start action [0000011-160630143249353-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2016-06-30 18:33:37,934 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@:start:] [***0000011-160630143249353-oozie-oozi-W@:start:***]Action status=DONE 2016-06-30 18:33:37,934 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@:start:] [***0000011-160630143249353-oozie-oozi-W@:start:***]Action updated in DB! 2016-06-30 18:33:38,013 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000011-160630143249353-oozie-oozi-W 2016-06-30 18:33:38,014 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000011-160630143249353-oozie-oozi-W@:start: 2016-06-30 18:33:38,042 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] Start action [0000011-160630143249353-oozie-oozi-W@hive-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2016-06-30 18:33:49,999 INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] checking action, hadoop job ID [job_1467297249897_0020] status [RUNNING] 2016-06-30 18:33:50,002 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] [***0000011-160630143249353-oozie-oozi-W@hive-node***]Action status=RUNNING 2016-06-30 18:33:50,002 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] [***0000011-160630143249353-oozie-oozi-W@hive-node***]Action updated in DB! 2016-06-30 18:33:50,017 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] No Notification URL is defined. Therefore nothing to notify for job 0000011-160630143249353-oozie-oozi-W@hive-node 2016-06-30 18:38:09,496 INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] action completed, external ID [job_1467297249897_0020] 2016-06-30 18:38:09,783 WARN HiveActionExecutor:523 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [12] 2016-06-30 18:38:10,580 INFO ActionEndXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] ERROR is considered as FAILED for SLA 2016-06-30 18:38:10,984 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@fail] Start action [0000011-160630143249353-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2016-06-30 18:38:11,002 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@fail] [***0000011-160630143249353-oozie-oozi-W@fail***]Action status=DONE 2016-06-30 18:38:11,003 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@fail] [***0000011-160630143249353-oozie-oozi-W@fail***]Action updated in DB! 2016-06-30 18:38:11,506 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@fail] No Notification URL is defined. Therefore nothing to notify for job 0000011-160630143249353-oozie-oozi-W@fail 2016-06-30 18:38:11,506 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000011-160630143249353-oozie-oozi-W 2016-06-30 18:38:11,506 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000011-160630143249353-oozie-oozi-W] ACTION[0000011-160630143249353-oozie-oozi-W@hive-node] No Notification URL is defined. Therefore nothing to notify for job 0000011-160630143249353-oozie-oozi-W@hive-node job.properties nameNode=hdfs://sandbox.hortonworks.com:8020 jobTracker=sandbox.hortonworks.com:8050 queueName=default examplesRoot=examples oozie.use.system.libpath=true oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/hive my Workflow is <workflow-app xmlns="uri:oozie:workflow:0.2" name="hive-wf"> <start to="hive-node"/> <action name="hive-node"> <hive xmlns="uri:oozie:hive-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/hive"/> <mkdir path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data"/> </prepare> <job-xml>hive-site.xml</job-xml> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <script>script.q</script> <param>INPUT=/user/${wf:user()}/${examplesRoot}/input-data/table</param> <param>OUTPUT=/user/${wf:user()}/${examplesRoot}/output-data/hive</param> </hive> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Hive failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app> ... View more

I started working with ooze. I am trying to run work flow examples in the examples folder. However, when I try to run a hive action I get error number E0729. In the log file the error is: org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate anonymous at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:266) at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:202) at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:402) at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:297) at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253) at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate anonymous at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:83) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) at com.sun.proxy.$Proxy28.open(Unknown Source) at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258) ... 12 more Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate anonymous at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:137) at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ... 20 more my job.properties is nameNode=hdfs://sandbox.hortonworks.com:8020 jobTracker=sandbox.hortonworks.com:8050 queueName=default examplesRoot=examples oozie.use.system.libpath=true oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/hive ... View more

I followed this link http://lecluster.delaurent.com/oozie-101-your-first-workflow-in-5-minutes/ to start with oozie. My job stays at Running state with the JA006 error code. I started my history server but, I still get the same error. How I can resolve this? I am using Oozie 4.2.0.2.3. This is my job.properties: nameNode=hdfs://sandbox.hortonworks.com:8020 jobTracker=sandbox.hortonworks.com:8050 queueName=default examplesRoot=examples oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/shell ... View more

In my data I have 8 different schemas. I want to create 8 different data frame for them and save them in 8 different tables in hive. So far I created a super bean class which holds shared attributes and each bean class extends it. Based on the type attribute I created different objects. The problem is I am unable to save them in different data frame. Is there any way I can do that? Here is my code so far, which works fine for one schema. xmlData.foreachRDD( new Function2<JavaRDD<String>, Time, Void>() { public Void call(JavaRDD<String> rdd, Time time) { HiveContext hiveContext = JavaHiveContext.getInstance(rdd.context()); // Convert RDD[String] to RDD[case class] to DataFrame JavaRDD<JavaRow> rowRDD = rdd.map(new Function<String, JavaRow>() { public JavaRow call(String line) throws Exception{ String[] fields = line.split("\\|"); //JavaRow is my super class JavaRow record = null; if(fields[2].trim().equalsIgnoreCase("CDR")){ record = new GPRSClass(fields[0], fields[1]); } if(fields[2].trim().equalsIgnoreCase("Activation")){ record = new GbPdpContextActivation(fields[0], fields[1], fields[2], fields[3]); } return record;}}); DataFrame df; df = hiveContext.createDataFrame(rowRDD, JavaRow.class); df.toDF().registerTempTable("Consumer"); System.out.println(df.count()+" ************Record Recived************"); df = hiveContext.createDataFrame(rowRDD, GPRSClass.class); hiveContext.sql("CREATE TABLE if not exists gprs_data ( processor string, fileName string, type string, version string, id string )STORED AS ORC "); df.save("/apps/hive/warehouse/data", "org.apache.spark.sql.hive.orc",SaveMode.Append); } return null; } }); ... View more

I am new in Oozie. I am using Hue 2.6.1-2950 and Oozie 4.2. I develop a spark program in java which gets the data from kafka topic and save them in hive table. I pass my arguments to my .ksh script to submit the job. It works perfect however, I have no idea how to schedule this using oozie and hue to run every 5 minutes. I have a jar file which is my java code, I have a consumer.ksh which gets the arguments from my configuration file and run my jar file using spark-submit command. Please give me suggestion how to this. ... View more

I am new in oozie. I have a java program which produce data into kafka topic(it is not map reduce job). I am trying to schedule it with ozzie. How ever, I am getting this error: JA009: Could not load history file hdfs://sandbox.hortonworks.com:8020/mr-history/tmp/hue/job_1459358290769_0012-1459533575025-hue-oozie%3Alauncher%3AT%3Djava%3AW%3DData+Producer%3AA%3DproduceDat-1459533591693-1-0-SUCCEEDED-default-1459533581542.jhist at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.loadFullHistoryData(CompletedJob.java:349) at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.<init>(CompletedJob.java:101) at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager$HistoryFileInfo. I read it can be permission or owner problem so, I changed the owner to mapred and give 777 permission. But I still I get the same error. I am using java action to schedule my jar file. ... View more