Oozie hive-add-partition keeps on running state forever

[#1]

I've installed a fresh HDP 2.6.1.0 and lost my past configuration files.

Tried to configure the Twitter pipeline but I'm stuck at Oozie Workflow.

When I launch the coordinator job to run every 60minutes, it hangs on running state forever. It simply doesn't do anything.

This also happened in the past when I changed some memory configurations, so I suspect it must be it...

I've tunned my memory configurations with the python script so I guess everything is ok...

Can someone give me some hints to resolve this issue?

Many thanks in advance.

Best regards

[#2]

@Hugo Felix

Add_partition if ACID is enabled, requires updating of Hive Metastore tables. There are known issues related to the transactions. Check hiveserver2 log to see if there are any transaction related exceptions.

[#3]

@sindhu many thanks for your answer... hiveserver2logs are clean...
Which logs can I check to tackle the problem?
I'll post here the yarn log which I think is clean.

yarn.txt

Also here it goes the oozie log:

2017-09-04 09:07:01,107  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@:start:] Start action [0000065-170901161649746-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2017-09-04 09:07:01,107  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@:start:] [***0000065-170901161649746-oozie-oozi-W@:start:***]Action status=DONE
2017-09-04 09:07:01,107  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@:start:] [***0000065-170901161649746-oozie-oozi-W@:start:***]Action updated in DB!
2017-09-04 09:07:01,139  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000065-170901161649746-oozie-oozi-W
2017-09-04 09:07:01,139  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000065-170901161649746-oozie-oozi-W@:start:
2017-09-04 09:07:01,154  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] Start action [0000065-170901161649746-oozie-oozi-W@hive-add-partition] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2017-09-04 09:07:03,202  INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] Trying to get job [job_1504271777639_0005], attempt [1]
2017-09-04 09:07:03,216  INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] checking action, hadoop job ID [job_1504271777639_0005] status [RUNNING]
2017-09-04 09:07:03,217  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] [***0000065-170901161649746-oozie-oozi-W@hive-add-partition***]Action status=RUNNING
2017-09-04 09:07:03,217  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] [***0000065-170901161649746-oozie-oozi-W@hive-add-partition***]Action updated in DB!
2017-09-04 09:07:03,221  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] No Notification URL is defined. Therefore nothing to notify for job 0000065-170901161649746-oozie-oozi-W@hive-add-partition
2017-09-04 09:17:12,662  INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] Trying to get job [job_1504271777639_0005], attempt [1]
2017-09-04 09:17:12,701  INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] checking action, hadoop job ID [job_1504271777639_0005] status [RUNNING]
2017-09-04 09:27:24,642  WARN ResumeXCommand:523 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[] E1100: Command precondition does not hold before execution, [workflow's status is RUNNING is not SUSPENDED], Error Code: E1100
2017-09-04 09:28:12,685  INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] Trying to get job [job_1504271777639_0005], attempt [1]
2017-09-04 09:28:12,704  INFO HiveActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[0000065-170901161649746-oozie-oozi-W@hive-add-partition] checking action, hadoop job ID [job_1504271777639_0005] status [RUNNING]
2017-09-04 09:38:24,684  WARN ResumeXCommand:523 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[hive-add-partition-wf] JOB[0000065-170901161649746-oozie-oozi-W] ACTION[] E1100: Command precondition does not hold before execution, [workflow's status is RUNNING is not SUSPENDED], Error Code: E1100
<br>

Best regards