It's a bug in Oozie. CoordActionCheckXCommand doesn't take care of SUSPENDED state. It only handles SUCCEEDED, FAILED and KILLED. protected Void execute() throws CommandException { try { InstrumentUtils.incrJobCounter(getName(), 1, getInstrumentation()); Status slaStatus = null; CoordinatorAction.Status initialStatus = coordAction.getStatus(); if (workflowJob.getStatus() == WorkflowJob.Status.SUCCEEDED) { coordAction.setStatus(CoordinatorAction.Status.SUCCEEDED); // set pending to false as the status is SUCCEEDED coordAction.setPending(0); slaStatus = Status.SUCCEEDED; } else { if (workflowJob.getStatus() == WorkflowJob.Status.FAILED) { coordAction.setStatus(CoordinatorAction.Status.FAILED); slaStatus = Status.FAILED; // set pending to false as the status is FAILED coordAction.setPending(0); } else { if (workflowJob.getStatus() == WorkflowJob.Status.KILLED) { coordAction.setStatus(CoordinatorAction.Status.KILLED); slaStatus = Status.KILLED; // set pending to false as the status is KILLED coordAction.setPending(0); } else { LOG.warn("Unexpected workflow " + workflowJob.getId() + " STATUS " + workflowJob.getStatus()); coordAction.setLastModifiedTime(new Date()); CoordActionQueryExecutor.getInstance().executeUpdate( CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_MODIFIED_DATE, coordAction); return null; } } } ... View more

You can find the indices and matches of the header with the finditer of the re package. Then, use that to process the rest: import reimport json thefile = open("file.txt")line = thefile.readline()iter = re.finditer("\w+\s+", line)columns = [(m.group(0), m.start(0), m.end(0)) for m in iter]records = [] while line: line = thefile.readline() record = {} for col in columns: record[col[0]] = line[col[1]:col[2]] records.append(record) print(json.dumps(records)) ... View more

This looks like a java memory error. The reason why a select * works, but a select column doesn't, is that the select * just pulls a row of data from HDFS rather than actually executing a map-reduce job. You might be able to solve the problem by increasing the maximum heap size: export HADOOP_CLIENT_OPTS="-Xmx512m" would set the heap size to 512m, for example. I hope this helps! Regards, Lewis Tech-consultant   ... View more

SHORT Cloudera has broken zookeeper 3.4.5-cdh5.4.0 in several places. Service is working but CLI is dead. No workaround other than rollback. LONG Assign a bounty on this ;-). I have stepped on this mine too and was angry enough to find the reason: Zookeeper checks JLine during ZooKeeperMain.run(). There is a try-catch block that loads a number of classes. Any exception during class loading fails the whole block and JLine support is reported to be disabled. But here is why this happens with CDH 5.4.0: Current opensource Zookeeper-3.4.6 works against jline-0.9.94. Has no such issue. In CDH 5.4 Cloudera has applied the following patch: roman@node4:$ diff zookeeper-3.4.5-cdh5.3.3/src/java/main/org/apache/zookeeper/ZooKeeperMain.java zookeeper-3.4.5-cdh5.4.0/src/java/main/org/apache/zookeeper/ZooKeeperMain.java 305,306c305,306 < Class consoleC = Class.forName("jline.ConsoleReader"); < Class completorC = --- > Class consoleC = Class.forName("jline.ConsoleReader"); > Class completorC = 316,317c316,317 < Method addCompletor = consoleC.getMethod("addCompletor", < Class.forName("jline.Completor")); --- > Method addCompletor = consoleC.getMethod("addCompleter", > Class.forName("jline.console.completer.Completer")); CDH 5.4 uses jline-2.11.jar for ZooKeeper and it has no jline.ConsoleReader class (from 2.11 it is jline.console.ConsoleReader). Jline 0.9.94 in turn has no jline.console.completer.Completer. So there is incompatibility with any existing JLine. Any Cloudera CDH 5.4 user can run zookeeper-client on his/her cluster and find it does not work. Open-source zookeeper-3.4.6 depends on jline-0.9.94 which has no such patches. Don't know why Cloudera engineers have done such a mine. I see no clean way to fix it with 3.4.5-cdh5.4.0. I stayed with 3.4.5-cdh5.3.3 dependency where I need CLI and have production clusters. It seemed to me both jline-0.9.94.jar and jline.2.11.jar in classpath for zookeeper will fix the problem. But just have found Cloudera made another 'fix' in ZK for CDH 5.4.0, they have renamed org.apache.zookeeper.JLineZNodeCompletor class to org.apache.zookeeper.JLineZNodeCompleter. But here is the code from ZooKeeperMain.java Class<?> completorC = Class.forName("org.apache.zookeeper.JLineZNodeCompletor"); And of course, it means practically it is not possible to start ZK CLI in CDH 5.4.0 proper way. Awful work. 😞 ... View more

Look in the nifi-app.log and the nifi-bootstrap.log, they'll provide more information on why NiFi didn't start up. The default location of these files will be in the nifi-root-directory/logs. ... View more

Download HDP Sandbox MySQL database (Should already be present in the sandbox) NiFi 0.6 or later (Download and install a new version of NIFI or use Ambari to install NIFI in the sandbox) Use this code:     insert into cdc_test values (3, 'cdc3', null, null); insert into cdc_test values (4, 'cdc3', null, null); insert into cdc_test values (5, 'cdc3', null, null); insert into cdc_test values (6, 'cdc3', null, null); insert into cdc_test values (7, 'cdc3', null, null); insert into cdc_test values (8, 'cdc3', null, null); insert into cdc_test values (9, 'cdc3', null, null); insert into cdc_test values (10, 'cdc3', null, null); insert into cdc_test values (11, 'cdc3', null, null); insert into cdc_test values (12, 'cdc3', null, null); insert into cdc_test values (13, 'cdc3', null, null);     I hope this helps! Regards, Lewis ... View more

Try and use it after this modification. I succeeded like this. [You] jdbc:hive2://zk=hadoopcluster01:2181,hadoopcluster02:2181/hiveserver2 [My Proposal] jdbc:hive2://zk=hadoopcluster01:2181/hiveserver2,hadoopcluster02:2181/hiveserver2 ... View more

I uninstall Cloudera Manager by reference to Cloudera manager uninstall. Then I install Cloudera Manager again. Hive works well. I hope this works for you as well. Regards, Lewis ... View more

thanks for this information, it is quite helpful!   ... View more