Member since
01-10-2020
26
Posts
0
Kudos Received
0
Solutions
03-04-2020
09:46 AM
Hi @Faerballert , How can I achieve the same delta changes on cassandra. could u please share some processors which I can use and if I'm using QueryDatabaseTable its asking for jdbc driver and jar details. So please share your thought
... View more
01-25-2020
06:08 AM
I’m the schema remove the second row in fields. So it’s just id1. Now do a test: use generateflowfile with the schema in contents. Then evaluatejson. In this processor click +, and make id1. The value for id1 is then $.value.id1 (this is the code to get the real value. Then route evaluatejson routes to an output port. Run. Inspect the item in the queue and confirm you see the attribute id1. This is a simple example to help you understand how json works, how to get the data from the value object, how to confirm schema is correct, and how to unit test by looking at flowfile in the queue.
... View more
01-23-2020
07:35 AM
It's a bug in Oozie. CoordActionCheckXCommand doesn't take care of SUSPENDED state. It only handles SUCCEEDED, FAILED and KILLED. protected Void execute() throws CommandException {
try {
InstrumentUtils.incrJobCounter(getName(), 1, getInstrumentation());
Status slaStatus = null;
CoordinatorAction.Status initialStatus = coordAction.getStatus();
if (workflowJob.getStatus() == WorkflowJob.Status.SUCCEEDED) {
coordAction.setStatus(CoordinatorAction.Status.SUCCEEDED);
// set pending to false as the status is SUCCEEDED coordAction.setPending(0);
slaStatus = Status.SUCCEEDED;
}
else {
if (workflowJob.getStatus() == WorkflowJob.Status.FAILED) {
coordAction.setStatus(CoordinatorAction.Status.FAILED);
slaStatus = Status.FAILED;
// set pending to false as the status is FAILED coordAction.setPending(0);
}
else {
if (workflowJob.getStatus() == WorkflowJob.Status.KILLED) {
coordAction.setStatus(CoordinatorAction.Status.KILLED);
slaStatus = Status.KILLED;
// set pending to false as the status is KILLED coordAction.setPending(0);
}
else {
LOG.warn("Unexpected workflow " + workflowJob.getId() + " STATUS " + workflowJob.getStatus());
coordAction.setLastModifiedTime(new Date());
CoordActionQueryExecutor.getInstance().executeUpdate(
CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_MODIFIED_DATE,
coordAction);
return null;
}
}
}
... View more
01-23-2020
05:26 AM
@Lewis @Shelton @Prakashcit Hi All, I did ranger re-installation and now it is working, but if i integrate hive with ldap, again getting same null pointer exception. Note: we integrated hive with ldap not ranger, just creating same ldap user in ranger (local) and trying to match it with LDAP user. whether it will create any problem? Kindly advice on this. select * from asop.test; Error: Error while compiling statement: FAILED: NullPointerException null (state=42000,code=40000)
... View more
01-22-2020
12:39 PM
@saivenkatg55 Your Out Of Memory (OOM) issue is occurring in the NiFi toolkit script and not in NiFi itself. Ambari will execute the NiFi toolkit before the NiFi service itself is ever started. If the NiFi toolkit fails, Ambari does not proceed any further. I can see you are running HDF 3.2.0 which exposes the heap memory settings under the NiFi configs in Ambari: Locate the above line and increase your heap settings here. Once the toolkit script finished running which must occur before NiFi is even started, that JVM is gone. So the memory allocation set here are short lived and will not impact available memory for your NiFi. Hope this helps, Matt
... View more
01-22-2020
06:06 AM
SHORT Cloudera has broken zookeeper 3.4.5-cdh5.4.0 in several places. Service is working but CLI is dead. No workaround other than rollback. LONG Assign a bounty on this ;-). I have stepped on this mine too and was angry enough to find the reason: Zookeeper checks JLine during ZooKeeperMain.run(). There is a try-catch block that loads a number of classes. Any exception during class loading fails the whole block and JLine support is reported to be disabled. But here is why this happens with CDH 5.4.0: Current opensource Zookeeper-3.4.6 works against jline-0.9.94. Has no such issue. In CDH 5.4 Cloudera has applied the following patch: roman@node4:$ diff zookeeper-3.4.5-cdh5.3.3/src/java/main/org/apache/zookeeper/ZooKeeperMain.java zookeeper-3.4.5-cdh5.4.0/src/java/main/org/apache/zookeeper/ZooKeeperMain.java
305,306c305,306
< Class consoleC = Class.forName("jline.ConsoleReader");
< Class completorC =
---
> Class consoleC = Class.forName("jline.ConsoleReader");
> Class completorC =
316,317c316,317
< Method addCompletor = consoleC.getMethod("addCompletor",
< Class.forName("jline.Completor"));
---
> Method addCompletor = consoleC.getMethod("addCompleter",
> Class.forName("jline.console.completer.Completer"));
CDH 5.4 uses jline-2.11.jar for ZooKeeper and it has no jline.ConsoleReader class (from 2.11 it is jline.console.ConsoleReader). Jline 0.9.94 in turn has no jline.console.completer.Completer. So there is incompatibility with any existing JLine. Any Cloudera CDH 5.4 user can run zookeeper-client on his/her cluster and find it does not work. Open-source zookeeper-3.4.6 depends on jline-0.9.94 which has no such patches. Don't know why Cloudera engineers have done such a mine. I see no clean way to fix it with 3.4.5-cdh5.4.0. I stayed with 3.4.5-cdh5.3.3 dependency where I need CLI and have production clusters. It seemed to me both jline-0.9.94.jar and jline.2.11.jar in classpath for zookeeper will fix the problem. But just have found Cloudera made another 'fix' in ZK for CDH 5.4.0, they have renamed org.apache.zookeeper.JLineZNodeCompletor class to org.apache.zookeeper.JLineZNodeCompleter. But here is the code from ZooKeeperMain.java Class<?> completorC = Class.forName("org.apache.zookeeper.JLineZNodeCompletor"); And of course, it means practically it is not possible to start ZK CLI in CDH 5.4.0 proper way. Awful work. 😞
... View more
01-12-2020
11:28 PM
@Lewis It is not an option available to me, I am facing the same error on Dev and QA both. It will be close to impossible to get an approval for the re installation of cloudera manager on both environments.
... View more