Member since
01-26-2022
82
Posts
2
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
701 | 09-08-2022 04:02 AM | |
1664 | 07-27-2022 03:29 AM | |
1227 | 06-28-2022 09:43 PM | |
1228 | 06-27-2022 08:42 AM | |
12733 | 06-16-2022 10:29 PM |
06-16-2022
08:40 AM
No sir, I have not done that.
... View more
06-16-2022
07:03 AM
Thanks sir. I'm not setting a secured one. Infact, i disable selinux.
... View more
06-16-2022
04:11 AM
I was trying to setup Nifi cluster. All these hosts pings one another. I have tried everything i could on the web, but no success. This is my configuration for master node: Thanks. MASTERNODE: nifi.state.management.embedded.zookeeper.start true nifi.zookeeper.connect.string =masternode:2181,workernode02:2181,workernode03:2181 nifi.cluster.is.node=true nifi.cluster.node.address=masternode nifi.cluster.node.protocol.port=9991 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file= nifi.cluster.protocol.is.secure=false nifi.cluster.load.balance.host=masternode nifi.cluster.load.balance.port=6342 nifi.remote.input.host=masternode nifi.remote.input.secure=false nifi.remote.input.socket.port=10000 nifi.remote.input.http.enabled=true nifi.remote.input.http.transaction.ttl=30 sec nifi.web.http.host=masternode nifi.web.http.port=9443 ------------------------------------------------------ WORKERNODE 02: nifi.state.management.embedded.zookeeper.start=true nifi.zookeeper.connect.string=masternode:2181,workernode02:2181,workernode03:2181 nifi.cluster.is.node=true nifi.cluster.node.address=workernode02 nifi.cluster.node.protocol.port=9991 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file = nifi.cluster.protocol.is.secure=false nifi.cluster.load.balance.host=workernode02 nifi.cluster.load.balance.port=6342 nifi.remote.input.host=workernode02 nifi.remote.input.secure=false nifi.remote.input.socket.port=10000 nifi.remote.input.http.enabled=true nifi.remote.input.http.transaction.ttl=30 sec nifi.web.http.host=workernode02 nifi.web.http.port=9443 ----------------------------------------- WORKERNODE 03: same as above except the host. ------------------------------------------------------------ state-maanagement.xml: <cluster-provider> <id>zk-provider</id> <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class> <property name="Connect String">masternode:2181,workernode02:2181,workernode03:2181</property> <property name="Root Node">/nifi</property> <property name="Session Timeout">10 seconds</property> <property name="Access Control">Open</property> </cluster-provider> ------------------------------------------------------- zookeeper.properties: server.1=masternode:2888:3888;2181 server.2=workernode02:2888:3888;2181 server.3=workernode03:2888:3888;2181 ------------------------------------------ ERROR: 2022-06-16 11:50:03,954 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again 2022-06-16 11:50:03,954 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered 2022-06-16 11:50:05,119 WARN [Heartbeat Monitor Thread-1] o.a.n.c.l.e.CuratorLeaderElectionManager Unable to determine leader for role 'Cluster Coordinator'; returning null org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /nifi/leaders/Cluster Coordinator at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2707) at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:242) at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:231) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:228) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:219) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:41) at org.apache.curator.framework.recipes.locks.LockInternals.getSortedChildren(LockInternals.java:154) at org.apache.curator.framework.recipes.locks.LockInternals.getParticipantNodes(LockInternals.java:134) at org.apache.curator.framework.recipes.locks.InterProcessMutex.getParticipantNodes(InterProcessMutex.java:170) at org.apache.curator.framework.recipes.leader.LeaderSelector.getLeader(LeaderSelector.java:337) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:281) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.verifyLeader(CuratorLeaderElectionManager.java:571) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.isLeader(CuratorLeaderElectionManager.java:525) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$LeaderRole.isLeader(CuratorLeaderElectionManager.java:466) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.isLeader(CuratorLeaderElectionManager.java:262) at org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.isActiveClusterCoordinator(NodeClusterCoordinator.java:824) at org.apache.nifi.cluster.coordination.heartbeat.AbstractHeartbeatMonitor.monitorHeartbeats(AbstractHeartbeatMonitor.java:132) at org.apache.nifi.cluster.coordination.heartbeat.AbstractHeartbeatMonitor$1.run(AbstractHeartbeatMonitor.java:84) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2022-06-16 11:50:11,732 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive 2022-06-16 11:50:11,733 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Archive cleanup completed for container default; will now allow writing to this container. Bytes used = 10.06 GB, bytes free = 7.63 GB, capacity = 17.7 GB
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
05-18-2022
06:44 PM
Thanks so much. But i have no control over the system generated json.
... View more
05-18-2022
06:25 AM
Hi, I would like to convert this json to csv. But it failed because of the special characters in the json content. Below is the json i would like to convert to csv. So the responseMessage value is where the problem is & its auto-generated in that format. Thank you. INPUT { "responseMessage": "hdhdhdhdh:dgdgdgdg "fdfdf" bbbbbbb", "result": "failed", "code": "34" } DESIRED OUTPUT { "responseMessage": "hdhdhdhdh dgdgdgdg fdfdf bbbbbbb", "result": "failed", "code": "34" }
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
05-10-2022
01:00 PM
Hi SAMSAL, Thank you so much. I got it working using your method at the moment. Please, can you explain this RE to me or point me to a suitable material where i can learn this particular Regular Expression. I really appreciate.
... View more
05-10-2022
05:49 AM
Hi, Please, is there a way to reference a key part of JSON file that can come in different structures. For example i want to get the value of "requestUid" from these different JSON structures just with a single processor. So, irrespective of JSON formats or structures sent, i would want to get the value of requestUid. Thanks. For example: 1) { "responseMetaData": { "type": "ACK", "userId": "SYSTEM", "requestUid": "1742963e-f02f-4db6-88e0-6af8c4d4825b",, "systemId": "CIF", "serviceId": "FR_CUS" }, "responseDataObject": { "result": "Accepted", "responseCode": "SUC", "responseMessage": "", "execTime": "1216" } } 2) { "responseMetaData": { "type": "ACK", "sendTime": "23/03/2019 09:16:25.305277", "timeZone": "4", "userId": "SYSTEM", "branchCompanyId": "784100", "branchLocationId": "784101", "systemId": "CIF", "serviceId": "FR_CUS" }, "responseDataObject": { "result": "Accepted", "responseCode": "SUC", "requestUid": "yywhhnan56289200200", "execTime": "1216" } }
... View more
Labels:
- Labels:
-
Apache NiFi
05-07-2022
02:12 AM
The only thing i guessed you can do is to increase "Run schedule" time so the processor will not be looking for what to do all the time but will run at the Run schedule time interval you specified.
... View more
05-07-2022
02:04 AM
If you do not mind, you can add MonitorActivity processor to the output end of the processor. This would generate two flow files(inactivity & activity restored) whenever your processor is down. In addition, connect the output end of MonitorActivity processor to PutEmail processor to configure an email Alert that would notify you when the processor is down & when activity has been restored. I hope this helps.
... View more