Member since
01-10-2020
52
Posts
1
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1488 | 08-25-2020 09:30 AM | |
2334 | 08-21-2020 08:31 PM | |
1789 | 05-28-2020 03:53 PM |
08-21-2020
03:26 PM
here is the error message resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdf/current/nifi-toolkit/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /grid/var/lib/nifi/conf/flow.xml.gz -s '[PROTECTED]' -l /usr/hdf/current/nifi/conf/login-identity-providers.xml -a /usr/hdf/current/nifi/conf/authorizers.xml -m -e '[PROTECTED]' -p '[PROTECTED]'' returned 255. 2020/08/21 09:07:10 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties 2020/08/21 09:07:10 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten 2020/08/21 09:07:10 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of login-identity-providers.xml 2020/08/21 09:07:10 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source login-identity-providers.xml and destination login-identity-providers.xml are identical [/usr/hdf/current/nifi/conf/login-identity-providers.xml] so the original will be overwritten 2020/08/21 09:07:10 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of authorizers.xml 2020/08/21 09:07:10 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source authorizers.xml and destination authorizers.xml are identical [/usr/hdf/current/nifi/conf/authorizers.xml] so the original will be overwritten 2020/08/21 09:07:10 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of flow.xml.gz 2020/08/21 09:07:10 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source flow.xml.gz and destination flow.xml.gz are identical [/grid/var/lib/nifi/conf/flow.xml.gz] so the original will be overwritten
... View more
08-21-2020
02:37 PM
NIFI flow.xml.gz is getting cleared out during NIFI restart HDF 3.4 NIFI 1.9 no configs have been changes but when restarted the existing flwo.xml.gz of size 70MB is getting overwritten. though restored from a backup and placed back in, again after restarting it's getting cleared. @MattWho please advice
... View more
Labels:
- Labels:
-
Apache NiFi
07-20-2020
03:04 PM
this is multi-az AWS cluster 3 nodes on zone 1 and 3 on zone 2
... View more
07-20-2020
03:03 PM
@MattWho 2020-07-20 17:15:58,357 INFO [Process Cluster Protocol Request-2] o.a.n.c.c.node.NodeClusterCoordinator Received Connection Request from qa-nifi-node-blue-02.abc.com:9091; responding with my DataFlow 2020-07-20 17:15:58,388 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.node.NodeClusterCoordinator Event Reported for qa-nifi-node-blue-02.abc.com:9091 -- Received first heartbeat from connecting node. Node connected. 2020-07-20 17:16:07,332 INFO [Process Cluster Protocol Request-2] o.a.n.c.c.node.NodeClusterCoordinator Status of qa-nifi-node-blue-02.abc.com:9091 changed from NodeConnectionStatus[nodeId=qa-nifi-node-blue-02.abc.com:9091, state=CONNECTED, updateId=21] to NodeConnectionStatus[nodeId=qa-nifi-node-blue-02.abc.com:9091, state=CONNECTING, updateId=22] 2020-07-20 17:16:09,000 WARN [Process Cluster Protocol Request-2] o.a.n.c.p.impl.SocketProtocolListener Failed processing protocol message from ip-10-175-123-222.us-west-2.compute.internal due to org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling protocol message in response to message type: CONNECTION_REQUEST due to java.net.SocketException: Broken pipe (Write failed) org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling protocol message in response to message type: CONNECTION_REQUEST due to java.net.SocketException: Broken pipe (Write failed) at org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:184) at org.apache.nifi.cluster.protocol.jaxb.JaxbProtocolContext$1.marshal(JaxbProtocolContext.java:86) at org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:182) 2020-07-20 17:16:09,009 INFO [Process Cluster Protocol Request-23] o.a.n.c.p.impl.SocketProtocolListener Finished processing request b20868cb-d4ba-41c4-90ba-07cddda92131 (type=HEARTBEAT, length=3465 bytes) from qa-nifi-node-blue-02.abc.com:9091 in 95 millis 2020-07-20 17:16:41,298 INFO [Process Cluster Protocol Request-24] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 6766e1e4-5181-48aa-9d05-e9c93617afcf (type=CLUSTER_WORKLOAD_REQUEST, length=85 bytes) from ip-10-175-123-222.us-west-2.compute.internal in 133 millis 2020-07-20 17:16:43,489 INFO [Process Cluster Protocol Request-25] o.a.n.c.p.impl.SocketProtocolLis stopped all nodes, started 1 node [CONNECTED, PRIMARY, COORDINATOR], then started node by node, and cluster came up.
... View more
07-20-2020
02:49 PM
NIFI 1.9
HDF 3.4.1
6 node cluster 16 core 64gb memory
5 zookeeper nodes 2 cores 8gb memory
restarted the new cluster, nodes taking very long to join the cluster. unless I bring node by node up, clustering is not happening.
nifi.cluster.node.connection.timeout 120 sec
nifi.cluster.node.max.concurrent.requests 400
nifi.cluster.node.protocol.max.threads 100
nifi.cluster.node.protocol.threads 50
nifi.cluster.node.read.timeout 120s
nifi.zookeeper.connect.timeout 60s
nifi.zookeeper.session.timeout 60s
nifi.cluster.load.balance.comms.timeout 60s
nifi.cluster.node.connection.timeout 120s
nifi.cluster.node.read.timeout 120s
memory 40GB
@MattWho could you advise here why the nodes are not joining the cluster
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
07-10-2020
12:49 AM
Thanks on hdf 3.4.1. cleared the users from the auth.xml and users.xml. back to normal operations. Appreciate your solution.
... View more
07-02-2020
12:54 PM
noticed below error from users.log but nothing changed from nifi configs but started noticing this error nifi-user.log:2020-07-02 12:16:13,857 INFO [NiFi Web Server-343673] o.a.n.w.a.c.ResourceNotFoundExceptionMapper org.apache.nifi.web.ResourceNotFoundException: Unable to find user with id '938eb61e-bbc4-383a-8475-aee80541b5a5'.. Returning Not Found response. @MattWho
... View more
Labels:
- Labels:
-
Apache NiFi
05-28-2020
03:53 PM
I am able to resolve the issue. Update nifi authorizers to use ranger ldap rather than file based.
... View more
05-27-2020
12:19 PM
@MattWho please advice
... View more