NiFi Kubernetes cluster - ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
Hi everybody!
I am struggling trying to create a NiFi Kubernetes cluster out of cetic helm chart.
After installation, I see a working zookeeper cluster (3 instances) and a 2 instances stateful set for NiFi. One instance is correctly elected as cluster coordinator, NiFi frontend shows me a 1 member cluster. The second instance's log keeps on popping messages such as:
2022-07-01 11:52:23,839 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:29,313 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:34,792 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
every five seconds.
I'm attaching the second pod's log and that pod's generated nifi.properties file.
Please help me 😞
nifi.properties:
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Core Properties #
nifi.flow.configuration.file=../data/flow.xml.gz
nifi.flow.configuration.archive.enabled=true
nifi.flow.configuration.archive.dir=../data/archive/
nifi.flow.configuration.archive.max.time=30 days
nifi.flow.configuration.archive.max.storage=500 MB
nifi.flow.configuration.archive.max.count=
nifi.flowcontroller.autoResumeState=true
nifi.flowcontroller.graceful.shutdown.period=10 sec
nifi.flowservice.writedelay.interval=500 ms
nifi.administrative.yield.duration=30 sec
# If a component has no work to do (is "bored"), how long should we wait before checking again for work?
nifi.bored.yield.duration=10 millis
nifi.authorizer.configuration.file=./conf/authorizers.xml
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.templates.directory=../data/templates
nifi.ui.banner.text=
nifi.ui.autorefresh.interval=30 sec
nifi.nar.library.directory=./lib
nifi.nar.library.directory.custom=
nifi.nar.library.autoload.directory=./extensions
nifi.nar.working.directory=./work/nar/
nifi.documentation.working.directory=./work/docs/components
####################
# State Management #
####################
nifi.state.management.configuration.file=./conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
# The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster.
nifi.state.management.provider.cluster=zk-provider
# Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server
nifi.state.management.embedded.zookeeper.start=false
# Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
# H2 Settings
nifi.database.directory=../data/database_repository
nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.directory=../flowfile_repository
nifi.flowfile.repository.partitions=256
nifi.flowfile.repository.checkpoint.interval=2 mins
nifi.flowfile.repository.always.sync=false
nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000
nifi.swap.in.period=5 sec
nifi.swap.in.threads=1
nifi.swap.out.period=5 sec
nifi.swap.out.threads=4
# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=1 MB
nifi.content.claim.max.flow.files=100
nifi.content.repository.directory.default=../content_repository
nifi.content.repository.archive.max.retention.period=3 days
nifi.content.repository.archive.max.usage.percentage=85%
nifi.content.repository.archive.enabled=true
nifi.content.repository.always.sync=false
nifi.content.viewer.url=/nifi-content-viewer/
# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
nifi.provenance.repository.debug.frequency=1_000_000
nifi.provenance.repository.encryption.key.provider.implementation=
nifi.provenance.repository.encryption.key.provider.location=
nifi.provenance.repository.encryption.key.id=
nifi.provenance.repository.encryption.key=
# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=../provenance_repository
nifi.provenance.repository.max.storage.time=10 days
nifi.provenance.repository.max.storage.size=8 GB
nifi.provenance.repository.rollover.time=30 secs
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=2
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
nifi.provenance.repository.journal.count=16
# Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable. Some examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
# but should provide better performance
nifi.provenance.repository.index.shard.size=500 MB
# Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
# Volatile Provenance Respository Properties
nifi.provenance.repository.buffer.size=100000
# Component Status Repository
nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository
nifi.components.status.repository.buffer.size=1440
nifi.components.status.snapshot.frequency=1 min
# Site to Site properties
nifi.remote.input.host=dashboard-nifi-1405-1.dashboard-nifi-1405-headless.default.svc.cluster.local
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10000
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs
# web properties #
nifi.web.war.directory=./lib
nifi.web.proxy.host=dashboard-nifi-1405.default.svc
nifi.web.https.port=8443
nifi.web.http.host=dashboard-nifi-1405-1.dashboard-nifi-1405-headless.default.svc.cluster.local
nifi.web.http.network.interface.default=eth0
nifi.web.https.host=
nifi.web.https.network.interface.default=
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
# nifi.web.proxy.context.path=
# security properties #
nifi.sensitive.props.key=changeMechangeMe
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=NIFI_PBKDF2_AES_GCM_256
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=
nifi.security.keystore=./conf/keystore.p12
nifi.security.keystoreType=PKCS12
nifi.security.keystorePasswd=1ff548a931ff221a92f866e5107c0f73
nifi.security.keyPasswd=1ff548a931ff221a92f866e5107c0f73
nifi.security.truststore=./conf/truststore.p12
nifi.security.truststoreType=PKCS12
nifi.security.truststorePasswd=1ca7ca6eb083ef4ecb8bf0983e2f43c9
nifi.security.user.login.identity.provider=single-user-provider
nifi.security.user.authorizer=single-user-authorizer
nifi.security.needClientAuth=
# Apache Knox SSO Properties #
nifi.security.user.knox.url=
nifi.security.user.knox.publicKey=
nifi.security.user.knox.cookieName=hadoop-jwt
nifi.security.user.knox.audiences=
# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.security.identity.mapping.value.dn=$1@$2
# nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
# nifi.security.identity.mapping.value.kerb=$1@$2
# cluster common properties (all nodes must have same values) #
nifi.cluster.protocol.heartbeat.interval=5 sec
nifi.cluster.protocol.is.secure=true
# cluster node properties (only configure for cluster nodes) #
nifi.cluster.is.node=true
nifi.cluster.node.address=dashboard-nifi-1405-1.dashboard-nifi-1405-headless.default.svc.cluster.local
nifi.cluster.node.protocol.port=6007
nifi.cluster.node.protocol.threads=10
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=5 sec
nifi.cluster.node.read.timeout=5 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=1 mins
nifi.cluster.flow.election.max.candidates=
# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=dashboard-nifi-1405-zookeeper:2181
nifi.zookeeper.connect.timeout=3 secs
nifi.zookeeper.session.timeout=3 secs
nifi.zookeeper.root.node=/nifi
# Zookeeper properties for the authentication scheme used when creating acls on znodes used for cluster management
# Values supported for nifi.zookeeper.auth.type are "default", which will apply world/anyone rights on znodes
# and "sasl" which will give rights to the sasl/kerberos identity used to authenticate the nifi node
# The identity is determined using the value in nifi.kerberos.service.principal and the removeHostFromPrincipal
# and removeRealmFromPrincipal values (which should align with the kerberos.removeHostFromPrincipal and kerberos.removeRealmFromPrincipal
# values configured on the zookeeper server).
nifi.zookeeper.auth.type=
nifi.zookeeper.kerberos.removeHostFromPrincipal=
nifi.zookeeper.kerberos.removeRealmFromPrincipal=
# kerberos #
nifi.kerberos.krb5.file=
# kerberos service principal #
nifi.kerberos.service.principal=
nifi.kerberos.service.keytab.location=
# kerberos spnego principal #
nifi.kerberos.spnego.principal=
nifi.kerberos.spnego.keytab.location=
nifi.kerberos.spnego.authentication.expiration=12 hours
# external properties files for variable registry
# supports a comma delimited list of file locations
nifi.variable.registry.properties=
nifi.web.http.network.interface.lo=lo
--------
app.log from the second node:
2022-07-01 11:51:11,944 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:11,945 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:12,303 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:17,309 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:17,310 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:17,935 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:22,942 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:22,943 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:23,442 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:28,447 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:28,447 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:28,849 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:33,853 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:33,854 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:34,316 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:39,326 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:39,326 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:39,853 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:44,863 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:44,863 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:45,397 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:47,203 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive
2022-07-01 11:51:47,204 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Archive cleanup completed for container default; will now allow writing to this container. Bytes used = 57.01 GB, bytes free = 60.86 GB, capacity = 117.87 GB
2022-07-01 11:51:50,405 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007; will use this address for sending heartbeat messages
2022-07-01 11:51:50,406 INFO [main] o.a.n.c.p.AbstractNodeProtocolSender Cluster Coordinator is located at dashboard-nifi-1348-0.dashboard-nifi-1348-headless.default.svc.cluster.local:6007. Will send Cluster Connection Request to this address
2022-07-01 11:51:50,833 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
2022-07-01 11:51:55,834 INFO [main] o.a.nifi.controller.StandardFlowService Flow controller will load local dataflow and suspend connection handshake until a cluster connection response is received.
2022-07-01 11:51:55,849 INFO [main] o.apache.nifi.controller.FlowController Cluster State changed from Not Clustered to Clustered
2022-07-01 11:51:55,852 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Primary Node; this node is an active participant in the election.
2022-07-01 11:51:55,853 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election.
2022-07-01 11:51:55,862 INFO [main] o.apache.nifi.controller.FlowController Will no longer send heartbeats
2022-07-01 11:51:55,868 INFO [main] o.a.n.wali.SequentialAccessWriteAheadLog Recovering records from Write-Ahead Log at ../flowfile_repository
2022-07-01 11:51:55,873 INFO [main] o.a.n.wali.SequentialAccessWriteAheadLog No Snapshot File to recover from at ../flowfile_repository. Now recovering records from 0 journal files
2022-07-01 11:51:55,875 INFO [main] o.a.n.wali.SequentialAccessWriteAheadLog Successfully recovered 0 records in 6 milliseconds. Now checkpointing to ensure that Write-Ahead Log is in a consistent state
2022-07-01 11:51:55,916 INFO [main] o.a.n.wali.SequentialAccessWriteAheadLog Checkpointed Write-Ahead Log with 0 Records and 0 Swap Files in 41 milliseconds (Stop-the-world time = 37 milliseconds), max Transaction ID -1
2022-07-01 11:51:55,919 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Successfully restored 0 FlowFiles and 0 Swap Files
2022-07-01 11:51:55,951 INFO [main] o.apache.nifi.controller.FlowController Performed initial validation of all components in 1 milliseconds
2022-07-01 11:51:55,952 INFO [main] o.apache.nifi.controller.FlowController Starting 0 processors/ports/funnels
2022-07-01 11:51:55,952 INFO [main] o.apache.nifi.controller.FlowController Started 0 Remote Group Ports transmitting
2022-07-01 11:51:56,016 INFO [main] org.apache.nifi.web.server.JettyServer Flow loaded successfully.
2022-07-01 11:51:56,028 INFO [main] org.apache.nifi.nar.NarAutoLoader Starting NAR Auto-Loader for directory ./extensions ...
2022-07-01 11:51:56,030 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs:
2022-07-01 11:51:56,031 INFO [main] org.apache.nifi.web.server.JettyServer https://10.1.2.67:8443/nifi
2022-07-01 11:51:56,031 INFO [main] org.apache.nifi.web.server.JettyServer https://127.0.0.1:8443/nifi
2022-07-01 11:51:56,070 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2022-07-01 11:51:56,071 INFO [main] org.apache.nifi.NiFi Started Application Controller in 94.386 seconds (94386630200 ns)
2022-07-01 11:51:56,436 WARN [Clustering Tasks Thread-1] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:01,904 WARN [Clustering Tasks Thread-1] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:07,311 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:12,773 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:18,360 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:23,839 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:29,313 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:34,792 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:40,163 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:45,210 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@2d198533 checkpointed with 2 Records and 0 Swap Files in 10 milliseconds (Stop-the-world time = 1 milliseconds, Clear Edit Logs time = 1 millis), max Transaction ID 1
2022-07-01 11:52:45,565 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:47,215 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive
2022-07-01 11:52:47,215 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Archive cleanup completed for container default; will now allow writing to this container. Bytes used = 57.01 GB, bytes free = 60.86 GB, capacity = 117.87 GB
2022-07-01 11:52:50,977 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
2022-07-01 11:52:56,333 WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message
