Member since
10-17-2022
3
Posts
0
Kudos Received
0
Solutions
11-16-2022
09:40 PM
Hi Team, We are Using Apache Nifi to Publish events to PubsubLite using (PublishGCPubSubLite 1.17.0) Nifi processor. GCP SA json files contains below field: { "type": "service_account", "project_id": "", "private_key_id": "", "private_key": "", "client_email": "", "client_id": "", "auth_uri": "", "token_uri": "", "auth_provider_x509_cert_url": "", "client_x509_cert_url": "" } We tried using type:"service_account" and it is getting failed to communicate. and we are getting the below error, PublishGCPubSubLite[id=7f9baf80-0184-1000-ffff-ffff8ec321b3] Failed to create Google Cloud PubsubLite Publisher: com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. - Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. From Nifi server we tested the connectivity and credential, it is working: ./google-cloud-sdk/bin/gcloud auth login --cred-file gcp.json Authenticated with service account credentials for: [****@****.iam.gserviceaccount.com]. Your current project is [None]. You can change this setting by running: $ gcloud config set project PROJECT_ID Please suggest how to make connection to PubsubLite on Apache Nifi
... View more
Labels:
- Labels:
-
Apache NiFi
11-16-2022
06:27 AM
I am facing similar issues. Please help me out if anyone has any solution to this.
... View more
10-17-2022
10:33 AM
Please help me to complete Nifi cluster setup. I can see Nifi is running on server, but GUI is not coming. **Java version:** openjdk version "1.8.0_302" OpenJDK Runtime Environment (build 1.8.0_302-b08) OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode) **NiFi version:** nifi-1.17.0 **Nifi.properties:** nifi.state.management.embedded.zookeeper.start=true nifi.remote.input.host=Svxxx.xyz.com nifi.remote.input.secure=true nifi.remote.input.socket.port=10443 nifi.remote.input.http.enabled=true nifi.web.https.host=Svxxx.xyz.com nifi.web.https.port=9443 nifi.web.proxy.host=localhost:9443,Svxxx.xyz.com:9443 nifi.sensitive.props.key=propkeywith12chars nifi.cluster.is.node=true nifi.cluster.node.address=Svxxx.xyz.com nifi.cluster.node.protocol.port=11443 nifi.cluster.load.balance.host=Svxxx.xyz.com nifi.cluster.load.balance.port=6342 nifi.zookeeper.connect.string=Svxxx.xyz.com:2181,Svxxx.xyz.com:2181,Svxxx.xyz.com:2181 zookeeper. properties: server.1=Svxxx.xyz.com:2888:3888;2181 server.2=Svxxx.xyz.com:2888:3888;2181 server.3=Svxxx.xyz.com:2888:3888;2181 **Changes made in state-management.xml:** <cluster-provider> <id>zk-provider</id> <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class> <property name="Connect String">Svxxx.xyz.com:2181,Svxxx.xyz.com:2181,Svxxx.xyz.com:2181</property> <property name="Root Node">/nifi</property> <property name="Session Timeout">10 seconds</property> <property name="Access Control">Open</property> </cluster-provider> **Firewall status:** disabled Also created SSL certificate using toolkit, and put those on respective servers. Replaced truststore.jks and keystore.jks also accordingly. **nifi-app.log:** 2022-10-17 22:00:05,669 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again 2022-10-17 22:00:05,670 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered 2022-10-17 22:00:11,403 WARN [Heartbeat Monitor Thread-1] o.a.n.c.l.e.CuratorLeaderElectionManager Unable to determine leader for role 'Cluster Coordinator'; returning null org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /nifi/leaders/Cluster Coordinator at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2480) at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:242) at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:231) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:228) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:219) at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:41) at org.apache.curator.framework.recipes.locks.LockInternals.getSortedChildren(LockInternals.java:154) at org.apache.curator.framework.recipes.locks.LockInternals.getParticipantNodes(LockInternals.java:134) at org.apache.curator.framework.recipes.locks.InterProcessMutex.getParticipantNodes(InterProcessMutex.java:170) at org.apache.curator.framework.recipes.leader.LeaderSelector.getLeader(LeaderSelector.java:337) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:281) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.verifyLeader(CuratorLeaderElectionManager.java:571) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.isLeader(CuratorLeaderElectionManager.java:525) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$LeaderRole.isLeader(CuratorLeaderElectionManager.java:466) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.isLeader(CuratorLeaderElectionManager.java:262) at org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.isActiveClusterCoordinator(NodeClusterCoordinator.java:824) at org.apache.nifi.cluster.coordination.heartbeat.AbstractHeartbeatMonitor.monitorHeartbeats(AbstractHeartbeatMonitor.java:132) at org.apache.nifi.cluster.coordination.heartbeat.AbstractHeartbeatMonitor$1.run(AbstractHeartbeatMonitor.java:84) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2022-10-17 22:00:12,371 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive 2022-10-17 22:00:12,371 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Archive cleanup completed for container default; will now allow writing to this container. Bytes used = 10.53 GB, bytes free = 26.46 GB, capacity = 36.99 GB 2022-10-17 22:00:14,203 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@22e813fc checkpointed with 1 Records and 0 Swap Files in 5 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID 3 2022-10-17 22:00:18,823 WARN [main] o.a.n.c.l.e.CuratorLeaderElectionManager Unable to determine leader for role 'Cluster Coordinator'; returning null org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /nifi/leaders/Cluster Coordinator
... View more
Labels:
- Labels:
-
NiFi Registry