About tkryazh

tkryazh · ‎04-09-2021

The solution is to use Hive3ConnectionPool from nifi-hive3-nar library Hive 1.* processors, e.g., GenerateTableFetch and SelectHiveQL, work with Hive3 connector.

tkryazh · ‎03-29-2021

I was missing zk connect string at conf/state-management.xml. Now it looks like this: ... <cluster-provider> <id>zk-provider</i> <class> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider </class> <property name="Connect String">my zk connect string</property> <property name="Root Node">/nifi</property> <property name="Session Timeout">30 seconds</property> <property name="Access Control">Open</property> </cluster-provider> ListHDFS works OK now. But GenerateTableFetch still throws: 2021-03-26 18:06:04,432 INFO [Timer-Driven Process Thread-2] org.apache.hive.jdbc.HiveConnection Will try to open client transport with JDBC Uri: jdbc:hive2://hive.server2.instance.uri=hiveserver1_hostname:10000;hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hiveserver1_hostname;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_HOST@DOMAIN/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 2021-03-26 18:06:04,432 INFO [Timer-Driven Process Thread-2] org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC Uri: jdbc:hive2://hive.server2.instance.uri=hiveserver1_hostname:10000;hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hiveserver1_hostname;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_HOST@DOMAIN/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 2021-03-26 18:06:04,432 INFO [Timer-Driven Process Thread-2] o.a.c.f.imps.CuratorFrameworkImpl Starting 2021-03-26 18:06:04,441 INFO [Timer-Driven Process Thread-2-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2021-03-26 18:06:04,446 ERROR [Timer-Driven Process Thread-2] o.a.nifi.dbcp.hive.HiveConnectionPool HiveConnectionPool[id=4cfbf700-6449-3d37-9ab1-263a7528f352] Error getting Hive connection: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper) org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper) Actually, it read Hive server2 uri from zk, but couldn't connect.

tkryazh · ‎03-17-2021

I am integrating Apache NiFi 1.9.2 (secure cluster) with HDP 3.1.4. HDP contains Zookeeper 3.4.6 with SASL auth (Kerberos). NiFi nodes successfully connect to this Zookeeper, sync flow and log heartbeats. Meanwhile, NiFi processors using Zookeeper are not able to connect. GenerateTableFetch throws: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper) ListHDFS fails attempting to write its state to ZooKeeper: o.apache.nifi.processors.hadoop.ListHDFS ListHDFS Failed to retrieve timestamp of last listing from the State Manager. Will not perform listing until this is accomplished. But GetHDFS (which doesn't communicate with zk) works OK. Enable HiveConnector works without errors. Thank you in before for any ideas and support.

Online	Offline
Last Visited	‎04-28-2021 07:23 AM

Member Since	‎03-17-2021 04:12 AM
Last Visited	‎04-28-2021 07:23 AM
Posts	3

Cloudera Community

Re: NiFi processors cannot connect to Zookeeper

Re: NiFi processors cannot connect to Zookeeper

Re: NiFi processors cannot connect to Zookeeper

NiFi processors cannot connect to Zookeeper