Hello Experts,   I am connecting Spark Thrift Server from Nifi through JDBC to run sql select query against delta tables. Normal select query is working fine. But now I want to implement pagination for my sql query for large tables.   I tried using OFFSET and LIMIT x,y keywords etc but both of them did not work with STS JDBC. Is there anyway to achieve pagination with thrift server jdbc sql? I am not sure if offset and/or limit keywords are not available or not working due to some version. Could you please suggest me on this?   Thanks Mahendra ... View more

Hello experts,   We have setup our nifi to consume message from kafka broker which has SSL as security protocol.   It worked well with pre-prod brokers, but facing below error in prod broker. (we have configured ssl context service with respective ssl certs)   Preprod nifi <-> Preprod kafka : working Prod nifi <-> Prod kafka : Not  working   telnet is working from Prod nifi to Prod kafka broker.   2023-07-17 15:30:58,814 WARN [Timer-Driven Process Thread-190] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-ProdOnHandR3NiFiConsumer-28, groupId=ProdOnHandR3NiFiConsumer] Connection to node -3 (kafka3.xxx.pivotal.emea.kaas.3stripes.net/xxx.128.xx.150:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. 2023-07-17 15:30:58,814 WARN [Timer-Driven Process Thread-190] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-ProdOnHandR3NiFiConsumer-28, groupId=ProdOnHandR3NiFiConsumer] Bootstrap broker kafka3.xxx.pivotal.emea.kaas.3stripes.net:9093 (id: -3 rack: null) disconnected 2023-07-17 15:30:59,816 WARN [Timer-Driven Process Thread-104] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-ProdSpmSeasonNiFiConsumer-26, groupId=ProdSpmSeasonNiFiConsumer] Connection to node -8 (kafka8.xxx.pivotal.emea.kaas.3stripes.net/xx.xxx.5.199:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. 2023-07-17 15:30:59,816 WARN [Timer-Driven Process Thread-104] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-ProdSpmSeasonNiFiConsumer-26, groupId=ProdSpmSeasonNiFiConsumer] Bootstrap broker kafka8.xxx.pivotal.emea.kaas.3stripes.net:9093 (id: -8 rack: null) disconnected 2023-07-17 15:31:00,819 WARN [Timer-Driven Process Thread-127] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-ProdSpmModelNiFiConsumer-25, groupId=ProdSpmModelNiFiConsumer] Connection to node -6 (kafka6.xxx.pivotal.emea.kaas.3stripes.net/xxx.128.xx.13:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.     Any idea what could be the reason for this?   Thanks Mahendra ... View more

Hello Experts,   I need to query delta table from apache nifi. Since Nifi cant connect delta table directly, I am trying it through Spark Thrift Server.   I have used 'Hive3ConnectionPool' controller service with 'SelectHive3QL' processor and used below jdbc url and it worked for me. jdbc:hive2://localhost:10002   But I dont know whether using jdbc:hive2//<host>:<port> is right approach or should I use jdbc:spark//<host>:<port> url? Will there be any performance difference between jdbc:hive2 & jdbc:spark jdbc url's with respect to performance or any other imp thing? I tried jdbc:spark//<host>:<port> url with ' DBCPConnectionPool 1.16.3 ' cntroller service but could not succeed with DatabricksJDBC jar. Any suggestion would be appreciated   Thanks Mahendra ... View more

Hello experts,   One of our client has data stored in delta table(delta lake), I am totally new to delta lake part. Do we have any option in nifi to read data from delta lake or is there any sample 3rd party implementation available for the same for nifi?   Thanks, Mahendra ... View more