How to access Spark Thrift server via jdbc from apache nifi?

hegdemahendra — Mon, 17 Jul 2023 04:48:39 GMT

Hello Experts,

I need to query delta table from apache nifi.
Since Nifi cant connect delta table directly, I am trying it through Spark Thrift Server.

I have used 'Hive3ConnectionPool' controller service with 'SelectHive3QL' processor and used below jdbc url and it worked for me.

jdbc:hive2://localhost:10002

But I dont know whether using jdbc:hive2//<host>:<port> is right approach or should I use jdbc:spark//<host>:<port> url?

Will there be any performance difference between jdbc:hive2 & jdbc:spark jdbc url's with respect to performance or any other imp thing?

I tried jdbc:spark//<host>:<port> url with 'DBCPConnectionPool 1.16.3' cntroller service but could not succeed with DatabricksJDBC jar.

Any suggestion would be appreciated

Thanks

Mahendra

Re: How to access Spark Thrift server via jdbc from apache nifi?

hegdemahendra — Wed, 19 Jul 2023 03:46:53 GMT

Old version of Databricks jdbc jar has the simba driver class, with that I am able to use jdbc:spark:// connection string.
But jdbc:hive2:// performance looks better with thrift for me

question Re: How to access Spark Thrift server via jdbc from apache nifi? in Support Questions

How to access Spark Thrift server via jdbc from apache nifi?

Re: How to access Spark Thrift server via jdbc from apache nifi?