question Nifi custom property for SelectHiveQL Database Connection URL in Archives of Support Questions (Read Only)

Nifi custom property for SelectHiveQL Database Connection URL

alarsen — Sun, 18 Aug 2019 00:31:05 GMT

We are trying making our processor using custom properties for environment independent.

In this case we are using a SelectHiveQL processor which is working if the Database Connection URL is hardcode.

If the Database Connection URL is a reference as done with Kerberos Principal and Kerberos Keytab it return a error, when running SelectHiveQL.

The pictures show the two scenarios.

Our hive url is as follow: jdbc:hive2://HOST:10000/default;principal=hive/_HOST@HOST.com

I have tried added the propery

How is it possible to do a custom property reference in Database Connection URL?

Re: Nifi custom property for SelectHiveQL Database Connection URL

mburgess — Mon, 23 Oct 2017 22:53:36 GMT

Edited due to Matt Clarke's comment below:

Support for Expression Language in SelectHiveQL properties was added to NiFi 1.3.0 under NIFI-3867. It is also available in HDF 3.0.x.

One (less-than-ideal) workaround is to set the property to something invalid but identifiable (like "@HIVE_URL@") and have a script to replace that value in the template before uploading to a particular environment.

Re: Nifi custom property for SelectHiveQL Database Connection URL

MattWho — Mon, 23 Oct 2017 23:06:37 GMT

@Matt Burgess @Anders Boje

NiFi-3867 is included in HDF 3.0.0 baseline:

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_release-notes/content/ch_hdf_relnotes.html

While HDF 3.0.0 is based off Apache NiFi 1.2.0, it included many additional fixes/enhancements that later went in to Apache NiFi 1.3.0. The above release notes show all additional Apache Jiras that were included on top of everything Apache NiFi 1.2.0 that makes up the HDF 3.0.0 release.

Thanks,

Matt

Re: Nifi custom property for SelectHiveQL Database Connection URL

alarsen — Tue, 24 Oct 2017 00:07:11 GMT

Thx for answer. We are using a older version today. As showed only some of the fields is supported for Expression Language.

HDF Version 2.1.1 - Powered by Apache NiFi - Version 1.1.0.2.1.2.0-10