Member since
06-23-2017
8
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2843 | 01-03-2019 03:04 PM |
02-10-2023
02:34 AM
To perform a data lookup using MS SQL as the datasource in NiFi, you can use the LookupRecord processor along with the DBCPConnectionPool and ScriptedLookupService. Here is an outline of the steps you can follow: Create a DBCPConnectionPool service: Go to the NiFi UI and click on the gear icon to access the Controller Services. Click on the "+" button and select "DBCPConnection Pool" Configure the connection pool with the details of your MS SQL database, such as the driver class name, URL, username, password, etc. Create a ScriptedLookupService: In the same Controller Services screen, click on the "+" button and select "ScriptedLookupService". In the ScriptedLookupService, you need to write a Javascript code that will be used to perform the lookup. You can refer to the example code provided in the forum topic How to get DBCP service inside ScriptedLookupService. Configure the LookupRecord processor: Add a LookupRecord processor to your data flow. In the Properties tab, set the "Lookup Service" to the ScriptedLookupService you created in step 2. In the same Properties tab, set the "Connection Pool" to the DBCPConnectionPool you created in step 1. Connect the LookupRecord processor to your data flow, with the input flow file containing the data that you want to enrich.
... View more
06-05-2022
06:38 PM
Excellent solution, works perfectly and the best part you can copy and paste to any flow. Thanks! @ozw1z5rd1
... View more
01-03-2019
03:04 PM
I figured out that if I use regex_replace, I can work around the data issue. select
xpath_string(regex_replace(xml_column, '\\?\<', ''), '//Root [@name="OfferId"]/@value')
from table;
... View more
07-19-2018
10:22 PM
This I believe, is in correlation to the following bug Link which was corrected in specific versions of Tez. If you are running Interactive query HDInsight in Azure, they are still using Tez .7.0 (not fixed). I did however, find this blog post that explains how to at least stop the queries from running. As far as I can tell, Tez will still show them as running, but it does free up your resources again. Edit: the follow up link is probably helpful 🙂 Fix
... View more