Member since
06-26-2015
515
Posts
138
Kudos Received
114
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2271 | 09-20-2022 03:33 PM | |
| 6048 | 09-19-2022 04:47 PM | |
| 3259 | 09-11-2022 05:01 PM | |
| 3734 | 09-06-2022 02:23 PM | |
| 5823 | 09-06-2022 04:30 AM |
03-22-2022
03:43 AM
Thanks for the explanation, @dmorvant .
From what you described, it seems to me that the ideal solution for this would be to use NiFi Parameters.
You could have 3 different Parameter Contexts, each of them with a different value for database.name.
Each one of the 3 Process Groups would use one of the Parameter Contexts. Then, when you create your DBCPConnectionPool controller service, you can specifyy the Database Connection URL parameter as, for example:
jdbc:postgresql://cdp.100.20.115.239.nip.io:5432/#{database.name}
With this, the controller service of each Process Group would connect to the database defined in the Parameter Context of that group. You don't even need a DBCPConnectionPoolLookup in this case.
Would that address you need?
What the NiFi version that you are using?
André
-- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-22-2022
03:25 AM
@AzfarB , Which other roles are running on the same node as this Solr role? Could you provide the output of a "sar -r 5 12" command? Cheers, André
... View more
03-21-2022
11:43 PM
1 Kudo
@pandu2022 , Have you updated the properties below for *all* the Impala service roles (ID, catalog and statestore)? principal: When using a load-balancer this should be of the form impala_test/<LB_fqdn>@<REALM>. If not using a LB, this should be impala_test/<host_fqdn>@<REALM>. be_principal: This is only necessary when a LB is being used and should be of the form impala_test/<host_fqdn>@<REALM>. Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-21-2022
11:22 PM
1 Kudo
@pandu2022 , Understood. Just keep in mind that not knowing any details makes it more difficult to help. What's your Kerberos KDC? (AD, MIT, FreeIPA, or other) When you changed the Impala principal name, did you create the new principal in Kerberos? You need to make sure that all the principals "impala_test/<host>" exist in the KDC for all the hosts. Did you regenerate the keytabs for all the Impala Daemons, Catalog and State Store with the new principal name? Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-21-2022
11:16 PM
Hi @dmorvant , The QueryDatabaseTableRecord processor is probably not the processor that you need. The main objective of this processor it to periodically query the same table looking for recent records added to the table that need to be ingested. Rather than triggered by an incoming flowfile, this processor is typically scheduled to run at regular intervals. It's also a stateful processor, which records the last record seen in the configured table. If you were able to change the configured database where it was looking for the table, it could confuse the processor and generate unexpected results. I believe that the processor that you're looking for is the ExecuteSQLRecord. This processor receives a flowfile with a SQL query and executes the query against the configured DBCP. If using DBCPConnectionPoolLookup, you can add a "database.name" attribute to the incoming flowfile to do what you want. If this is not what you need, it would help to understand what's your end goal so that we can thing of other ways to do what you need. Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-21-2022
08:52 PM
Did you change this configuration in Cloudera Manager? Can you share screenshots of your configuration? André
... View more
03-21-2022
03:23 PM
@Katja , I'm not totally sure but it may be possible that bind variables are accepted inside WITH clauses. Would you be able to try the alternative below? With variable1 as
(
SELECT orders.order_id, orders.order_date
FROM quest_stage.orders orders
)
Select *
from variable1
WHERE order_date = (CAST(? AS TIMESTAMP)) Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-21-2022
03:19 PM
@asomrak , Please have a look at the individual nodes and look for a directory called /tmp/scm_prepare_node.* Inside that directory there should be a log file called scm_prepare_node.log, which could tell you more about the error. If you can't figure out the root cause from that log's content, please share the file here so that we can take a look. Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-21-2022
03:12 PM
1 Kudo
@RobinRo , Yes, it does support other names. "cdsw" is just an example, but you can replace it with your preferred string. Cheers, André -- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
03-21-2022
03:07 PM
@Tra , Could you please explains what your configuration is and which steps you executed to get to this point? Cheers, André
... View more