Member since
03-10-2017
68
Posts
25
Kudos Received
16
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
77 | 07-26-2022 05:44 AM | |
74 | 07-25-2022 05:49 AM | |
97 | 07-25-2022 03:45 AM | |
92 | 07-25-2022 03:22 AM | |
101 | 06-21-2022 08:22 AM |
07-26-2022
08:36 AM
At moment you can not run stored procedure using ExecuteSQL, however, you can use ExecureScript to do the same with stored procedure wrapped in script with MySQL engine or any other client you can use please refer : https://community.cloudera.com/t5/Support-Questions/Does-ExecuteSQL-processor-allow-to-execute-stored-procedure/m-p/158922 If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you
... View more
07-26-2022
05:44 AM
This is related to Jira https://issues.apache.org/jira/browse/NIFI-9993 , fixed in the latest release 1.16.3 If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan
... View more
07-25-2022
05:49 AM
I mean you can have as many CaptureChangeMySQL for a number of different MySQL DB's , Yes your design is alinged with what I have updated before. If you found this response assisted with your issue, please take a moment and click on "Accept as Solution" below this post. Thank you
... View more
07-25-2022
03:45 AM
Answer to your queries : 1. You have to use the two CaptureChangeMySQL Processors if you wish to connect two different MySQL database services. 2. Since you can get the change records from two CaptureChangeMySQL processors for two different source MySQL DB's then you can insert those records into another MySQL DB using PutSQL processor. Thank you
... View more
07-25-2022
03:22 AM
2 Kudos
In nifi there is no provision to restrict resources based upon process groups or processors, so the same goes for threads. but what you can do with your current request on limiting threads, you can run the processors which are part of one process group with default concurrent task 1 settings if not already done. Thank You.
... View more
07-14-2022
05:09 AM
1 Kudo
ExtractText processor can help you grab needed information from json content and store in the flow file attribute for reuse after InvokeHTTP. Thank You
... View more
06-21-2022
08:22 AM
There is no inbuilt one but you can please refer suggestion posted by one of NiFi Engineers about running Stored procedure at https://stackoverflow.com/questions/36306764/how-to-use-procedure-call-in-apache-nifi-executesql-processor-with-both-in-and If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan
... View more
06-21-2022
08:01 AM
1 Kudo
DBCPConnectionPool provides Database Connection Pooling Service. Connections can be asked from pool and returned after usage by SQL processor which are responsible to run the SQL queries . You can useListDatabaseTables processor along withDBCPConnectionPool controller to get the table metadata If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan
... View more
06-21-2022
07:51 AM
I would check Hive metastore logs to get more details on aborted t ransactions with respect to transaction ids since transactions are aborted at the Hive end and not the NiFi end.
... View more
06-16-2022
06:51 AM
I suggest you have a technical support case opened with the Cloudera Technical Support team so we can dive further. Thank You.
... View more
06-14-2022
05:21 AM
So java.net.BindException: Cannot assign requested address is an error occurred while attempting to bind a socket to provided hostname and port. Typically, the port is in use, or the requested hostname could not be assigned, Are you sure port is free and no issues with the hostname you are using ?
... View more
06-14-2022
05:15 AM
I per MS SQL docs I found this example String connectionUrl = "jdbc:sqlserver://localhost:1433;" + "databaseName=AdventureWorks;integratedSecurity=true;" + "encrypt=true; trustServerCertificate=false;" + "trustStore=storeName;trustStorePassword=storePassword"; Can you please check ? also the URL which you have used works outside of nifi ?
... View more
06-14-2022
02:56 AM
1 Kudo
Details you can only get through UI or API , considering web endpoint is reachable, Seems in this case NiFi service is up but UI is not available or NiFi service is completely down , can be confirm by checking its process status. So you need find way to bring back the NIFI service along with UI in minimalistic load by stopping all flows by setting nifi.flowcontroller.autoResumeState to false and start the service, by doing this nifi will start with all the queued data but it will not do any processing, this can give you the UI but since you have said that are too much data so startup will take time. If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan
... View more
06-14-2022
02:18 AM
How you are passing SSL details in DBCPConnectionPool used by ConvertJSONToSQL? in configured JDBC URL?
... View more
06-13-2022
02:49 AM
Ok, so you got the copy file in NiFi using List/FetchFile then you changed the new directory location where it has to be written using UpdateAttribute then you have used DuplicateFlowFile to create duplicates, till now what is happening since ListFile is running on Primary node (And it should be running on Primary ) rest all downstream flow will be processed/computed on Primary that why you see DuplicateFlowFile creating duplicated on node which is Primary in NiFi Cluster. I do not see any issue here. If you wish to have flow files on All NiFi nodes redistributed after DuplicateFlowFile then you need to set Load Balancing at queue connection on the connection "Round Robin " between DuplicateFlowFile & PutFile, by doing this duplicates flow files will be distributed among nifi nods in round-robin manner one by one and will be written by PutFile back on each nifi nodes where files got distributed after DuplicateFlowFile. Now one question you mention 7 HDFS data nodes, does that means you have NiFi service running as well on the same 7 data nodes ? and having 7 node nifi cluster? in that case, only all 7 nodes will receive each file copy. I see
... View more
06-09-2022
06:21 AM
The error says the Connection pool is not able to find configured driver classpath name into provided database client-jar driver. Please check and make sure configured driver classpath name is present on the provided jar file.
... View more
06-09-2022
06:09 AM
1 Kudo
Please refer answer posted by MattB on UTC time,https://stackoverflow.com/questions/65620632/why-do-executesqlrecord-and-csvrecordsetwriter-updated-the-time-zone-of-datetime. Thank You.
... View more
06-09-2022
06:00 AM
Sorry some confusion here ,so you saying The file is placed by a third party onto an SFTP location on one node? so the file is on SFTP server at this stage but only on node ? and then you fetch the file into NiFi using List/FetchSFTP processor running on Primary node only ? then ? what is the end goal "we want the file to be available on all nodes so that the receiving team can access it". What do you mean by All nodes is these NiFi nodes or SFTP nodes ?
... View more
06-06-2022
08:11 AM
Why do you want to do that ? NiFi cluster is not like HDFS where data will be replicated across nodes NiFi is not a storage system. What NiFi provides as , let's say you have 3 node cluster and you receive 100 files on one node assuming Processor which one is receiving file running on Primary node,now you want to processor these all 100 files in distributed manner using all 3 nodes , then you need to use Load Balancing at queue connection to distribute 100 file on 3 nodes , 33/33/34 files on each node then rest further downstream processing will be done on all 3 nodes. Example flow design flow be, ListSFTP (Primary)-->Connection set to LB at queue tion --> FetchSFTP, in this use case ListSFTP will list the file details on Primary node then FetchSFTP will fetch the file from SFTP server on All 3 nodes in a distributed manner. Thanks
... View more
05-30-2022
05:15 AM
1 Kudo
SSLHandshakeException: PKIX path building failed, which means one or more certificated chain is missing thus SSL handshake is failing, Can you check and confirm if NiFi and NiFi Registry are both signed by same certificate signing Authority ? if not then please make sure you have imported/exchanged complete trusted chain in truststore file used by NiFi and NiFi Registry vice versa
... View more
05-27-2022
06:23 AM
Please make sure SSL certificates are created with the following settings https://docs.cloudera.com/cfm/2.1.4/cfm-security/topics/cfm-security-tls-certificate-requirements-recommendations.html
... View more
05-19-2022
05:18 AM
I believe MergeContent Merging Strategy is set to Defragment ? If it is set to Defragment then MergeContent will wait to receive all flow file part of fragment.count. so what happened to the remaining 2 flow files? which r elationships they move? failure? and terminated on Processor itself assuming as per screenshot . If you wish to merge with reamaning 8 then Merging Strategy needs to be changed but that could be lead to case where all 8 records can not be merged together all the time since there is no correlation, or if you just want to merge all records irrespective of failure or success , then you need to have failure or success both relationships configured for MergeContent . Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
05-16-2022
06:55 AM
1 Kudo
With LB in place, you are reaching NiFi as anonymous when LB has not been configured to pass the user identity to NiFi that what I understand from looking the log snippet.. Please refer this page to check if proxy details are configured correctly https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration
... View more
04-25-2022
05:02 AM
Have you tested ListenFTP without setting SSLContextService? just to make sure supplied username and password is correct?
... View more
04-20-2022
05:01 AM
It seems you misunderstood the usage of UpdateCounter, UpdateCounter is only used for debugging purposes, when you use UpdateCounter in flow then counter option at NiFi--Humburgenr Menu will give you the counter details.. you can not reuse or call these details on any other processor.
... View more
04-13-2022
06:16 AM
There are two aspects of your question " How do you configure a PutMongo NiFi Processor " 1. Connection details: Connection URL, User Auth, Wire encryption (SSL ) 2. Actual Query: make sure query syntax is correct. If you can post an error snipped and snap from the processor config to be able to isolate if this query issue or connection issue. Assume if this is with query-related issue , then you can verify if the same query works from the generic mongo db client? if not then same query will not work from nifi:PutMongo as well. Thank You.
... View more
04-13-2022
05:53 AM
You can receive values from http end point by using LisntHTTP in nifi which receives values through HTTP headers and stored in flow file attribute.
... View more
04-12-2022
07:24 AM
1 Kudo
NiFi as a service alone can not autoscale irrespective of the type public/private cloud provided until it is customized and managed by different automation and scaling applications over the public cloud. But Cloudera offers NiFi as Data flow Expreinace in the public cloud which does support autoscaling Please check https://docs.cloudera.com/dataflow/cloud/release-notes/topics/cdf-whats-new.html for more info. Thank You.
... View more
04-12-2022
07:06 AM
also, you mentioned " Processor doesn't read saved attribute" Which specific processor you are complaining about ?
... View more
04-12-2022
07:04 AM
trying to understand your flow pipeline with respect to your use case explanations, You have two upstream queued for InvokeHTTP , in one using ExtractText you are extracting text , which will be stored in flow file as an attribute then running HTTP Method from InvokeHTTP and configured Attributes to Send for the attributed which is created by ExtractText ? What GenreateFlowFile os doing in this flow? what evaluation is configured with EvaluateJsonPath ? Please provide more details
... View more