Member since
07-27-2015
92
Posts
4
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4624 | 12-15-2019 07:05 PM |
11-01-2019
05:23 PM
@Former Member Thanks, I solved this issue when i follow your point. Paul
... View more
11-01-2019
12:26 AM
Hello, My new CDF cluster include 3 nifi nodes. I enable SSL in nifi . but there is not SSL on LDAP. I don't what is the reason got the error: 2019-11-01 13:42:28,489 WARN org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator: Failed to replicate request GET /nifi-api/flow/current-user to xx-cmf02.beta1.nn/10.202.252.92:8080 due to java.net.ConnectException: Failed to connect to xx-cmf02.beta1.nn/10.202.252.92:8080
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:242)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:160)
......
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) The behavior is very strange that one node access else node 8080 port but i did not to config it. So, what did i miss it? Thank you for your advice first. Paul
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera Data Flow
11-01-2019
12:13 AM
@Former Member Thank you, I'm doing what you point me to do.
... View more
10-31-2019
02:23 AM
Hello!
Urgent problem I'm working on Cloudera Flow Management 1.0.1 and to evaluate the feasibility to move HDF to CDF. I'm meeting a problem that is how to config the Multi-Tenant Authorization with kerberos/LDAP in cloudera manager. Could you please help me for the following questions ? For HDF, there is apache ranger can config and implement the Multi-Tenant Authorization. It can be config in Ranger Admin GUI. How can I do the Multi-Tenant Authorization like HDF with CFM?
Thanks,
Paul
... View more
Labels:
- Labels:
-
Cloudera Data Flow
11-06-2016
07:09 AM
@mclark Thanks for your detailed answers. Paul
... View more
10-27-2016
01:29 AM
@mclark Thanks for your detailed answers. For us, we expect there are like 2 flow files will be routed to a single Node, and other 2 flow files be routed to another Node. For load balance: This is perfect for handling the execute sql that select 100W records. To load balance the SQl flow file , the constant of 100 flow files of RPG is so large. My question 1. how to configure the 100 files to 2 files etc in one node that you mentioned. 2. how many RPG is recommend in one cluster with 3 node? 3. Is there another method to implement load balance of ExecuteSQL ? Thanks. Paul
... View more
10-26-2016
10:39 AM
@mclark Thanks for your reply. Yes, I follow your picture of nifi data flow. got the load balance smartly of data, But It seem your picture be implement in root flow. Because input port => more_table_ino... must be placed to root process group. the picture will very confused if to do more table ingest . Because the best way for me is every department or every business one process group. So could you give some advise to avoid the issue? Thanks, Paul
... View more
10-18-2016
01:33 AM
@mclark Every node distribute run different SQL that meet my requirement. I do this nifi flow picture. the GenerateTableFetch execute on primary node to keep the sql is not replicate. I try to distribute the sql statements then send it to a remote progress group, But the processor on one node got all sql statements in the queued. So I cant got effect that distribute sql statements to every node. So, could you give me some advice to implement execute different sql distribute? Thanks
... View more
10-17-2016
08:03 AM
Hi This is very emergency for me.Why should choice the cluster mode If I cannot distribute sql statement to every node in the cluster? Who can give me any idea? Thanks in advance!
... View more
10-16-2016
02:57 AM
1 Kudo
Hi:
From google :
If you are using NiFi 1.0 you can use the GenerateTableFetch processor. It allows you to choose the "page" (aka partition) size, and will generate SQL statements, each of which will grab one "page" of data. You can route those into ExecuteSQL and it will retrieve smaller sets of results at a time. If you have a NiFi cluster, you can route GenerateTableFetch into a Remote Process Group that points at an Input Port on the same cluster, (which will distribute the SQL statements across the cluster), then the Input Port can be connected to the ExecuteSQL. This allows you to fetch rows from a table in parallel
I have three nodes on my nifi cluster, I follow the post, put GenerateTableFetch on primary node(test01) to execute, and then send to remote process group on the same cluster. then output port to ExecuteSQL.
But the actual behavior is ExecuteSQL was execute just in one node (test02, or test 03, or tes01).
My question is how to fetch rows from a table in parallel(test02, and test 03, and tes01) .
Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
- « Previous
-
- 1
- 2
- Next »