Member since
07-27-2015
92
Posts
4
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3008 | 12-15-2019 07:05 PM |
05-30-2020
07:08 PM
1 Kudo
Hello, @Bender Yes, I have got the link [2] from your reply Thank you very much! Paul
... View more
05-26-2020
07:58 PM
HELLO, I remembered that I could download the newest version of CDF about one year ago. For me, I found that I couldn't free download the newest CDF now again. The download page mentioned it is just for customers. So my question is how could we become a customer? What policy would authorize to the customer? then I could download the newest version of CDF again.
... View more
12-17-2019
07:30 PM
Hi @alim Is there any way to work with CaptureChangeMySQL and EnforceOrder in cluster environment for better performance?
... View more
12-15-2019
07:05 PM
Oh, it is network connection issue. it gone away when i add oracle.jdbc.ReadTimeout=120000 and oracle.net.CONNECT_TIMEOUT=10000 to config of DBCPConnectionPool.
... View more
11-12-2019
05:20 AM
I'm sorry. Correct the above description. The behavior is the ExcuteSQL processor will execute very slowly seem hanged. but sometime execute very fast. I believe that is not database issue. So my questions is: 1. what is the impact? is dbcp connections pool issue of config? or is the dbcp connections pool lookup wrong behavior? 2. I'm afraid the impact of this will be magnified when we face to increased database connection pool config to 500+. Is that the right thing to worry about? Could help me to answer my questions? Thanks Paul
... View more
11-09-2019
02:27 AM
Hello, I am working with 3 nodes CFM cluster. the version of CFM is 1.0.1. The below pic shows the steps: the below pic is config of ExecuteSQL: the below pic is lookup service: the below is the config of dbcpconnetionpool: The behavior is the ExcuteSQL processor will execute very slowly seem hanged. but sometime execute very fast. I believe that is database issue. So my questions is: 1. what is the impact? is dbcp connections pool issue of config? or is the dbcp connections pool lookup wrong behavior? 2. I'm afraid the impact of this will be magnified. when increased database connection pool to 500+. Could help me to answer my questions? Thanks Paul
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
11-05-2019
06:05 PM
@MattWho Follow your points i got win. Thank you a lot. Paul
... View more
11-04-2019
06:25 PM
@MattWho Thanks for your detail answers. I almost to get win. Unfortunately, I can not get sync policy between NIFI and NIFI Registry with my ldap account. I must to config my node identity as a user like CN=arch-fndtf04.beta1.fn, OU=NIFI and grant it proxy access policy, so can import the bucket and commit the version. If i remove the user CN=arch-fndtf04.beta1.fn, OU=NIFI or i remove the proxy access policy of it in NIFI Registry. The NIFI GUI will show the "?" on top left corner of process group picture. Could you help me how to avoid the issue? Paul
... View more
11-03-2019
02:43 AM
Update: the below is the log: 2019-11-03 20:06:10,531 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using X509IdentityProvider
2019-11-03 20:06:10,531 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Adding credentials claim to SecurityContext to be authenticated. Credentials extracted by X509IdentityProvider: AuthenticationRequest{username='CN=arch-fndtf04.beta1.fn, OU=NIFI', credentials=[PROTECTED], details=null}
2019-11-03 20:06:10,531 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Credentials already extracted for [org.apache.nifi.registry.web.security.authentication.AuthenticationRequestToken$1@39a29a41], skipping credentials extraction filter using JwtIdentityProvider
2019-11-03 20:06:10,532 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.ResourceAuthorizationFilter Request filter authorization check is not required for this HTTP Method on this resource. Allowing request to proceed. An additional authorization check might be performed downstream of this filter.
2019-11-03 20:06:10,688 INFO [NiFi Registry Web Server-12] o.a.n.r.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos service ticket login not supported by this NiFi Registry. Returning Conflict response.
2019-11-03 20:06:10,691 DEBUG [NiFi Registry Web Server-12] o.a.n.r.w.m.IllegalStateExceptionMapper
java.lang.IllegalStateException: Kerberos service ticket login not supported by this NiFi Registry
at org.apache.nifi.registry.web.api.AccessResource.createAccessTokenUsingKerberosTicket(AccessResource.java:285) ~[classes/:na]
......
2019-11-03 20:06:10,721 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using X509IdentityProvider
2019-11-03 20:06:10,722 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Adding credentials claim to SecurityContext to be authenticated. Credentials extracted by X509IdentityProvider: AuthenticationRequest{username='CN=arch-fndtf04.beta1.fn, OU=NIFI', credentials=[PROTECTED], details=null}
2019-11-03 20:06:10,722 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Credentials already extracted for [org.apache.nifi.registry.web.security.authentication.AuthenticationRequestToken$1@2929ad59], skipping credentials extraction filter using JwtIdentityProvider
2019-11-03 20:06:10,723 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.ResourceAuthorizationFilter Request filter authorization check is not required for this HTTP Method on this resource. Allowing request to proceed. An additional authorization check might be performed downstream of this filter.
2019-11-03 20:06:10,784 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using X509IdentityProvider
2019-11-03 20:06:10,784 DEBUG [NiFi Registry Web Server-17] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using X509IdentityProvider
2019-11-03 20:06:10,784 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Adding credentials claim to SecurityContext to be authenticated. Credentials extracted by X509IdentityProvider: AuthenticationRequest{username='CN=arch-fndtf04.beta1.fn, OU=NIFI', credentials=[PROTECTED], details=null}
2019-11-03 20:06:10,784 DEBUG [NiFi Registry Web Server-17] o.a.n.r.w.s.a.IdentityFilter Adding credentials claim to SecurityContext to be authenticated. Credentials extracted by X509IdentityProvider: AuthenticationRequest{username='CN=arch-fndtf04.beta1.fn, OU=NIFI', credentials=[PROTECTED], details=null}
2019-11-03 20:06:10,784 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.IdentityFilter Credentials already extracted for [org.apache.nifi.registry.web.security.authentication.AuthenticationRequestToken$1@7bf82c3a], skipping credentials extraction filter using JwtIdentityProvider
2019-11-03 20:06:10,784 DEBUG [NiFi Registry Web Server-17] o.a.n.r.w.s.a.IdentityFilter Credentials already extracted for [org.apache.nifi.registry.web.security.authentication.AuthenticationRequestToken$1@69275fb3], skipping credentials extraction filter using JwtIdentityProvider
2019-11-03 20:06:10,785 DEBUG [NiFi Registry Web Server-19] o.a.n.r.w.s.a.ResourceAuthorizationFilter Request filter authorization check is not required for this HTTP Method on this resource. Allowing request to proceed. An additional authorization check might be performed downstream of this filter.
2019-11-03 20:06:10,785 DEBUG [NiFi Registry Web Server-17] o.a.n.r.w.s.a.ResourceAuthorizationFilter Request filter authorization check is not required for this HTTP Method on this resource. Allowing request to proceed. An additional authorization check might be performed downstream of this filter. The below is my configurations: nifi-registry.properties nifi.registry.db.directory=
nifi.registry.db.driver.class=org.h2.Driver
nifi.registry.db.driver.directory=
nifi.registry.db.maxConnections=5
nifi.registry.db.password=UqZCvEAQeGvUUIGH||82ibCgtpV4JUhkFCnxQkW7kXxkmkHrc
nifi.registry.db.password.protected=aes/gcm/256
nifi.registry.db.sql.debug=false
nifi.registry.db.url=jdbc:h2:/var/lib/nifiregistry/database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
nifi.registry.db.url.append=
nifi.registry.db.username=nifireg
nifi.registry.initial.admin.identity=*******
nifi.registry.kerberos.krb5.file=/etc/krb5.conf
nifi.registry.kerberos.service.keytab.location=/var/run/cloudera-scm-agent/process/238-nifiregistry-NIFI_REGISTRY_SERVER/nifiregistry.keytab
nifi.registry.kerberos.spnego.authentication.expiration=12 hours
nifi.registry.kerberos.spnego.keytab.location=/var/run/cloudera-scm-agent/process/238-nifiregistry-NIFI_REGISTRY_SERVER/nifiregistry.keytab
nifi.registry.providers.configuration.file=/var/run/cloudera-scm-agent/process/238-nifiregistry-NIFI_REGISTRY_SERVER/providers.xml
nifi.registry.security.authorizer=managed-authorizer
nifi.registry.security.authorizers.configuration.file=/var/run/cloudera-scm-agent/process/238-nifiregistry-NIFI_REGISTRY_SERVER/authorizers.xml
nifi.registry.security.identity.provider=ldap-provider
nifi.registry.security.identity.providers.configuration.file=/var/run/cloudera-scm-agent/process/238-nifiregistry-NIFI_REGISTRY_SERVER/identity-providers.xml
nifi.registry.security.keyPasswd=cpDNEjgeOtHgUKBg||/TtGPhbQyltKWVvH9Cj7rj3ZVYZO
nifi.registry.security.keyPasswd.protected=aes/gcm/256
nifi.registry.security.keystore=/var/lib/nifiregistry/cert/keystore.jks
nifi.registry.security.keystorePasswd=QgccvlFai9XXLFUB||Pgu0W6X+BYYSPCiu1drPcqtWIru7
nifi.registry.security.keystorePasswd.protected=aes/gcm/256
nifi.registry.security.keystoreType=jks
nifi.registry.security.needClientAuth=true
nifi.registry.security.truststore=/var/lib/nifiregistry/cert/truststore.jks
nifi.registry.security.truststorePasswd=TKpFfRmNkxQD5xqg||IY8IZookjPjKpGiKiTplZpvmkMRB
nifi.registry.security.truststorePasswd.protected=aes/gcm/256
nifi.registry.security.truststoreType=jks
nifi.registry.sensitive.props.additional.keys=nifi.registry.db.password
nifi.registry.web.http.host=
nifi.registry.web.http.port=
nifi.registry.web.https.host=arch-fndtf03.beta1.fn
nifi.registry.web.https.port=18433
nifi.registry.web.jetty.threads=200
nifi.registry.web.jetty.working.directory=/var/lib/nifiregistry/work/jetty
nifi.registry.web.war.directory=/opt/cloudera/parcels/CFM-1.0.1.0/REGISTRY/lib identity-providers.xml: <identityProviders>
<provider>
<identifier>kerberos-identity-provider</identifier>
<class>org.apache.nifi.registry.web.security.authentication.kerberos.KerberosIdentityProvider</class>
<property name="Authentication Expiration">12 hours</property>
<property name="Default Realm"></property>
<property name="Enable Debug">false</property>
</provider>
<provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.registry.security.ldap.LdapIdentityProvider</class>
<property name="User Search Base">***</property>
<property name="Connect Timeout">10 secs</property>
<property encryption="aes/gcm/256" name="Manager Password">**</property>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">**</property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Identity Strategy">USE_USERNAME</property>
<property name="User Search Filter">cn={0}</property>
<property name="Authentication Expiration">12 hours</property>
<property name="Read Timeout"></property>
<property name="Url">**</property>
</provider>
</identityProviders> authorizations.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies>
<policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="2dbc92a2-b091-3616-8e88-5078b9103b04" resource="/tenants" action="D">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="01b87cb5-c0b6-342d-b108-d8bc03ab5cde" resource="/policies" action="D">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="9d182b11-ebe3-3a7a-8731-98ce6d6e44fd" resource="/buckets" action="R">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="dfbf3c51-fdec-3328-b169-3b54eb033147" resource="/buckets" action="W">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="05b96464-9ec8-312a-8459-67812a8b48c1" resource="/buckets" action="D">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="2fd3fcf5-b10f-33fa-8d8e-b262fa34815e" resource="/actuator" action="R">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="2f470357-e82c-38ee-8062-ab6388d6ec75" resource="/actuator" action="W">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="3ee4703f-94ca-33c2-8060-17f5d313f560" resource="/actuator" action="D">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="0eaa47b9-e409-304e-8682-30d1b0d86d05" resource="/swagger" action="R">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="cf4d8390-5ac7-3ff0-82ce-a274b5f88b21" resource="/swagger" action="W">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="ac587f43-6e1c-3890-81fd-83b4df2e678e" resource="/swagger" action="D">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
<policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53"/>
</policy>
</policies>
</authorizations> users.xml cat users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="d9e3d4d3-e7d2-3c6e-9a70-2602c3265b53" identity="****"/>
</users>
</tenants> could you please point me what i missed? Thanks, Paul
... View more
11-01-2019
07:44 PM
1 Kudo
Hello, I am working with cloudera flow manager version 1.0.1. I cannot got login page when i enable ssl & ldap in nifi registry instead of this page, and there is an node identity. I have checked the config that may be correctly. and try more time such as remove /var/lib/nifiregister/* or /var/run/cloudera-scm-agent/process/***-nifiregistry-NIFI_REGISTRY_SERVER/* the blow pic shows the login user is node id. the related ldap info is below, but there is not userDN config option in cloudera manager. there is same setting between nifi and nifi registry. 2019-11-02 10:24:59,167 INFO org.springframework.ldap.core.support.AbstractContextSource: Property 'userDn' not set - anonymous context will be used for read-write operations The behavior is very strange. Who could help me what i missed? Thanks, Paul
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)