Member since
01-27-2021
21
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1718 | 09-04-2021 02:39 AM |
10-17-2022
09:07 PM
Hi @tovganesh I tested dbt via kerberos profiles and when I debug the project I got these following error 1 check failed: dbt was unable to connect to the specified database. The database returned the following error: >Runtime Error (('Unspecified GSS failure. Minor code may provide more information', 851968), ('Server not found in Kerberos database', -1765328377)) Check your database credentials and try again. For more information, visit: https://docs.getdbt.com/docs/configure-your-profile And this is the profiles config impala_testing: outputs: dev: type: impala host: cdpgateway port: 21000 dbname: testdb schema: testdb user: admin password: admin auth_type: GSSAPI kerberos_service_name: None use_http_transport: true # default value: true use_ssl: true # TLS should always be used with LDAP to ensure secure transmission of credentials, default value: true retries: 3 target: dev
... View more
10-16-2022
10:53 PM
Hi @tovganesh our private cluster using kerberos and tls, how to setup dbt profile for this requirement ? Thanks
... View more
10-11-2022
09:10 PM
Hi @tovganesh Using RHEL 7 and CDP with python v3.6 ok i will try first
... View more
10-11-2022
08:11 PM
Hi @sdairs1 we try to setup from pip, and get below error : ERROR: Could not find a version that satisfies the requirement dbt-core>=1.1.0 any help appreciate
... View more
09-20-2021
06:51 AM
hi @GangWar Disable TLS from CDP ? i'm using AutoTLS for CDP Cluster only , CDSW not enable TLS
... View more
09-18-2021
08:46 AM
Hi All, if there is an example for NIFI User Group with ranger configuration it is very helpful
... View more
09-18-2021
08:43 AM
Hi All, I try to running pyspark example code from CDSW template our environment : CDSW 1.9.2 ,CDP Private Cloud Base 7.1.6 pi.py # # Estimating $\pi$ # # This is the simplest PySpark example. It shows how to estimate $\pi$ in parallel # using Monte Carlo integration. If you're new to PySpark, start here! from __future__ import print_function import sys from random import random from operator import add from pyspark.sql import SparkSession spark = SparkSession\ .builder\ .appName("PythonPi")\ .getOrCreate() partitions = int(sys.argv[1]) if len(sys.argv) > 1 else 2 n = 100000 * partitions def f(_): x = random() * 2 - 1 y = random() * 2 - 1 return 1 if x ** 2 + y ** 2 < 1 else 0 count = spark.sparkContext.parallelize(range(1, n + 1), partitions).map(f).reduce(add) print("Pi is roughly %f" % (4.0 * count / n)) spark.stop() and we got this message from session logs: Failed to get auth cookie contents data = {"cookies":[],"err":"http: named cookie not present","user":"cdsw"} pyspark code just runs without finishing, please help
... View more
Labels:
09-15-2021
07:49 AM
Hi @MattWho this is the sample from ldapsearch : distinguishedName: CN=wbivp,OU=DEVELOPER,DC=msad,DC=local,DC=co memberOf: CN=cdpconsultant,OU=Groups,OU=DEVELOPER,DC=msad,DC=local,DC=co my username wbivp , with group cdpconsultant from this information, please advise to fill in the configuration required by NIFI
... View more
09-09-2021
07:23 AM
hi @MattWho , this is my authorizer.xml , for now i still cannot get authorize within group <authorizers> <userGroupProvider> <identifier>cm-user-group-provider</identifier> <class>org.apache.nifi.authorization.CMUserGroupProvider</class> <property name="Knox Nodes Properties Location">/var/run/cloudera-scm-agent/process/1546339525-nifi-NIFI_NODE/knox-conf/knox-gateway.properties</property> <property name="NiFi Registry Nodes Properties Location">/var/run/cloudera-scm-agent/process/1546339525-nifi-NIFI_NODE/nifiregistry-conf/peer.properties</property> <property name="NiFi Group">nifi</property> <property name="Infer Unqualified Hostnames">false</property> <property name="NiFi Nodes Properties Location">/var/run/cloudera-scm-agent/process/1546339525-nifi-NIFI_NODE/nifinode-conf/peer.properties</property> </userGroupProvider><userGroupProvider> <identifier>composite-user-group-provider</identifier> <class>org.apache.nifi.authorization.CompositeUserGroupProvider</class> <property name="User Group Provider 1">ldap-user-group-provider</property> <property name="User Group Provider 2">cm-user-group-provider</property> </userGroupProvider><userGroupProvider> <identifier>ldap-user-group-provider</identifier> <class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class> <property name="Connect Timeout">10 secs</property> <property name="TLS - Client Auth"></property> <property name="TLS - Protocol"></property> <property name="User Identity Attribute">cn</property> <property name="Group Name Attribute">cn</property> <property name="User Search Scope">ONE_LEVEL</property> <property name="Group Object Class">group</property> <property name="Url">ldap://msad.local.co:389</property> <property name="TLS - Keystore Type"></property> <property name="User Search Base">OU=DEVELOPER,DC=msad,DC=local,DC=co</property> <property name="Group Membership - Enforce Case Sensitivity">false</property> <property name="Authentication Strategy">SIMPLE</property> <property name="Group Search Base"></property> <property name="Group Member Attribute - Referenced User Attribute"></property> <property name="Group Member Attribute"></property> <property name="TLS - Keystore"></property> <property name="TLS - Truststore"></property> <property name="Group Search Scope">ONE_LEVEL</property> <property encryption="aes/gcm/256" name="Manager Password">Yf41</property> <property name="User Group Name Attribute"></property> <property name="TLS - Truststore Password"></property> <property name="User Object Class">user</property> <property name="Referral Strategy">FOLLOW</property> <property name="Page Size"></property> <property name="Read Timeout">10 secs</property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="TLS - Keystore Password"></property> <property name="TLS - Shutdown Gracefully"></property> <property name="Sync Interval">30 mins</property> <property name="Manager DN">CN=CDP Admin,OU=DEVELOPER,DC=msad,DC=local,DC=co</property> <property name="User Search Filter"></property> <property name="TLS - Truststore Type"></property> <property name="Group Search Filter"></property> </userGroupProvider> <authorizer> <identifier>ranger-provider</identifier> <class>org.apache.nifi.ranger.authorization.ManagedRangerAuthorizer</class> <classpath>/var/run/cloudera-scm-agent/process/1546339525-nifi-NIFI_NODE/hadoop-conf</classpath> <property name="Ranger Security Config Path">/var/run/cloudera-scm-agent/process/1546339525-nifi-NIFI_NODE/ranger-nifi-security.xml</property> <property name="User Group Provider">composite-user-group-provider</property> <property name="Ranger Admin Identity">host_ranger</property> <property name="Ranger Service Type">nifi</property> <property name="Ranger Audit Config Path">/var/run/cloudera-scm-agent/process/1546339525-nifi-NIFI_NODE/ranger-nifi-audit.xml</property> <property name="Ranger Application Id">Cluster1_nifi</property> <property name="Ranger Kerberos Enabled">true</property> </authorizer> </authorizers>
... View more