Cloudera Community
Shelton
user rank
Master Mentor

Re: Atlas: How to generate lineage between hive ex...

by Master Mentor in Support Questions
‎05-23-2017 02:59 PM
1 Kudo
‎05-23-2017 02:59 PM
1 Kudo
@Qinglin Xia What is the version of Atlas and HDP ?? I encountered the same issue on 2.5.0.0 but see this output with 2.5.3 on how to reproduce lineage. Make sure that the Ranger Plugins for Atlas ,hbase,hive,hdfs,kafka have been enable with corresponding Ranger Policies !! see the Additional Requirements for Atlas with Ranger and Kerberos Have a look at the attached document running Atlas in a kerberized the only difference if your environment is not kerberiszed s not to run the kinit:-) ... View more

Re: I have just setup kerberos witn hadoop. unable...

by Master Mentor in Support Questions
‎05-22-2017 06:50 PM
‎05-22-2017 06:50 PM
@priyanshu hasija, I assume you are on linux Check the valid principal for the hdfs keytab [root@toronto ~]# klist -kt /etc/security/keytabs/hdfs.headless.keytab Keytab name: FILE:/etc/security/keytabs/hdfs.headless.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 05/08/17 23:33:51 hdfs-has@HASIJA.COM 1 05/08/17 23:33:51 hdfs-has@HASIJA.COM 1 05/08/17 23:33:51 hdfs-has@HASIJA.COM 1 05/08/17 23:33:51 hdfs-has@HASIJA.COM 1 05/08/17 23:33:51 hdfs-has@HASIJA.COM Kinit using the hdfs keytab and principal [root@toronto ~]# kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-has@HASIJA.COM Local file system [root@toronto ~]# ls anaconda-ks.cfg authorized_keys install.log install.log.syslog Now copy to hdfs [root@toronto ~]# hdfs dfs -copyFromLocal authorized_keys /user/admin Check the copy was successful [root@toronto ~]# hdfs dfs -ls /user/admin Found 1 items -rw-r--r-- 3 hdfs hdfs 405 2017-05-22 20:41 /user/admin/authorized_keys There you go ... View more

Re: Single Datanode start fails ?

by Master Mentor in Support Questions
‎05-18-2017 05:35 AM
‎05-18-2017 05:35 AM
@btandel Have a look at this ... View more

Re: Kerberized Clusters - Can you have multiple ke...

by Master Mentor in Support Questions
‎05-16-2017 05:32 PM
‎05-16-2017 05:32 PM
@Debra Montague Make sure the encryption types are the same , also check that the KVNO (key version number) is the same ! See the syntax here Link ... View more

Re: How to import hive metadata in Apache Atlas ??

by Master Mentor in Support Questions
‎05-11-2017 10:34 AM
‎05-11-2017 10:34 AM
@subash sharma Did you succeed with the hive metadata import in Atlas? ... View more

Re: Sync ldap problem

by Master Mentor in Support Questions
‎04-26-2017 12:26 PM
‎04-26-2017 12:26 PM
Hi all, My problem has been resolved ! I had to ask the client to install a AD Explorer and figured out the correct settings and not only changed the baseDn all the group and user attributes ! authentication.ldap.baseDn=DC=hq,DC=uk,DC=com authentication.ldap.bindAnonymously=false authentication.ldap.dnAttribute=organizationalPerson authentication.ldap.groupMembershipAttr=member authentication.ldap.groupNamingAttr=cn authentication.ldap.groupObjectClass=group authentication.ldap.managerDn=cn=svc-hadoop-ldap,ou=Data Lake,ou=Applications,ou=Administrative,dc=hq,dc=uk,dc=com authentication.ldap.managerPassword=/etc/ambari-server/conf/ldap-password.dat authentication.ldap.primaryUrl=fake.uk.com:389 authentication.ldap.referral=ignore authentication.ldap.useSSL=false authentication.ldap.userObjectClass=organizationalPerson authentication.ldap.usernameAttribute=sAMAccountName This pulled out the desired users and group ambari-server sync-ldap --groups groups.txt Using python /usr/bin/python Syncing with LDAP... Enter Ambari Admin login: admin Enter Ambari Admin password:Syncing specified users and groups....Completed LDAP Sync.Summary: memberships: removed = 0 created = 4 users: updated = 0 removed = 0 created = 1 groups: updated = 0 removed = 0 created = 1 ... View more

Re: Sync ldap problem

by Master Mentor in Support Questions
‎04-24-2017 07:19 PM
‎04-24-2017 07:19 PM
@Vipin Rathor Sorry to get back this late I have just had acess again and I have change my baseDn to "DC=hq,DC=uk,DC=com" but that doesn't still pull the desired group. This is making me mad ... View more

Re: kerberos setup on server

by Master Mentor in Support Questions
‎04-23-2017 08:02 PM
‎04-23-2017 08:02 PM
@Keshav Piplani, According to your setup I assume you have 4 clusters farm1.L3-ds.com farm2.L3-ds.com farm3.L3-ds.com farm4.L3-ds.com If so then you should create a domain realm LS-DS.COM (Uppercase)in the KDC in each of the above domains Update the krb5.conf on all the 4 KDC's to reflect the 4 KDC's Setup cross realm trust between 4 MIT KDC Check this autosetup ... View more

Sync ldap problem

by Master Mentor in Support Questions
‎04-21-2017 07:23 PM
‎04-21-2017 07:23 PM
ad-browser.pngHi all, I have a very Bizarre situation while running sync-ldap for Ambari The group does exist in the LDAP but I get an exception ! The contents of the groups.txt is hadoop_administrators # ambari-server sync-ldap --users users.txt --groups groups.txt 21 Apr 2017 13:38:12,563 ERROR [pool-16-thread-6] LdapSyncEventResourceProvider:457 - Caught exception running LDAP sync. org.apache.ambari.server.AmbariException: Couldn't sync LDAP group hadoop_administrators,it doesn't exist at org.apache.ambari.server.security.ldap.AmbariLdapDataPopulator.synchronizeLdapGroups(AmbariLdapDataPopulator.java:253) at org.apache.ambari.server.controller.AmbariManagementControllerImpl.synchronizeLdapUsersAndGroups(AmbariManagementControllerImpl.java:4775) at org.apache.ambari.server.controller.internal.LdapSyncEventResourceProvider.syncLdap(LdapSyncEventResourceProvider.java:487) at org.apache.ambari.server.controller.internal.LdapSyncEventResourceProvider.processSyncEvents(LdapSyncEventResourceProvider.java:445) at org.apache.ambari.server.controller.internal.LdapSyncEventResourceProvider.access$000(LdapSyncEventResourceProvider.java:65) at org.apache.ambari.server.controller.internal.LdapSyncEventResourceProvider$1.run(LdapSyncEventResourceProvider.java:257) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Here is my ambari.properties authentication.ldap.alternateUserSearchEnabled=true authentication.ldap.alternateUserSearchFilter=(&(userPrincipalName={0})(objectClass=person)) authentication.ldap.baseDn=OU=Users,OU=Enterprise,DC=hq,DC=uk,DC=com authentication.ldap.bindAnonymously=false authentication.ldap.dnAttribute=distinguishedName authentication.ldap.groupMembershipAttr=member authentication.ldap.groupNamingAttr=cn authentication.ldap.groupObjectClass=group authentication.ldap.managerDn=cn=svc-hadoop-ldap,OU=Data Lake,OU=Applications,OU=Administrative,DC=hq,DC=uk,DC=com authentication.ldap.managerPassword=/etc/ambari-server/conf/ldap-password.dat authentication.ldap.primaryUrl=mboro:389 authentication.ldap.referral=ignore authentication.ldap.useSSL=false authentication.ldap.userObjectClass=person authentication.ldap.usernameAttribute=sAMAccountName Attached is a screenshot of my AD explorer CN=svc-hadoop-ldap,OU=Data Lake,OU=Applications,OU=Administrative,DC=hq,DC=k,DC=grp I have only 4 users in the LDAP group hadoop_administrators,these users were synced correctly but the process couldn't pull the group. I appreciate any help. ... View more

Re: Hive Beeline Connection Error

by Master Mentor in Support Questions
‎04-13-2017 04:57 AM
‎04-13-2017 04:57 AM
@zkfs Is your cluster kerberized ? If so see below Try this $ beeline Beeline version 1.2.1000.2.5.3.0-37 by Apache Hive beeline> ! connect jdbc:hive2://london.nakuru.com:10000/;principal=hive/london.nakuru.com@NAKURU.COM Connecting to jdbc:hive2://london.nakuru.com:10000/;principal=hive/london.nakuru.com@NAKURU.COM Enter username for jdbc:hive2://london.nakuru.com:10000/;principal=hive/london.nakuru.com@NAKURU.COM:xx Enter password for jdbc:hive2://london.nakuru.com:10000/;principal=hive/london.nakuru.com@NAKURU.COM:xx Connected to: Apache Hive (version 1.2.1000.2.5.3.0-37) Driver: Hive JDBC (version 1.2.1000.2.5.3.0-37) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://london.nakuru.com:10000/> show databases; +----------------+--+ | database_name | +----------------+--+ | default | | mbuzi | | slum | +----------------+--+ 3 rows selected (2.863 seconds) ... View more
Contact Me
Online
Offline
Last Visited
‎06-05-2025 02:03 PM
Community Statistics
Member Since ‎01-19-2017 04:35 AM
Last Visited ‎06-05-2025 02:03 PM
Posts 3,676
Kudos received 627