Member since
02-22-2016
25
Posts
8
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3770 | 08-18-2019 11:01 PM |
08-18-2019
11:01 PM
Issue was resolved. No need to configure cross-realm trust rather, try logging in to Ambari URL via browser or API using curl. This will generate a OAuth token for the user which will then be used for authentication to access ADLS. So Kerberos token for hadoop authentication and Oauth token needed for ADLS authentication. Thus each time , when you create HDInsight cluster, ensure you create token for the user to access ADLS from Ambari API.
... View more
09-14-2018
10:03 AM
We are working with HDInsight Spark cluster with ADLS as its primary storage . Now, we need to join HDInsight cluster to a AD domain for user authentciation and make it enterprise ready.
Read that HDInsight only allows domain joining via Azure ADDS. Our onprem enterprise AD domain domainA.com is already in sync with Azure AD using Azure connect and ADDS was created in Azure for HDInsights with a custom domain- domainB.com , enabled password hash sync for Kerberos.
We were able to join the cluster to newly created ADDS domain domainB.com successfully and all hadoop services are running and in good health. We are able to login to cluster using onprem AD credentials in domainA.com as they are in sync with azure ad.
But the issue is, we are able to access hadoop services including HDFS,Hive,etc only when logged into cluster as users created in Azure ADDS domain domainB.com and same access is not available for users in enterprise AD domainA.com though they are synced to Azure AD.
So the issue is not due to ADLS store connectivity, because adls is accessible for users in azure AD / ADDS domain and not for enterprise AD users in different domain.
When tried to access ADLS using
hadoop fs -ls / or
hdfs dfs -ls adl:/// or
hadoop fs -ls adl://home or
hadoop fs -ls adl://datalakestorename.azuredatalake.net/ ,
the error thrown is as follows:
ERROR: secure.AbstractCredentialServiceCaller: Token does not exist in Tokenmanager(Response code 404) ls: Error fetching access token
Is this can happen due to difference in two domains- Azure ADDS and onprem AD. Do we need to configure anything like cross realm trust , in this PaaS manually to make it work. We are totally stuck with this issue.
Please help ASAP if anyone has encountered similar issues.
... View more
Labels:
- Labels:
-
Apache Ambari
02-02-2017
09:18 AM
Can you please share any documents to install solr in HDP-2.5.3 and also the steps to configure it with ranger and atlas.
... View more
02-02-2017
07:20 AM
we havent installed solr. is it mandatory to install solr if we are not using search?
... View more
02-02-2017
07:19 AM
Following is the atlas-application-properties content: atlas.audit.hbase.tablename=ATLAS_ENTITY_AUDIT_EVENTS
atlas.audit.hbase.zookeeper.quorum=hortonworks.example.com,hdp-dn02.example.com,hdp-dn03.example.com
atlas.audit.zookeeper.session.timeout.ms=1000
atlas.auth.policy.file=/etc/atlas/conf/policy-store.txt
atlas.authentication.keytab=/etc/security/keytabs/atlas.service.keytab
atlas.authentication.method.file=true
atlas.authentication.method.file.filename=/etc/atlas/conf/users-credentials.properties
atlas.authentication.method.kerberos=false
atlas.authentication.method.ldap=false
atlas.authentication.method.ldap.ad.base.dn= atlas.authentication.method.ldap.ad.bind.dn=
atlas.authentication.method.ldap.ad.bind.password= atlas.authentication.method.ldap.ad.default.role=ROLE_USER atlas.authentication.method.ldap.ad.domain= atlas.authentication.method.ldap.ad.referral=ignore atlas.authentication.method.ldap.ad.url= atlas.authentication.method.ldap.ad.user.searchfilter=(sAMAccountName={0}) atlas.authentication.method.ldap.base.dn=
atlas.authentication.method.ldap.bind.dn= atlas.authentication.method.ldap.bind.password= atlas.authentication.method.ldap.default.role=ROLE_USER
atlas.authentication.method.ldap.groupRoleAttribute=cn atlas.authentication.method.ldap.groupSearchBase= atlas.authentication.method.ldap.groupSearchFilter= atlas.authentication.method.ldap.referral=ignore atlas.authentication.method.ldap.type=none
atlas.authentication.method.ldap.url= atlas.authentication.method.ldap.user.searchfilter=
atlas.authentication.method.ldap.userDNpattern=uid= atlas.authentication.principal= atlas
atlas.authorizer.impl=simple atlas.cluster.name=HSBC atlas.enableTLS=false atlas.graph.index.search.backend=solr5 atlas.graph.index.search.solr.mode=cloud atlas.graph.index.search.solr.zookeeper-url=hdp-dn02.example.com:2181,hdp-dn03.example.com:2181,hortonworks.example.com:2181 atlas.graph.storage.backend=hbase
atlas.graph.storage.hbase.table=atlas_titan atlas.graph.storage.hostname=hortonworks.example.com,hdp-dn02.example.com,hdp-dn03.example.com
atlas.kafka.auto.commit.enable=false atlas.kafka.bootstrap.servers=hortonworks.example.com:6667 atlas.kafka.hook.group.id=atlas atlas.kafka.zookeeper.connect=hdp-dn02.example.com:2181,hdp-dn03.example.com:2181,hortonworks.example.com:2181
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.session.timeout.ms=400 atlas.kafka.zookeeper.sync.time.ms=20 atlas.lineage.schema.query.hive_table=hive_table where __guid='%s'\, columns atlas.lineage.schema.query.Table=Table where __guid='%s'\, columns
atlas.notification.create.topics=true atlas.notification.embedded=false
atlas.notification.replicas=1 atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES atlas.rest.address=http://hdp-dn02.example.com:21000 atlas.server.address.id1=hdp-dn02.example.com:21000 atlas.server.bind.address=hdp-dn02.example.com atlas.server.ha.enabled=false
atlas.server.http.port=21000
atlas.server.https.port=21443
atlas.server.ids=id1
atlas.solr.kerberos.enable=false
... View more
02-02-2017
07:15 AM
Hi, The cluster is not kerberised but installed ranger and atlas. As we are not having plan of search in atlas, we haven't installed solr. Atlas UI is not accessible in our cluster. We could found the following errors in application log: 2017-02-02 12:07:06,497 WARN - [main:] ~ FAILED o.e.j.w.WebAppContext@63d75942{/,file:/usr/hdp/2.5.3.0-37/atlas/server/webapp/atlas/,STARTING}{/usr/hdp/current/atlas-server/server/webapp/atlas}: java.lang.ExceptionInInitializerError (AbstractLifeCycle:212)
java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
Caused by: java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.solr.Solr5Index
at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:55)
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:421)
Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at hdp-dn02.example.com:2181: cluster not found/not ready
at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:290)
at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:467) Please help.
... View more
Labels:
- Labels:
-
Apache Atlas
02-02-2017
05:21 AM
thank you,my issue has been resolved with negotiate option.
... View more
02-01-2017
10:54 AM
@slachterman thank you it worked with negotiate. Now when I disabled the kerberos and tried the same rest api command , same exception recreated. command: curl -u keyadmin:keyadmin1 -X GET http://<ranger-KMS-server>:9292/kms/v1/keys/names Exception: Authentication required-This request requires HTTP authentication. Please advice
... View more
01-30-2017
06:17 AM
@vperiasamy I tried kiniting the keyadmin principal. But still facing the same authentication error.
... View more
01-27-2017
02:32 PM
1 Kudo
Hi, While executing the following Ranger KMS rest API command, we have encountered the exception: command: curl -u admin:admin -X GET http://<ranger-KMS-server>:9292/kms/v1/keys/names Exception: Authentication required-This request requires HTTP authentication. We have created the keyadmin principal with the password keyadmin1 as configured in kms-properties. We can create keys and list keys via Ranger KMS UI. Please advice a solution ASAP.
... View more
Labels:
- Labels:
-
Apache Ranger