Member since
11-06-2015
44
Posts
9
Kudos Received
0
Solutions
07-03-2019
06:47 AM
@Debabrata Ghosh, integrating external Kafka to Atlas should be possible, In addition In addition you will need to create and provide appropriate permissions in Kafka for topics ATLAS_HOOKS and ATLAS_ENTITIES which will be used by Atlas and Atlas hook services for respective users. Additionally, kafka configurations will need to be put in place in Atlas and the Atlas hook services in the atlas-application.properties atlas.kafka.bootstrap.servers and atlas.kafka.zookeeper.connect. Mainly, Kafka brokers configuration and Kafka-Zookeeper quorum should be added in the atlas-application.properties to get started with. If Kafka cluster is kerberized then Atlas will need to use the same KDC as Kafka cluster.
... View more
01-28-2019
06:39 AM
1 Kudo
@Nagaraju V, you will need to ensure that the Ranger-Admin service has appropriate read-write permissions for both the keystore and truststore. You can check ranger-admin logs for more details, in addition to the catalina.out file, there will be an additional file created like xa_portal.log or ranger_admin.log, you can check the file for more errors as to what is causing the ranger-admin to not respond.
... View more
10-29-2018
06:54 AM
1 Kudo
Hi @Shesh Kumar, The issue seems probably related to using a different auto-increment being used than the one which is set to default in the database, hence recommend to use the auto_increment property instead of the foreign_key_check property for the database. Instead of using set FOREIGN_KEY_CHECKS=0; you can use set AUTO_INCREMENT=1; in the sql file. you will get the same result.
... View more
09-03-2018
12:19 PM
Pankaj Singh, Are both Ranger and Ranger database configured in the same environment, or same cluster ? This can be attributed to a delay in response from the host where Ranger-DB is required to be configured.
... View more
03-13-2018
10:07 AM
@GN_Exp, has the Knox certificate been imported in Ranger truststore ? If not here are some links you can follow: installing ranger-knox plugin. ranger-knox-repo-test-connection-failure. do-we-need-to-add-knox-self-signed-cert-into-range.
... View more
03-12-2018
06:11 AM
@GN_Exp, looks like the url for Knox present with Ranger is pointing to localhost, can you try using the actual f.q.d.n of the Knox host, even if both Ranger and Knox services are on the same host. Also as @Deepak Sharma suggested, can you check if the LDAP server being used by Knox service is started and working.
... View more
02-09-2018
12:31 PM
@Bramantya Anggriawan, If the files are already present, then can you check whether there is a log-directory defined for Atlas-Metadata server, you can check this in Ambari - atlas-env.sh section, also check if the log-directory path actually exists
... View more
02-08-2018
09:32 AM
@Bramantya Anggriawan, if the files are present under /usr/hdp/current/atlas-server, there is a chance that there may be an ownership issue for the files, can you check the ownership of the files by using the below commands: ls -altr /usr/hdp/current/atlas-server/bin/ ls -altr /usr/hdp/current/atlas-server/conf/
... View more
01-04-2018
06:05 AM
@Karthick Raja, To install mrjob package from python-pip can you try the steps shared on the mrjob site here
... View more
01-04-2018
04:49 AM
@Karthick Raja, yes you will need to restart the services in the listed order.
... View more
01-02-2018
11:15 AM
@Vijay Mishra, The mentioned properties should now be available on Ambari Ranger-configurations, and you might not need to add the properties to ranger-admin*.sh script manually. Glad to know the issue was fixed.
... View more
01-02-2018
06:51 AM
@Karthick Raja, can you try below commands in listed order, from Ambari UI: Restart Zookeeper service. Restart Ambari-Infra service. Restart Ranger service. This should do the needful.
... View more
01-01-2018
03:44 PM
@Karthick Raja, looks like service Ambari-Infra is down, which is required for Ranger service to start properly, the error messages relate to connection with Infra-Solr server is failing to create collection for ranger-audits, can you try starting Ambari-Infra service and then start Ranger service again.
... View more
11-24-2017
01:15 PM
@Nisha, If you want you use AD users system-wide for Hadoop, Knox and Ranger as well, you can think to configure Kerberos via AD itself, you can refer AD integration with Kerberos. Additionally you can configure Linux System service to use AD users and groups which can be then used configured in Hadoop as well. Please refer below articles: HDP-2.6 Security Labs Hadoop Group Mapping using LDAP/AD Setup SSSD for AD Hope this helps.
... View more
11-23-2017
11:10 AM
Nisha, Thanks for the details, (Please mention the Ambari / HDP version you will be using.) So as I understand, your goal seems to be to access Web-HDFS, Hive etc via Knox which is to be configured for both AD and kerberos authentication. Additionally you also want the user to be authenticated by Ranger as well ?
... View more
11-22-2017
02:03 PM
Nisha, from the above it does not seem clear what is the target goal, Are you trying to access Hive via Knox which is authenticated via Kerberos where Knox is coupled with AD authentication ? For "Now Kerberos would validate KNOX service with its keytab as per this link." link seems to be missing. Additionally, Kindly specify the version of Ambari/HDP you are planning to use.
... View more
11-02-2017
08:52 AM
Avdhoot Patankar, which file are the capture6.png logs from ? The error message indicates that a SSL certificate was expected and it is missing in one of the required configuration, which results in authentication failure
... View more
11-02-2017
05:08 AM
Avdhoot Patankar, the issue seems to be similar to the one posted here -> insufficient-permissions-error-while-login-to-nifi. Can you try to follow the steps shared by Geoffrey Shelton Okot, which point to this document.
... View more
10-16-2017
06:56 PM
@Sean Roberts, did you try curl with -v option ?
... View more
10-16-2017
05:49 AM
@Sean Roberts, Can you try the same curl request with the verbose option and share the response. Looks like the response, GSSException:Failure unspecified at GSS-API level (Mechanism level:Requestis a replay (34)) implies that the request carries the same token which was previously used for a different request and the connection doesn't seem to be closed. In that case you can destroy the current ticket and try with a fresh ticket.
... View more
09-15-2017
06:26 AM
@Jacqualin jasmin, you can do ssh / login to the terminal of the host where Ranger-Usersync is installed and execute the below command and check the output: curl -iv http://<VIP of the LB>:<Port of LB for Ranger>
... View more
09-13-2017
11:10 AM
@Jacqualin jasmin, currently multiple entries are not supported for policymgr_external_url ( External URL ) parameter. For configuration with LB using a vip, Can you do a curl call from Ranger-Usersync host to Ranger-Admin LB url and check whether it is able to connect to the LB.
... View more
09-11-2017
09:20 AM
@Sanaz Janbakhsh, If /var/log/ranger/admin is empty or is not available, logs for Ranger will be available under /usr/hdf/current/ranger-admin/ews/logs on the host where Ranger-Admin is installed in case of a multi-node setup. For Nifi audit not working, can you check the Nifi logs, looks like Nifi is unable to Audit to Solr, what is the Solr you are using, is it installed from Ambari as Ambari-Infra service ?
... View more
09-11-2017
05:13 AM
Hi @Sanaz Janbakhsh, Is the audit functionality working for other services, As Nifi plugin is able to download pollicies, Ranger authorization seems to be working fine. Can you check Nifi log for audit related errors, seems Nifi is not able to write audits to Solr ? Also do mention the Ambari / HDF versions you are using.
... View more
09-07-2017
05:23 AM
Hi @Sanaz Janbakhsh, after enabling Nifi for Ranger, can you confirm Nifi is able to download policies from Ranger, if it is doing so Nifi service should be registered under Audits -> Plugins tab in Ranger-UI. For nifi plugin to work, you can also get help from below articles, https://community.hortonworks.com/articles/60842/hdf-20-defining-nifi-policies-in-ranger.html https://community.hortonworks.com/articles/57980/hdf-20-apache-nifi-integration-with-apache-ambarir.html https://community.hortonworks.com/content/kbentry/60001/hdf-20-integrating-secured-nifi-with-secured-range.html
... View more
08-31-2017
06:25 PM
Hi @Vijay Kiran, can you please add which version of Ranger have you installed, is this a manual installation or Ranger was installed through Ambari (do mention Ambari version also if in use) and How did you enable Ranger plugin for Hive ?
... View more
04-10-2017
05:00 AM
2 Kudos
How to set up a Ranger enabled cluster using Ambari-blueprint. In order to setup a cluster using Ambari blueprints we will need a minimum three node cluster. Lets say this cluster contains three hosts as below: secure-ranger-a.com secure-ranger-b.com secure-ranger-c.com Pre-requisites: To setup a Ranger enabled cluster we need to install ambari-server on one of the hosts which can be the called the gateway node and ambari-agents should be installed on all the nodes including the gateway node.All the ambari-agents should be configured to point to the ambari-server on the gateway node. Note : The operating system used for this setup is Cent-OS. Lets consider the gateway node to be simple-ranger-a.com, to install ambari-server on this host we need to add ambari repository to the OS repositories. To update the OS repository we add ambari.repo file to /etc/yum.repos.d/ directory. Next we do a refresh of the OS repositories using command yum clean all Then we run the command yum install -y ambari-server ambari-agent, this will install both ambari-server and ambari-agent on the gateway host. Then we run ambari-server setup -s to setup ambari-server silently with default values, and ambari-server start to start the ambari-server To install ambari-agent on the all hosts we copy the ambari.repo file to each of the hosts /etc/yum.repos.d/ and do yum clean all and yum install -y ambari-agent. To make ambari-agent point to the ambari-server on the gateway node we run the command ambari-agent reset <FQDN of the gateway node> in this case we need to run ambari-agent reset simple-ranger-a.com on all hosts where agent is to be installed. We now need to setup the Ranger-database as it is a pre-requisite to install Ranger service, here is a nice document to configure Ranger with mysql, postgres or oracle , which can be referred to setup the database of choice. To enable security on the cluster we need to setup kerberos on the cluster, here is a nice document to setup the same. Now we will need to create the blueprints and run them on the Ambari server.you can use the below blueprints and modify accordingly. Save the attached hdp.txt and rename the .txt extention to .json Save the attached cluster.txt and rename the .txt extention to .json Save the attached blueprint.txt and rename the .txt extention to .json After modifying the above jsons to use required repositories and hosts, we should now save the jsons on the gateway host. Now we can submit the blueprints to be processed on the ambari-server using below commands. curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d @<path to hdp.json> "http://<gateway-host>:8080/api/v1/stacks/HDP/versions/2.6/operating_systems/redhat6/repositories/HDP-2.6" curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d @<path to blueprint.json> "http://<gateway-host>:8080/api/v1/blueprints/myblueprint" curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d @<path to cluster.json> "http://<gateway-host>:8080/api/v1/clusters/myblueprint" On successful execution of above commands we get response message in below format: { "href" : "http://<gateway-host>:8080/api/v1/clusters/myblueprint/requests/1", "Requests" : { "id" : 1, "status" : "Accepted" } } We can now log-in to Ambari and check the background operations to see the components being installed and started on all the hosts.
... View more
- Find more articles tagged with:
- ambari-blueprint
- FAQ
- Ranger
- ranger-plugins
- Security
Labels:
03-20-2017
11:34 AM
2 Kudos
How to set up a Ranger enabled cluster using Ambari-blueprint.
In order to setup a cluster using Ambari blueprints we will need a minimum three node cluster. Lets say this cluster contains three hosts as below: simple-ranger-a.com simple-ranger-b.com simple-ranger-c.com
Pre-requisites: To setup a Ranger enabled cluster we need to install ambari-server on one of the hosts can be the called the gateway node let and ambari-agents should be installed on all the nodes.All the ambari-agents should be configured to point to the ambari-server on the gateway node. Note : The operating system used for this setup is Cent-OS. Lets consider the gateway node to be simple-ranger-a.com, to install ambari-server on this host we need to add ambari repository to the OS repositories. To update the OS repository we add ambari.repo file to /etc/yum.repos.d/ directory. Next we do a refresh of the OS repositories using command yum clean all Then we run the command yum install -y ambari-server ambari-agent, this will install both ambari-server and ambari-agent on the gateway host. Then we run ambari-server setup -s to setup ambari-server silently with default values, and ambari-server start to start the ambari-server To install ambari-agent on the all hosts we copy the ambari.repo file to each of the hosts /etc/yum.repos.d/ and do yum clean all and yum install -y ambari-agent. To make ambari-agent point to the ambari-server on the gateway node we run the command ambari-agent reset <FQDN of the gateway node> in this case we need to run ambari-agent reset simple-ranger-a.com on all hosts where agent is to be installed. We now need to setup the Ranger-database as it is a pre-requisite to install Ranger service, here is a nice document to configure Ranger with mysql, postgres or oracle , which can be referred to setup the database of choice. Now we will need to create the blueprints and run them on the Ambari server.you can use the below blueprints and modify accordingly. Save the attached hdp.txt and rename the .txt extention to .json Save the attached cluster.txt and rename the .txt extention to .json Save the attached blueprint.txt and rename the .txt extention to .json
After modifying the above jsons to use required repositories and hosts, we should now save the jsons on the gateway host. Now we can submit the blueprints to be processed on the ambari-server using below commands.
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d @<path to hdp.json> "http://<gateway-host>:8080/api/v1/stacks/HDP/versions/2.6/operating_systems/redhat6/repositories/HDP-2.6" curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d @<path to blueprint.json> "http://<gateway-host>:8080/api/v1/blueprints/myblueprint" curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d @<path to cluster.json> "http://<gateway-host>:8080/api/v1/clusters/myblueprint" On successful execution of above commands we get response message in below format: { "href" : "http://<gateway-host>:8080/api/v1/clusters/myblueprint/requests/1", "Requests" : { "id" : 1, "status" : "Accepted" } } We can now log-in to Ambari and check the background operations to see the components being installed and started on all the hosts.
... View more
- Find more articles tagged with:
- ambari-blueprint
- How-ToTutorial
- Ranger
- ranger-plugins
- Sandbox & Learning
Labels:
02-24-2017
04:25 AM
@Normen Zoch, if you can share the version of Ambari and HDP you are trying to setup as well!!
... View more