Member since
06-26-2019
68
Posts
8
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3426 | 04-20-2018 09:51 PM | |
4234 | 02-08-2018 01:27 AM | |
1316 | 01-31-2018 06:39 PM | |
5345 | 01-31-2018 07:27 AM | |
1380 | 01-17-2018 09:37 PM |
01-31-2018
06:39 PM
This problem is solved using the solution outlined https://community.hortonworks.com/content/supportkb/150187/unable-to-view-ranger-audit-when-ssl-is-enabled-on.html
... View more
01-31-2018
06:29 PM
I checked my certificate and under Extended Key Usage it has both server Authentication and Client Authentication as a value.
... View more
01-31-2018
06:25 PM
@vperiasamy am trying to understand what the relevance of the note at the bottom of this solution . Is that solution upto date ? Note: while creating the client certs, make sure you provide extension as"usr_cert"and server cert as"server_cert", other wise 2 WAY SSL communication would fail.
... View more
01-31-2018
06:21 PM
We are not on a kerberos environment yet. in terms of errors in /var/log/ranger/admin/xa_portal.log 2018-01-31 00:00:17,150 [http-bio-6182-exec-2] ERROR org.apache.ranger.common.ServiceUtil (ServiceUtil.java:1376) - Unauthorized access. Unable to get client certificate. serviceName=HadoopCluster_hbase
2018-01-31 00:00:17,151 [http-bio-6182-exec-2] INFO org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request failed. loginId=null, logMessage=Unauthorized access - unable to get client ce
rtificate
javax.ws.rs.WebApplicationException
at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:325)
at org.apache.ranger.common.ServiceUtil.isValidateHttpsAuthentication(ServiceUtil.java:1377)
at org.apache.ranger.rest.ServiceREST.getServicePoliciesIfUpdated(ServiceREST.java:2567)
at org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
2018-01-31 00:00:17,151 [http-bio-6182-exec-2] INFO org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:326) - Operation error. response=VXResponse={org.apache.ranger.view.VXResponse@2a28b481statu
sCode={1} msgDesc={Unauthorized access - unable to get client certificate} messageList={[VXMessage={org.apache.ranger.view.VXMessage@6f0ff521name={OPER_NOT_ALLOWED_FOR_ENTITY} rbKey={xa.error.oper_not_all
owed_for_state} message={Operation not allowed for entity} objectId={null} fieldName={null} }]} }
javax.ws.rs.WebApplicationException
at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:325)
at org.apache.ranger.common.ServiceUtil.isValidateHttpsAuthentication(ServiceUtil.java:1377)
at org.apache.ranger.rest.ServiceREST.getServicePoliciesIfUpdated(ServiceREST.java:2567)
at org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
in /var/log/ranger/kms/kms.log 2018-01-31 00:00:17,544 ERROR PolicyRefresher - PolicyRefresher(serviceName=HadoopCluster_kms): failed to refresh policies. Will continue to use last known version of policies (-1)
java.lang.IllegalArgumentException: SSLContext must not be null
at com.sun.jersey.client.urlconnection.HTTPSProperties.<init>(HTTPSProperties.java:106)
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.ja
(don't think this is used anywhere .. and can be ignored)
2018-01-31 00:00:17,529 WARN FSInputChecker - Problem opening checksum file: file:/etc/ranger/HadoopCluster_kms/cred.jceks. Ignoring exception:
java.io.FileNotFoundException: /etc/ranger/HadoopCluster_kms/.cred.jceks.crc (Permission denied)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
in /var/log/hadoop/hdfs/hadoop-hdfs-namednode 2018-01-31 18:17:05,296 WARN client.RangerAdminRESTClient (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(162)) - Error getting policies. secureMode=false, user=hdfs (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, serviceName=HadoopCluster_hadoop
2018-01-31 18:17:06,824 INFO BlockStateChange (BlockManager.java:computeReplicationWorkForBlocks(1653)) - BLOCK* neededReplications = 0, pendingReplications = 0.
2018-01-31 18:17:08,325 WARN mortbay.log (Slf4jLog.java:warn(76)) - SSL renegotiate denied: java.nio.channels.SocketChannel[connected local=/192.168.10.20:50470 remote=/192.168.10.20:45972]
2018-01-31 18:17:08,333 WARN mortbay.log (Slf4jLog.java:warn(76)) - SSL renegotiate denied: java.nio.channels.SocketChannel[connected local=/192.168.10.20:50470 remote=/192.168.10.20:45970]
In the ranger admin UI logged in as the keyadmin user > service manager > edit kms service > test connection rg.apache.ranger.plugin.client.HadoopException: {
"RemoteException" : {
"message" : "User:keyadmin not allowed to do 'GET_KEYS'",
"exception" : "AuthorizationException",
"javaClassName" : "org.apache.hadoop.security.authorize.AuthorizationException"
}
}.
{
"RemoteException" : {
"message" : "User:keyadmin not allowed to do 'GET_KEYS'",
"exception" : "AuthorizationException",
"javaClassName" : "org.apache.hadoop.security.authorize.AuthorizationException"
}
}.
... View more
01-31-2018
07:27 AM
The responses above helped me with the problems i had, however the right answer is that when using blueprints in version 2.6 onwards when the vdf file is registered, we have to specify the repositories in that file. That input is then used to create another ambari-hdp-repo-1.repo which will then be subsequently used.
... View more
01-31-2018
07:21 AM
@amarnath reddy pappu can you please elaborate on the Note: about providing extension as "usr_cert" and "server_cert" ? I have a wildcard certificate and after following all the above steps and also with modifications as mentioned by @Luis Vazquez the plugins don't show up in the ranger ui and the error is keyadmin is not allowed to do "GET_KEYS". the documentation on setting this up correctly using CA signed certs is suprisingly sparse.
... View more
01-31-2018
04:38 AM
Hi I have a CA signed wildcard cert for my company like *.mycompany.com and am attempting to set it up for the cluster ssl setup. I have it setup successfully for all components except solr and ranger. Specific to ranger my intention to use the CA signed cert and key for ALL the ranger plugins and the ranger admin . I understand that without kerberos there can only be 2 way ssl. After following the steps as documented here ranger admin serves up properly however 1. during ranger admin client install the solr cloud cannot create the ranger-audit collection because the cert that it is trying to verify tries to pick up the ip instead of the hostname which i will try and follow up with this 2. None of the hdfs/hbase/hive plugins appear in the ranger admin and when I attempt to test connection in the kms view of ranger admin the test fails saying that keyadmin user has no authorization for "GET keys" so my question is that will the above setup work i.e can i use the same keystore for all plugins and the ranger ui using the wildcard certificate and then use the same truststore for all ? we maintain our own network level security . I am on the hdp 2.6.4 stack
... View more
Labels:
- Labels:
-
Apache Ranger
01-31-2018
12:15 AM
Hi So I am attempting to use my CA signed cert for ranger auditing. Although I don't have the complete setup running yet one of the issues I am facing is that ranger cannot initiate the solr collection because of the following error Note that this is a CA issued wildcard cert for *.my-company.com and it works properly across certs and other products. Why is it that it is trying to use the ip address rather than the hostname which would probably then give the right result. I have looked around in the exported blueprint and I don't any reference to the ip ; just the hostname which all end with *.my-company.com and thus they should be resolved. Am using solr cloud so the ranger.audit.solr.urls = "" and the ranger.audit.solr.zookeepers="server1.my-company.com:2181,server2.my-company.com:2181,server3.my-company.com:2181/infra-solr" No live SolrServers available to handle this request:[https://192.168.10.20:8886/solr]
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://192.168.10.20:8886/solr]
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:891)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:827)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at org.apache.ambari.logsearch.solr.commands.AbstractSolrRetryCommand.createAndProcessRequest(AbstractSolrRetryCommand.java:43)
at org.apache.ambari.logsearch.solr.commands.AbstractRetryCommand.retry(AbstractRetryCommand.java:45)
at org.apache.ambari.logsearch.solr.commands.AbstractRetryCommand.run(AbstractRetryCommand.java:40)
at org.apache.ambari.logsearch.solr.AmbariSolrCloudClient.listCollections(AmbariSolrCloudClient.java:102)
at org.apache.ambari.logsearch.solr.AmbariSolrCloudClient.createCollection(AmbariSolrCloudClient.java:109)
at org.apache.ambari.logsearch.solr.AmbariSolrCloudCLI.main(AmbariSolrCloudCLI.java:473)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://192.168.10.20:8886/solr
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
... 11 more
Caused by: javax.net.ssl.SSLException: Certificate for <192.168.10.20> doesn't match common name of the certificate subject: *.my-company.com
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:172)
at org.apache.http.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:61)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:140)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:114)
at org.apache.http.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:569)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:544)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:482)
... 15 more
... View more
Labels:
- Labels:
-
Apache Ranger
-
Apache Solr
01-17-2018
09:51 PM
Thanks for your inputs @Jay Kumar SenSharma and @Aditya Sirna but for me updating to the latest version 2.6.1 was a possibility and that worked. As feedback the blueprint installation will work but in case of failures like starting services for instance the metrics monitor because of python dependencies can there be a hook in the process or really a part of the installation itself which does that ? Otherwise we are basically left with a cluster which is installed but cannot be started. If certain components fail to install (say on a node) and as a consequence the subsequent packages then would it be possible to restart the cluster provisioning request from that point ? Is it possible to have a heirarchy of component installation and start ? i.e is it really necessary to install the metrics monitor before and start it before the core services ?
... View more
01-17-2018
09:37 PM
resolved the above oozie problem ; the problem was a corrupt bigtop-tomcat in our repository
... View more