Hi, I've configured usersync with LDAP (IPA backend) successfully and also LDAP for the Admin UI. What I want to achieve: All the users from the cluster are sync with usersync. Admin group can login to Ranger Admin and create rules Auditer can login to Ranger and see only the Audit tabs Standard user cannot login to Ranger Admin UI. Current state: Usersync is OK Admin group is OK Auditer can login OK Standard user CAN login to Ranger Admin and see only "Resource Based Policies" and "Reports" KO I've tried to filter users who can login to Ranger Admin via LDAP: ranger.ldap.user.searchfilter: (&(uid={0})(memberOf=cn=my_admin_team,cn=groups,cn=accounts,dc=example,dc=com)) but this does not work. I've also tried to remove the users from the permissions (Settings -> Permissions -> Resource Based Policies) as explained here: https://community.hortonworks.com/questions/62605/permissions-for-using-ranger.html but after restarting the Ranger service, all users are back. Looks like the permission is not persisted, or is overriden by the service restart. Note that this standard user can delete policies of other users. Questions: Can we filter which users/groups can login to Ranger Admin UI without changing the permissions (Settings -> Permissions) but only with the properties in Ambari (Ranger -> Config -> Advanced -> LDAP settings ? Is is a limitation with that version of Ranger? How can we remove the default matching for every user with permissions in "Resource Based Policies" and "Reports" HDP 2.6.5, ranger 0.7 from HDP 2.6.5 Thanks ... View more

Hi, We hit a limitation of flume where it failed processing a file bigger than 2GB. I had this stack trace 29 Nov 2017 05:07:03,582 WARN [1030545479@qtp-1230732238-0] (org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost:236) - Deserializer threw unexpected exception. java.lang.NegativeArraySizeException at org.apache.commons.io.output.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:322) at org.apache.flume.source.http.BLOBHandler.getEvents(BLOBHandler.java:83) at org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost(HTTPSource.java:228) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) The problem comes from the BLOBHandler hortonworks-flume. All the flume version are affected including apache flume or cloudera version. We are using HDP 2.3.4 with flume based on cloudera plus our patches. The exact issue is the function toByteArray() which create an array with the maximum size of Integer.MAX_VALUE (=2GB of file) To give more details about the flume pipeline: HTTPSource --> Memory channel --> HDFS sink Are you aware of this issue? Vincent ... View more

Hi, We upgrade our jdk from 1.8.0_112 to 1.8.0_121 and after rebooting the host, ambari-server is not able to received data from the agents. Ambari-server logs is full of: 29 May 2017 20:06:38,009 WARN [qtp-ambari-agent-1091] nio:726 - handle failed java.lang.NoClassDefFoundError: Could not initialize class sun.security.ssl.SupportedEllipticCurvesExtension at sun.security.ssl.HelloExtensions.<init>(HelloExtensions.java:82) at sun.security.ssl.HandshakeMessage$ClientHello.<init>(HandshakeMessage.java:245) at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:220) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) at sun.security.ssl.Handshaker$1.run(Handshaker.java:966) at sun.security.ssl.Handshaker$1.run(Handshaker.java:963) at java.security.AccessController.doPrivileged(Native Method) at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416) at org.eclipse.jetty.io.nio.SslConnection.process(SslConnection.java:375) at org.eclipse.jetty.io.nio.SslConnection.access$900(SslConnection.java:48) at org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.fill(SslConnection.java:715) at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1044) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) In ambari-server UI, all the services and hosts are yellow with the question mark "?". After digging a bit, I found a similar issue https://tickets.puppetlabs.com/browse/SERVER-1720 and it appears to be an improvement of the default strength of EC cryptography they made in jdk 1.8.0_121 https://bugs.openjdk.java.net/browse/JDK-8148912 To fix the issue, I had to revert back to jdk 1.8.0_112 and change the link of /usr/java/default, stopping ambari-server, running "ambari-server setup -j /usr/java/default" and starting again it went back fine. To be sure it was not a configuration problem, I tried to upgrade once again to JDK 1.8.0_121 but I hit the same issue. QUESTIONS: Are you aware of the problem with ambari 2.2.2, and can you confirm the behavior? As we MUST upgrade our java version, which version of ambari work with JDK > 1.8.0_121 ? ... View more

Hi, I have ambari 2.2.2 running. When I've installed the cluster I set up only a local repository for RHEL6. Now I want to add a host with RHEL7 and I add HDP and HDP-UTILS repo but ambari cannot validate it. From the developer console, I have that error: { "status" : 500, "message" : "An internal system exception occurred: Stack data, stackName=HDP, stackVersion= 2.3, osType=redhat7, repoId= HDP-2.3.6.0-3796" } However, I'm confident that those repos are reachable and properly configured as we used them for other cluster. My question: How the validation process works? Second problem, if I skip the repository validation, ambari will not pushed my HDP and HDP-UTILS repo to the host when I install a component, but the public repo. I discover that I have to edit manually the table "metainfo" with the following value to be able to push my repository config to the RHEL7 host. repo:/HDP/2.3/redhat7/HDP-2.3:baseurl | http://mycompany.com/cobbler/repo_mirror/hdp-2_3_6-centos7 repo:/HDP/2.3/redhat7/HDP-UTILS-1.1.0.20:baseurl | http://mycompany.com/cobbler/repo_mirror/HDP-UTILS-1_1_0_20-centos7 Why the metainfo table is so important? Thanks ... View more

Hi, We found an issue after re-installing ambari-metrics in the HDP cluster. We first install ambari-metrics in 50 hosts in a new cluster, then install kafka in 20 hosts and storm in the other 30. Each kafka broker has its own group_config to override the broker.id, broker.rack and host.name. Same with storm where we overridden for each supervisor and nimbus the storm.local.hostname property. Everythings were good at that point. We faced some issue with ambari-metrics and failed to solve them (that's another story). We decide to re-install the component. We proceed like that: Remove znode /ams-hbase-unsecured from the embedded zookeeper (localhost:21181) Stop all ambari-metrics instances Delete ambari-metrics services from ambari Remove all rpm related to ambari-metrics Remove all folder related to ambari-metrics and hbase(/var/lib/ambari-metrics-collector, ...) Re-install ambari-metrics service from ambari. After re-installing with success, we discover that each group_config for kafka and storm were overridden with all the same value. All had the broker.id set to 1, broker.rack = rack1 and host.name= myhostname1.mycompany.com. Same for storm. We test once more the procedure in another cluster with the same configuration and it's happened exactly the same. Our cluster: HDP 2.5.0.0-1245 with kafka 0.10, Storm 1.0 Ambari 2.4.0.1-1 Did someone faced the same issue? Can someone confirm that is a bug in ambari? Thanks ... View more