Hi ,   I am getting below error when i ran the below command    1st question bash-4.1$ yarn logs -applicationId application_55261206146721_39976 18/05/23 23:10:20 INFO client.RMProxy: Connecting to ResourceManager at xxxxxxxxxxxxxxxxxxxxxxxxx /tmp/logs/yarn/logs/ application_55261206146721_39976 does not exist. Log aggregation has not completed or is not enabled.   i already enabled    yarn.log-aggregation-enable=true yarn.nodemanager.remote-app-log-dir=/tmp/logs yarn.log-aggregation.retain-seconds=3 days     2nd question: does below command syntax  supported in cloudera?    yarn logs -applicationId application_332332323073474_0002 -show_application_log_info yarn logs -applicationId application_1 332332323073474_0002 -show_container_log_info   Regards Jacqueline   ... View more

Can someone distinguish yarn container vs yarn Child Regards JJ ... View more

Hi Gurus. Our Ambari Metrics collector goes down. Our is Kerberos env. Ambari Version is :Version2.4.2.0 i see below errors in ambari-metrics-collector.log i see belwo error FATAL org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer: Error starting ApplicationHistoryServer org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.MetricsSystemInitializationException: Error creating Metrics Schema in HBase using Phoenix. Caused by: org.apache.phoenix.exception.PhoenixIOException: SYSTEM.CATALOG Caused by: org.apache.hadoop.hbase.TableNotFoundException: SYSTEM.CATALOG in hbase-ams-master-xxxxxxxxxx.log 2017-12-23 07:40:57,717 ERROR [xxxxxx,61300,1513971592600_ChoreService_1] master.BackupLogCleaner: Failed to get hbase:backup table, therefore will keep all files org.apache.hadoop.hbase.TableNotFoundException: hbase:backup I implemented below procedure but no luck https://community.hortonworks.com/questions/73577/problem-with-ambari-metrics.html 1) Turn on Maintenance mode 2) Stop Ambari Metrics 3) hadoop fs -rmr /ams/hbase/* 4) rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/* 5) [zk: localhost:2181(CONNECTED) 0] ls / [zk: localhost:2181(CONNECTED) 1] rmr /ams-hbase-unsecure 6) Start Ambari Metrics 7) Turn off Maintenance mode My setting are : hbase.rootdir = hdfs://hpdprod:8020/user/ams/hbase hbase.cluster.distributed=true timeline.metrics.service.operation.mode=distributed Any help is really appreciate Thanks JJ ... View more

unix ldapsearch workfine ,but ranger ldaptool is failing below ldapsearch works fine : ldapsearch -h free-ipa-dev-01.uat.txdc.datastax.com -x -b "dc=txdc,dc=datastax,dc=com" -W hadoopadmin but ranger ldaptool is failing : [root@dev-rng-001 ~]# cd /usr/hdp/current/ranger-usersync/ldaptool [root@dev-rng-001 ldaptool]# ./run.sh -d users Ldap url [ldap://ldap.example.com:389]: ldaps://free-ipa-dev-01.uat.txdc.datastax.com:636 Bind DN [cn=admin,ou=users,dc=example,dc=com]: hadoopadmin Bind Password: User Search Base [ou=users,dc=example,dc=com]: dc=txdc,dc=datastax,dc=com User Search Filter [cn=user1]: cn=* Reading ldap properties from input.properties ERROR: Failed to perfom ldap bind. Please verify values for ranger.usersync.ldap.binddn and ranger.usersync.ldap.ldapbindpassword javax.naming.CommunicationException: simple bind failed: free-ipa-dev-01.uat.txdc.datastax.com:636 [Root exception is javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target] Can you please help Regards JJ ... View more

Right now we have only one topic , with 3 partitions and with a replicaiton factor of 3. My log.dirs has 4 mount point or paths or directories. Only 1 mount point is being used out of 4 . Other 3 mount point or paths or directories , mentioned in log.dirs are not been used. How can i make all the mount / directories , get used. Thanks JJ ... View more

Hi , policymgr_external_url parameter ( External URL ) , when pointed the load balancer vip , Ranger sink with Active Directory isnot happening. We have successfully setup Ranger Admin HA . External URL was pointing to load balancer vip.With this new configuration, Ranger sink with Active Directory is not happening. if we replace the load balance vip with actual host name , able get all the Active Directory user list. is there any problem with setting up load balancer vip ? , i am not getting. http://dev-rag.dataquest.com:6080” is not working for AD user sink. can we have multiple hostname for policymgr_external_url parameter ( External URL ) like“ http://dev-rag-001.dataquest.com,dev-rag-002.dataquest.com:6080” ? please share your thoughts and experiences Regards JJ ... View more

Ranger is not getting updated with all the Active Directory accounts. select * from x_portal_user; ---> does not show Active Directory accounts. ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add User Group Info -- Connection reset Kerberos was successfully implemented. After enabling kerberos following values we set for ranger. 1) ranger.usersync.kerberos.keytab : /etc/security/keytabs/rangerusersync.service.keytab 2) ranger.usersync.kerberos.principal : rangerusersync/_HOST@DEV.DATAQUEST.COM 3) ranger.usersync.policymgr.username : rangerusersync But klist for keytab show the principal as : rangerusersync/rng-node1.dev.dataquest.com@DEV.DATAQUEST.COM Note : we do not have any ssl implementation and certificate or keystore adn thruststore is not needed for us. below is the error message from usersync.log ``` 07 Sep 2017 05:02:42 ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add User Group Info : com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Connection reset at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.getUsergroupInfo(PolicyMgrUserGroupBuilder.java:567) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.access$500(PolicyMgrUserGroupBuilder.java:72) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$2.run(PolicyMgrUserGroupBuilder.java:539) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$2.run(PolicyMgrUserGroupBuilder.java:535) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addUserGroupInfo(PolicyMgrUserGroupBuilder.java:535) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addOrUpdateUser(PolicyMgrUserGroupBuilder.java:340) at org.apache.ranger.ldapusersync.process.LdapUserGroupBuilder.updateSink(LdapUserGroupBuilder.java:327) at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:58) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:706) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:249) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) ... 15 more 07 Sep 2017 05:02:42 INFO LdapUserGroupBuilder [UnixUserSyncThread] - groupSearch is enabled, would search for groups and compute memberships 07 Sep 2017 05:02:43 INFO LdapUserGroupBuilder [UnixUserSyncThread] - LDAPUserGroupBuilder.getGroups() completed with group count: 0 ``` Any help is really appreciated. ... View more

Hi Experts, Can i more the 1 Active Name Node in HDP Cluster? . Prefer to have 1 Stand by Name node for multiple Active Name Node. Does this configuration is possible? Thanks JJ ... View more

Hi , How can we modify below properties to new location hadoop_pid_dir_prefix yarn_pid_dir_prefix hive_pid_dir hcat_pid_dir All these parameters are grayed out and does not let me to modify. I want to modify their exisitng configuration to my custom folder / directory. Thanks JJ ... View more

how can we modify the below ranger parameters. •ranger_admin_log_dir •ranger_pid_dir •ranger_usersync_log_dir •ranger.tagsync.logdir Rigth now in ambari these are grayed out and does not let me to modify From the os command prompt ,i tried to modify below two files /usr/hdp/current/ranger-admin/ews/ranger-admin-services.sh /usr/hdp/current/ranger-usersync/ranger-usersync-services.sh example from pidf=/var/run/ranger/rangeradmin.pid to pidf=/var/log/hdp/ranger/run/rangeradmin.pid There after Ranger servers failed to come up Your help is needed ... View more

Hi , ambari-admin-password-reset : say command not found. How to reset ambari admin password? Do we need to run from any particular directory / path ? we are running Ambari Version 2.4.2.0 I have seen couple of links where they set ambari password from mysql . Is it true? Regards JJ ... View more