from some unclear reason when we start the ambari agent on master machine its failed from the log we can see that: ERROR 2017-10-02 11:58:42,597 script_alert.py:123 - [Alert][hive_server_process] Failed with result CRITICAL: ['Connection failed on host machine-master01.pop.com:10000 (Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py", line 211, in execute\n ldap_password=ldap_password)\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hive_check.py", line 79, in check_thrift_port_sasl\n timeout=check_command_timeout)\n File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__\n self.env.run()\n File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run\n what cause this problem? ... View more

we try to start the hive server on master01 machine as the following: [root@master01 hive]# su hive [hive@master01 hive]$ /usr/bin/hive --service hiveserver2 but hive command not print anything and hive server still stop what are the option to check why hive not start? ... View more

we start the spark history as the following /usr/hdp/2.6.0.3-8/spark2/sbin/start-history-server.sh from the log spark-root-org.apache.spark.deploy.history.HistoryServer-1-master01 we get many lines like that Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=root, access=READ, inode="/spark2-history/application_1497173286109_0003_2.inprogress":hdfs:hadoop:-rwxrwx--- we try hdfs dfs -chown spark /spark2-history but not help not sure if we need to see the folder ? # ls /spark2-history/ ls: cannot access /spark2-history/: No such file or directory please advice what is solution in order to strat the spark history ... View more

spark history server not started and from the logs under /var/log/spark2 we see the following 17/10/31 21:00:23 ERROR FsHistoryProvider: Exception encountered when attempting to load application log hdfs://hdfsha/spark2-history/application_1507958402099_0822_1 org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=READ, inode="/spark2-history/application_1507958402099_0822_1":bipflop:hadoop:-rwxrwx--- at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:219) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1955) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1939) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPathAccess(FSDirectory.java:1913) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2000) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1969) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1882) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:699) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:376) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2345) please advice how to resolve this ? do you think that we need to do restore to postgresql from backup in order to solve this? ... View more