Member since
03-14-2019
12
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3815 | 09-23-2020 03:51 AM |
11-11-2020
12:37 AM
Hi @GangWar , I have one doubt if I run "ambari-server reset" I am getting the following. "This will remove all cluster host and configuration information from the database. You will be required to re-configure the Ambari server and re-run the cluster wizard." Is it safe to run this and also what are the poststeps that need to be performed. Also you mentioned I can see more details on this in stderr logs. Can you please guide me to the path of the stderr logs. Thanks and Regards, Ananya
... View more
11-04-2020
02:49 AM
Hi Experts, Please help here. Thanks and Regards, Ananya
... View more
10-22-2020
03:54 AM
Hi Experts, We are using following products. HDP 3.1.5 We have integration with SAP HANA for which SAP HANA spark controller service is installed in ambari. The service however always show stopped and whenever I try to start it manually from ambari I am getting the following error. "404 status code received on PUT method for API: /api/v1/clusters/Hadoop_UAT/services/SPARKCONTROLLER" "Error message: The specified resource doesn't exist: Service not found, clusterName=Hadoop_UAT, serviceName=SPARKCONTROLLER" However when we go from the host and from the list of services from the host try to start it the service get started on OS level however from ambari it still shows as stopped. So basically ambari is not able to detect the state of the service. How to solve the issue any idea. Thanks and Regards, Ananya
... View more
Labels:
10-06-2020
01:59 AM
Hi @GangWar , Thank you very much for your prompt response. However there is no issue with the curl in our servers. I had investigated further and found out that the /var/log/hadoop/hdfs/namenode.log is indicating the following error. 2020-10-06 16:19:01,877 ERROR namenode.NameNode (NameNode.java:main(1715)) - Failed to start namenode. org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /master/hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not exist during upgrade. Finalize or rollback first. at org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:411) at org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:418) at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:438) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:310) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1090) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:714) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:632) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:694) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:910) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1643) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1710) 2020-10-06 16:19:01,879 INFO util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /master/hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not exist during upgrade. Finalize or rollback first. 2020-10-06 16:19:01,879 INFO timeline.HadoopTimelineMetricsSink (AbstractTimelineMetricsSink.java:getCurrentCollectorHost(291)) - No live collector to send metrics to. Metrics to be sent will be discarded. This message will be skipped for the next 20 times. 2020-10-06 16:19:01,880 INFO namenode.NameNode (LogAdapter.java:info(51)) - SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at H8CC0328.stb.gov.sg/10.21.21.85 Also I tried to Finalize my upgrade but finalize HDFS is failing.The error messages are attached. Please help. Thanks and Regards, Ananya
... View more
10-05-2020
11:35 PM
Hi Experts, Please help here.I am stuck in between my upgrade. Thanks and Regards, Ananya
... View more
10-05-2020
01:56 AM
Hi Experts, We are currently in middle of upgrade from HDP 2.6.5 to HDP 3.1.5. Currently we are just before finalizing the upgrade state. As instructed we wanted to make sure everything is fine before we finalize the upgrade. However when we checked we found that the HDFS namenode is not starting. We have HA configuration for our HDFS and both the ZKFAILOVERCONTROLLER is running but HDFS namenode or standby namenode is not starting and it keep on trying without any success. I am attaching the logs and screenshots for your analysis.Please help. Thanks and Regards, Ananya
... View more
Labels:
- Labels:
-
HDFS
-
Hortonworks Data Platform (HDP)
09-23-2020
03:51 AM
1 Kudo
Hi Experts, My issue is resolved now. I had raised a ticket with cloudera and they provided my with a modified yumrpm.py file which I replaced with my existing file. This new python had option to skip in case it encounters a blank package. Following are the steps to resolve my issue. 1) verify ambari-agent version is 2.6.2.2 by `ambari-agent --version` on problematic node 2) stop ambari-agent on problematic node 3)go to /usr/lib/ambari-agent/lib/resource_management/core/providers/package/ ~> cd /usr/lib/ambari-agent/lib/resource_management/core/providers/package/ 4) take bacup of yumrpm.py ~> mv yumrpm.py /tmp 5) download the attached yumrpm.py and keep it in /user/home/ananya 6) do an diff of ~> diff /tmp/yumrpm.py /user/home/ananya/yumrpm.py >>>(diff loooks similar too this) 263,267c263,264 < if len(name) == 0: < Logger.info("Skipping installation of empty package") < else: < Logger.info("Installing package %s ('%s')" % (name, string_cmd_from_args_list(cmd))) < self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput()) --- > Logger.info("Installing package %s ('%s')" % (name, string_cmd_from_args_list(cmd))) > self.checked_call_with_retries(cmd, sudo=True, > logoutput=self.get_logoutput()) <<<< 7) move the yumrpm.py to /usr/lib/ambari-agent/lib/resource_management/core/providers/package/ mv /user/home/ananya/yumrpm.py /usr/lib/ambari-agent/lib/resource_management/core/providers/package/ 😎 start ambari-agent Thanks and Regards, Ananya
... View more
09-17-2020
03:59 AM
Hi @Bender Frenec, Really appreciate your prompt response. I have tried your suggestion. 1. I tried to deregister the 2.6.5 version but the deregister button is disabled and it is saying can not deregister as the version already installed on some nodes. 2. I checked and found no old transaction by running following command on my problematic node. ls -lart /var/lib/yum/transaction* Anything else I can check. Thanks and Regards, Ananya
... View more
09-16-2020
04:06 AM
Hi @Bender Frenec, I am on the installation binary phase and the upgrade is not yet started. If I check on the host where the installation is failing as well as where installation is successful. Both servers still got the packages for HDP 2.3.4. Is it okay to remove them before the upgrade is completed and we finalize the upgrade. Please advise. Thanks and Regards, Ananya
... View more
09-15-2020
10:34 PM
Hi Experts, Any idea on this issue please. We are currently stuck with the upgrade and can not proceed further. Thanks and Regards, Ananya
... View more