Member since
02-27-2017
171
Posts
9
Kudos Received
0
Solutions
02-22-2017
06:56 AM
Hi, I am trying to setup SSL between different HDP components. For that i need to modify some files i.e. hdfs-site.xml, core-site.xml etc. So i stopped HDFS, YARN, MAPREDUCE services from Ambari dashboard and made changes to these config files from ambari. When i am trying to restart services it is showing error and not able to restart. I checked Ambari-server logs and found below 22 Feb 2017 06:50:03,692 ERROR [ambari-heartbeat-processor-0] HeartbeatProcessor:554 - Operation failed - may be retried. Service component host: DATANODE, host: slave1 Action id 120-0 and Task id 961
22 Feb 2017 06:50:03,701 ERROR [ambari-heartbeat-processor-0] HeartbeatProcessor:554 - Operation failed - may be retried. Service component host: NAMENODE, host: master Action id 120-0 and Task id 958 22 Feb 2017 06:50:03,715 ERROR [ambari-heartbeat-processor-0] HeartbeatProcessor:554 - Operation failed - may be retried. Service component host: DATANODE, host: slave2 Action id 120-0 and Task id 959 My ambari-agents are running on all the nodes of the cluster(2 slave nodes and 1 namenode). I have no idea why it is not taking heartbeats. Any help would be appreciated? Thanks Rahul
... View more
Labels:
- Labels:
-
Apache Ambari
02-21-2017
02:44 PM
@Jay SenSharma Thoughts?
... View more
02-21-2017
02:44 PM
I am trying to setup SSL for HDFS on 4 Node cluster(Edge Node, Master Node and 2 Slave Nodes). I am trying to follow the link mentioned below to setup SSL between different hadoop services. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_Security_Guide/content/create-internal-ca.html i want to use the option of Creating and Setting internal CA repository(openssl). I have created the key and certificate for each node in cluster(including edge node).I am finding to hard to understand that where should i be keeping the truststore and keystore across different machines of the cluster? Shall i put it only on Namenode only or all node? Will Edge Node be part of setting up this SSL? Please suggest the way to setup SSL in case of MultiNode Cluster?
... View more
Labels:
- Labels:
-
Apache Ambari
02-16-2017
09:56 AM
@Jay SenSharma Following the link below i have added all the resp. properties in custom core-site.xml(Ambari) But no success. http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_ambari_views_guide/content/_configuring_your_cluster_for_files_view.html i have added 4 more properties named as hadoop.proxyuser.root.groups=*
hadoop.proxyuser.root.hosts=*
hadoop.proxyuser.admin.groups=*
hadoop.proxyuser.admin.hosts=*
... View more
02-16-2017
09:12 AM
@Jay SenSharma Error mentioned below 16 Feb 2017 09:11:22,733 ERROR [ambari-client-thread-10030] ContainerResponse:537 - Mapped exception to response: 500 (Internal Server Error)
org.apache.ambari.view.hive2.utils.ServiceFormattedException
at org.apache.ambari.view.hive2.resources.jobs.viewJobs.JobControllerImpl.setupQueryFile(JobControllerImpl.java:270)
at org.apache.ambari.view.hive2.resources.jobs.viewJobs.JobControllerImpl.setupQueryFileIfNotPresent(JobControllerImpl.java:178)
at org.apache.ambari.view.hive2.resources.jobs.viewJobs.JobControllerImpl.afterCreation(JobControllerImpl.java:164)
at org.apache.ambari.view.hive2.resources.jobs.viewJobs.JobResourceManager.create(JobResourceManager.java:56)
at org.apache.ambari.view.hive2.resources.jobs.JobService.create(JobService.java:523)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
... View more
02-16-2017
08:50 AM
@Jay SenSharma Yes. I even have given 777 as permission as well hadoop fs -ls /user
Found 8 items drwxrwxrwx - admin hadoop 0 2017-02-16 07:33 /user/admin It looks good but donot know why it is not working.
... View more
02-16-2017
07:32 AM
@Jay SenSharma I have created a new View named File View New and Hive View New in Local Cluster. My checks are passed now and i am able to get into hdfs directories. But There are 2 things which i observed 1) When i try to open file stored in HDFS then i am getting the same errors 16 Feb 2017 07:27:58,295 ERROR [ambari-client-thread-7609] ContainerResponse:537 - Mapped exception to response: 500 (Internal Server Error)
org.apache.ambari.view.commons.exceptions.ServiceFormattedException 2) Under Hive View when i run a any command then i am seeing error like "Could not write to /user/admin/hive/jobs(HDFSSApiException). Although i have created directory using below mentioned commands but still not able to see why this is happening. I am using below mentioned link to create admin directory in HDFS. And i am getting 2nd error as mentioned in below link https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_ambari_views_guide/content/troubleshooting.html
... View more
02-16-2017
07:14 AM
5 Kudos
@Nikita Kiselev Try below commands and check
sudo su - hdfs hdfs dfs -mkdir /user/admin hdfs dfs -chown root:hdfs /user/admin
... View more
02-16-2017
06:26 AM
@Jay SenSharma I understood that. I am managing cluster using Ambari and i deployed the same using Ambari as well.(4 Node). But is there a separate way to identify if it is a local or Remote? Because as mentioned in link, in case of local cluster File View should be able to communicate with HDFS on its own. Please suggest. Thanks Rahul
... View more
02-16-2017
06:15 AM
@Jay SenSharma Error logs from ambari-server.log are below 16 Feb 2017 06:13:39,297 ERROR [ambari-client-thread-6610] ContainerResponse:537 - Mapped exception to response: 500 (Internal Server Error)
org.apache.ambari.view.commons.exceptions.ServiceFormattedException
at org.apache.ambari.view.commons.hdfs.HdfsService.getApi(HdfsService.java:95)
at org.apache.ambari.view.commons.hdfs.UserService.homeDir(UserService.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMetho
dDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
... View more