Cloudera Community
daniel_zafar
user rank
Contributor
My Accepted Solutions
Show All

Issues adding RegionServers to datanodes

by Contributor in Support Questions
‎08-14-2018 11:17 PM
‎08-14-2018 11:17 PM
Hello, I just installed Hbase RegionServers to 2 of my datanodes. I'm getting the following error when I start them on HDP 3.0: esource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-regionserver/conf start regionserver' returned 127. -bash: /usr/hdp/current/hbase-regionserver/bin/hbase-daemon.sh: No such file or directory What am I missing? I installed them through Ambari. ... View more

Spark2 Phoenix Plugin with Zeppelin

by Contributor in Support Questions
‎08-14-2018 04:42 PM
‎08-14-2018 04:42 PM
Hello, I'm looking to read tables from Phoenix using Zeppelin. I don't want to use the JDBC connection in favor of the plugin described here: https://phoenix.apache.org/phoenix_spark.html I believe I am not specifying the dependencies correctly because even though I added this to Zeppelin Interpreter (see zeppelin-phoenix-config.png). I'm still getting the following error: import org.apache.spark.SparkContext import org.apache.spark.sql.SQLContext import org.apache.phoenix.spark._ error: object phoenix is not a member of package org.apache How do I fix this? I've also added /usr/hdp/current/phoenix-server/lib/:/usr/hdp/current/phoenix-client/lib/:/usr/hdp/current/phoenix-server/ to both spark.driver.extraClassPath and spark.executor.extraClassPath in Spark2's spark.defaults.conf. Thanks in advance! P.S. I'm trying to figure all this out so that I can write a sparklyr extension to interact with Hbase data through Phoenix. Any pointers toward that goal are appreciated as well. ... View more

Re: Timeline Service V2.0 Reader not starting

by Contributor in Support Questions
‎08-13-2018 05:51 PM
‎08-13-2018 05:51 PM
@Akhil S Naik That worked! Nice job! ... View more

Re: Timeline Service V2.0 Reader not starting

by Contributor in Support Questions
‎08-13-2018 03:33 PM
‎08-13-2018 03:33 PM
@Akhil S Naik Hm, I actually have all of those, any other ideas? <br>[root@NCIENSPK01 ~]# cd /usr/hdp/3.0.0.0-1634/hadoop-yarn [root@NCIENSPK01 hadoop-yarn]# ls -l total 12544 drwxr-xr-x 2 root root 93 Aug 10 17:51 bin lrwxrwxrwx 1 root root 16 Aug 10 17:51 conf -> /etc/hadoop/conf drwxr-xr-x 3 root root 30 Aug 10 17:51 etc -rw-r--r-- 1 root root 3097487 Jul 12 16:02 hadoop-yarn-api-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 38 Aug 10 17:51 hadoop-yarn-api.jar -> hadoop-yarn-api-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 110407 Jul 12 16:02 hadoop-yarn-applications-distributedshell-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 64 Aug 10 17:51 hadoop-yarn-applications-distributedshell.jar -> hadoop-yarn-applications-distributedshell-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 55399 Jul 12 16:02 hadoop-yarn-applications-unmanaged-am-launcher-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 69 Aug 10 17:51 hadoop-yarn-applications-unmanaged-am-launcher.jar -> hadoop-yarn-applications-unmanaged-am-launcher-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 300539 Jul 12 16:02 hadoop-yarn-client-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 41 Aug 10 17:51 hadoop-yarn-client.jar -> hadoop-yarn-client-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 2814263 Jul 12 16:02 hadoop-yarn-common-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 41 Aug 10 17:51 hadoop-yarn-common.jar -> hadoop-yarn-common-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 224598 Jul 12 16:02 hadoop-yarn-registry-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 43 Aug 10 17:51 hadoop-yarn-registry.jar -> hadoop-yarn-registry-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 294798 Jul 12 16:02 hadoop-yarn-server-applicationhistoryservice-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 67 Aug 10 17:51 hadoop-yarn-server-applicationhistoryservice.jar -> hadoop-yarn-server-applicationhistoryservice-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 1336803 Jul 12 16:02 hadoop-yarn-server-common-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 48 Aug 10 17:51 hadoop-yarn-server-common.jar -> hadoop-yarn-server-common-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 1293507 Jul 12 16:02 hadoop-yarn-server-nodemanager-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 53 Aug 10 17:51 hadoop-yarn-server-nodemanager.jar -> hadoop-yarn-server-nodemanager-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 2172705 Jul 12 16:02 hadoop-yarn-server-resourcemanager-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 57 Aug 10 17:51 hadoop-yarn-server-resourcemanager.jar -> hadoop-yarn-server-resourcemanager-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 149224 Jul 12 16:02 hadoop-yarn-server-router-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 48 Aug 10 17:51 hadoop-yarn-server-router.jar -> hadoop-yarn-server-router-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 94342 Jul 12 16:02 hadoop-yarn-server-sharedcachemanager-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 60 Aug 10 17:51 hadoop-yarn-server-sharedcachemanager.jar -> hadoop-yarn-server-sharedcachemanager-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 43865 Jul 12 16:02 hadoop-yarn-server-tests-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 47 Aug 10 17:51 hadoop-yarn-server-tests.jar -> hadoop-yarn-server-tests-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 93081 Jul 12 16:02 hadoop-yarn-server-timeline-pluginstorage-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 64 Aug 10 17:51 hadoop-yarn-server-timeline-pluginstorage.jar -> hadoop-yarn-server-timeline-pluginstorage-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 80404 Jul 12 16:02 hadoop-yarn-server-web-proxy-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 51 Aug 10 17:51 hadoop-yarn-server-web-proxy.jar -> hadoop-yarn-server-web-proxy-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 103378 Jul 12 16:02 hadoop-yarn-services-api-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 47 Aug 10 17:51 hadoop-yarn-services-api.jar -> hadoop-yarn-services-api-3.1.0.3.0.0.0-1634.jar -rw-r--r-- 1 root root 539627 Jul 12 16:02 hadoop-yarn-services-core-3.1.0.3.0.0.0-1634.jar lrwxrwxrwx 1 root root 48 Aug 10 17:51 hadoop-yarn-services-core.jar -> hadoop-yarn-services-core-3.1.0.3.0.0.0-1634.jar drwxr-xr-x 2 root root 4096 Aug 10 17:51 lib drwxr-xr-x 2 root root 49 Aug 10 17:51 sbin drwxr-xr-x 3 root root 4096 Aug 10 17:51 timelineservice drwxr-xr-x 3 root root 16 Aug 10 17:51 webapps drwxr-xr-x 5 root root 51 Aug 10 17:51 yarn-service-examples Then in timeline service: [root@NCIENSPK01 hadoop-yarn]# ls timelineservice hadoop-yarn-server-timelineservice-3.1.0.3.0.0.0-1634.jar hadoop-yarn-server-timelineservice-hbase-common-3.1.0.3.0.0.0-1634.jar hadoop-yarn-server-timelineservice-hbase-coprocessor.jar hadoop-yarn-server-timelineservice-hbase-client-3.1.0.3.0.0.0-1634.jar hadoop-yarn-server-timelineservice-hbase-common.jar hadoop-yarn-server-timelineservice.jar hadoop-yarn-server-timelineservice-hbase-client.jar ... View more

Re: Timeline Service V2.0 Reader not starting

by Contributor in Support Questions
‎08-11-2018 01:21 AM
‎08-11-2018 01:21 AM
@amarnath reddy pappu Good idea! This is what I got: [root@DRIVER_NODE ambari-agent]# ./ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/lib64/qt-3.3/bin:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/maven/bin:/root/bin:/opt/maven/bin:/opt/maven/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.0.0-1634/hadoop-yarn/timelineservice/; /usr/hdp/3.0.0.0-1634/hbase/bin/hbase --config /usr/hdp/3.0.0.0-1634/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s' Error: Could not find or load main class org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator ... View more

Re: Timeline Service V2.0 Reader not starting

by Contributor in Support Questions
‎08-11-2018 12:39 AM
‎08-11-2018 12:39 AM
Interestingly I don't have anything in the logs from today at all. I have tried to restart it today.... I am looking in: /var/log/hadoop-yarn/yarn/hadoop-yarn-timelinereader-MYNODE01.log Is that right? ... View more

Timeline Service V2.0 Reader not starting

by Contributor in Support Questions
‎08-11-2018 12:08 AM
1 Kudo
‎08-11-2018 12:08 AM
1 Kudo
Everything in my HDP is working except Timeline Service V2.0 Reader. When I try to start it I get the following error log: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 108, in <module> ApplicationTimelineReader().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 51, in start hbase(action='start') File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 80, in hbase createTables() File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 147, in createTables logoutput=True) File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run returns=self.resource.returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 308, in _call raise ExecuteTimeoutException(err_msg) resource_management.core.exceptions.ExecuteTimeoutException: Execution of 'ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/lib64/qt-3.3/bin:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/maven/bin:/root/bin:/opt/maven/bin:/opt/maven/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.0.0-1634/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.0.0-1634/hbase/bin/hbase --config /usr/hdp/3.0.0.0-1634/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'' was killed due timeout after 300 seconds ... View more
Labels:

Re: Nodemanager bad health and connection refused

by Contributor in Support Questions
‎08-11-2018 12:05 AM
‎08-11-2018 12:05 AM
So I resolved all this. I just followed the steps here to remove all my packages, then deleted the contents of my files: rm -rf /usr/hdp/ Then in Ambari I used the "Start all Services" command and it went through and installed everything again for me. Then to solve the nodemanager issue I did the spark-yarn- install which jave me the missing jar that I needed and then just copied that dir: /usr/hdp/3.0.0.0-.../spark2/aux/ to all the other nodes in my cluster. Now all my nodemanagers are coming up and things are looking good. I'm creating another post about resolving my Timeline Service V2.0 issue which is somehow still persisting. ... View more

Re: Nodemanager bad health and connection refused

by Contributor in Support Questions
‎08-10-2018 09:20 PM
‎08-10-2018 09:20 PM
@Jay Kumar SenSharma I'm definitely in a jam now. Really hoping you can help me. A bit scared to touch anything at this point. ... View more

Re: Nodemanager bad health and connection refused

by Contributor in Support Questions
‎08-10-2018 07:29 PM
‎08-10-2018 07:29 PM
@Jay Kumar SenSharma When I try to start services now I'm getting: For HDFS Client Install RuntimeError: Failed to execute command '/usr/bin/yum -y install hadoop_3_0_0_0_1634', exited with code '1', message: 'Error unpacking rpm package hadoop_3_0_0_0_1634-3.1.0.3.0.0.0-1634.x86_64' For Hive Client Install RuntimeError: Failed to execute command '/usr/bin/yum -y install hive_3_0_0_0_1634-hcatalog', exited with code '1', message: 'Error unpacking rpm package hadoop_3_0_0_0_1634-3.1.0.3.0.0.0-1634.x86_64 ... View more
Contact Me
Online
Offline
Last Visited
‎03-21-2019 06:40 PM
Community Statistics
Member Since ‎08-08-2018 10:30 PM
Last Visited ‎03-21-2019 06:40 PM
Posts 49
Kudos received 2