Member since
03-14-2016
4721
Posts
1111
Kudos Received
874
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2825 | 04-27-2020 03:48 AM | |
| 5486 | 04-26-2020 06:18 PM | |
| 4668 | 04-26-2020 06:05 PM | |
| 3706 | 04-13-2020 08:53 PM | |
| 5607 | 03-31-2020 02:10 AM |
05-12-2019
10:51 PM
@Madhura Mhatre Can you please run the following query inside your MySQL DB so that we can findout which database ( ambaridb/rangerdb/hivedb) is actually utilizing more memory? Please share the output. mysql> SELECT table_schema "DB Name", ROUND(SUM(data_length + index_length) / 1024 / 1024, 1) "DB Size in MB" FROM information_schema.tables GROUP BY table_schema; . then accordingly we can check if there is a possibility to clear some unwanted data. Like in case of ambaridb it may be the old unwanted alerts and operational data that might be consuming most of the db space. In such cases Ambari Provides a utility to purge the historical unwanted data: # ambari-server db-purge-history --cluster-name YOUR_CLUSTER_NAME --from-date 2018-08-01 https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/administering-ambari/content/amb_purge_ambari_server_database_history.html . Also to reclaim diskspace used by MySQL DB sometimes you might need to do "OPTIMIZE TABLE <tablename>" on the DB https://www.percona.com/blog/2013/09/25/how-to-reclaim-space-in-innodb-when-innodb_file_per_table-is-on/
... View more
05-07-2019
01:33 AM
@Joha Hazimeh Your driverName seems to be slightly different. You are using "org.apache.hadoop.hive.jdbc.HiveDriver" so can you please try using the "org.apache.hive.jdbc.HiveDriver" class name to see if it works. then see if it works? There is a simple example available here you might want to quickly test it: https://github.com/jaysensharma/MiddlewareMagicDemos/tree/master/HDP_Ambari/Hive/HiveJavaClient Also can you please let us know how exactly are you running it? Like how you are setting the classpath... may be some incorrect/different version of JARs are there in your classpath.
... View more
04-06-2019
08:10 PM
@Michael Bronson Yes, it can be done something as described in: https://community.hortonworks.com/questions/244121/api-how-to-set-both-thrift-server-in-ambari-to-mai.html?childToView=244130#answer-244130
... View more
04-06-2019
08:08 PM
1 Kudo
@Michael Bronson Find the hostnames where the "SPARK2_THRIFTSERVER" server is running: # curl -H "X-Requested-By: ambari" -u admin:admin -X GET "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts?(host_components/HostRoles/component_name=SPARK2_THRIFTSERVER)&minimal_response=true" | grep host_name | awk -F":" '{print $2}' | awk -F"\"" '{print $2}' Example Output: newhwx3.example.com newhwx5.example.com Once we know the hosts where the "SPARK2_THRIFTSERVER" is running then we can run the following command by replacing the host newhws3 and newhwx5 to turn ON the maintenance mode for it. # curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn ON Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"ON"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx3.example.com/host_components/SPARK2_THRIFTSERVER"
# curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn ON Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"ON"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx5.example.com/host_components/SPARK2_THRIFTSERVER" . Turn OFF maintenance Mode for Spark2 thrift server on newhwx3 and newhws5 # curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn OFF Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"OFF"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx3.example.com/host_components/SPARK2_THRIFTSERVER"
# curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn OFF Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"OFF"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx5.example.com/host_components/SPARK2_THRIFTSERVER" .
... View more
04-06-2019
07:54 PM
1 Kudo
@Michael Bronson Example Putting a Service (Spark2) in Maintenance Mode using API call: Turning ON maintenance Mode for Spark2 Service # curl -sH "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn On Maintenance Mode for Spark2"},"Body":{"ServiceInfo":{"maintenance_state":"ON"}}}' http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/services/SPARK2 Turning OFF maintenance Mode for Spark2 Service # curl -sH "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn Off Maintenance Mode for Spark2"},"Body":{"ServiceInfo":{"maintenance_state":"OFF"}}}' http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/services/SPARK2 Please change the cluster name "NewCluster" and ambari hostname + credentials according to yoiur env. .
... View more
04-01-2019
10:38 PM
1 Kudo
@Alessandro As we see the following error while installing the SHELL interpreter. org.sonatype.aether.RepositoryException: Cannot fetch dependencies for org.apache.zeppelin:zeppelin-shell:0.8.0
at org.apache.zeppelin.dep.DependencyResolver.getArtifactsWithDep(DependencyResolver.java:179)
.
.
Caused by: java.lang.NullPointerException
at org.sonatype.aether.impl.internal.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:352) . This can happen if you do not have internet access (or if you have soem n/w issue_ on the Zeppelin server host. Can you please isolate that? You can also refer to the following article for similar issues: https://community.hortonworks.com/content/kbentry/84498/zeppelin-does-not-get-installed-properly-without-i.html . Basically We need to have the dependenices (jars) under "/usr/hdp/current/zeppelin-server/local-repo" and the interpreters inside the "/usr/hdp/current/zeppelin-server/interpreter/" dir if he interpreter installation is successful without internet issue.
... View more
03-23-2019
07:54 PM
1 Kudo
@Alessandro Volcich The hadoop command line utility also provides an option to get the correct classpath produced something like following which you can use: Example: # export CLASSPATH=`hadoop classpath`:.:
# echo $CLASSPATH
# javac -d wordcount_classes WordCount.java .
... View more
03-22-2019
05:43 AM
1 Kudo
@Franklin Please try this: 1. On ambari agent host please find this script "/usr/lib/ambari-agent/lib/ambari_agent/CustomServiceOrchestrator.py" and then search for a string with 'if the return type is not used' # grep -A2 'if the return type is not used' /usr/lib/ambari-agent/lib/ambari_agent/CustomServiceOrchestrator.py
# if the return type is not used
base_dir = self.file_cache.get_service_base_dir(command)
script_path = self.resolve_script_path(base_dir, script) . 2. Just add a new line on all agent hosts self.file_cache.get_host_scripts_base_dir(command) there as following Just below the commented line: # grep -A2 'if the return type is not used' /usr/lib/ambari-agent/lib/ambari_agent/CustomServiceOrchestrator.py
# if the return type is not used
self.file_cache.get_host_scripts_base_dir(command)
base_dir = self.file_cache.get_service_base_dir(command)
script_path = self.resolve_script_path(base_dir, script) . **NOTE:** Python is Indentation sensitive so you will need to put the same indentation while adding the line there (else it might cause some syntax error) 3. Clear the Agent Cache Dir (or move it as backup) # mv /var/lib/ambari-agent/cache /var/lib/ambari-agent/cache_OLD . 4. Restart the ambari server and agent. # ambari-server restart
# ambari-agent restart . 5. Now check the host script dir if it has pulled file the custom scripts from ambari server host. # ls -lart /var/lib/ambari-agent/cache/host_scripts/ . It looks related to: https://issues.apache.org/jira/browse/AMBARI-25123
... View more
03-21-2019
01:05 AM
@Saul Ramirez Looks like your Public_hostname for the agent is not right. As we see thsat the public_hostname is being taken as : \'publicHostname\': \'<!doctype html public "-//w3c//dtd xhtml 1.0 transitional//en" "http://www.w3.org/tr/xhtml1/dtd/xhtml1-transitional.dtd">\\r\
<html xmlns="http://www.w3.org/1999/xhtml">\\r\
<head>\\r\
<meta content="text/html; charset=utf-8" http-equiv="content-type" />\\r\
<meta content="no-cache" http-equiv="pragma" />\\r\
<title>waiting...</title>\\r\
<script type="text/javascript">\\r\ . Thats why you are getting too long column value for public_hostname org.eclipse.persistence.exceptions.DatabaseException\
Internal Exception: org.postgresql.util.PSQLException: ERROR: el valor es demasiado largo para el tipo character varying(255)\
Error Code: 0\
Call: UPDATE hosts SET total_mem = ?, host_attributes = ?, public_host_name = ? WHERE (host_id = ?)\
\\tbind => [4 parameters bound]\ . Can you please run the following command on yourt host which you re trying to add to see if it is returning correct FQDN? # hostname -f
You can refer to the following Article which i wrote sometime back to fidn the difference between Public_hostname and hostname and how to fix it: https://community.hortonworks.com/content/kbentry/42872/why-ambari-host-might-have-different-public-host-n.html
... View more
03-15-2019
09:13 AM
@Paul Heinzlreiter Great to know that it resolved your issue. It will be great if you can mark this HCC thread as "Answered" by clicking on "Accept" button on the correct answer.
... View more