Member since
03-14-2016
4721
Posts
1111
Kudos Received
874
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2450 | 04-27-2020 03:48 AM | |
4888 | 04-26-2020 06:18 PM | |
3977 | 04-26-2020 06:05 PM | |
3220 | 04-13-2020 08:53 PM | |
4928 | 03-31-2020 02:10 AM |
05-07-2019
07:06 AM
@Victor L The error indicates that you might be running "NiFI" server as "root" user and the "root" user does not have read/write permission on the HDFS directory "/data/clustering/temp/" where your NiFi processor is trying to write as the Dir permission on your HDFS is as "hdfs:hdfs:drwxr-xr-x": So you may need to choose the options like Change the permission of "/data/clustering/temp/" HDFS directory recursively so that the "root" user cvan write on HDFS. # su - hdfs -c "hdfs dfs -chown -R root:hadoop /data/" .
... View more
05-07-2019
01:33 AM
@Joha Hazimeh Your driverName seems to be slightly different. You are using "org.apache.hadoop.hive.jdbc.HiveDriver" so can you please try using the "org.apache.hive.jdbc.HiveDriver" class name to see if it works. then see if it works? There is a simple example available here you might want to quickly test it: https://github.com/jaysensharma/MiddlewareMagicDemos/tree/master/HDP_Ambari/Hive/HiveJavaClient Also can you please let us know how exactly are you running it? Like how you are setting the classpath... may be some incorrect/different version of JARs are there in your classpath.
... View more
04-29-2019
11:15 PM
@Daniel L. Can you share this data so that we can find out what might be wrong. 1. Restart ambari-agent on a problematic host and then collect the full log after it got restarted for 3-5 minutes after restart. 2. Also please share the output of the following commands: # hostname -f
# cat /etc/hosts
# cat /etc/ambari-agent/conf/ambari-agent.ini | grep 'hostname = ' 3. From Agent host are you able to make this call to the Ambari FQDN to see if you are able to connect to correct Ambari Server on port 8440 from problematic agent host. Or if the Ambari server / Agent FQDN got changed? # telnet $AMBARI_FQDN 8440
(OR)
# nc -v $AMBARI_FQDN 8440 . 4. When you restart ambari-agent please collect few minutes log of ambari-server as we for the same time when you tried to restart the agent so that we can see if the ambari-agent is sending the registration request/ Heartbeat properly to the Ambari Server correctly or not?
... View more
04-06-2019
08:10 PM
@Michael Bronson Yes, it can be done something as described in: https://community.hortonworks.com/questions/244121/api-how-to-set-both-thrift-server-in-ambari-to-mai.html?childToView=244130#answer-244130
... View more
04-06-2019
08:08 PM
1 Kudo
@Michael Bronson Find the hostnames where the "SPARK2_THRIFTSERVER" server is running: # curl -H "X-Requested-By: ambari" -u admin:admin -X GET "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts?(host_components/HostRoles/component_name=SPARK2_THRIFTSERVER)&minimal_response=true" | grep host_name | awk -F":" '{print $2}' | awk -F"\"" '{print $2}' Example Output: newhwx3.example.com newhwx5.example.com Once we know the hosts where the "SPARK2_THRIFTSERVER" is running then we can run the following command by replacing the host newhws3 and newhwx5 to turn ON the maintenance mode for it. # curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn ON Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"ON"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx3.example.com/host_components/SPARK2_THRIFTSERVER"
# curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn ON Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"ON"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx5.example.com/host_components/SPARK2_THRIFTSERVER" . Turn OFF maintenance Mode for Spark2 thrift server on newhwx3 and newhws5 # curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn OFF Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"OFF"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx3.example.com/host_components/SPARK2_THRIFTSERVER"
# curl -H "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn OFF Maintenance Mode for Spark2 Thrift Server"},"Body":{"HostRoles":{"maintenance_state":"OFF"}}}' "http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/hosts/newhwx5.example.com/host_components/SPARK2_THRIFTSERVER" .
... View more
04-06-2019
07:54 PM
1 Kudo
@Michael Bronson Example Putting a Service (Spark2) in Maintenance Mode using API call: Turning ON maintenance Mode for Spark2 Service # curl -sH "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn On Maintenance Mode for Spark2"},"Body":{"ServiceInfo":{"maintenance_state":"ON"}}}' http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/services/SPARK2 Turning OFF maintenance Mode for Spark2 Service # curl -sH "X-Requested-By: ambari" -u admin:admin -X PUT -d '{"RequestInfo":{"context":"Turn Off Maintenance Mode for Spark2"},"Body":{"ServiceInfo":{"maintenance_state":"OFF"}}}' http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/services/SPARK2 Please change the cluster name "NewCluster" and ambari hostname + credentials according to yoiur env. .
... View more
04-01-2019
10:38 PM
1 Kudo
@Alessandro As we see the following error while installing the SHELL interpreter. org.sonatype.aether.RepositoryException: Cannot fetch dependencies for org.apache.zeppelin:zeppelin-shell:0.8.0
at org.apache.zeppelin.dep.DependencyResolver.getArtifactsWithDep(DependencyResolver.java:179)
.
.
Caused by: java.lang.NullPointerException
at org.sonatype.aether.impl.internal.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:352) . This can happen if you do not have internet access (or if you have soem n/w issue_ on the Zeppelin server host. Can you please isolate that? You can also refer to the following article for similar issues: https://community.hortonworks.com/content/kbentry/84498/zeppelin-does-not-get-installed-properly-without-i.html . Basically We need to have the dependenices (jars) under "/usr/hdp/current/zeppelin-server/local-repo" and the interpreters inside the "/usr/hdp/current/zeppelin-server/interpreter/" dir if he interpreter installation is successful without internet issue.
... View more
03-23-2019
08:11 PM
@krishna raj Have you tried setting this property to true? "hive.lazysimple.extended_boolean_literal": Default Value: false Added in: Hive 0.14 with HIVE-3635 LazySimpleSerDe uses this property to determine if it treats 'T', 't', 'F', 'f', '1', and '0' as extended, legal boolean literals, in addition to 'TRUE' and 'FALSE'. The default is false , which means only 'TRUE' and 'FALSE' are treated as legal boolean literals. See: https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties . Also please refer to the following link to know how to set "hive.lazysimple.extended_boolean_literal" to "true" while creating the Hive Tables. Like: CREATE TABLE ........ [TBLPROPERTIES ("hive.lazysimple.extended_boolean_literal"="true")]; https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL .
... View more
03-23-2019
07:54 PM
1 Kudo
@Alessandro Volcich The hadoop command line utility also provides an option to get the correct classpath produced something like following which you can use: Example: # export CLASSPATH=`hadoop classpath`:.:
# echo $CLASSPATH
# javac -d wordcount_classes WordCount.java .
... View more
03-22-2019
05:43 AM
1 Kudo
@Franklin Please try this: 1. On ambari agent host please find this script "/usr/lib/ambari-agent/lib/ambari_agent/CustomServiceOrchestrator.py" and then search for a string with 'if the return type is not used' # grep -A2 'if the return type is not used' /usr/lib/ambari-agent/lib/ambari_agent/CustomServiceOrchestrator.py
# if the return type is not used
base_dir = self.file_cache.get_service_base_dir(command)
script_path = self.resolve_script_path(base_dir, script) . 2. Just add a new line on all agent hosts self.file_cache.get_host_scripts_base_dir(command) there as following Just below the commented line: # grep -A2 'if the return type is not used' /usr/lib/ambari-agent/lib/ambari_agent/CustomServiceOrchestrator.py
# if the return type is not used
self.file_cache.get_host_scripts_base_dir(command)
base_dir = self.file_cache.get_service_base_dir(command)
script_path = self.resolve_script_path(base_dir, script) . **NOTE:** Python is Indentation sensitive so you will need to put the same indentation while adding the line there (else it might cause some syntax error) 3. Clear the Agent Cache Dir (or move it as backup) # mv /var/lib/ambari-agent/cache /var/lib/ambari-agent/cache_OLD . 4. Restart the ambari server and agent. # ambari-server restart
# ambari-agent restart . 5. Now check the host script dir if it has pulled file the custom scripts from ambari server host. # ls -lart /var/lib/ambari-agent/cache/host_scripts/ . It looks related to: https://issues.apache.org/jira/browse/AMBARI-25123
... View more