Member since
08-30-2017
6
Posts
0
Kudos Received
0
Solutions
01-15-2018
11:39 AM
I have resolved the issue by placing ACTIVEAMBARIHOST with headnode FQDN name as from Edge Node it was not reflecting.
... View more
01-14-2018
02:41 PM
Changes in system_action_definitions.xml is not picking up by ambari-server even after restart. Even I have kept the <actionName.py> file in the respective diretory. Could you please help me out?
... View more
01-14-2018
10:44 AM
@Aditya Sirna After going through the Ambari API documentation, what I found that the given curl command to execute the custom script action needs to be modified as I have tested and it's working fine. curl -u <username>:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute an action", "action" : "<action_name>"}, "Requests/resource_filters":[{"service_name" : "", "component_name":"", "hosts":"<comma_separated_host_names>"}]}' http://ambari_host:port/api/v1/clusters/cluster_name/requests
For specific components manageability: curl -k -i -u <ambari_admin_user>:<ambari_admin_password> -H 'X-Requested-By: ambari' -X
POST -d '{"RequestInfo":
{"query":"Hosts/host_name.in(<comma separated list of
hosts to be added, if one node give only one
name>)"},
"Body":{"host_components":[{"HostRoles":{"component_name":"BIGSQL_WORKER"}}
]}}'
http://<ambari-host>:
<ambari-port>/api/v1/clusters/<cluster_name>/hosts?
curl -k -i -u <ambari_admin_user>:<ambari_admin_password> -H 'X-Requested-By: ambari' -X
PUT -d '{ "HostRoles": {"state":"INSTALLED" } }' http://<ambari-host>:<ambari-port>/api/v1/clusters/<cluster-name>/host_components?HostRoles/state=INIT
... View more
01-13-2018
04:24 PM
@Aditya Sirna The mentioned command in the post is not working: curl -u <username>:<password>-X POST -H 'X-Requested-By:ambari'-d'{"RequestInfo":{"context":"Execute an action", "action" : "install_kerberos_package", "service_name" : "", "component_name":"", "hosts":"<comma-separated-hosts>"}}' http://<ambari-host>:<ambari-port>/api/v1/clusters/<cluster-name>/requests
I am working in a Azure HDInsight Hadoop Cluster having 2 headnodes [primary and secondary] and 2 worker nodes and 1 edge node. I have configured the script and restarted ambari server to reflect the changes as you've mentioned but want to execute the script specific to primary head node. By executing the command hostname -f, I got the FQDN of that host and execute the following command from edge node: curl -u admin:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute an action", "acton" : "validate_nfs_server_export_path", "service_name" : "", "component_name":"", "hosts":"hn0-kalyan.qjhddkgqet0efmcgr4zymppb4e.cx.internal.cloudapp.net"}}' http://headnodehost:8080/api/v1/clusters/cluster_name/requests But when I saw the AMBARI UI, it shows executed on all the all nodes which it should not be.
... View more
08-30-2017
10:39 AM
I have provisioned a Azure HDInsight Hadoop [3.6] and Spark Cluster [2.1] cluster and did some configuration changes to add some custom attributes [core-site.xml, mapred-site.xml, spark-env.sh, spark-defaults.sh] After doing those changes I am restarting those services in the following order: stop_service() {
if [ -z "$1" ];
then echo "[`date`] [${USER}] Need service name to stop service"
exit 1
fi
SERVICENAME=$1
echo "[`date`] [${USER}] Stopping $SERVICENAME"
if [[ $SERVICENAME =~ SPARK.* ]]; then
curl -u $USERID:$PASSWD -sS -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"$SERVICENAME"}},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME"
else
curl -u $USERID:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Stop $SERVICENAME via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' http://$ACTIVEAMBARIHOST:$PORT/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME
fi
sleep 10
}
start_service() {
if [ -z "$1" ]; then
echo "[`date`] [${USER}] Need service name to start service"
exit 1
fi
sleep 10
SERVICENAME=$1
echo "[`date`] [${USER}] Starting $SERVICENAME"
if [[ $SERVICENAME =~ SPARK.* ]]; then
startResult=$(curl -u $USERID:$PASSWD -sS -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"$SERVICENAME"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME")
else
startResult=$(curl -u $USERID:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start $SERVICENAME via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' http://$ACTIVEAMBARIHOST:$PORT/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME)
fi
if ([[ $startResult == *"500 Server Error"* ]] || [[ $startResult == *"400 Bad Request"* ]]) || [[ $startResult == *"internal system exception occurred"* ]]; then
sleep 60
echo "[`date`] [${USER}] Retry starting $SERVICENAME"
if [[ $SERVICENAME =~ SPARK.* ]]; then
startResult=$(curl -u $USERID:$PASSWD -sS -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"$SERVICENAME"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME")
else
startResult=$(curl -u $USERID:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start $SERVICENAME via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' http://$ACTIVEAMBARIHOST:$PORT/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME)
fi
fi
echo "$startResult"
}
Note: I have to use "RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME" .... for SPARK2 service to stop otherwise Spark Thrift server still shows in stale state and need to restart. [Spark maintenance mode is On] stop_service SPARK2 stop_service OOZIE stop_service HIVE stop_service MAPREDUCE2 stop_service YARN stop_service HDFS start_service HDFS start_service YARN start_service MAPREDUCE2 start_service HIVE start_service OOZIE start_service SPARK2 [Spark maintenance mode is Off] After doing all this restart surprisingly all the services came up successfully but sometimes services goes to unknown state and I can see yellow question mark in Ambari UI. [stop and start service both returns 200 in place of 202 (accepted)] But if I run the same script again to provision another cluster with same type it works. Why this kind of inconsistency there and what is the best way to restart a service if I did some configuration changes?
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Spark
08-30-2017
10:34 AM
I have provisioned a Azure HDInsight Hadoop [3.6] and Spark Cluster [2.1] cluster and did some configuration changes to add some custom attributes [core-site.xml, mapred-site.xml, spark-env.sh, spark-defaults.sh] After doing those changes I am restarting those services in the following order: stop_service() {
if [ -z "$1" ];
then echo "[`date`] [${USER}] Need service name to stop service"
exit 1
fi
SERVICENAME=$1
echo "[`date`] [${USER}] Stopping $SERVICENAME"
if [[ $SERVICENAME =~ SPARK.* ]]; then
curl -u $USERID:$PASSWD -sS -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"$SERVICENAME"}},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME"
else
curl -u $USERID:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Stop $SERVICENAME via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' http://$ACTIVEAMBARIHOST:$PORT/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME
fi
sleep 10
}
start_service() {
if [ -z "$1" ]; then
echo "[`date`] [${USER}] Need service name to start service"
exit 1
fi
sleep 10
SERVICENAME=$1
echo "[`date`] [${USER}] Starting $SERVICENAME"
if [[ $SERVICENAME =~ SPARK.* ]]; then
startResult=$(curl -u $USERID:$PASSWD -sS -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"$SERVICENAME"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME")
else
startResult=$(curl -u $USERID:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start $SERVICENAME via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' http://$ACTIVEAMBARIHOST:$PORT/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME)
fi
if ([[ $startResult == *"500 Server Error"* ]] || [[ $startResult == *"400 Bad Request"* ]]) || [[ $startResult == *"internal system exception occurred"* ]]; then
sleep 60
echo "[`date`] [${USER}] Retry starting $SERVICENAME"
if [[ $SERVICENAME =~ SPARK.* ]]; then
startResult=$(curl -u $USERID:$PASSWD -sS -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"$SERVICENAME"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' "https://$CLUSTERNAME.azurehdinsight.net/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME")
else
startResult=$(curl -u $USERID:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start $SERVICENAME via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' http://$ACTIVEAMBARIHOST:$PORT/api/v1/clusters/$CLUSTERNAME/services/$SERVICENAME)
fi
fi
echo "$startResult"
}
Note: I have to use "RequestInfo":{"context":"_PARSE_.STOP.$SERVICENAME" .... for SPARK2 service to stop otherwise Spark Thrift server still shows in stale state and need to restart. [Spark maintenance mode is On] stop_service SPARK2 stop_service OOZIE stop_service HIVE stop_service MAPREDUCE2 stop_service YARN stop_service HDFS start_service HDFS start_service YARN start_service MAPREDUCE2 start_service HIVE start_service OOZIE start_service SPARK2 [Spark maintenance mode is Off] After doing all this restart surprisingly all the services came up successfully but sometimes services goes to unknown state and I can see yellow question mark in Ambari UI. [stop and start service both returns 200 in place of 202 (accepted)] But if I run the same script again to provision another cluster with same type it works. Why this kind of inconsistency there and what is the best way to restart a service if I did some configuration changes?
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Spark
