Member since
11-19-2015
158
Posts
25
Kudos Received
21
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
11914 | 09-01-2018 01:27 AM | |
1143 | 09-01-2018 01:18 AM | |
3804 | 08-20-2018 09:39 PM | |
512 | 07-20-2018 04:51 PM | |
1523 | 07-16-2018 09:41 PM |
04-18-2018
09:41 PM
Probably a better question for the Ambari mailing lists. https://ambari.apache.org/mail-lists.html Also lots of issues tagged in the Ambari JIRA for 3.0.0 - https://issues.apache.org/jira/browse/AMBARI-23611?jql=project%20%3D%20AMBARI%20AND%20fixVersion%20%3D%203.0.0
... View more
04-10-2018
08:35 PM
Yes, the commands work the same assuming you have winutils.exe on your PATH as well as HADOOP_HOME and HADOOP_CONF_DIR defined as environment variables. Windows is not as stable or as supported as Linux, however.
... View more
04-09-2018
03:12 PM
Hello, it seems you duplicated this post. https://community.hortonworks.com/questions/183845/hdpcd-exam-issue.html Please remove this one.
... View more
04-09-2018
03:09 PM
@bob rabih sh scripts are meant to be ran from *nix machines, not from Windows. Use the appropriate bat files in the kafka\bin\windows folder.
... View more
04-06-2018
04:47 PM
@Rajesh Reddy - No, wget only tests HTTP/S connections. Not plain TCP that Kafka and Zookeeper are using
... View more
04-05-2018
07:57 PM
Well, can you issue "kafka-topics --zookeeper $ZK_VIP" or "kafka-console-producer --bootstrap-server $KAFKA_VIP" commands without error?
... View more
04-03-2018
06:57 PM
You should consider just using Kafka for all ingestion. Run Kafka Connect locally. Point it at a directory. http://kafka.apache.org/documentation/#connect https://github.com/jcustenborder/kafka-connect-spooldir Alternative solutions include Fluentd or Filebeat
... View more
04-02-2018
09:12 PM
If you are using a Hortonworks offering, then you can use Apache NiFi. https://community.hortonworks.com/articles/97773/how-to-retrieve-files-from-a-sftp-server-using-nif.html Otherwise, according to that documentation, if you have Sqoop 1.99.7, then yes, it appears to be possible.
... View more
03-30-2018
03:06 AM
@Geoffrey Shelton Okot - That requires installing sh module into the Ambari python installation, which I do not want to maintain. If I could do that, then I would use a Python Hive driver.
... View more
03-28-2018
08:37 PM
Ambari alerts only check process and port health, yes? There are no smoke tests being ran by Ambari unless a manual service check is ran?
I was able to write a bash script to detect a timeout from a simple COUNT query, but later found out that Ambari only accepts Python scripts as alerts.
What are my options if I would like to periodically run a Hive query and do a validity check against a Hive table?
This script works on it's own in bash, but I need help calling it / re-doing it using Python if I want to use Ambari alerts
#!/usr/bin/env bash
set -u
TIMEOUT=1m
BEELINE=/usr/hdp/current/hive-client/bin/beeline
RUNAS=hive
TEZ_QUEUE=infrastructure
OUTPUT_FILE=/tmp/hivecanary.out
CONNECT_URI=jdbc:hive2://localhost:10000?tez.queue.name=infrastructure
if [[ ! -x $BEELINE ]]; then
echo CRITICAL
exit 2
fi
if [[ -f $OUTPUT_FILE ]]; then
rm -f "$OUTPUT_FILE"
fi
echo "$BEELINE --showHeader=false --outputformat=tsv2 -n hive -u 'jdbc:hive2://localhost:10000?tez.queue.name=infrastructure' -e 'SELECT COUNT(*) FROM default.customers'" | bash &
a=0
sleep 10
while true;do
a=$(($a+1))
PID=$(ps aux | grep beeline | grep -v grep | grep 'SELECT COUNT' | awk '{ print $2 }')
if [ -z "$PID" ]; then
echo OK
exit 0
else
if [ "$a" -gt 60 ]; then
echo CRITICAL
exit 1
fi
fi
sleep 1
done
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive