Member since
01-11-2016
18
Posts
23
Kudos Received
0
Solutions
09-03-2018
07:24 AM
You have tried an ambari upgrade or HDP upgrade??
... View more
07-02-2018
08:50 AM
Please add the relevant details like screenshot, logs and hdp-select outputs.
... View more
07-02-2018
08:43 AM
Clearly a problem with ssh key. Please use the correct key available for the host with correct permission(400) set on that.
... View more
07-02-2018
08:35 AM
also check the usual stuff like 1. /etc/hosts/, 2. stop iptables, 3.telnet to 8440 from agents, 4. connection established at 8441
... View more
07-02-2018
08:33 AM
1 Kudo
Once u have started ambari-server in debug mode. please check the following in new host agent log. 2018-06-20 17:30:03,018 - IP address forward resolution check started.
2018-06-20 17:30:03,018 - All hosts resolved to an IP address.
2018-06-20 17:30:03,018 - IP address forward resolution check completed.
2018-06-20 17:30:03,019 - Host checks completed. 2018-06-20 17:30:03,019 - Structured output: {'host_resolution_check': {'failed_count': 0, 'exit_code': 0, 'success_count': 21, 'failures': [], 'message': 'All hosts resolved to an IP address.', 'hosts_with_failures': []}} 2018-06-20 17:30:03,019 - Action afix 'post_actionexecute' not present If it is still stuck at host-check. the agent log should contain 2018-06-20 17:31:25,502 DEBUG [ambari-client-thread-86] BaseProvider:331 - Skipping property for resource as not in requestedIds, resourceType=Task, propertyId=Tasks/role, value=check_host
2018-06-20 17:31:25,503 DEBUG [ambari-client-thread-86] BaseProvider:331 - Skipping property for resource as not in requestedIds, resourceType=Task, propertyId=Tasks/command, value=ACTIONEXECUTE 2018-06-20
17:31:25,503 DEBUG [ambari-client-thread-86] BaseProvider:308 - Setting property for resource, resourceType=Task, propertyId=Tasks/status, value=QUEUED Please revert what do you see in agent logs??
... View more
07-02-2018
08:18 AM
It is nothing to do with the spark. may be package-mismatch on libcurl
do a complete uninstall of curl and try a fresh install.
by
wget http://curl.haxx.se/download/curl-7.48.0.tar.gz
tar -xvzf curl-7.48.0.tar.gz
cd curl-7.48.0
./configure
sudo make
sudo make install If you are particular about the version, please refer https://curl.haxx.se/download.html . or sudo apt-get update
sudo apt-get install curl
sudo apt-get install libcurl3 please change apt-get according to your Linux flavor
... View more
03-26-2018
11:33 AM
First Thing you should go for Core-spark processing use. Extraction/Transformation . Then you should start using spark-sql. I would suggest, take a dataset, have an overview of the data. frame a most difficult question you want to extract from it. apply action/transformation to reach to answer. After that only try spark streaming. one of the best tutorial videos for this https://www.youtube.com/watch?v=2-d8WwdtlUI&list=PLf0swTFhTI8rDQXfH8afWtGgpOTnhebDx . Hope this answer your question.
... View more
03-26-2018
09:29 AM
6 Kudos
There are lots of websites where you can find the IOT datasets. just import the data and play around with it in spark. 1. https://data.world/datasets/iot 2. https://www.datasciencecentral.com/profiles/blogs/great-sensor-datasets-to-prepare-your-next-career-move-in-iot-int 3. https://www.quora.com/Where-can-I-find-a-good-Internet-of-Things-dataset 4. https://old.datahub.io/dataset/knoesis-linked-sensor-data 5. https://archive.ics.uci.edu/ml/datasets/Gas+Sensor+Array+Drift+Dataset 6. https://www.microsoft.com/en-us/download/details.aspx?from=http%3A%2F%2Fresearch.microsoft.com%2Fen-us%2Fdownloads%2Fb16d359d-d164-469e-9fd4-daa38f2b2e13%2F&id=52367 7. https://carto.com/blog/forty-brilliant-open-data-projects-preparing-smart-cities-2018/
... View more
03-19-2018
12:20 PM
These Happens due to DB related Problem. Please see If you can connect to the DB Host from Ambari Host (If You have a external database.). Please share the logs for more debugging.
... View more
10-26-2017
11:52 AM
1 Kudo
I installed hue 2.6 . [root@ctr-e134-1499953498516-253920-01-000007 hue]# rpm -qa|grep -i hue hue-beeswax-2.6.1.2.6.3.0-227.x86_64 hue-oozie-2.6.1.2.6.3.0-227.x86_64 hue-pig-2.6.1.2.6.3.0-227.x86_64 hue-2.6.1.2.6.3.0-227.x86_64 hue-hcatalog-2.6.1.2.6.3.0-227.x86_64 hue-common-2.6.1.2.6.3.0-227.x86_64 hue-server-2.6.1.2.6.3.0-227.x86_64 Here is my hue.ini [root@ctr-e134-1499953498516-253920-01-000007 hue]# find / -name hue.ini /tmp/hue.ini /etc/hue/conf.empty/hue.ini hue-ini.txt I tried to restart hue [root@ctr-e134-1499953498516-253920-01-000007 hue]# service hue restart Shutting down hue: [FAILED] Starting hue: [ OK ] [root@ctr-e134-1499953498516-253920-01-000007 hue]# service hue start Starting hue: [ OK ] [root@ctr-e134-1499953498516-253920-01-000007 hue]# service hue restart Shutting down hue: [ OK ] Starting hue: [ OK ] [root@ctr-e134-1499953498516-253920-01-000007 hue]# service hue status supervisor is stopped The UI is Unreachable. hence I checked the logs. in the error .log [26/Oct/2017 09:20:46 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:20:47 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:20:49 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:20:50 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver has restarted more than 3 times in the last 4 seconds
[26/Oct/2017 09:20:50 +0000] supervisor ERROR Exception in supervisor main loop
Traceback (most recent call last):
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main
wait_loop(sups, options)
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 431, in wait_loop
shutdown(sups) # shutdown() exits the process
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
sys.exit(1)
SystemExit: 1
[26/Oct/2017 09:21:00 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:21:02 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:21:03 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:21:05 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver has restarted more than 3 times in the last 4 seconds
[26/Oct/2017 09:21:05 +0000] supervisor ERROR Exception in supervisor main loop
Traceback (most recent call last):
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main
wait_loop(sups, options)
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 431, in wait_loop
shutdown(sups) # shutdown() exits the process
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
sys.exit(1)
SystemExit: 1
[26/Oct/2017 09:41:48 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:41:50 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:41:51 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 09:41:53 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver has restarted more than 3 times in the last 4 seconds
[26/Oct/2017 09:41:53 +0000] supervisor ERROR Exception in supervisor main loop
Traceback (most recent call last):
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main
wait_loop(sups, options)
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 431, in wait_loop
shutdown(sups) # shutdown() exits the process
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
sys.exit(1)
SystemExit: 1
[26/Oct/2017 11:17:22 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:17:24 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:17:26 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:17:27 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver has restarted more than 3 times in the last 5 seconds
[26/Oct/2017 11:17:28 +0000] supervisor ERROR Exception in supervisor main loop
Traceback (most recent call last):
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main
wait_loop(sups, options)
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 431, in wait_loop
shutdown(sups) # shutdown() exits the process
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
sys.exit(1)
SystemExit: 1
[26/Oct/2017 11:17:34 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:17:36 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:17:37 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:17:39 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver has restarted more than 3 times in the last 4 seconds
[26/Oct/2017 11:17:39 +0000] supervisor ERROR Exception in supervisor main loop
Traceback (most recent call last):
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main
wait_loop(sups, options)
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 431, in wait_loop
shutdown(sups) # shutdown() exits the process
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
sys.exit(1)
SystemExit: 1
[26/Oct/2017 11:29:36 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:29:38 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:29:39 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
[26/Oct/2017 11:29:41 +0000] supervisor ERROR Process /usr/lib/hue/build/env/bin/hue runcpserver has restarted more than 3 times in the last 4 seconds
[26/Oct/2017 11:29:41 +0000] supervisor ERROR Exception in supervisor main loop
Traceback (most recent call last):
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 414, in main
wait_loop(sups, options)
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 431, in wait_loop
shutdown(sups) # shutdown() exits the process
File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
sys.exit(1)
SystemExit: 1 Any help is appreciated
... View more
Labels:
- Labels:
-
Apache Ambari
-
Cloudera Hue
07-03-2017
11:38 AM
1 Kudo
Creating a new cluster, I will verify and update in this thread
... View more
07-03-2017
09:26 AM
4 Kudos
OS:- Centos7, Centos 6 I was trying to generate code-coverage report from ambari using cobertura 2.1.1. The steps to do that involves the following:- 1. Stop ambari. 2.Instrument the target jars. sudo su - -c "/tmp/coverage-tmp/cobertura-2.1.1/cobertura-instrument.sh --datafile /tmp/coverage-tmp/cobertura-2.1.1/cobertura-data.ser --includeClasses org.apache.ambari.* --excludeClasses net.sourceforge.cobertura.* --excludeClasses .*[tT][eE][sS][tT].* --excludeClasses .*shaded.* --excludeClasses .*org.apache.ambari.server.controller.* --auxClasspath /usr/lib/ambari-server/jna-4.1.0.jar:/usr/lib/ambari-server/activation-1.1.jar:/usr/lib/ambari-server/guava-16.0.jar:/usr/lib/ambari-server/oro-2.0.8.jar:/usr/lib/ambari-server/ambari-metrics-common-2.5.2.0.95.jar:/usr/lib/ambari-server/guice-persist-3.0.jar:/usr/lib/ambari-server/ambari-server-2.5.2.0.95.jar:/usr/lib/ambari-server/guice-servlet-3.0.jar:/usr/lib/ambari-server/ambari-views-2.5.2.0.95.jar:/usr/lib/ambari-server/jetty-6.1.26.jar:/usr/lib/ambari-server/ant-1.6.5.jar:/usr/lib/ambari-server/h2-1.4.193.jar:/usr/lib/ambari-server/ant-launcher-1.7.1.jar:/usr/lib/ambari-server/jetty-io-8.1.19.v20160209.jar:/usr/lib/ambari-server/antlr-2.7.7.jar:/usr/lib/ambari-server/jline-2.11.jar:/usr/lib/ambari-server/aopalliance-1.0.jar:/usr/lib/ambari-server/guice-3.0.jar:/usr/lib/ambari-server/mimepull-1.9.3.jar:/usr/lib/ambari-server/apache-log4j-extras-1.2.17.jar:/usr/lib/ambari-server/hadoop-annotations-2.6.0.jar:/usr/lib/ambari-server/apacheds-i18n-2.0.0-M19.jar:/usr/lib/ambari-server/gson-2.2.2.jar:/usr/lib/ambari-server/mina-core-2.0.9.jar:/usr/lib/ambari-server/apacheds-kerberos-codec-2.0.0-M19.jar:/usr/lib/ambari-server/hadoop-auth-2.7.2.jar:/usr/lib/ambari-server/api-asn1-api-1.0.0-M26.jar:/usr/lib/ambari-server/hadoop-common-2.7.2.jar:/usr/lib/ambari-server/api-asn1-ber-1.0.0-M26.jar:/usr/lib/ambari-server/jackson-xc-1.9.9.jar:/usr/lib/ambari-server/api-i18n-1.0.0-M26.jar:/usr/lib/ambari-server/java-xmlbuilder-0.4.jar:/usr/lib/ambari-server/api-ldap-model-1.0.0-M26.jar:/usr/lib/ambari-server/javax.inject-1.jar:/usr/lib/ambari-server/api-util-1.0.0-M26.jar:/usr/lib/ambari-server/jsch-0.1.42.jar:/usr/lib/ambari-server/asm-3.3.1.jar:/usr/lib/ambari-server/json-smart-1.1.1.jar:/usr/lib/ambari-server/avro-1.7.4.jar:/usr/lib/ambari-server/jsr305-1.3.9.jar:/usr/lib/ambari-server/c3p0-0.9.5.2.jar:/usr/lib/ambari-server/jsr311-api-1.1.1.jar:/usr/lib/ambari-server/cglib-2.2.2.jar:/usr/lib/ambari-server/kerberos-client-2.0.0-M19.jar:/usr/lib/ambari-server/cloning-1.9.2.jar:/usr/lib/ambari-server/javax.json-1.0.4.jar:/usr/lib/ambari-server/commonj.sdo-2.1.1.jar:/usr/lib/ambari-server/javax.persistence-2.1.1.jar:/usr/lib/ambari-server/commons-beanutils-1.7.0.jar:/usr/lib/ambari-server/guice-assistedinject-3.0.jar:/usr/lib/ambari-server/commons-beanutils-core-1.8.0.jar:/usr/lib/ambari-server/javax.servlet-3.0.0.v201112011016.jar:/usr/lib/ambari-server/commons-cli-1.3.1.jar:/usr/lib/ambari-server/javax.servlet-api-3.1.0.jar:/usr/lib/ambari-server/commons-codec-1.8.jar:/usr/lib/ambari-server/jackson-jaxrs-1.9.9.jar:/usr/lib/ambari-server/commons-collections-3.2.1.jar:/usr/lib/ambari-server/jaxb-api-2.2.2.jar:/usr/lib/ambari-server/commons-compress-1.5.jar:/usr/lib/ambari-server/guice-multibindings-3.0.jar:/usr/lib/ambari-server/commons-configuration-1.6.jar:/usr/lib/ambari-server/jul-to-slf4j-1.7.2.jar:/usr/lib/ambari-server/commons-csv-1.1.jar:/usr/lib/ambari-server/jaxb-impl-2.2.3-1.jar:/usr/lib/ambari-server/commons-digester-1.8.jar:/usr/lib/ambari-server/jcip-annotations-1.0.jar:/usr/lib/ambari-server/commons-httpclient-3.1.jar:/usr/lib/ambari-server/commons-io-2.1.jar:/usr/lib/ambari-server/log4j-1.2.17.jar:/usr/lib/ambari-server/commons-lang-2.5.jar:/usr/lib/ambari-server/jdk.tools-1.7.jar:/usr/lib/ambari-server/commons-logging-1.1.1.jar:/usr/lib/ambari-server/jersey-client-1.19.jar:/usr/lib/ambari-server/commons-math3-3.1.1.jar:/usr/lib/ambari-server/jersey-core-1.19.jar:/usr/lib/ambari-server/commons-net-1.4.1.jar:/usr/lib/ambari-server/jersey-guice-1.19.jar:/usr/lib/ambari-server/curator-client-2.7.1.jar:/usr/lib/ambari-server/jersey-json-1.19.jar:/usr/lib/ambari-server/curator-framework-2.7.1.jar:/usr/lib/ambari-server/jersey-multipart-1.19.jar:/usr/lib/ambari-server/curator-recipes-2.7.1.jar:/usr/lib/ambari-server/libpam4j-1.8.jar:/usr/lib/ambari-server/ecj-3.5.1.jar:/usr/lib/ambari-server/jersey-server-1.19.jar:/usr/lib/ambari-server/eclipselink-2.6.2.jar:/usr/lib/ambari-server/mailapi-1.5.2.jar:/usr/lib/ambari-server/ehcache-2.10.0.jar:/usr/lib/ambari-server/htrace-core-3.1.0-incubating.jar:/usr/lib/ambari-server/metrics-jvm-3.1.0.jar:/usr/lib/ambari-server/httpclient-4.2.5.jar:/usr/lib/ambari-server/metrics-core-3.1.0.jar:/usr/lib/ambari-server/httpcore-4.2.4.jar:/usr/lib/ambari-server/jackson-mapper-asl-1.9.13.jar:/usr/lib/ambari-server/jackson-annotations-2.1.4.jar:/usr/lib/ambari-server/jersey-servlet-1.19.jar:/usr/lib/ambari-server/jackson-core-asl-1.9.9.jar:/usr/lib/ambari-server/netty-3.7.0.Final.jar:/usr/lib/ambari-server/jets3t-0.9.0.jar:/usr/lib/ambari-server/jetty-client-8.1.19.v20160209.jar:/usr/lib/ambari-server/stax-api-1.0-2.jar:/usr/lib/ambari-server/jetty-continuation-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-http-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-security-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-server-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-servlet-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-servlets-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-util-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-webapp-8.1.19.v20160209.jar:/usr/lib/ambari-server/jetty-xml-8.1.19.v20160209.jar:/usr/lib/ambari-server/jsp-2.1-glassfish-2.1.v20100127.jar:/usr/lib/ambari-server/validation-api-1.1.0.Final.jar:/usr/lib/ambari-server/jsp-api-2.1-glassfish-2.1.v20100127.jar:/usr/lib/ambari-server/mchange-commons-java-0.2.11.jar:/usr/lib/ambari-server/nimbus-jose-jwt-3.9.jar:/usr/lib/ambari-server/objenesis-2.1.jar:/usr/lib/ambari-server/objenesis-tck-1.2.jar:/usr/lib/ambari-server/paranamer-2.3.jar:/usr/lib/ambari-server/postgresql-9.3-1101-jdbc4.jar:/usr/lib/ambari-server/protobuf-java-2.5.0.jar:/usr/lib/ambari-server/quartz-2.2.1.jar:/usr/lib/ambari-server/quartz-jobs-2.2.1.jar:/usr/lib/ambari-server/slf4j-api-1.7.2.jar:/usr/lib/ambari-server/slf4j-log4j12-1.7.2.jar:/usr/lib/ambari-server/smtp-1.5.2.jar:/usr/lib/ambari-server/snappy-java-1.0.4.1.jar:/usr/lib/ambari-server/snmp4j-1.10.1.jar:/usr/lib/ambari-server/spring-aop-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-asm-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-beans-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-context-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-core-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-expression-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-jdbc-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-ldap-core-2.0.4.RELEASE.jar:/usr/lib/ambari-server/velocity-1.7.jar:/usr/lib/ambari-server/spring-security-config-3.1.2.RELEASE.jar:/usr/lib/ambari-server/xmlenc-0.52.jar:/usr/lib/ambari-server/spring-security-core-3.1.2.RELEASE.jar:/usr/lib/ambari-server/cobertura-2.1.1.jar:/usr/lib/ambari-server/spring-security-kerberos-core-1.0.1.RELEASE.jar:/usr/lib/ambari-server/spring-web-3.0.7.RELEASE.jar:/usr/lib/ambari-server/spring-security-kerberos-web-1.0.1.RELEASE.jar:/usr/lib/ambari-server/xz-1.2.jar:/usr/lib/ambari-server/spring-security-ldap-4.0.4.RELEASE.jar:/usr/lib/ambari-server/zookeeper-3.4.6.jar:/usr/lib/ambari-server/spring-security-web-3.1.2.RELEASE.jar:/usr/lib/ambari-server/spring-tx-3.0.7.RELEASE.jar:/usr/lib/oracle/11.2/client64/lib/ojdbc5.jar:/usr/lib/oracle/11.2/client64/lib/ojdbc6.jar:/usr/lib/oracle/11.2/client64/lib/xstreams.jar /usr/lib/ambari-server/ambari-metrics-common-2.5.2.0.95.jar /usr/lib/ambari-server/ambari-server-2.5.2.0.95.jar /usr/lib/ambari-server/ambari-views-2.5.2.0.95.jar" root The target jars are :- 1. /usr/lib/ambari-server/ambari-metrics-common-2.5.2.0.95.jar 2./usr/lib/ambari-server/ambari-server-2.5.2.0.95.jar 3./usr/lib/ambari-server/ambari-views-2.5.2.0.95.jar The instrumentation completes with out giving any error:- 2017-07-02 21:16:24,491|INFO|MainThread|machine.py:159 - run()||GUID=a3384dea-adb2-47cc-9c0b-8460a0582a3c|This is MOTD message, added for testing in qe infra
2017-07-02 21:16:24,949|INFO|MainThread|machine.py:159 - run()||GUID=a3384dea-adb2-47cc-9c0b-8460a0582a3c|Cobertura 2.1.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
2017-07-02 21:16:42,679|INFO|MainThread|machine.py:159 - run()||GUID=a3384dea-adb2-47cc-9c0b-8460a0582a3c|[INFO] Cobertura: Coverage data file /tmp/coverage-tmp/cobertura-2.1.1/cobertura-data.ser either does not exist or is not readable. Creating a new data file.
2017-07-02 21:16:44,442|INFO|MainThread|machine.py:159 - run()||GUID=a3384dea-adb2-47cc-9c0b-8460a0582a3c|[INFO] Cobertura: Saved information on 2089 classes.
2017-07-02 21:16:44,916|INFO|MainThread|machine.py:159 - run()||GUID=a3384dea-adb2-47cc-9c0b-8460a0582a3c|[INFO] Cobertura: Loaded information on 2089 classes.
2017-07-02 21:16:46,618|INFO|MainThread|machine.py:159 - run()||GUID=a3384dea-adb2-47cc-9c0b-8460a0582a3c|[INFO] Cobertura: Saved information on 2089 classes. As soon as I start ambari-server, it failed:- 2017-07-02 21:16:46,716|INFO|MainThread|machine.py:145 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|RUNNING: sudo su - -c "ambari-server start" root
2017-07-02 21:16:46,771|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|######## Hortonworks #############
2017-07-02 21:16:46,772|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|This is MOTD message, added for testing in qe infra
2017-07-02 21:16:46,794|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Using python /usr/bin/python
2017-07-02 21:16:46,794|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Starting ambari-server
2017-07-02 21:16:47,831|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Ambari Server running with administrator privileges.
2017-07-02 21:16:47,832|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Organizing resource files at /var/lib/ambari-server/resources...
2017-07-02 21:16:47,832|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Ambari database consistency check started...
2017-07-02 21:16:47,832|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Server PID at: /var/run/ambari-server/ambari-server.pid
2017-07-02 21:16:47,832|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Server out at: /var/log/ambari-server/ambari-server.out
2017-07-02 21:16:47,832|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Server log at: /var/log/ambari-server/ambari-server.log
2017-07-02 21:16:53,880|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|Waiting for server start.........Unable to determine server PID. Retrying...
2017-07-02 21:16:59,917|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|......Unable to determine server PID. Retrying...
2017-07-02 21:17:05,958|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|......Unable to determine server PID. Retrying...
2017-07-02 21:17:05,961|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|ERROR: Exiting with exit code -1.
2017-07-02 21:17:05,961|INFO|MainThread|machine.py:159 - run()||GUID=a282a54e-6b04-4612-a53f-ac7023a29608|REASON: Ambari Server java process died with exitcode 1. Check /var/log/ambari-server/ambari-server.out for more information.
And in the /var/log/ambari-server/ambari-server.out:- hence I checked for this class in ambari-server.jar Question:- This has only happened in centos7 and centos6 os. Ambari-server was fine in Debian os. Is ambari-server jar is getting corrupted by instrumentation command. How do I further debug on this?
... View more
Labels:
- Labels:
-
Apache Ambari
06-14-2017
06:17 AM
2 Kudos
By default ambari background operations tab display the:- 1. Start/Stop of Service/components. 2. Installation of components. I want to create a dummy task(request) along with display message to be shown in background operations tab which should always pass. for example PUT URl:- http://<cluster_ip>:8080/api/v1/clusters/<cluster_name>/requests
{
"RequestInfo": {
"context": "Update Successful",
"command": "HDFS_SERVICE_CHECK"
},
"Requests/resource_filters": [
{
"service_name": "HDFS"
}
]
}
Screenshot:- Here I am reusing "HDFS_SERVICE_CHECK" to display my own message. But it may fail if there is a problem related to HDFS. hence I wanted to create my own task which will not change the state of the cluster and should always pass.
... View more
Labels:
- Labels:
-
Apache Ambari
10-19-2016
11:16 AM
1 Kudo
Can you please provide little more details. like what version of Ambari,HDP ? any screenshot that constitute the error?
... View more
08-31-2016
11:42 AM
4 Kudos
Introduction Ambari 2.4 introduces the Hue-to-Views Migration tool, which is specifically designed to migrate existing Hue artifacts to an Ambari View. Currently in Hue the hive queries are saved in ‘Saved Queries’ in Hue UI. These queries can be invoked by the user by going to Saved Queries in Hue UI. HUETOAMBARI_MIGRATION would help in migrating these saved Hive artifacts from Hue to Ambari Hive View. Once the migration is done, the Saved Queries can be viewed from Ambari Hive View. Currently HUETOAMBARI_MIGRATION would help in migrating the following artifacts from Hue to Ambari Views. Hive - Saved Queries, Query History Pig - Saved Scripts, Pig Jobs The view would use JDBC to connect to Hue database to obtain the artifacts. Once obtained, these would be stored in the Ambari View Server database. Today, the view migrates the scripts one user at a time. The user to be migrated would be specified during the migration. Note:- This view does not validate scripts. Non-working Hue scripts will not work in Ambari Views. Pre-requisites Ambari Server serving the HUETOAMBARI_MIGRATION view should have connectivity to Hue server Only users with 'Cluster Admin' privilege can do the migration Ambari and Hue JDBC connection details are required while configuring the view The user whose artifacts are being migrated should also exist in Ambari with correct permissions to the respective View (Hive / Pig) Hive and Pig view should be pre-configured and in a workings state Following article describes how to use HUETOAMBARI_MIGRATION view. Migrating Hue Artifacts to an Ambari View View Creation: - 1.1 Create an Instance by going admin > manage ambari > views 1.2 Give a suitable Name 1.3 Enter view Parameters To show how to fill up the view-parameters we have assumed the following and we will use this while writing the examples in the table 1.Hue Server url :- c6401.ambari.apache.org 2.Hue NameNode URI:-c6402.ambari.apache.org 3.NameNode port:- 50070 4.Hue Database Name(mysql):- Huedb 5.Hue Database username(mysql):- hue 6.Hue Database Password(mysql):- hue 7.Ambari Hostname:-c6402.ambari.apache.org 8.Ambari Database Name(postgresql):- ambari 9.Ambari Database username(postgresql):- ambari 10.Ambari Database Password(postgresql):- bigdata The table below describes each of the parameters Property Description Syntax Example Hue Http URL Https url where Hue server is located <hue http url> http://c6401.ambari.apache.org:8000/ Hue Server hostname Hostname of Hue Server <hue hostname> c6401.ambari.apache.org Ambari http URL Http url where Ambari Server <ambari http url> http://c6402.ambari.apache.org:8080/ Ambari Server hostname Hostname of Ambari server <ambari hostname> c6402.ambari.apache.org Webhdfs URI(Hue) Namenode URI of Hue webhdfs://<hostname>:50070 webhdfs://c6402.ambari.apache.org:50070 Webhdfs URI(Ambari) NameNode URI of Ambari webhdfs://<hostname>:50070 webhdfs://c6402.ambari.apache.org:50070 Hue Database Driver JDBC Driver to access Hue DB <db driver> com.mysql.jdbc.Driver Hue JDBC URL JDBC Url to access Hue DB jdbc:<dbtype>://<hostname>/<db name> jdbc:mysql://c6401.ambari.apache.org/huedb Hue Database Username Hue Database Username <db username> hue Hue Database Password Hue Database Password <db password> hue Ambari Database Driver JDBC Driver to access Ambari DB <db driver> org.postgresql.Driver Ambari JDBC URL JDBC Url to access Ambari DB jdbc:<dbtype>://<hostname>/<db name> jdbc:postgresql://c6402.ambari.apache.org:5432/ambari Ambari Database Username Ambari Database Username <db username> ambari Ambari Database Password Ambari Database Password <db password> bigdata Kerberos enabled on Ambari cluster?(y/n) (y/n) for Kerberos <y/n> n principal name (if kerberos is enabled) If Kerberos enable, Principal Name
must be given or else it should be left blank <principal username> ambari-cl1 **For HA Setups: The Hue-to-Views Migration View does not support HA. You must provide the current active namenode for the target cluster regarding the Webhdfs URI for Ambari. After submitting all the view parameters. click on save. wait for the instance to successfully created. once the instance is successfully created . click on "Go to instance" 2.Work flow of the view 2.1 Configuration check If entered values in view parameters are correct then configuration check will succeed and home page of the view will open, incase of any wrong entries in view parameters configuration check will fail and proper stack trace will be shown In case of any wrong entries in view parameter 2.2 Hive Migration Click on Hive → HiveHistory Query to transfer executed queries. Click on Hive → HiveSaved Query to transfer saved queries.
fill up the details 1. User Name (Hue User Name) 1.1 In Ambari also the user should exist in the same name. In this case "hue". 1.2 Make sure that the user in Ambari has the proper access rights to hive and pig views. Here ,the user "hue" exist in Ambari and has the right to access "HiveView123","hive_1" and "pig_view". 2. Instance Name (Hive View Instance where the queries need to be transferred) **Make sure the user has permission to access the view instance.In this case target instance is "hive_1" to which user hue has proper access rights. 3. Start Date (Date from which queries need to be transferred - excluding) 4. End Date (Date to which queries need to be transferred - excluding) Click ‘Submit’. The transfer takes places and the below report is generated 2.3 Pig Migration Click on PigSaved Script to transfer saved pig jobs Click on PigJob to transfer executed pig jobs fill up the details 1. User Name (Hue User Name) ** The queries will be migrating to the same username in Ambari view (make sure that the user exist in Ambari and has the proper right to access Pig views ) 2. Instance Name (Pig View where the queries need to be transferred) 3. Start Date (Date from which queries need to be transferred - excluding) 4. End Date (Date to which queries need to be transferred - excluding) Click ‘Submit’. The transfer takes places and the below report is generated The migration reports shows the Total number of query selected according your your search criteria(start date, end date ),Number of query actually transferred,Total time taken. You can verify the migration by login into the hive view or pig view. Happy migration...
... View more
- Find more articles tagged with:
- ambari-view
- Cloud & Operations
- data-migration
- How-ToTutorial
- hue
- Migration
- new-feature
Labels:
