Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

mrdavidsheard · ‎11-09-2017

Thanks for the feedback. We used both QueryDatabaseTable and ExecuteSQL. The CLOB is large and our quick and dirty workaround is to create a view but I had to limit the column to 1000 (see below) as the processors still had an error with the max 4000 that Oracle could handle (converting a CLOB). DBMS_LOB.SUBSTR(TEXT_DOC, 1000, 1)

mrdavidsheard · ‎11-09-2017

Hi, I have an Oracle table that has a CLOB. I can't pull them back using any of the processors that I try. Any way around this other than using SQOOP? NiFI version 1.2.0.3.0.1.1-5 Cheers

mrdavidsheard · ‎10-10-2017

Thanks for the response. Any indication to when 3.0.2 is to be released? We don't have internet access, except via a proxy, so we can't get a list of jars and poms required. Any chance of doing a `ls` and sending those, or is that pushing things?

mrdavidsheard · ‎10-10-2017

I am running HDF 3.0.1 Storm is up and running and I can see it in Ambari. I can add it to my service pool in SAM but in my application it shows that the status of Storm is not running. And I have no idea why not. It's a 3 node cluster with storm on node 2 and Streamline on node 1

mrdavidsheard · ‎10-10-2017

I get the following error with HDF 3.0.1 when deploying an application. I'm on a single server that uses a proxy. I can see it is a known error but no real work around - https://community.hortonworks.com/questions/118255/streaming-analytics-manager-fail-to-deploy-applica.html The issue is STORM-2598. But Storm 1.2.0 is not out yet. Help! ERROR: Topology submission failed due to: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1

mrdavidsheard · ‎09-01-2017

Excellent, thanks for the link Appreciated

mrdavidsheard · ‎08-30-2017

Hi I'm looking to capture data linage in Atlas for our Java mapreduce jobs loading data from HDFS into Elasticsearch. Even an API example. We can track it from sqoop to hdfs but not the next step. App

mrdavidsheard · ‎05-02-2017

Resolved with a work around only. Changed master to true in config on servers and restarted service on server. However if I restart elasticsearch via Ambari the same thing happens, that is it reverts to master false. A permanent fix would be nice.

mrdavidsheard · ‎05-02-2017

Hi going to the URL http://<server>:5000/app/kibana I get the error plugin:elasticsearch Service Unavailable. Metron, Elasticsearch and Kibana are up and managed by Ambari. I can connect to elasticsearch. I am running Kibana 4.5.1 and Metron 0.40 and elasticsearch 2.3.3 elasticsearch.yml node: data: true master: false can't find where to change master: true in Ambari. Elasticsearch installed via Ambari

mrdavidsheard · ‎12-30-2016

Worked like a charm. Appreciated

mrdavidsheard · ‎12-29-2016

Note Metrics Monitor starts but Metrics Collector doesn't. Version2.4.2.0

mrdavidsheard · ‎12-29-2016

Tried to start ambari metric service after install but it errors saying a missing directory, see below. ls /usr/lib/ams-hbase conf docs hbase-webapps lib But no bin directory? raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname)) resource_management.core.exceptions.Fail: Applying File['/usr/lib/ams-hbase/bin/hadoop'] failed, parent directory /usr/lib/ams-hbase/bin doesn't exist

mrdavidsheard · ‎12-28-2016

Ok sorted yum remove ambari-agent on the servers where I did a manual install and rerun using Ambari

mrdavidsheard · ‎12-28-2016

I notice it launches 2 /usr/bin/python /usr/lib/python2.6/site-packages/ambari_agent/main.py scripts ?? [root@ec2-xxxxx ec2-user]# ps aux | grep ambari root 16253 0.0 0.1 385276 17736 ? Sl 04:19 0:00 /usr/bin/python /usr/lib/python2.6/site-packages/ambari_agent/main.py start --expected-hostname=ec2-xxxxx.ap-southeast-2.compute.amazonaws.com root 16344 0.0 0.0 112648 960 pts/3 S+ 04:39 0:00 grep --color=auto ambari [root@ec2-xxxx ec2-user]# ps aux | grep ambari root 16253 0.0 0.1 385276 17736 ? Sl 04:19 0:00 /usr/bin/python /usr/lib/python2.6/site-packages/ambari_agent/main.py start --expected-hostname=ec2-xxxx.ap-southeast-2.compute.amazonaws.com root 16346 0.0 0.0 112648 956 pts/3 S+ 04:39 0:00 grep --color=auto ambari

mrdavidsheard · ‎12-28-2016

Checked https://community.hortonworks.com/questions/145/openssl-error-upon-host-registration.html And I'm using java 8 on both. Tried using ssh keys and also manual install of ambari-agent but no luck. I can get the server running and the agents on both servers but no luck via ambari installing HDP. I can see the agent startup when lauching from Ambari but then it can't connect to the server

mrdavidsheard · ‎12-28-2016

Still having issue ========================== Creating target directory... ========================== Command start time 2016-12-28 04:19:41 Connection to ec2-54-206-122-4.ap-southeast-2.compute.amazonaws.com closed. SSH command execution finished host=ec2-xxxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:41 ========================== Copying ambari sudo script... ========================== Command start time 2016-12-28 04:19:41 scp /var/lib/ambari-server/ambari-sudo.sh host=ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:41 ========================== Copying common functions script... ========================== Command start time 2016-12-28 04:19:41 scp /usr/lib/python2.6/site-packages/ambari_commons host=ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:42 ========================== Copying OS type check script... ========================== Command start time 2016-12-28 04:19:42 scp /usr/lib/python2.6/site-packages/ambari_server/os_check_type.py host=ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:42 ========================== Running OS type check... ========================== Command start time 2016-12-28 04:19:42 Cluster primary/cluster OS family is redhat7 and local/current OS family is redhat7 Connection to ec2-xxxxx.ap-southeast-2.compute.amazonaws.com closed. SSH command execution finished host=ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:42 ========================== Checking 'sudo' package on remote host... ========================== Command start time 2016-12-28 04:19:42 Connection to ecxxxxxx.ap-southeast-2.compute.amazonaws.com closed. SSH command execution finished host=ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:43 ========================== Copying required files... Ambari repo file not found: /etc/yum.repos.d/ambari.repo ========================== Copying setup script file... ========================== Command start time 2016-12-28 04:19:43 scp /usr/lib/python2.6/site-packages/ambari_server/setupAgent.py host=ec2-xxxxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:43 ========================== Running setup agent script... ========================== Command start time 2016-12-28 04:19:43 ('INFO 2016-12-28 04:14:20,333 NetUtil.py:62 - Connecting to https://ec2-xxxxxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca WARNING 2016-12-28 04:14:20,334 NetUtil.py:93 - Failed to connect to https://ec2-xxxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca due to [Errno 111] Connection refused WARNING 2016-12-28 04:14:20,334 NetUtil.py:116 - Server at https://ec2-xxxxxxxx.ap-southeast-2.compute.amazonaws.com:8440 is not reachable, sleeping for 10 seconds... INFO 2016-12-28 04:14:20,334 HeartbeatHandlers.py:115 - Stop event received INFO 2016-12-28 04:14:20,334 NetUtil.py:122 - Stop event received INFO 2016-12-28 04:14:20,334 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-12-28 04:14:20,334 ExitHelper.py:67 - Cleanup finished, exiting with code:0 INFO 2016-12-28 04:14:22,156 main.py:223 - Agent died gracefully, exiting. INFO 2016-12-28 04:14:22,157 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-12-28 04:19:44,977 main.py:90 - loglevel=logging.INFO INFO 2016-12-28 04:19:44,977 main.py:90 - loglevel=logging.INFO INFO 2016-12-28 04:19:44,977 main.py:90 - loglevel=logging.INFO INFO 2016-12-28 04:19:44,978 DataCleaner.py:39 - Data cleanup thread started INFO 2016-12-28 04:19:44,979 DataCleaner.py:120 - Data cleanup started INFO 2016-12-28 04:19:44,980 DataCleaner.py:122 - Data cleanup finished INFO 2016-12-28 04:19:45,022 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2016-12-28 04:19:45,023 main.py:349 - Connecting to Ambari server at https://ec2-xxxxxxxx.ap-southeast-2.compute.amazonaws.com:8440 (xxxxxxx) INFO 2016-12-28 04:19:45,023 NetUtil.py:62 - Connecting to https://ec2-xxxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca WARNING 2016-12-28 04:19:45,024 NetUtil.py:93 - Failed to connect to https://ec2-xxxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca due to [Errno 111] Connection refused WARNING 2016-12-28 04:19:45,025 NetUtil.py:116 - Server at https://ec2-xxxxxxx.ap-southeast-2.compute.amazonaws.com:8440 is not reachable, sleeping for 10 seconds... ', None) ('INFO 2016-12-28 04:14:20,333 NetUtil.py:62 - Connecting to https://ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca WARNING 2016-12-28 04:14:20,334 NetUtil.py:93 - Failed to connect to https://ec2-xxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca due to [Errno 111] Connection refused WARNING 2016-12-28 04:14:20,334 NetUtil.py:116 - Server at https://ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com:8440 is not reachable, sleeping for 10 seconds... INFO 2016-12-28 04:14:20,334 HeartbeatHandlers.py:115 - Stop event received INFO 2016-12-28 04:14:20,334 NetUtil.py:122 - Stop event received INFO 2016-12-28 04:14:20,334 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-12-28 04:14:20,334 ExitHelper.py:67 - Cleanup finished, exiting with code:0 INFO 2016-12-28 04:14:22,156 main.py:223 - Agent died gracefully, exiting. INFO 2016-12-28 04:14:22,157 ExitHelper.py:53 - Performing cleanup before exiting... INFO 2016-12-28 04:19:44,977 main.py:90 - loglevel=logging.INFO INFO 2016-12-28 04:19:44,977 main.py:90 - loglevel=logging.INFO INFO 2016-12-28 04:19:44,977 main.py:90 - loglevel=logging.INFO INFO 2016-12-28 04:19:44,978 DataCleaner.py:39 - Data cleanup thread started INFO 2016-12-28 04:19:44,979 DataCleaner.py:120 - Data cleanup started INFO 2016-12-28 04:19:44,980 DataCleaner.py:122 - Data cleanup finished INFO 2016-12-28 04:19:45,022 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2016-12-28 04:19:45,023 main.py:349 - Connecting to Ambari server at https://ec2-xxxxx.ap-southeast-2.compute.amazonaws.com:8440 (54.206.122.4) INFO 2016-12-28 04:19:45,023 NetUtil.py:62 - Connecting to https://ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca WARNING 2016-12-28 04:19:45,024 NetUtil.py:93 - Failed to connect to https://ec2-xxxxxx.ap-southeast-2.compute.amazonaws.com:8440/ca due to [Errno 111] Connection refused WARNING 2016-12-28 04:19:45,025 NetUtil.py:116 - Server at https://ec2-xxxxx.ap-southeast-2.compute.amazonaws.com:8440 is not reachable, sleeping for 10 seconds... ', None) Connection to ec2-xxxxx.ap-southeast-2.compute.amazonaws.com closed. SSH command execution finished host=ec2-xxxxx.ap-southeast-2.compute.amazonaws.com, exitcode=0 Command end time 2016-12-28 04:19:47 Registering with the server... Registration with the server failed.

mrdavidsheard · ‎12-26-2016

Tried both public and private in GUI also in hosts file with no luck

mrdavidsheard · ‎12-26-2016

yes I can even ssh key-less

mrdavidsheard · ‎12-26-2016

This is the error - Registering with the server... Registration with the server failed.

mrdavidsheard · ‎12-26-2016

Hi, Except for the instance that has Ambari Server and Agent running I can't install HDP 2.5 using Ambari 2.5. I've installed the agents manually successfully. I'm not sure I ave the /etc/hosts file right. I'm using <private IP> <pribate DNS> and the <private DNS> in the Ambari GUI. But they fail to install all except the instance that has Ambari Server runnning. I'm using Centos 7. Any ideas?

mrdavidsheard · ‎09-27-2016

Thanks for your reply. We are running on 2.5. We found the jars in the path issue and resolved the first issue. The select works fine. But something like count, which uses map-reduce just doesn't return. It works like a charm in the 'hive' shell. Any thoughts?

mrdavidsheard · ‎09-27-2016

we are having the same issue. Any suggestions would be appreciated

mrdavidsheard · ‎09-19-2016

It seems it isn't happy with using a function such as max 😞 Work around (schema and table name withheld): select * from (select nextval from <schema>.<table_name> order by nextval desc) where rownum=1

mrdavidsheard · ‎09-19-2016

Any advice, please. I get an error doing any sort of select statement. E.G. select max(nextval) from <schema>.<table_name> But I get an error below (schema and table name withheld: Unable to execute SQL select query select max(nextval) from <schema>.<table_name>; due to org.apache.nifi.processor.exception.ProcessException: java.sql.SQLSyntaxErrorException: ORA-00911: invalid character ; routing to failure: org.apache.nifi.processor.exception.ProcessException: java.sql.SQLSyntaxErrorException: ORA-00911: invalid character

Online	Offline
Last Visited	‎11-10-2017 03:31 AM

Date Registered	‎09-01-2016 03:50 AM
Last Visited	‎11-10-2017 03:31 AM
Total Messages Posted	27
Total Kudos Received	4

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: NiFi executesql invalid character exception

Re: Oracle CLOB not support by any prosessor in Ni...

Oracle CLOB not support by any prosessor in NiFi

Re: SAM Error Deploying due to Dependency Resoluti...

SAM and Storm not playing

SAM Error Deploying due to Dependency Resolution E...

Re: Atlas Elasticsearch Hook

Atlas Elasticsearch Hook

Re: metron UI plugin:elasticsearch Service Unavail...

metron UI plugin:elasticsearch Service Unavailable

Re: AMBARI_METRIC won't start after install

Re: AMBARI_METRIC won't start after install

AMBARI_METRIC won't start after install

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Install HDP 2.5 using Ambari 2.4 on AWS Fails

Install HDP 2.5 using Ambari 2.4 on AWS Fails

Re: Zeppelin Hive Interpreter - add dependencies p...

Re: Zeppelin Hive Interpreter - add dependencies p...

Re: NiFi executesql invalid character exception

NiFi executesql invalid character exception